Enter code Log in

Aim and Purposes of SIGdial: SIGdial aims to encourage researchers and educators to- promote development and distribution of reusable discourse processing components; explore techniques for evaluation of dialogue systems; share resources and data among the international community; encourage empirical methods in research, ...
Extrinsic Evaluation of Dialog State Tracking and Predictive Metrics for Dialog Policy Optimization

Year: 2014

During the recent Dialog State Tracking Challenge (DSTC), a fundamental question was raised: “Would better performance in dialog state tracking translate to better performance of the optimized policy by reinforcement learning?” Also, during the challenge system evaluation, another non-trivial question arose: “Which evaluation metric and schedule would best predict improvement in overall dialog performance?” This paper aims to answer these questions by applying an off-policy reinforcement learning method to the output of each challenge system. The results give a positive answer to the first question. Thus the effort to separately improve the performance of dialog state tracking as carried out in the DSTC may be justified. The answer to the second question also draws several insightful conclusions on the characteristics of different evaluation metrics and schedules.

agentura Motiv P s.r.o.
Řehořova 726/14, 618 00 Brno

E-mail: helpdesk@motivp.com
Tel: +420 545 234 698
Copyright © 1996-2018, agentura Motiv P s.r.o. All rights reserved.

Agentura Motiv P používá na svém webu www.MotivP.com soubory cookie. Používáním webu vyjadřujete souhlas.