Dagstuhl Perspectives Workshop — Conversational Agents: A Framework for Evaluation (CAFE) .
Please note that the event you are currently viewing has already passed and is now considered archived.
| Dates | August 25-30, 2024 |
|---|---|
| Location | Schloss Dagstuhl, Germany |
| Organizer | Schloss Dagstuhl - Leibniz-Zentrum fur Informatik |
| Topics | |
| Conversational Agents (CA) as frontends to Information Retrieval (IR) and Recommender Systems (RS) become more popular in everyday life, with a wider range of users and usages. The latest developments in Large Language Models (LLMs) will have tremendous consequences, especially for the workplace and education. In this Dagstuhl Perspectives Workshop, we want to focus on the evaluation of these conversational systems, as appropriate methods are still missing. The quality of these systems is limited in terms of personalization, veracity and correctness, bias, transparency, trustworthiness, and understandability. Thus, evaluation methods must address these shortcomings. Furthermore, user- and usage-oriented aspects should become a more prominent and integral component in evaluations, as the user population as well as the tasks these systems are used for become more heterogeneous. For this reason, the topic-centric view of relevance has to be extended to a broad range of facets which are important for the different usage scenarios. Therefore, suitable evaluation criteria have to be specified, which form the basis for defining appropriate measures. Most importantly, the range of evaluation methods must be revisited and extended, as popular methods like the Cranfield approach or crowdsourcing must be complemented by new evaluation methods and strategies specifically tailored to this new type of system. | |
Similar events
2026 IEEE/MTT-S International Microwave Symposium - IMS 2026
June 7-12, 2026
Boston, Massachusetts, United States
June 7-12, 2026
Boston, Massachusetts, United States
2036 IEEE/MTT-S International Microwave Symposium - IMS 2036
June 1-6, 2036
Boston, Massachusetts, United States
June 1-6, 2036
Boston, Massachusetts, United States
2031 IEEE/MTT-S International Microwave Symposium - IMS 2031
June 1-6, 2031
Boston, Massachusetts, United States
June 1-6, 2031
Boston, Massachusetts, United States
2026 IEEE/PES Transmission and Distribution Conference and Exposition (T&D)
May 4-7, 2026
Chicago, Illinois, United States
May 4-7, 2026
Chicago, Illinois, United States
