DialDoc Workshop
About
Welcome to the Third DialDoc Workshop co-located with ACL 2023.
The DialDoc workshop focuses on Document-Grounded Dialogue and Conversational Question Answering. Given the vast amount of content created every day in various mediums, it is a meaningful yet challenging task not only to make such content accessible to end users via various conversational interfaces, but also to make sure the responses provided by the models are grounded and faithful with respect to the knowledge sources.
Special Theme For our third workshop, we would like to highlight the theme of Factual Consistency. With recent advances of large language models, a major issue may arise when systems generate responses that include factual inconsistencies with respect to external sources, creating implications on user trust and safety. We seek submissions that tackle challenges around this issue, including, but not limited to, automatic evaluation methods, human evaluation, modeling techniques and datasets.
Shared Task We also host a shared task on grounded dialogue based on multilingual documents, which aims to extend the current advances to languages other than English.
Topics of interests include, but are not limited to:
- Document-grounded dialogue and conversational machine reading;
- Knowledge-grounded dialogue generation with pre-trained language models;
- Open domain dialogue and conversational QA;
- Topical open-domain conversational chat;
- Parsing semi-structured document content for dialogue and conversational QA, table reading;
- Evaluation for document-grounded dialogue;
- Interpretability and faithfulness in dialogue modeling;
- Dialogue summarization and query-based summarization.
Program
- 08:55 - 09:00: Opening Remarks
- 09:00 - 09:35: Invited Talk I by Hannaneh Hajishirz - “Improved Factual Precision in Long-form Text Generation with Fine-grained Evaluation and Feedback”
- 09:35 - 10:30: Oral Presentations I - (Session Chair: Nouha Dziri)
- 09:35 - 09:55: Follow the Knowledge: Structural Biases and Artefacts in Knowledge Grounded Dialog Datasets
- 09:55 - 10:15: Revisiting Sentence Union Generation as a Testbed for Text Consolidation
- 10:15 - 10:30: Graph-Guided Unsupervised Knowledge Identification for Dialogue Agents
- 10:30 - 10:50: Coffee Break
- 10:50 - 11:25: Invited Talk II by Rui Yan - “Recent Progress of Conversational AI in the Open Domain”
- 11:25 - 12:30: Short Presentations - (Session Chair: Cam-Tu Nguyen)
- 11:25 - 11:35: ConvRGX: Recognition, Generation, and Extraction for Self-trained Conversational Question Answering
- 11:35 - 11:45: Position Matters! Empirical Study of Order Effect in Knowledge-grounded Dialogue
- 11:45 - 11:55: C-PMI: Conditional Pointwise Mutual Information for Turn-level Dialogue Evaluation
- 11:55 - 12:25: Shared Task Paper Presentations
- Cross-lingual Data Augmentation for Document-grounded Dialog Systems in Low Resource Languages
- Enhanced Training Methods for Multiple Languages
- SLDT: Sequential Latent Document Transformer for Multilingual Document-based Dialogue
- Language-Agnostic Transformers and Assessing ChatGPT-Based Query Rewriting for Multilingual Document-Grounded QA
- 12:30 - 13:30: Lunch break
- 13:30 - 14:00: Shared Task Results - (Session Chair: Bowen Yu)
- 13:30 - 13:40: Shared Task Overview
- 13:40 - 14:00: Shared Task Paper Presentations
- Exploration of Multilingual Prompts in Document-grounded Dialogue
- Enhancing Multilingual Document-Grounded Dialogue Using Cascaded Prompt-Based Post-Training Models
- 14:00 - 14:35: Invited Talk III by Xiang Ren - “Reflex or Reflect: When Do Language Tasks Need Slow Reasoning? “
- 14:35 - 14:50: Coffee break
- 14:50 - 16:05: Oral Presentations II and Best Paper Award - (Session Chair: Hui Wan)
- 14:50 - 15:10: AlignScore: Evaluating Factual Consistency with A Unified Alignment Function
- 15:10 - 15:30: A Dialogue System for Assessing Activities of Daily Living: Improving Consistency with Grounded Knowledge
- 15:30 - 15:50: Ontologically Faithful Generation of Non-Player Character Dialogues
- 15:50 - 16:10: MoQA: Benchmarking Multi-Type Open-Domain Question Answering
- 16:10 - 16:45: Invited Talk IV by Greg Durrett - “Assessing LLM Faithfulness: Lessons from Political Fact-checking”
- 16:45 - 16:50: Closing Remarks
Call for Papers
We welcome submissions of original work as long or short papers, as well as non-archival papers. We also accept papers submitted via ARR as either archival or non-archival submissions. All accepted papers will be presented at the workshop.
We will have the Best Paper Award and Best Student Paper Award, which will be announced during the workshop.
Submission Instructions
Formatting Guidelines: We accept long (eight pages plus unlimited references) and short (four pages plus unlimited references) papers, which should conform to ARR CFP guidelines.
Non-Archival Submissions: The accepted papers can opt to be non-archival.
The submission site on OpenReview is available here.
Review Process
All submissions will be peer-reviewed by at least two reviewers. The reviewing process will be two-way anonymized. Authors are responsible for anonymizing their submissions.
Important Dates
- Direct paper submissions: May 2, 2023
(April 24, 2023) - Paper submissions with ARR reviews: May 10, 2023
(April 24, 2023) - Notification of Acceptance: May 23, 2023
May 22, 2023 - Camera-ready Paper Due Date: May 30, 2023
- Pre-recorded video due: June 12, 2023
- Workshop Date: July 13, 2023
Shared Task
Leaderboard
The competition is hosted on the Tianchi platform - Leaderboard!
Please also join our Google Group to further communications.
Task
We invite the participants to join us to tackle a challenging Document-grounded Dialogue task in a multilingual setting! In this competition, you will be given a conversational query and a set of domain documents in Vietnamese and French. Your goal is to generate a piece of text that answers the query in the target language.
To assess your performance, we will use the evaluation metrics such as token-level F1, SacreBleu and Rouge-L. You will be scored based on how accurately your generated response matches the ground-truth answer and how well it aligns with the target language. We will provide training data for this task, including 3,446 turns in Vietnamese and 3,510 turns in French. We have also organized the currently available Chinese and English document-grounded dialogue data. We hope that participants can leverage the linguistic similarities, for example, a large number of Vietnamese words are derived from Chinese, and English and French both belong to the Indo-European language family, to improve their models’ performance in Vietnamese and French. Please note that no additional human annotated data is allowed.
Baseline
We have also provided a baseline model, which can be found at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/acl23doc2dial.
Prizes
Cash prizes will be awarded to the top-performing participants. The winners will be determined based on their scores on the evaluation metrics. Additionally, we will require all winners to submit a technical paper describing their methods and approaches.
The prize pool is $7000 in total:
- 1st Place: $3000;
- 2nd Place: $1600;
- 3rd Place: $1000;
- 4th Place: $800;
- 5th Place: $600.
Important Dates
The challenge includes leaderboards for two task settings with two phases, Dev (TestDev) and Test phase,
- Training data & Dev data release: February 17, 2023
- Test data release: March 25, 2023
- Winners announced: April 07, 2023
- Paper submission: May 2nd, 2023
(April 24, 2023)
Invited Speakers
Organization
Workshop Organizers
Shared Task Organizers
Program Committee
Srijan Bansal (Carnegie Mellon University)
Daniele Bonadiman (Amazon)
Pengshan Cai (University of Massachusetts - UMass Amherst)
Danish Contractor (IBM Research AI)
Le Anh Cuong (Ton Duc Thang University, Vietnam
Nico Daheim (Technische Universität Darmstadt)
Sam Davidson (UC Davis)
Wanyu Du (University of Virginia)
Hao Fang (Microsoft)
Jatin Ganhotra (IBM Research AI)
Chang Gao (The Chinese University of Hong Kong)
Chulaka Gunasekara (IBM Research AI)
Ankita Gupta (University of Massachusetts - UMass Amherst)
Han He (Emory University)
Xiangkun Hu (Amazon)
Etsuko Ishii (Hong Kong University of Science and Technology)
Ehsan Kamalloo (University of Alberta)
Zichao Li (McGill / Mila)
Xing Han Lu (McGill / Mila)
Tran Thi Oanh (Vietnam National University, Vietnam)
Baolin Peng (Microsoft)
Xuan-Hieu Phan (Vietnam National University, Vietnam)
Kun Qian (Columbia University)
Le Hoang Quynh (Vietnam National University, Vietnam)
Min Yang (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
Bowen Yu (DAMO Academy Alibaba Group)
Contact
Please join our Google Group for the updates!
Please email us at dialdoc2023-organizers@@googlegroups.com for questions and suggestions.