Dear colleagues, As announced a few weeks ago, the fourth iteration of the GEM workshop will be held as part of ACL <https://2025.aclweb.org/>, July 27–August 1st, 2025. This year we’re planning a major upgrade to the GEM workshop, which we dub GEM2, through the introduction of a large dataset of 1B model predictions together with prompts and gold standard references, encouraging researchers from all backgrounds to submit work on meaningful, efficient and robust evaluation of LLMs.
In this second CfP, we are happy to announce that (i) the large datasets of model predictions have been released (DOVE <https://slab-nlp.github.io/DOVE/> and DataDecide <https://huggingface.co/datasets/allenai/DataDecide-eval-instances>), and (ii) GEM2 will host the ReproNLP <https://repronlp.github.io/> shared task results. Important Dates - April 11: Direct paper submission deadline (ARR). - May 5: Pre-reviewed (ARR) commitment deadline. - May 19: Notification of acceptance. - June 6: Camera-ready paper deadline. - July 7: Pre-recorded videos due. - July 31 - August 1: Workshop at ACL in Vienna. Please check the GEM website <https://gem-benchmark.com/workshop> for submission links, templates, and more details. For any questions, please email [email protected]. best, simon *ADAPT Research Centre / Ionaid Taighde ADAPT* *School of Computing, Dublin City University, Glasnevin Campus / Scoil na Ríomhaireachta, Campas Ghlas Naíon, Ollscoil Chathair Bhaile Átha Cliath*
_______________________________________________ Corpora mailing list -- [email protected] https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/ To unsubscribe send an email to [email protected]
