Call for Participants LongEval: Longitudinal Evaluation of Model Performance CLEF 2024 Lab
9-12 September, Grenoble, France -------------------------------------------------------- ** Registration open, training data available ** The CLEF 2024 LongEval lab is motivated by recent research showing that the performance of information retrieval and text classification models drops as the test data becomes more distant in time from the training data. LongEval differs from traditional IR and classification shared tasks with special considerations on evaluating models that mitigate performance drop over time. It encourages participants to develop temporal information retrieval systems and longitudinal text classifiers that survive through dynamic temporal text changes, introducing time as a new dimension for ranking models performance. The lab consists of two tasks: * Task 1. LongEval-Retrieval: The goal of Task 1 is to propose an information retrieval system which can handle changes over the time. The proposed retrieval system should follow the temporal timewise evolution of Web documents. Contact: [email protected] * Task 2. LongEval-Classification: The goal of Task 2 is to propose a temporal persistence classifier which can mitigate performance drop over short and long periods of time compared to a test set from the same time frame as training. Contact: [email protected] Registration ------------ Please use the CLEF registration form (https://clef2024-labs-registration.dei.unipd.it/ ) to register for the tasks. The registration closes on Monday, 22 April 2024. Timeline -------- April 2024: Test data release 22 April 2024: Registration closes 6 May 2024: End of Evaluation Cycle [submission of runs] 31 May 2024: Submission of Participant Papers [CEUR-WS] 31 May–21 June 2024: Review process of participant papers July 2024: Camera ready paper submission September 2024: CLEF Conference Organisers ---------- Rabab Alkhalifa, Hsuvas Borkakoty, Romain Deveaud, Alaa El-Ebshihy, Luis Espinosa-Anke, Gabriela Gonzalez-Saez, Petra Galuščáková, Lorraine Goeuriot, Maria Liakata, Harish Tayyar Madabushi, Philippe Mulhem, Florina Piroi, Martin Popel, Christophe Servan, Arkaitz Zubiaga. _______________________________________________ Corpora mailing list -- [email protected] https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/ To unsubscribe send an email to [email protected]
