[Apologies for multiple postings]
Covid-19 MLIA @ Eval - Community Evaluation Effort for MultiLingual
Information Access (MLIA) on Covid-19
******* ROUND II *******
Call for Participation in the Second Evaluation Round: Runs due by 15
September-15 October 2021
* Website: http://eval.covid19-mlia.eu/ <http://eval.covid19-mlia.eu/>
* Registration: http://eval.covid19-mlia.eu/registration/
<http://eval.covid19-mlia.eu/registration/>
* Twitter: @covid19mlia
* Email: i...@covid19-mlia.eu
*********************
*Overall Description*
Covid-19 MLIA @ Eval organizes a community evaluation effort aimed at
accelerating the creation of resources and tools for improved
MultiLingual Information Access (MLIA) in the current emergency
situation with a reference to a general public use case:
“Sofia has heard that a drug has been experimented in different
countries and she would like to have a consolidated and trustworthy view
of the main findings, whether the drug is effective or not, and whether
there are any adverse effects”.
Distillation for the general public also implies a level of
specialist-non-specialist communication, when the aggregated sources
contain both disseminative and specialised sources. Therefore, the
general public would need to understand medical expertise by using their
correspondent in the "everyday " language or by using an appropriately
calibrated language for the communication to be effective.
Covid-19 MLIA @ Eval adopts an incremental and iterative evaluation
methodology to enable the release of intermediate (but functional)
resources and to progressively (iteration-after-iteration) move towards
finally consolidated tools and resources. We plan for three rounds,
tentatively one and half-month long each.
Each round consists of the following phases:
* data release
* submission of participants’ runs
* ground-truth creation (when needed)
* scoring of participants’ runs
* writing of the rolling technical report
* round wrap virtual meeting
An integral part of the Covid-19 MLIA @ Eval approach is the sharing of
information and ideas among the participants. This happens via the
rolling technical report where participants describe their solutions as
the evaluation rounds progress and the virtual meetings at the end of
each round where participants briefly present the main highlights of
what worked and what did not work and interactively discuss together in
order to share ideas and improve for the next round.
Moreover, to facilitate the exchange of resources and components and to
jointly work together for improving MLIA technologies for Covid-19,
participants are provided with a dedicated git repository where to push
and share the outcomes of your participation in the different rounds,
i.e. runs, code, (language) resources, and a technical report. To
facilitate sharing and re-use, all the contents of the repositories are
released under the Creative Commons Attribution-ShareAlike 4.0
International License.
*Tasks*
Covid-19 MLIA @ Eval offered the following tasks for its first round and
will continue for this second round:
* *Information Extraction <http://eval.covid19-mlia.eu/task1/>*
* *Multilingual Semantic Search <http://eval.covid19-mlia.eu/task2/>*
* *Machine Translation <http://eval.covid19-mlia.eu/task3/>*
We offer tasks in the following languages: Arabic, English, French,
German, Greek, Italian, Spanish, Swedish.
Registration for the tasks is open at:
http://eval.covid19-mlia.eu/registration/
<http://eval.covid19-mlia.eu/registration/>
*Important Dates - Second Round*
- Registration Opens: 14 June 2021
- Second Round Starts: 21 June 2021 (data release)
- Submission of Runs: 15 September - 15 October 2021 (depending on task)
- Submission of Technical Report: 19 November 2021
- Round II Wrap-up Virtual Meeting: 30 November-2 December 2021 (exact
date TBD)
- Second Round Ends: 2 December 2021
More detailed dates are provided on the Web pages dedicated to each
specific task, since the internal scheduling of each task may differ
slightly to better match the task design.
*Organizers*
_Overall_
Khalid Choukri, ELDA, France
Nicola Ferro, University of Padua, Italy
_Data Acquisition and Engineering_
Miltos Deligiannis, ILSP/Athena RC, Greece
Marwa Hadj Salah, ELDA, France
Guillaume Jacquet, JRC, Italy
Vassilis Papavassiliou, ILSP/Athena RC, Greece
Stelios Piperidis, ILSP/Athena RC, Greece
Prokopis Prokopidis, ILSP/Athena RC, Greece
_Information Extraction_
Cyril Grouin, LISN, France
Thierry Declerck, DFKI, Germany
Pierre Zweigenbaum, LISN, France
_Multilingual Semantic Search_
Giorgio Maria Di Nunzio, University of Padua, Italy
Maria Eskevich, CLARIN ERIC
_Machine Translation_
Francisco Casacuberta, Universitat Politècnica de València, Spain
Miguel Domingo, Universitat Politècnica de València, Spain
Mercedes García-Martínez, Pangeanic, Spain
Manuel Herranz, Pangeanic, Spain
*Supporters *
Covid-19 MLIA @ Eval is an evaluation effort promoted by several
communities which are closely working together.
* European Commission <https://ec.europa.eu/>
* European Language Resources Coordination (ELRC)
<http://www.lr-coordination.eu/>
* European Language Resources Association (ELRA)
<http://www.elra.info/en/>
* European Research Infrastructure for Language Resources and
Technology (CLARIN) <https://www.clarin.eu/>
* CLEF Initiative <http://www.clef-initiative.eu/>
We hope that other communities will join us in this shared effort. You
are very welcome!
Any question? Please email us at i...@covid19-mlia.eu
<mailto:i...@covid19-mlia.eu>.
_______________________________________________
Mt-list site list
Mt-list@eamt.org
http://lists.eamt.org/mailman/listinfo/mt-list
