Apologies if you receive multiple copies of this CFP
Covid-19 MLIA @ Eval
Call for Participation in the First Evaluation Round - Round Starts on 23
October 2020
Website: http://eval.covid19-mlia.eu/
Registration: http://eval.covid19-mlia.eu/registration/
Twitter: @covid19mlia
Email: i...@covid19-mlia.eu
Overall Description
Covid-19 MLIA @ Eval organizes a community evaluation effort aimed at
accelerating the creation
of resources and tools for improved MultiLingual Information Access (MLIA) in
the current
emergency situation with a reference to a general public use case:
“Sofia has heard that a drug has been experimented in different countries and
she would like to
have a consolidated and trustworthy view of the main findings, whether the drug
is effective or
not, and whether there are any adverse effects”.
Distillation for the general public also implies a level of
specialist-non-specialist
communication, when the aggregated sources contain both disseminative and
specialised sources.
Therefore, the general public would need to understand medical expertise by
using their
correspondent in the "popular" language or by using an appropriately calibrated
language for the
communication to be effective.
Covid-19 MLIA @ Eval adopts an incremental and iterative evaluation methodology
to enable the
release of intermediate (but functional) resources and to progressively
(iteration-after-iteration) move towards finally consolidated tools and
resources. We plan for
three rounds, tentatively one and half-month long each.
Each round consists of the following phases:
* - data release
* - submission of participants’ runs
* - ground-truth creation (when needed)
* - scoring of participants’ runs
* - writing of the rolling technical report
* - round wrap virtual meeting
An integral part of the Covid-19 MLIA @ Eval approach is the sharing of
information and ideas
among the participants. This happens via the rolling technical report where
participants
describe their solutions as the evaluation rounds progress and the virtual
meetings at the end
of each round where participants briefly present the main highlights of what
worked and what did
not work and interactively discuss together in order to share ideas and improve
for the next
round.
Moreover, to facilitate the exchange of resources and components and to jointly
work together
for improving MLIA technologies for Covid-19, participants are provided with
a dedicated git
repository where to push and share the outcomes of your participation in the
different rounds,
i.e. runs, code, (language) resources, and a technical report. To facilitate
sharing and re-use,
all the contents of the repositories are released under the Creative Commons
Attribution-ShareAlike 4.0 International License.
Tasks
Covid-19 MLIA @ Eval offers the following tasks for its first round:
* - Information Extraction. More information
at: http://eval.covid19-mlia.eu/task1/
* - Multilingual Semantic Search. More information
at: http://eval.covid19-mlia.eu/task2/
* - Machine Translation. More information
at: http://eval.covid19-mlia.eu/task3/
Registration for the tasks is open
at: http://eval.covid19-mlia.eu/registration/
Important Dates - First Round
*
* - Registration Opens: 19 October 2020
* - Registration Closes: 14 May 2021
* - First Round Starts: 23 October 2020 (data release)
* - Submission of Technical Report: 8 January 2021
* - Round Wrap-up Virtual Meeting: 11-15 January 2021 (exact date TBD)
* - First Round Ends: 15 January 2021
More detailed dates are provided on the Web pages dedicated to each specific
task, since the
internal scheduling of each task may differ slightly to better match the task
design.
Organizers
Overall
* - Khalid Choukri, ELDA, France
* - Nicola Ferro, University of Padua, Italy
Information Extraction
* - Cyril Grouin, LIMSI, France
* - Thierry Declerck, DFKI, Germany
* - Pierre Zweigenbaum, LIMSI, France
Multilingual Semantic Search
* - Giorgio Maria Di Nunzio, University of Padua, Italy
* - Maria Eskevich, CLARIN ERIC
Machine Translation
* - Francisco Casacuberta, Universitat Politècnica de València, Spain
* - Miguel Domingo, Universitat Politècnica de València, Spain
* - Mercedes García-Martínez, Pangeanic, Spain
* - Manuel Herranz, Pangeanic, Spain
SUPPORTERS
Covid-19 MLIA @ Eval is an evaluation effort promoted by several communities
which are closely
working together.
* - European Commission https://ec.europa.eu/
* - European Language Resources Coordination
(ELRC), http://www.lr-coordination.eu/
* - European Language Resources Association (ELRA) http://www.elra.info/en/
* - European Research Infrastructure for Language Resources and Technology
(CLARIN) https://www.clarin.eu/
* - CLEF Initiative http://www.clef-initiative.eu/
We hope that other communities will join us in this shared effort. You are very
welcome!
ADDITIONAL INFORMATION
You can find additional information on our
Website: http://eval.covid19-mlia.eu/
You can follow updates through our Twitter account @covid19mlia
Any requests or comments? Drop us an email at: i...@covid19-mlia.eu
_______________________________________________
Mt-list site list
Mt-list@eamt.org
http://lists.eamt.org/mailman/listinfo/mt-list
