[Apologies if you receive multiple copies of this call]
===========================================================
Second Call for Participation
FAQ Retrieval Using Noisy Queries
(at Forum for Information Retrieval Evaluation 2013)
4th - 6th December 2013
India International Center
New Delhi
http://www.isical.ac.in/~fire/faq-retrieval/2013/faq-retrieval.html
==========================================================
With the increasing use of smart phones and the popularity of dialog based
systems such as that by Samsung (S-voice), Siri (iOS) there is renewed
research
interest in improving speech recognition systems. However, dialog systems in
mobile phones are still very error prone and are do not work well with
all speaker accents. As a result, due to the high error rate in
transcription,
they sometimes do not work as well even in simple IR tasks.
A recent study conducted found that response times to voice activated
dialog systems were significantly longer than traditional text based
queries because users frequently required to correct speech transcription
errors made by the system!
Keeping this problem in mind, in FIRE 2013, we propose a Noisy Queries
based FAQ Retrieval task. The traditional task for SMS Based FAQ Retrieval
at FIRE, will now not only include noisy SMS queries but also queriesgenerated
using an actual speech transcription system.
The goal of this task is to find a question Q* from corpora of FAQs
(Frequently asked questions) that best answers/matches the Noisy query S.
SMS queries which are written in "SMS language" tend to be noisy as
users try and compress text by omitting letters, using slang, etc., due to
a cap on the length of messages (160 characters constitutes one SMS), lack
of screen space (which makes reading large amounts of text difficult), etc.
The messages also frequently contain unintended typographical errors due
to the small size of keypads on mobile phones as also the poor language
skills of the users. The presence of such noise makes this task different
and more challenging than traditional QA retrieval tasks.In addition, for
FIRE 2013 we will also include queries generated using a speech
transcription
system.
The task comprises mono-, cross-language FAQ Retrieval sub-tasks.
All participating teams also need to submit a working note that outlines
the approach followed by them. The Working note is due on November
15, 2013. The post-proceedings will be published in Lecture Notes in
Computer Science (http://www.springer.com/computer/lncs), Springer.
Authors will be invited to submit a revised and expanded version of
their working note to be published in the post-proceedings.
To register and participate please send an email to [email protected]
IMPORTANT DATES:
Training data release: Sept 7,2013
Test data release: Sept 7,2013
Run Submission:Oct 30,2013
Qrel Release:Oct 30,2013
Working Note Due: Nov 15,2013
TASK COORDINATORS:
Danish Contractor, [email protected]
Kanika Narang : [email protected]
L.V. Subramaniam, [email protected]
Deepak S Padmanabhan, [email protected]
Nitendra Rajput [email protected]
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
