Dear MT Community, Modifications to the schedule of current NIST machine translation commitments have necessitated yet another schedule change for OpenMT. The upside is that this will allow more time for training and development as well as more time to create larger test set. The test set size for the CTS genre is now 25K instead of 5K.
Revised schedule: * Registration deadline September 9, 2014 * Dry run Nov 3-7, 2014 * Eval (audio) Jan 12-17, 2015 * Eval (text) Jan 19-24, 2015 * Workshop April 2015 (in Washington DC area, exact date TBD) Since the eval is now in early 2015, we shall refer to it as OpenMT15 :-) For more information, please visit the website at http://www.nist.gov/itl/iad/mig/openmt15.cfm Best Regards, Audrey -- Audrey Tong NIST Multimodal Information Group 100 Bureau Drive, Stop 8940 Gaithersburg, MD 20899 U.S.A. Tel: 301-975-6091 Fax: 301-670-0939 From: <Tong>, Audrey Tong <[email protected]<mailto:[email protected]>> Date: Friday, February 28, 2014 1:54 PM To: MT_LIST <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: mt_poc <[email protected]<mailto:[email protected]>>, Audrey Tong <[email protected]<mailto:[email protected]>> Subject: Re: Announcing NIST OpenMT14 Dear MT Community, We want to let you know that we have revised the evaluation plan. The main changes (some prompted by feedback) include: * Clarification on the constrained training data condition - For OpenMT14, all data regardless of the source language that are listed in the LDC license can be used for the constrained training data condition. This is a less stricter interpretation than the one used in OpenMT12 to encourage participants to try new ideas. * Test set size for the text genre (SMS/chat) is significantly larger than initially planned (25K instead of 5K) * A revised evaluation schedule to give participants more time to prepare for the evaluation (see Schedule below) Schedule: * February 1, 2014: Initial draft evaluation plan available * February 1 - June 13, 2014: Registration period * February - June 13, 2014: Training data available from LDC with incremental releases of new data until the end of May 2014 * June 16 - 20, 2014: Dry run period * July 14 - 18, 2014: Main evaluation period for audio track * July 21 - 25, 2014: Main evaluation period for text track * August 4 - 15, 2014: Human assessment period * August 28 - 29, 2014: Workshop in the Washington DC area The updated evaluation plan has been posted on the OpenMT14 page http://www.nist.gov/itl/iad/mig/openmt14.cfm We encourage early registration so the training data can be used fully. Let us know if you have any questions at [email protected]<mailto:[email protected]>. Best Regards, Audrey -- Audrey Tong NIST Multimodal Information Group 100 Bureau Drive, Stop 8940 Gaithersburg, MD 20899 U.S.A. Tel: 301-975-6091 Fax: 301-670-0939 From: <Tong>, Audrey Tong <[email protected]<mailto:[email protected]>> Date: Friday, January 31, 2014 3:19 PM To: MT_LIST <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: Audrey Tong <[email protected]<mailto:[email protected]>> Subject: Announcing NIST OpenMT14 Dear MT Community, We are pleased to announce the next NIST Open Machine Translation evaluation, OpenMT14. Continuing the tradition of investigating the limits of translation technologies and measurement techniques, OpenMT14 will focus on system performance on informal data genres and explore common MT measurement methods on these data types. Highlights of OpenMT14 include: * Evaluation on informal data genres (SMS/Chat, Conversational Telephone Speech) for Arabic-to-English and Chinese-to-English, * Inclusion of audio input track, and * Explore common MT measurement techniques on these informal data genres. Schedule (Tentative): * February 1, 2014: Initial draft evaluation plan available * February 1 - April 30, 2014: Registration period * February - April, 2014: Training data available from LDC with incremental releases of new data until the end of April 2014 * TBD: Dry run period * May 12 - 16, 2014: Main evaluation period for audio track; output due at NIST May 16, 11.59am ET * May 19 - 23, 2014: Main evaluation period for text track; output due at NIST May 23, 11.59am ET * May 28 - June 18, 2014: Human assessment period * July, 2014: Workshop in the Washington DC area Updates will be posted to the OpenMT14 website http://www.nist.gov/itl/iad/mig/openmt14.cfm If you have any questions or comments, please email us at [email protected]<mailto:[email protected]>. Feel free to pass this on to those who may be interested. Best Regards, Audrey -- Audrey Tong NIST Multimodal Information Group 100 Bureau Drive, Stop 8940 Gaithersburg, MD 20899 U.S.A. Tel: 301-975-6091 Fax: 301-670-0939
_______________________________________________ Mt-list site list [email protected] http://lists.eamt.org/mailman/listinfo/mt-list
