Apologies for this second message. This one is the same except without the evaluation plan attached, as I have received a number of bounces.
Kay. From: Peterson, Kay Sent: Tuesday, January 26, 2010 4:02 PM To: 'metricsmt_l...@nist.gov'; 'mt-list@eamt.org' Subject: Announcing NIST MetricsMATR10 Dear MT and MT metric community, We're pleased to announce the second NIST Metrics for Machine Translation Challenge, MetricsMATR10, which will continue the quest for improved automated machine translation metrology. MetricsMATR10 will join efforts with the Workshop on Statistical Machine Translation (WMT) as the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR10<http://www.statmt.org/wmt10>, a workshop of ACL 2010<http://www.acl2010.org/>. MetricsMATR10 will represent the "evaluation task" of the workshop. MetricsMATR10 will be implemented similarly to the last MetricsMATR challenge, MetricsMATR08. Participants will submit their metric(s) to NIST for installation. NIST will run them on an evaluation set of machine translations that remains at NIST, and report correlations with human assessments. Because of the collaboration with WMT, NIST will also be able to evaluate the metrics against human assessments of the machine translations from the WMT10 translation task. The MetricsMATR10 website<http://www.nist.gov/itl/iad/mig/metricsmatr10.cfm> will be up very soon; you will find the evaluation plan and registration form there. For the time being, the evaluation plan is attached. Important dates: Immediately: MetricsMATR08 development data available as a re-release for MetricsMATR10 March 26 2010: Metric submission commitment due at NIST March 26 - May 14 2010: Metric submission period; metrics must be installed and operational at NIST by May 14, 5pm EDT June 18 2010: Informal metric descriptions due at NIST July 15 - 16 2010: Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR10 (at ACL 2010 in Uppsala, Sweden) Feel free to pass this announcement on to those outside of the immediate MT community who might be interested in participating. Please contact mt_...@nist.gov<mailto:mt_...@nist.gov> if you have any questions. Best, Kay / NIST Open MT Team. -- Kay Peterson NIST ITL IAD Multimodal Information Group http://www.nist.gov/itl/iad/mig (+1) 301-975-2960 Phone (+1) 301-670-0939 Fax
_______________________________________________ Mt-list mailing list
