[Mt-list] Announcement: A NIST Evaluation of MT Metrics

[Mark Przybocki]

We are pleased to announce a new NIST effort designed to *evaluate 
metrics *that are employed for the evaluation of MT technology.

   See:  http://www.nist.gov/speech/tests/metricsmatr/  for current 
plans and the evaluation specification document.

The Basics:

(1) NIST will distribute development-data to metric developers

- development-data consists of system translations, references 
translations and human assessments of adequacy and preferences.

(2) Metric developers create innovative MT metrics that address the 
flaws of current MT metrics

(3) NIST installs the new metric software locally, and processes a large 
evaluation-data set

- evaluation-data will be similar to but more expansive than the 
development data, and is not to be released.

(4) Reports of correlation studies will be discussed at the AMTA-2008 
Workshop:  "Metrics MATR: The Metrics for MAchine TRanslation challenge".

For more specific information please visit the link above, and feel free 
to address related comments or questions to "[EMAIL PROTECTED]".

Thank You.
Mark Przybocki  (for the NIST Open MT Team)

--
//  Mark Przybocki - NIST,  Phone: 301/975-3347  FAX: 301/670-0939  \\

_______________________________________________
Mt-list mailing list