Dear all,
This is perhaps a good time to explain what has happened with respect to the
MT Certification Initiative.
History.
Around the time of the 1997 Summit, several angry MT vendors approached the
IAMT Board and asked for some action to help inform the general public about
types and levels of MT products. They did so in reaction to one or two MT
companies' products that, in the angry vendors' view, were not really MT
systems, but rather translation aids, although the products advertised MT
of many languages at very low prices.
The IAMT Board instituted a panel to come up with what what initially called
a Seal of Approval but is now being called a Certification. This panel
contains MT experts from North America and Europe (listed below). Over the
past two years, the panel has come up with a set of criteria that define
various classes of language processing software. The panel also came up with
a procedure by which the Certification can be brought into common use. Both
these aspects are described below.
An abbreviated version of the full discussion by the Certification Panel is
available upon request. Please write to [EMAIL PROTECTED]
It must be noted briefly but seriously that the point of the Certification
is to provide useful guidance to non-expert users buying software. It is
not a research topic for experts and others building or studying MT technology.
Furthermore, the IAMT cannot allow itself to become partisan in the wars of
commerce. Therefore the Certification cannot be an evaluation; evaluation is
too difficult and controversial, and too easily slanted toward one or another
application. In summary, the Certification has to be simple, fair, and cheap
to perform.
Certification Panel.
The panel grew slowly over the two years, starting with:
Eduard Hovy, USC Information Sciences Institute, Los Angeles, CA (chair)
Laurie Gerber, recently of SYSTRAN, La Jolla, CA
John Hutchins, current President of IAMT, Norwich, England
Sharon O'Brien, ALPNET, Washington, DC
John O'Hara, recently of Lernout & Hauspie, San Diego, CA
Joerg Schuetz, IAI, Saarbruecken, Germany
Muriel Vasconcellos, MTNI Newsmagazine editor, San Diego, CA
John White, Litton PRC, Washington, DC
with additional comments from
Jeff Allen, ELRA, Paris, France
Marjorie Leon, Pan American Health Organization, Washington, DC
Helen McKay, ToggleText, Black Rock, Australia
Alan Melby, Brigham Young University, Provo, UT
L. Chris Miller, MCS, Washington, DC
Reba Rosenbluth, recently from SYSTRAN, La Jolla, CA
and several others.
Categories.
After extensive discussion, the Certification Panel has created the following
categories for the certification:
IAMT Category: Translation Support Tool, Electronic Dictionary
IAMT Category: Translation Support Tool, Terminology Management System
IAMT Category: Translation Support Tool, Translation Memory System
IAMT Category: Translation Support Tool, Foreign Language Authoring System
IAMT Category: Translation Support Tool, Translator Workstation
IAMT Category: Translation Support Tool, ...(for other tools)
IAMT Category: Machine Translation System (basic level)
IAMT Category: Machine Translation System (standard level)
IAMT Category: Machine Translation System (advanced level)
Criteria for the categories:
Criterion: Translation Support Tool
A software system is a "translation support tool" if it takes
input one word or phrase at a time and produces proposed translated
equivalents, which users must then assemble into sentences and texts.
Criterion: Machine Translation System
A software system is a "machine translation system" if it takes
input in the form of full sentences at a time and generates
corresponding full sentences (not necessarily of good quality).
Criterion: Translation Support Tool, Electronic Dictionary
A translation support tool is an "Electronic Dictionary" if it
consists mainly of a bilingual or multilingual dictionary together
with a dictionary lookup facility or mechanism.
Criterion: Translation Support Tool, Terminology Management System
A translation support tool is a "Terminology Management System" if
it consists mainly of methods to help the user construct a multilingual
terminology dictionary, together with a dictionary lookup facility
implemented as a database (and not simply as a word-list), and
includes multiple fields utilized according to standard terminology
practice.
Criterion: Translation Support Tool, Translation Memory System
A translation support tool is a "Translation Memory System" if it
consists mainly of methods to help the user construct and use a
collection of sets of previously translated text (ranging in length
from phrases to whole texts) together with one or more of a text
alignment facility, a text lookup facility, and a storage management
facility.
Criterion: Translation Support Tool, Foreign Language Authoring System
A translation support tool is a "Foreign Language Authoring System"
if it consists of lexicons, phrases, and even full text examples
that assist the user to write documents such as business letters,
contracts, etc., in a language in which they are not fluent.
Criterion: Translation Support Tool, Translator Workstation
A translation support tool is a "Translator Workstation" if it
consists of several Translation Support Tools integrated into a
single framework.
Criterion: Machine Translation System (basic level)
A machine translation system is an "Basic level system" if its
largest dictionary contains fewer than 50,000 entries OR if has
very limited facilities for users to extend the dictionary OR if
its translation capability is restricted to the translation of
single-clause (basic) sentences. Systems at this level are
primarily meant for home use.
Criterion: Machine Translation System (standard level)
A machine translation system is a "Standard level system" if it
contains a dictionary of more than 50,000 root entries AND it is
not restricted to the translation of single-clause (basic) sentences
AND it provides facilities for the user to extend the dictionary.
Systems at this level are primarily meant for home use or
standalone office use.
Criterion: Machine Translation System (advanced level)
A machine translation system is an "Advanced level system" if its
smallest dictionary contains more than 75,000 root entries AND it
is not restricted to the translation of single-clause sentences
AND it provides facilities for the user to extend the dictionary
AND it supports a configuration in which more than one client can
be networked with a single central server computer. Systems at
this level are primarily meant for office use with networking
capabilities.
Certification agencies.
Actually certifying MT systems is a lot of work. Someone has to inspect
every version of every product to be certified, and decide how to certify
it. Agencies exist that certify software and other products, upon payment
of a suitable fee.
Someone also has to periodically check all MT products on the market, to
see that the certification is being applied only where allowed and paid for.
Our model is that the MT vendor would be willing to pay a one-time fee of
a few hundred dollars per version of software to be certified. This model
has to be verified and refined with MT vendors and the certification agency.
I currently have a list of 5 potential certification agencies, and the name
of one individual.
Procedure for establishment.
The current plan for establishing the certification is as follows:
1. Define the certification criteria.
IAMT certification panel
2. Write to a selected group of MT vendors, requesting sample products
for certification.
Eduard Hovy
3. Certify the products, ironing out problems with criteria.
John Hutchins
4. Investigate potential certification alternatives, and select one.
IAMT certification panel
5. Establish certification procedure with agency/individual selected.
Set up monitoring board/panel that liaises with IAMT.
IAMT certification panel
6. Announce to IAMT membership and MT vendors.
IAMT certification panel
Letter to MT vendors.
According to step 2, a draft letter has been created to be sent to MT
vendors. It is attached below.
At this time, the letter is awaiting approval from the IAMT Board. Since
the Certification panel included no representatives from Asia, it was
felt that the panel does not really reflect MT and the IAMT fully. Upon
acceptance by the IAMT Board, the letter can be sent out.
To:
Tokyo Institute of Technology, Tokyo: Hozumi Tanaka [EMAIL PROTECTED]
U of Sheffield, Sheffield: Yorick Wilks [EMAIL PROTECTED]
U of Maryland, College Park: Bonnie Dorr: [EMAIL PROTECTED]
USC/ISI, Marina del Rey: Eduard Hovy [EMAIL PROTECTED]
USC/ISI, Marina del Rey: Kevin Knight [EMAIL PROTECTED]
CMU, Pittsburgh: Jaime Carbonell [EMAIL PROTECTED]
CMU, Pittsburgh: Robert Frederking [EMAIL PROTECTED]
CMU, Pittsburgh: Eric Nyberg [EMAIL PROTECTED]
NMSU, New Mexico: Sergei Nirenburg [EMAIL PROTECTED]
University of Essex: Doug Arnold [EMAIL PROTECTED]
ISSCO, Geneva: Maghia King [EMAIL PROTECTED]
CST, Copenhagen: Bente Maegaard [EMAIL PROTECTED]
BYU, Provo: Alan Melby: [EMAIL PROTECTED]
CEC, Brussel: Dimitrios Theologitis [EMAIL PROTECTED]
ELRA, Paris: Jeff Allen: [EMAIL PROTECTED]
FIDUL, Washington: John White [EMAIL PROTECTED]
PAHO, Washington: Marjorie Leon: [EMAIL PROTECTED]
ALPNET, "Sharon O'Brien" [EMAIL PROTECTED]
Transcend: http://www.translc.com/main.htm
SDL, Tara O'Leary, [EMAIL PROTECTED]
SAKHR, Cairo: [EMAIL PROTECTED] http://www.sakhr.com
LANT, Belgium: Herman Cayers [EMAIL PROTECTED])
TRADOS: Jochen Hummel [EMAIL PROTECTED]
Henri Broekmate [EMAIL PROTECTED]
ATRIL, Madrid: Emilio Benito [EMAIL PROTECTED]
STAR, Ramsen Switzerland: Hanspeter Siegrist [EMAIL PROTECTED]
Corel: Anna Morilla [EMAIL PROTECTED]
Katy Murphy [EMAIL PROTECTED]
Language Partners International, Chicago: Brian Briggs [EMAIL PROTECTED]
Softissimo, Paris: Th�o Hoffenberg [EMAIL PROTECTED]
LanguageForce, Los Angeles: Yuri Mordovskoi [EMAIL PROTECTED]
SYSTRAN, La Jolla: Reba Rosenbluth [EMAIL PROTECTED]
SYSTRAN, La Jolla: Denis Gachot [EMAIL PROTECTED]
Logos, New Jersey: W. Scott Bennett [EMAIL PROTECTED]
L&H, Boston: Gaston Bastiaens [EMAIL PROTECTED]
L&H, San Diego: John O'Hara [EMAIL PROTECTED]
NEC, Tokyo: Akitoshi Okumura [EMAIL PROTECTED]
ToggleText, Australia: Helen McCay [EMAIL PROTECTED]
TRANSPARENT: [EMAIL PROTECTED] http://www.transparent.com/
Accent Software International: Todd Oseth [EMAIL PROTECTED]
Translation Experts Ltd., England: Nenad Koncar [EMAIL PROTECTED]
Translation Service, European Commission: Dorothy Senez [EMAIL PROTECTED]
Dear language technology specialist,
In response to repeated requests, the International Association for Machine
Translation (IAMT), with its daughter organizations the Association for
Machine Translation in the Americas (AMTA), the European Association for
Machine Translation (EAMT), and the Asian Association for Machine
Translation (AAMT) have formed a panel to create a certification of
standards for commercial machine translation (MT) and related systems.
This certification will tell naive buyers something about the products
they are looking at in order to help them make a somewhat more informed
choice.
IAMT, AMTA, EAMT, and AAMT are non-profit associations by and for
professionals in the field of machine translation and translation
technologies. The IAMT counts several hundred members worldwide, drawn
from academia (universities and research laboratories), users (professional
translators and companies using translation services), and commercial
machine translation system developers and vendors.
The international Certification Panel, which is formulating suitable
certification standards, reports to the IAMT. This panel is not doing its
work for the benefit of any single company or organization. It is not being
paid. The certification is an action being taken by the Associations for
the benefit of Machine Translation and multilingual language technology as
a whole.
Though this is a slow process, the Certification panel has created a first
version of the certification criteria, attached below. We are now inviting
comments and software products to be certified according to the criteria,
in order to test the adequacy of their definition. The certifier will be
John Hutchins, noted MT scholar and current president of the IAMT.
We are asking for your help. As a translation technology professional, you
know whether such a certification would be useful to you, and if so, what
it should contain to be most useful. Please read the rest of this letter
and then send us your thoughts and suggestions. Also, if you would like
to help the process (and in so doing have your product certified free of
charge), please contact John Hutchins at
[EMAIL PROTECTED]
to determine how to send him a copy of the product.
Proposed categories for the certification:
IAMT Category: Translation Support Tool, Electronic Dictionary
IAMT Category: Translation Support Tool, Terminology Management System
IAMT Category: Translation Support Tool, Translation Memory System
IAMT Category: Translation Support Tool, Foreign Language Authoring System
IAMT Category: Translation Support Tool, Translator Workstation
IAMT Category: Translation Support Tool, ...(for other tools)
IAMT Category: Machine Translation System (basic level)
IAMT Category: Machine Translation System (standard level)
IAMT Category: Machine Translation System (advanced level)
Criteria for the categories:
Criterion: Translation Support Tool
A software system is a "translation support tool" if it takes
input one word or phrase at a time and produces proposed translated
equivalents, which users must then assemble into sentences and texts.
Criterion: Machine Translation System
A software system is a "machine translation system" if it takes
input in the form of full sentences at a time and generates
corresponding full sentences (not necessarily of good quality).
Criterion: Translation Support Tool, Electronic Dictionary
A translation support tool is an "Electronic Dictionary" if it
consists mainly of a bilingual or multilingual dictionary together
with a dictionary lookup facility or mechanism.
Criterion: Translation Support Tool, Terminology Management System
A translation support tool is a "Terminology Management System" if
it consists mainly of methods to help the user construct a multilingual
terminology dictionary, together with a dictionary lookup facility
implemented as a database (and not simply as a word-list), and
includes multiple fields utilized according to standard terminology
practice.
Criterion: Translation Support Tool, Translation Memory System
A translation support tool is a "Translation Memory System" if it
consists mainly of methods to help the user construct and use a
collection of sets of previously translated text (ranging in length
from phrases to whole texts) together with one or more of a text
alignment facility, a text lookup facility, and a storage management
facility.
Criterion: Translation Support Tool, Foreign Language Authoring System
A translation support tool is a "Foreign Language Authoring System"
if it consists of lexicons, phrases, and even full text examples
that assist the user to write documents such as business letters,
contracts, etc., in a language in which they are not fluent.
Criterion: Translation Support Tool, Translator Workstation
A translation support tool is a "Translator Workstation" if it
consists of several Translation Support Tools integrated into a
single framework.
Criterion: Machine Translation System (basic level)
A machine translation system is an "Basic level system" if its
largest dictionary contains fewer than 50,000 entries OR if has
very limited facilities for users to extend the dictionary OR if
its translation capability is restricted to the translation of
single-clause (basic) sentences. Systems at this level are
primarily meant for home use.
Criterion: Machine Translation System (standard level)
A machine translation system is a "Standard level system" if it
contains a dictionary of more than 50,000 root entries AND it is
not restricted to the translation of single-clause (basic) sentences
AND it provides facilities for the user to extend the dictionary.
Systems at this level are primarily meant for home use or
standalone office use.
Criterion: Machine Translation System (advanced level)
A machine translation system is an "Advanced level system" if its
smallest dictionary contains more than 75,000 root entries AND it
is not restricted to the translation of single-clause sentences
AND it provides facilities for the user to extend the dictionary
AND it supports a configuration in which more than one client can
be networked with a single central server computer. Systems at
this level are primarily meant for office use with networking
capabilities.
Questions:
1. Which of your systems would you like to have classified? (not more
than two, please). Please contact John Hutchins to determine how
to deliver him the system.
2. Are the classification categories good? Any comments?
3. What additional information should the Statement contain?
4. Who should the Statement be sent to?
5. Anything else?
Please send your comments (email preferred) to
Dr. Eduard Hovy, AMTA
USC Information Sciences Institute
4676 Admiralty Way
Marina del Rey, CA 90292-6695
USA
email: [EMAIL PROTECTED]
If you wish it, your comments will be held confidential. Please indicate
this in your reply.
Sincerely,
Eduard Hovy
for the IAMT Certification Panel
----------------------------------------------------------------------------
Eduard Hovy
email: [EMAIL PROTECTED] USC Information Sciences Institute
tel: 310-822-1511 ext 731 4676 Admiralty Way
fax: 310-823-6714 Marina del Rey, CA 90292-6695
project homepage: http://www.isi.edu/natural-language/nlp-at-isi.html
--
For MT-List info, see http://www.eamt.org/mt-list.html
