Hi Julian There was a project at the recent machine translation marathon on the automatic analysis of MT errors. This built on the work of two recent open source tools (Addicter and Hjerson). See 'Automatic MT error analysis' in http://mtmarathon2011.fbk.eu/Projects
You could also check out the recent discussion on this list of the 'Goodness' paper, which was concerned with estimating the confidence of a machine translation output, cheers - Barry On Monday 26 September 2011 11:50:21 Julian Myerscough wrote: > Thanks to everyone for your replies. > > I am not sure if the intent of my original question was clear. What I > would like to do is check newly translated (by a human) sentences > against a set of "approved" human translations with a view to finding > errors or inconsistencies (more emphasis on errors). For example, in the > following human translated English -> Dutch translation, "clockwise" is > mis-translated as "anti-clockwise". > > close firmly by turning the knob clockwise ; > sluit het instrument stevig door de knop tegen de klok in te draaien ; > > This is the kind of error I want to check for, but ideally I also want > to check for antonyms, inconsistent usage of verbs, inconsistent usage > of nouns etc. > > I could imagine this might possible by "mining" the phrase-table file > and to find the minimum probability of each part of a sentence > translation (antonyms might have a negative probability for example). > ...but wondering if there's an easier or better way to do it. > > Looking forward to your thoughts > > Julian > > Victor Chahuneau wrote: > > What you are looking for is the perplexity of the language model (from > > with you can derive the probability of the sentence). Depending on the > > toolkit you are using for language modelling, there are options to score > > a sequence of sentences or a single sentence. e.g. SRILM (ngram) > > -ppl: text file to compute perplexity from > > IRSTLM (compile-lm) > > --score|-s [yes|no] (computes log-prob scores from standard input) > > --eval|-e text-file (computes perplexity of text-file and returns) > > > > Le 23 sept. 2011 à 10:27, Julian Myerscough a écrit : > >> Hi folks, > >> > >> I would like to check newly translated sentences against an existing > >> language model to see how "likely" (or more importantly "unlikely") they > >> are (how well they fit into the existing model). > >> > >> Could you give me any pointers? > >> > >> Cheers > >> > >> Julian > >> > >> _______________________________________________ > >> Moses-support mailing list > >> [email protected] > >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
