I think that we should consider a tighter coupling of the XML markup and the training process. It seems logical to markup the training items along with the test items. We could add a preprocessing where we replace XML delineated phrases in training items with their tag. Since the tags would then be part of the Giza training / phrase extraction they would be included in larger phrases, and get reordering probabilities. This might be useful for putting in placeholders for names and numbers.
--Chris On Dec 12, 2007, at 11:12 AM, Hieu Hoang wrote: > yo loic > > i'm not the xml preprocessing guy for moses, but i don't think > marking up the input does, or should, change the phrase table. > > yes - the markup is primarily used to tell the decoder to avoid > certain words, or give extra info to translate them. > > but it might be a good idea to learn from the markup > > Loic Dugast <[EMAIL PROTECTED]> wrote: Hello > > I have a short question concerning the XML markup in Moses. > What happens in the phrase table if I mark up some entity as say, > type=person>Britney Spears ? > > Is there a way to generate such phrase pair as: > Britney Spears is a great singer > type=person>Britney Spears est une grande chanteuse ? > ... while actually learning: is a great singer > est une grande chanteuse ? > > (allright , with low probabilities associated) > > Or should I consider this being two different issues: 1) marking some > phrases as entities (with XML markup, in order for example not to > translate > them) > 2) generalise the relevant phrase pairs (by adding an ENTITY_PERSON > *factor* > maybe) ? > > Thanks > > Loic > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > Hieu Hoang > http//www.hoang.co.uk/hieu > > Support the World Aids Awareness campaign this month with Yahoo! > for Good > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
