Hi, if you use "-xml-input exclusive", then only the specified XML translation is used, and there is no chance for scoring functions to override this decision.
-phi On Thu, May 1, 2014 at 11:08 AM, Tom Hoar <[email protected]> wrote: > Thanks Hieu and Christophe, > > My intention behind the question is to understand how things actually work > now. I'm not trying to steer how it works one way or another. Also, I would > expect that tokens in the forced translations that are OOV in the parallel > corpus and/or LM corpus would cause unpredictable randomized output. So, > let's not worry about those cases. > > That said, let's look at a hypothetical example. What if we're translating > EN-ES and a hypothetical SMT model (phrase table, distortion table and > language model) are all working with in-vocab tokens. The normal and correct > input/output would be: > > % echo 'the fat black cat sleeps' | moses -f moses.ini > el gran gato negro dureme > > Then, we want to force the translation to this incorrect output: > > % echo 'the <xml translation="gran negro gato">fat black cat</xml> sleeps' | > moses -f moses.ini -xml-input exclusive > el gran negro gato dureme > > Is there a chance that the distortion model could override the > forced/incorrect translation and cause the first/correct output? Would > adding the XML attribute prob="1.00000" force the intended translation > regardless of the distortion table and language model? > > Tom > > > > On 05/01/2014 09:37 PM, Hieu Hoang wrote: > > I'm not sure. > > Ideally IMO, the reordering model should be used, even if the translation > comes from XML. The reordering model just gives a score to the translation, > just like any other feature function eg. LM, word penalty. > > However, there might be an optimization where the reordering model is cached > with the phrase-table. So if a rule is used multiple times, the reodering > model only need to be looked up once. The optimization might have forgotten > about XML, OOV etc. > > Please let me know what you find out, and if it's important to you to have > it 1 way or the other. > > > On 1 May 2014 11:10, Christophe Servan <[email protected]> wrote: >> >> As far as I understand, if the phrase table is ignored, the reordering >> model is ignored too. >> Maybe someone like Hieu can answer this specific point more precisely. >> >> >> 2014-05-01 11:44 GMT+02:00 Tom Hoar >> <[email protected]>: >> >>> Yes, the link's descriptions are good explanations of the differences >>> between "exclusive", "inclusive", constraint", ignore" and "pass-through." >>> All descriptions, however, refer to the "phrase table" (t-table), which to >>> my understanding does not include the distortion table. For example, >>> "exclusive" says, "Any phrases from the phrase table that overlap with that >>> span are ignored." There is no information about the effects of the >>> reordering/distortion table. >>> >>> >>> >>> >>> On 05/01/2014 02:33 PM, Christophe Servan wrote: >>> >>> Hi Tom, >>> As far as I understand there are diffrent ways to use it very well >>> explained there: >>> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc11 >>> Now ansering your questions, >>> >>>> I have two questions about how the -xml-input entries in the markup tag >>>> work. >>>> >>>> Are the entries applied before the distortion table with the distortion >>>> table affecting the result, or are the entries applied as a net effect >>>> after >>>> the distortion table? >>> >>> To me, it is another kind of decoding process, especially when the switch >>> and the option "exclusive" are used. They seem to bypass completely the >>> decoding process which uses the phrase-table and the distortion model. >>> If you want to simply add a new translation hypothesis which does not >>> already exist in your phrase table, use the "inclusive" option instead of >>> "exclusive". Both processes will be use at the same time. >>> But you will have no guaranty that your translation hypothesis, proposed >>> with the xml-markup switch, is the one chosen by the decoder. >>> As far as I know, the probability you gave in the xml tag correspond >>> jointly to all the features and weights associated to the hypothesis. >>>> >>>> Do the entries override or supplement the weightings in the loaded SMT >>>> model's t-table/distortion table combination? >>> >>> As I said, as far as I know, the exclusive mode, simply override the >>> phrase-table and the distortion model, if you still want use them, you can >>> use the mode "inclusive" for example. >>> >>> Best, >>> >>> Christophe >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > > > -- > Hieu Hoang > Research Associate > University of Edinburgh > http://www.hoang.co.uk/hieu > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
