Hi, if you use the new training code that is stored away in the mt3_chart branch, you can run
train-factored-phrase-model.perl with the additional option: -score-options "--WordAlignment FILE" which will generate a word alignment file (with the specified file name) for all the phrase pairs. If you run the decoder with the "-t" switch, you will get a phrase alignment. Given the phrase alignment, you can refer to the generated word alignment file and lookup up the word alignment for each phrase pair. -phi On Thu, Nov 5, 2009 at 7:17 PM, Hieu Hoang <[email protected]> wrote: > i wrote it to do some other stuff so the alignment info was stored > inefficiently. my bad > > Joerg Tiedemann wrote: >> OK - thanks! >> I don't really need it. And I could just get the word alignment directly >> from the phrase table I guess (using the segmentation info to get the >> phrases used). Why did it take so much memory to include it? Because of >> caching phrase translations? >> >> Jorg >> >> Barry Haddow wrote: >> >>> Hi Joerg >>> >>> The word alignment info code got removed as it was using too much memory. If >>> you really need it, then you could go back in svn to the time before the >>> multi-threaded code was merged in (before r2477, I think) >>> >>> regards >>> Barry >>> >>> On Thursday 05 November 2009 13:53, Jörg Tiedemann wrote: >>> >>>> Hmm ... >>>> I just tried the -print-alignment-info flag and did not get any word >>>> alignment info. I checked again and I do have the alignment fields in my >>>> phrase table. I also tried a binarized version including word alignments >>>> but nothing happens. Is there anything else I have to switch on to get >>>> the "verbose" output? I have no problem to get the phrase-mappings (like >>>> 0-3=0-1) in an n-best list when switching it on with >>>> -include-alignment-in-n-best. >>>> >>>> here is my command >>>> moses -f moses.ini -use-alignment-info -print-alignment-info < input >>>> >>>> this is what I get (for a very small test corpus ...): >>>> .... >>>> Collecting options took 2.470 seconds >>>> Search took 8.030 seconds >>>> BEST TRANSLATION: wir sind , die sich aus der gemeinsamen währung , die >>>> weit von uns , um die vorteile und flexibilität einer gemeinsamen >>>> währung einkerkert uns künstliche straitjacket|UNK|UNK|UNK , die über >>>> die völker europas . [11111111111111111111111111111111111111111111] >>>> [total=-257.879] <<-28.000, -34.000, -100.000, -9.775, -1.609, -5.646, >>>> -13.490, -1.609, -4.243, -247.338, -78.555, -108.830, -33.287, -47.820, >>>> 23.998>> >>>> wir sind , die sich aus der gemeinsamen währung , die weit von uns , um >>>> die vorteile und flexibilität einer gemeinsamen währung einkerkert uns >>>> künstliche straitjacket , die über die völker europas . >>>> Translation took 8.040 seconds >>>> Finished translating >>>> .... >>>> >>>> and my phrase-table looks like this: >>>> ! " ||| ! " ||| (0) (1) ||| (0) (1) ||| 0.75 0.0320773 1 0.595087 2.718 >>>> ! the total ||| denn die vorgesehene ||| () (1) () ||| () (1) () ||| >>>> 0.125 1.86531e-09 0.25 2.46276e-08 2.718 >>>> ! the total ||| denn die ||| () (1) () ||| () (1) ||| 0.0163934 >>>> 1.86531e-09 0.25 0.000386013 2.718 >>>> .... >>>> >>>> and I use the latest Moses (checked out yesterday) >>>> Well, it's not so important. Just wanted to know if there's something >>>> wrong with my command line options ... >>>> >>>> Jörg >>>> >>>> >>>> (By the way, the multi-threaded decoder works like a charm!) >>>> >>>> Nicola Bertoldi wrote: >>>> >>>>> Waleed, >>>>> if you are interested in a word-to-word alignment (instead of >>>>> phrase-to-phrase alignment) you could give a look to this feature of >>>>> Moses >>>>> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc6 >>>>> >>>>> Pay attention that you have to re-train your phrase tables including the >>>>> word-to-word alignments; see also >>>>> http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases >>>>> and >>>>> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#BinaryPhraseTable >>>>> >>>>> regards, >>>>> Nicola >>>>> >>>>> On 11/1/09 4:33 PM, "Felipe Sánchez Martínez" <[email protected]> >>>>> wrote: >>>>> >>>>> Hello, >>>>> >>>>> To obtain the correspondence between source phrases and target phrases >>>>> run the decoder with -report-segmentation. Also note that in >>>>> phrase-based SMT each target phrases is the translation of a source >>>>> phrases, there are no target phrases "aligned" with null. >>>>> >>>>> Regards >>>>> -- >>>>> Felipe >>>>> >>>>> El dom, 01-11-2009 a las 17:12 +0200, Waleed Oransa escribió: >>>>> >>>>>> Hello all, >>>>>> >>>>>> I would like to know if moses can output the mapping between input >>>>>> words and output words. In other way, how can I know the source word >>>>>> (or null) for each word in the translated sentence. I appreciate your >>>>>> prompt response. Thank you! >>>>>> >>>>>> Waleed >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>> -- >>>>> Felipe Sánchez Martínez <[email protected]> >>>>> Departamento de Lenguajes y Sistemas Informáticos >>>>> Universidad de Alicante, E-03071 Alicante (Spain) >>>>> Tel.: +34 965 903 400, ext: 2966 Fax: +34 965 909 326 >>>>> http://www.dlsi.ua.es/~fsanchez >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
