Hi Hieu and moses-support, Sorry for the missing detail, but I forgot to say that I used also --translation-factors 0-0 option at training step.
mosesdecoder/scripts/training/train-model.perl --mgiza --external-bin-dir /usr/local/bin/ --corpus factored-corpus/proj-syndicate.1000 --root-dir unfactored/ --f de --e en --lm 0:3:/home/ezgi/factored-corpus/surface.lm:0 >& unfactored/training.out as indicated at http://www.statmt.org/moses/?n=Moses.FactoredTutorial#ntoc1and mosesdecoder/scripts/training/train-model.perl --mgiza --external-bin-dir /usr/local/bin/ --corpus factored-corpus/proj-syndicate.1000 --root-dir unfactored/ --f de --e en --lm 0:3:/home/ezgi/factored-corpus/surface.lm:0 --translation-factors 0-0 >& unfactored/training.out gave the same phrase table (named as unfactored/model/phrase-table.gz). On Sun, Feb 10, 2013 at 9:37 PM, Hieu Hoang <[email protected]> wrote: > Hi Ezgi > > > On 06/02/2013 09:00, ezgi yıldırım wrote: > > Hi Moses-support, > > I am currently studying on factored models for English-Turkish language > pair, but I am stuck at getting a model trained on surface factors of the > corpus as explained there ( > http://www.statmt.org/moses/?n=Moses.FactoredTutorial). > > I used the command below. > > mosesdecoder/scripts/training/train-model.perl --mgiza > --external-bin-dir /usr/local/bin/ --corpus > factored-corpus/proj-syndicate.1000.clean --root-dir unfactored/ --f de --e > en --lm 0:3:/home/ezgi/factored-corpus/surface.lm:0 > > But I got unfactored/model/phrase-table.gz instead of > unfactored/model/phrase-table.0-0.gz. > > Their may be small changes to the training script or mistakes in tutorial. > If you are sure this is happening, let me know and i'll change the tutorial. > > In the tutorial ( http://www.statmt.org/moses/?n=Moses.FactoredTutorial#ntoc1), the part of moses.ini file that point to the phrase table is stated as [ttable-file] 0 0 5 /.../unfactored/model/*phrase-table.0-0.gz* and the elements of phrase table generated by training an unfactored model are like frage ||| issue ||| 0.25 0.285714 0.25 0.166667 2.718 frage ||| question ||| 0.75 0.555556 0.75 0.416667 2.718 as I also expect. Is there a contradiction between the training script and the tutorial or am I missing some critical details? Besides, my phrase table is composed of these sample lines below. > > selbst|selbst|adv|adv wenn|wenn|kous|kous die|d|art|art.def.e ||| > even|even|rb if|if|in the|the|dt ||| 0.5 0.0683685 1 0.138547 2.718 ||| 0-0 > 1-1 2-2 ||| 2 1 1 > selbst|selbst|adv|adv wenn|wenn|kous|kous sie|sie|pper|pper.nom ||| > even|even|rb if|if|in they|they|prp ||| 0.5 0.128719 1 0.102041 2.718 ||| > 0-0 1-1 2-2 ||| 2 1 1 > selbst|selbst|adv|adv wenn|wenn|kous|kous ||| even|even|rb if|if|in ||| > 0.333333 0.20595 0.666667 0.238095 2.718 ||| 0-0 1-1 ||| 6 3 2 > selbst|selbst|adv|adv wenn|wenn|kous|kous ||| even|even|rb when|when|wrb > ||| 1 0.056391 0.333333 0.0396825 2.718 ||| 0-0 1-1 ||| 1 3 1 > selbst|selbst|adv|adv ||| ,|,|, even|even|rb ||| 0.125 0.263158 0.0769231 > 0.273778 2.718 ||| 0-1 ||| 8 13 1 > > do your files > factored-corpus/proj-syndicate.1000.clean.en > factored-corpus/proj-syndicate.1000.clean.de > have factors? If so, the phrase-table will probably come out with factors > All the files are from http://www.statmt.org/moses/download/. I got them by using wget http://www.statmt.org/moses/download/factored-corpus.tgz, thus they certainly have factors. Ezgi > > Actually it does not recognize real surface form (first factor) and then > takes the whole tagged form as surface form. I have the same problem with > my English-Turkish corpus. Please enlighten me on this problem, and if you > have more detailed tutorial for factored model like baseline systems ( > http://www.statmt.org/moses/?n=Moses.Baseline), could you share it? > > Thanks in advance, > Ezgi > > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
