Hello again! First thank you very much for your answers. I'm still looking into solving this thing with the old phrase table, but I also tried to re-generate it. I have the data so it shouldn't be any problem to regenerate the phrase table.
Still, I'm stuck with the mkcls program. I've followed the steps and installed everything that was needed and managed to create a phrase table on a 10k phrases dataset. But when I use a larger dataset (600k phrases) the mkcls freezes like this: $MOSES_SCRIPTS/training/train-model.perl -scripts-root-dir $MOSES_SCRIPTS -corpus dict.train -f gr -e ph -lm 0:3:/home/cucu/speechRoot/tools/wordsPhonetization/devel/phonetizer1/dict.train.ph.lm Using SCRIPTS_ROOTDIR: /home/applications/moses/scripts Using single-thread GIZA (1) preparing corpus @ Fri Dec 16 14:37:30 EET 2011 Executing: mkdir -p ./corpus (1.0) selecting factors @ Fri Dec 16 14:37:30 EET 2011 (1.1) running mkcls @ Fri Dec 16 14:37:30 EET 2011 /home/applications/giza-pp/GIZA++-v2/mkcls -c50 -n2 -pdict.train.gr-V./corpus/gr.vcb.classes opt ./corpus/gr.vcb.classes already in place, reusing (1.1) running mkcls @ Fri Dec 16 14:37:30 EET 2011 /home/applications/giza-pp/GIZA++-v2/mkcls -c50 -n2 -pdict.train.ph-V./corpus/ph.vcb.classes opt Executing: /home/applications/giza-pp/GIZA++-v2/mkcls -c50 -n2 - pdict.train.ph -V./corpus/ph.vcb.classes opt WARNING: StatVar.cc I've waited for it a couple of hours although when this step only takes 1.5 minutes on the machine I was previously using. I think it might be something related to the compiler (i'm using g++ 4.5.2). I need to mention again that if I use only 10k phrases instead of 600k phrases everything works fine... Any ideas? Horia On 16 December 2011 11:39, Barry Haddow <[email protected]> wrote: > Hi Horia > > The phrase table format has changed so you might have to recreate or > convert > it. You should first try changing the phrase-table line in your moses.ini > from > > 0 0 5 <ttable-path> > > to > > 0 0 0 5 <ttable-path> > > The leading '0' signifies that the table is in text format. The new text > format > for phrase tables is given here > http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases > so you should be able to convert your old one with a simple script, > > cheers - Barry > > On Thursday 15 December 2011 14:05:09 Horia Cucu wrote: > > Hi, > > > > I have a phrase table created with a version of moses compiled in 2007. > > I've just installed moses on my computer and tried to decode using this > old > > phrase table and I'm stuck here: > > > > ... > > ... > > Loading lexical distortion models...have 0 models > > Start loading LanguageModel > > /home/cucu/speechRoot/tools/wordsPhonetization/dist/dict.train.ph.lm : > > [0.000] seconds > > Finished loading LanguageModels : [0.000] seconds > > Warning: Phrase table specification in old 4-field format. Assuming > binary > > phrase tables (type 1)! > > Start loading PhraseTable > > > /home/cucu/speechRoot/tools/wordsPhonetization/filteredTable/phrase-table.0 > > -0.1 > > > > : [0.000] seconds > > > > filePath: > > > /home/cucu/speechRoot/tools/wordsPhonetization/filteredTable/phrase-table.0 > > -0.1 Finished loading phrase tables : [0.000] seconds > > IO from STDOUT/STDIN > > Created input-output object : [0.000] seconds > > Translating line 0 in thread id 3074714480 > > Translating: c a s ă > > > > bin ttable does not exist -> create it > > distinct source phrases: 19 distinct first words of source phrases: 8 > > number of phrase pairs (line count): 41 > > reading bin ttable > > size of OFF_T 8 > > binary phrasefile loaded, default OFF_T: -1 > > Check weightT.size() == scoreVector.size() failed in > > moses/src/TargetPhrase.cpp:155 > > ./translateWordList.sh: line 16: 21578 Aborted moses -f > > filteredTable/moses.ini < $WORDLIST.gr > $WORDLIST.ph > > > > any ideas? > > horia > > > > -- > Barry Haddow > University of Edinburgh > +44 (0) 131 651 3173 > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
