Hi Ken The binarisation looks to have worked fine. The directory listing that you give below is what is expected.
What happens when you run with the binarised model? cheers - Barry On 28/02/13 15:18, Ken Fasano wrote: > > Hi, Barry! > > I have worked through your Moses Baseline tutorial and have everything > set up to the point where I have completed the Tuning task. The > directory structure is the same as described in the tutorial. > > I’ve gotten now to the Testing phase, and find that Moses is unable to > load the phrase tables – I am running it on a VMWare-virtualized > Ubuntu Linux (all software is the most recent) with (only) 2.5 GB RAM, > since the host machine is an i5 running Windows 7 32-bit with 3GB RAM > (not enough, I know, but that’s not my choice L ) Loading the tables > into memory freezes at between 45% and 55%. > > I am trying to determine if binarization helps. I’ve run the > binarization steps verbatim according to the tutorial (everything has > been run verbatim), and that seems to run successfully – except that > at the end the expected files “phrase-table” and “reordering-table” do > not appear; instead, ten files appear (see below). At this point I’m > not sure how to continue. Any suggestions would be greatly appreciated! > > This is what is in the original train/model directory: > > *~/working/train/model$ ll* > > *total 368372* > > *drwxrwxr-x 2 kfasano kfasano 4096 Feb 27 12:56 ./* > > *drwxrwxr-x 6 kfasano kfasano 4096 Feb 27 12:36 ../* > > *-rw-rw-r-- 1 kfasano kfasano 9960340 Feb 27 12:37 > aligned.grow-diag-final-and* > > *-rw-rw-r-- 1 kfasano kfasano 51375529 Feb 27 12:47 extract.inv.sorted.gz* > > *-rw-rw-r-- 1 kfasano kfasano 47599852 Feb 27 12:48 extract.o.sorted.gz* > > *-rw-rw-r-- 1 kfasano kfasano 52040032 Feb 27 12:48 extract.sorted.gz* > > *-rw-rw-r-- 1 kfasano kfasano 15748031 Feb 27 12:38 lex.e2f* > > *-rw-rw-r-- 1 kfasano kfasano 15748031 Feb 27 12:38 lex.f2e* > > *-rw-rw-r-- 1 kfasano kfasano 1298 Feb 27 12:56 moses.ini* > > *-rw-rw-r-- 1 kfasano kfasano 133315658 Feb 27 12:54 phrase-table.gz* > > *-rw-rw-r-- 1 kfasano kfasano 51391247 Feb 27 12:56 > reordering-table.wbe-msd-bidirectional-fe.gz* > > Here is a copy of what happens (all command lines are entered on one > line, although they may split in the output): > > ~/working$ *~/mosesdecoder/bin/processPhraseTable -ttable 0 0 > train/model/phrase-table.gz -nscores 5 -out binarised-model/phrase-table* > > processing ptree for train/model/phrase-table.gz > > ..................................................[phrase:500000] > > ..................................................[phrase:1000000] > > ..................................................[phrase:1500000] > > ..................................................[phrase:2000000] > > .......................................distinct source phrases: > 2395401 distinct first words of source phrases: 45688 number of phrase > pairs (line count): 7749919 > > Count of lines with missing alignments: 0/7749919 > > WARNING: there are src voc entries with no phrase translation: count 11722 > > There exists phrase translations for 33966 entries > > ~/working$ *~/mosesdecoder/bin/processLexicalTable -in > train/model/reordering-table.wbe-msd-bidirectional-fe.gz -out > binarised-model/reordering-table* > > processLexicalTable v0.1 by Konrad Rawlik > > processing train/model/reordering-table.wbe-msd-bidirectional-fe.gz to > binarised-model/reordering-table.* > > ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................~/working$ > > > > ~/working$ *cd binarised-model* > > *~/working/binarised-model$ ll* > > *total 1335328* > > *drwxrwxr-x 2 kfasano kfasano 4096 Feb 28 10:04 ./* > > *drwxrwxr-x 5 kfasano kfasano 4096 Feb 28 09:22 ../* > > *-rw-rw-r-- 1 kfasano kfasano 365508 Feb 28 09:50 phrase-table.binphr.idx* > > *-rw-rw-r-- 1 kfasano kfasano 74339520 Feb 28 09:50 > phrase-table.binphr.srctree.wa* > > *-rw-rw-r-- 1 kfasano kfasano 712942 Feb 28 09:50 > phrase-table.binphr.srcvoc* > > *-rw-rw-r-- 1 kfasano kfasano 458015919 Feb 28 09:50 > phrase-table.binphr.tgtdata.wa* > > *-rw-rw-r-- 1 kfasano kfasano 622345 Feb 28 09:50 > phrase-table.binphr.tgtvoc* > > *-rw-rw-r-- 1 kfasano kfasano 365508 Feb 28 10:00 > reordering-table.binlexr.idx* > > *-rw-rw-r-- 1 kfasano kfasano 521575608 Feb 28 10:00 > reordering-table.binlexr.srctree* > > *-rw-rw-r-- 1 kfasano kfasano 309996760 Feb 28 10:00 > reordering-table.binlexr.tgtdata* > > *-rw-rw-r-- 1 kfasano kfasano 712942 Feb 28 10:00 > reordering-table.binlexr.voc0* > > *-rw-rw-r-- 1 kfasano kfasano 622345 Feb 28 10:00 > reordering-table.binlexr.voc1* > > *Thank you!* > > *Ken Fasano* > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
