Yes, that worked for me as well, thank you. There is a little improvement in speed but not that much actually (about 5% faster using 30 threads).
2016-10-04 11:44 GMT+02:00 Hieu Hoang <[email protected]>: > yes - the script expects the files to be gzipped. > It runs ok for me. I executed this: > > MOSES_DIR=~/workspace/github/mosesdecoder.perf > > $MOSES_DIR/scripts/generic/binarize4moses2.perl > --phrase-table=phrase-table.gz > --lex-ro=reordering-table.wbe-msd-bidirectional-fe.gz > --output-dir=integrated_phrase-reordering/ --num-lex-scores=6 > > Got this: > > Executing: gzip -dc phrase-table.gz | /home/hieu/workspace/github/ > mosesdecoder.perf/scripts/generic/../../contrib/sigtest-filter/filter-pt > -n 0 | gzip -c > ./tmp.14373/pt.gz > ... > Reading phrase table finished, writing remaining files to disk. > > $ ll integrated_phrase-reordering/ > total 24688 > drwxrwxr-x 2 hieu hieu 4096 Oct 4 10:38 ./ > drwxrwxr-x 5 hieu hieu 4096 Oct 4 10:42 ../ > -rw-rw-r-- 1 hieu hieu 917861 Oct 4 10:42 Alignments.dat > -rw-rw-r-- 1 hieu hieu 2267885 Oct 4 10:42 cache > -rw-rw-r-- 1 hieu hieu 76 Oct 4 10:42 config > -rw-rw-r-- 1 hieu hieu 3146720 Oct 4 10:42 probing_hash.dat > -rw-rw-r-- 1 hieu hieu 333856 Oct 4 10:42 source_vocabids > -rw-rw-r-- 1 hieu hieu 18429920 Oct 4 10:42 TargetColl.dat > -rw-rw-r-- 1 hieu hieu 121401 Oct 4 10:42 TargetVocab.dat > > > On 04/10/2016 09:06, Vito Mandorino wrote: > > The command was > > perl /home/Moses/mosesdecoder/scripts/generic/binarize4moses2.perl > --phrase-table=/home/vito/phrase-table.sorted > --lex-ro=/home/vito/reordering-table.sorted > --output-dir=/home/vito/integrated_phrase-reordering/ > --num-lex-scores=6 > > The tables in the command are sorted with LC_ALL . I attach them in .gz > format. Should one use the .gz format also in the command above? > > Vito > > > -- *M**. Vito MANDORINO -- Chief Scientist* [image: Description : Description : lingua_custodia_final full logo] *The Translation Trustee* *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux* *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89* *Email :* *[email protected] <[email protected]>* *Website :* *www.linguacustodia.finance <http://www.linguacustodia.com/>*
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
