hi marie
On 5 August 2014 13:58, Marie Sayadchi <[email protected]> wrote: > hi > thanks for your reply. > > > *Are the binary files in filesystem cache?* > im not sure how to put them there or how to check it! I would appreciate > if you could supply more details on this. > before you run the decoder, run this first cat /home/lingua/smt/moses-binaries/test-new-moses/new/binariesed/phrase-table/*.dat > /dev/null cat /home/lingua/smt/moses-binaries/test-new-moses/new/binariesed/reordering-table.bin* > /dev/null cat /home/lingua/smt/Multisensor/news-models/news.en.truecase.irstlm.blm > /dev/null > > > * what happens if you fun the decoding again with same sentence and > moses-bin.ini ?* > this log is with moses-bin.ini file, yes with binaries it is ok in > loading, but still slow in translation, and also I would like to know why > normal model(non-binaries) are so slow in loading compared to older > versions? > i haven't monitored the loading time of the memory phrase table because i think most people are more interested in the binary version. Running with the binary version isn't much slower and is sometimes faster, according to my tests: http://www.statmt.org/moses/RELEASE-2.1/Mosesv2.1.1releasenotes.pdf > thanks > > > On Tue, Aug 5, 2014 at 2:45 PM, Hieu Hoang <[email protected]> wrote: > >> from the log file, the loading seemed to have taken 1 second, which is >> fast and normal. >> >> the translation of each sentence took 30+ sec each. Are the binary files >> in filesystem cache? what happens if you fun the decoding again with same >> sentence and moses-bin.ini ? >> >> >> >> >> On 5 August 2014 13:33, Marie Sayadchi <[email protected]> wrote: >> >>> Dear all >>> >>> I am using moses RELEASE-2.1 binaries. >>> my training data is very small corpus of 170k sentences. >>> I following instructions from " >>> http://www.statmt.org/moses/?n=Moses.Baseline"; for training and testing. >>> the problem is that when I test the system, loading time is very long >>> and after 30min the phrase-table is still loading .. the models are small >>> and it should not take that long, also translation itself is very slow. >>> same model loads relatively fast and works fine with my older version of >>> moses(before 2012). both were using only one cpu and almost same amount of >>> memory. >>> my question is : >>> how can I get the same decoding speed as before; with the moses 2.1 , >>> and what is wrong with my current settings? >>> >>> >>> binarising the models helped with initial loading, but the translation >>> is still much faster with my old moses. >>> please see my moses.ini and translation log files in the attachment. >>> the command that I am using: >>> training: >>> nohup nice >>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/scripts/training/train-model.perl >>> -root-dir new/ -corpus ./all -f de -e en -max-phrase-length 5 -alignment >>> grow-diag-final-and -reordering msd-bidirectional-fe -lm >>> 0:3:/home/lingua/smt/Multisensor/news-models/news.en.truecase.irstlm.blm >>> -external-bin-dir >>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/training-tools >>> -mgiza mgiza -snt2cooc snt2cooc >& ./new/training.out >>> >>> binarising: >>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/bin/processLexicalTable >>> -in ./new/model/reordering-table.wbe-msd-bidirectional-fe.gz -out >>> ./new/binariesed/reordering-table >>> >>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/bin/processLexicalTable >>> -in ./new/model/reordering-table.wbe-msd-bidirectional-fe.gz -out >>> ./new/binariesed/reordering-table >>> >>> testing: >>> nohup nice >>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/bin/moses >>> -f ./new/model/moses-bin.ini <./newstest2013.tok.norm.de > out 2> log >>> >>> Best, >>> Marieh >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> >> >> -- >> Hieu Hoang >> Research Associate >> University of Edinburgh >> http://www.hoang.co.uk/hieu >> >> > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
