Re: [Moses-support] When using moses, cpu is are not stable.

[Marcin Junczys-Dowmunt]

I forgot to add that I am using cube-pruning, with stack decoding moses uses pretty much all the available cores in one process. Cube-pruning seems to be so fast that lockage becomes an issue. W dniu 15.01.2014 09:40, Marcin Junczys-Dowmunt pisze: From my experience more than 16 threads per moses process causes performance to decrease. I have 64 cores, too. I use GNU parallel to get around this. cat input.txt | parallel --pipe -k -j 4 ./mosesdecoder/bin/moses -f moses.ini -threads 16 > out.txt This launches 4 moses processes a 16 threads and translates blocks of roughly 1MB per processes. You only see output after a block has been translated. You can set block size by using for instance --block 10K , then it will create and kill the moses processes much faster and output intermediate results. You can get GNU parallel from http://ftp.gnu.org/gnu/parallel/parallel-latest.tar.bz2 . Make sure to delete /etc/parallel/config after installation. Best, Marcin W dniu 15.01.2014 08:24, Hansung Cho pisze: hello. To change the option "-threads 64" then monitoring at server CPUs status. but, result is the same.. Try the other way? Attach the CPUs status picture thank you Hansung Cho 2014/1/15 Tom Hoar <tah...@precisiontranslationtools.com> To change the threads, run with '-threads x' added to your command line: moses -f /path/to/moses.ini -threads 20 On 01/14/2014 11:41 PM, Philipp Koehn wrote: Hi, you can try specifying more threads than the number of CPUs. I have no idea if this speeds things up or causes blockage in your setup. Just try it out and see what happens. -phi On Tue, Jan 14, 2014 at 4:31 AM, Hansung Cho <gyu...@gmail.com> wrote: My server configuration are multiple CPUs 16 core and running thread 16. How do I adjust the moses running thread? I need your help... Thank you 2014/1/14 Tom Hoar <tah...@precisiontranslationtools.com> With a system as complex as Moses, why would you think the CPU usage would be steady? Each sentence task goes through many stages with different CPU demands which probably map to the variation in each cycle. You can also look first at your threading configuration. If your system has multiple CPUs (cores) and Moses is running multi-threaded, I suspect the cycles' frequencies are related to the activities on each thread. On 01/14/2014 01:09 PM, Hansung Cho wrote: hello..  I'm using mosesdecoding  When monitoring a physical server, Server cpu is not very stable. However, the state is constantly moving cpu Look at the picture below Is it because the moses system cache?? Should any part option of the adjustment? My moses commit a1584c608f8bbd83921896bac9eac97d9effc0c8 Options related to the cache are "clean-lm-cache" ( default ==1 ) Will be waiting for your input.  Thank you Hansung cho _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support