More details... Our NFS is mounted at /mnt/storage .
The command to run moses server is: /mnt/storage/Common/mosesdecoder3/bin/mosesserver -f /mnt/storage/Common/models/translation_models/moses3_model/mert-work/moses.ini mark-unknown -xml-input -threads 18 exclusive Here is the complete moses.ini (excluding comments): [input-factors] 0 [mapping] 0 T 0 [distortion-limit] 6 [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/phrase-table.gz input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz Distortion KENLM lazyken=0 name=LM0 path=/mnt/storage/Common/models/language_models/langmod.5gram.blm order=5 [threads] 6 [weight] LexicalReordering0= 0.0268311 -0.0146878 0.0261305 0.0380759 0.0118265 0.07479 Distortion0= 0.074665 LM0= 0.0972206 WordPenalty0= -0.18469 PhrasePenalty0= -0.212528 TranslationModel0= 0.0184105 0.091358 0.112618 0.0161682 UnknownWordPenalty0= 1 On Wednesday, July 22, 2015, Oren <[email protected]> wrote: > Yes, we use a lot of RAM in our setup. But the improved response time > justifies it. > > Our language is on a nfs, bit we've been working this way with moses 1 for > some time with no problems (ten different machines using the same language > model file over nfs). Same for the reordering model. Neither of them is in > memory. Raising these models into memory will require raising our already > excessive RAM requirements... > > Thanks again for the help. > > On Wednesday, July 22, 2015, Barry Haddow <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> Hi Oren >> >> I'm not aware of any threading problems with PhraseDictionaryMemory, but >> not many people use it since it takes up too much memory. Moses command >> line runs multi-threaded using a thread pool. >> >> Is your language model on a local file system? Running it over nfs can be >> bad. What about your reordering model? Are you using the compact or the >> memory version? >> >> cheers - Barry >> >> On 22/07/15 15:09, Oren wrote: >> >> We have no swapping issues. I was asking if the use of an in-memory >> translation model might cause multithreading problems. >> >> >> I'm not sure how to replicate the problem on cmd moses, since it's >> purely a multithreading issue. Ca you run cmd moses multithreaded? >> >> I can't attach the complete moses.ini because it's on a separate >> network... But I copied below the stuff that looked relevant. I also tried >> to change the [threads] setting to 18, with no apparent effect. >> >> [input-factors] >> 0 >> >> [mapping] >> 0 T 0 >> >> [distortion-limit] >> 6 >> >> [feature] >> UnknownWordPenalty >> WordPenalty >> PhrasePenalty >> PhraseDictionaryMemory name=TranslationModel0 num-features=4 >> path=<path>/phrase-table.gz input-factor=0 output-factor=0 >> LexicalReordering <parameters> >> Distortion >> KENLM lazyken=0 name=LM0 path=<path> order=5 >> >> [threads] >> 6 >> >> [weight] >> >> <weight parameters> >> >> On Tuesday, July 21, 2015, Hieu Hoang <[email protected]> wrote: >> >>> is it possible you can make your moses.ini file available for us to see? >>> >>> do you know if the same problem occurs if you use the command line >>> moses, rather than mosesserver? >>> >>> >>> Hieu Hoang >>> Researcher >>> New York University, Abu Dhabi >>> http://www.hoang.co.uk/hieu >>> >>> On 21 July 2015 at 18:07, Barry Haddow <[email protected]> >>> wrote: >>> >>>> >>>> On 21/07/15 14:51, Oren wrote: >>>> >>>> I am using the in-memory mode, using about 50GB of RAM. (No swap issues >>>> as far as I can tell.) Could that cause issues? >>>> >>>> >>>> Yes, swapping would definitely cause issues - was that your question? >>>> >>>> >>>> >>>> I looked at the commit you linked to, but it doesn't seem to be >>>> something configurable beyond the -threads switch. Am I missing something? >>>> >>>> >>>> The commit enables you to set the maximum number of connections to be >>>> the same as the maximum number of threads. >>>> >>>> >>>> On Tuesday, July 21, 2015, Barry Haddow <[email protected]> >>>> wrote: >>>> >>>>> Hi Oren >>>>> >>>>> Does your host have 18 threads available? It could also be that >>>>> xmlrpc-c is limiting the number of connections - this can now be >>>>> configured: >>>>> >>>>> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523 >>>>> >>>>> Slowdowns in Moses are often caused by disk access bottlenecks. You >>>>> can use --minphr-memory and --minlexr-memory to make sure that the phrase >>>>> and reordering tables are loaded in to memory, rather than being access >>>>> on-demand. Make sure your host has enough RAM and is not swapping. As I >>>>> mentioned before there are various ways to make your models smaller ( >>>>> http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a >>>>> big difference to speed depending on your setup. >>>>> >>>>> cheers - Barry >>>>> >>>>> On 21/07/15 09:30, Oren wrote: >>>>> >>>>> Hi Barry, >>>>> >>>>> Thanks for the quick response. >>>>> >>>>> I added the switch "-threads 18" to the command to raise moses >>>>> server. The slowness issue persists but in a different form. Most requests >>>>> return right away, even under heavy load, but some requests (about 5%) >>>>> take >>>>> far longer - about 15-20 seconds. >>>>> >>>>> Perhaps there are other relevant switches? >>>>> >>>>> Thanks again. >>>>> >>>>> On Monday, July 20, 2015, Barry Haddow <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Oren >>>>>> >>>>>> The threading model is different. In v1, the server created a new >>>>>> thread for every request, v3 uses a thread pool. Try increasing the >>>>>> number >>>>>> of threads. >>>>>> >>>>>> Also, make sure you use the compact phrase table and KenLM as they >>>>>> are normally faster, and pre-pruning your phrase table can help, >>>>>> >>>>>> cheers - Barry >>>>>> >>>>>> On 20/07/15 12:01, Oren wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> We are in the process of migrating from Moses 1 to Moses 3. We have >>>>>> noticed a significant slowdown when sending many requests at once to >>>>>> Moses >>>>>> Server. The first request will actually finish about 25% faster that a >>>>>> single request using Moses 1, but as more requests accumulate there is a >>>>>> marked slowdown, until requests take 5 times longer or more. >>>>>> >>>>>> Is this a known issue? Is it specific to Moses Server? What can we >>>>>> do about it? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Oren. >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing >>>>>> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing >>>>> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >>>>> >>>>> >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>>> >>> >>
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
