Hi Oren
You can fit a lot of model in 50G RAM. It's worth looking at the compact
phrase and reordering models, pruning options, and kenlm quantisation,
and you might fit the models on one machine.
As to the slowdown, a delay of 20s or so suggests that it's waiting on
I/O, or perhaps xmlrpc-c is queueing up requests or connections.
cheers - Barry
On 23/07/15 15:06, Oren wrote:
More details...
Our NFS is mounted at /mnt/storage .
The command to run moses server is:
/mnt/storage/Common/mosesdecoder3/bin/mosesserver -f
/mnt/storage/Common/models/translation_models/moses3_model/mert-work/moses.ini
mark-unknown -xml-input -threads 18 exclusive
Here is the complete moses.ini (excluding comments):
[input-factors]
0
[mapping]
0 T 0
[distortion-limit]
6
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/phrase-table.gz
input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
Distortion
KENLM lazyken=0 name=LM0
path=/mnt/storage/Common/models/language_models/langmod.5gram.blmorder=5
[threads]
6
[weight]
LexicalReordering0= 0.0268311 -0.0146878 0.0261305 0.0380759
0.0118265 0.07479
Distortion0= 0.074665
LM0= 0.0972206
WordPenalty0= -0.18469
PhrasePenalty0= -0.212528
TranslationModel0= 0.0184105 0.091358 0.112618 0.0161682
UnknownWordPenalty0= 1
On Wednesday, July 22, 2015, Oren <[email protected]
<mailto:[email protected]>> wrote:
Yes, we use a lot of RAM in our setup. But the improved response
time justifies it.
Our language is on a nfs, bit we've been working this way with
moses 1 for some time with no problems (ten different machines
using the same language model file over nfs). Same for the
reordering model. Neither of them is in memory. Raising these
models into memory will require raising our already excessive RAM
requirements...
Thanks again for the help.
On Wednesday, July 22, 2015, Barry Haddow
<[email protected]
<javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote:
Hi Oren
I'm not aware of any threading problems with
PhraseDictionaryMemory, but not many people use it since it
takes up too much memory. Moses command line runs
multi-threaded using a thread pool.
Is your language model on a local file system? Running it over
nfs can be bad. What about your reordering model? Are you
using the compact or the memory version?
cheers - Barry
On 22/07/15 15:09, Oren wrote:
We have no swapping issues. I was asking if the use of
an in-memory translation model might cause multithreading
problems.
I'm not sure how to replicate the problem on cmd moses, since
it's purely a multithreading issue. Ca you run cmd moses
multithreaded?
I can't attach the complete moses.ini because it's on a
separate network... But I copied below the stuff that looked
relevant. I also tried to change the [threads] setting to 18,
with no apparent effect.
[input-factors]
0
[mapping]
0 T 0
[distortion-limit]
6
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=<path>/phrase-table.gz input-factor=0 output-factor=0
LexicalReordering <parameters>
Distortion
KENLM lazyken=0 name=LM0 path=<path> order=5
[threads]
6
[weight]
<weight parameters>
On Tuesday, July 21, 2015, Hieu Hoang <[email protected]>
wrote:
is it possible you can make your moses.ini file available
for us to see?
do you know if the same problem occurs if you use the
command line moses, rather than mosesserver?
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
On 21 July 2015 at 18:07, Barry Haddow
<[email protected]> wrote:
On 21/07/15 14:51, Oren wrote:
I am using the in-memory mode, using about 50GB of
RAM. (No swap issues as far as I can tell.) Could
that cause issues?
Yes, swapping would definitely cause issues - was
that your question?
I looked at the commit you linked to, but it doesn't
seem to be something configurable beyond the
-threads switch. Am I missing something?
The commit enables you to set the maximum number of
connections to be the same as the maximum number of
threads.
On Tuesday, July 21, 2015, Barry Haddow
<[email protected]> wrote:
Hi Oren
Does your host have 18 threads available? It
could also be that xmlrpc-c is limiting the
number of connections - this can now be configured:
https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
Slowdowns in Moses are often caused by disk
access bottlenecks. You can use --minphr-memory
and --minlexr-memory to make sure that the
phrase and reordering tables are loaded in to
memory, rather than being access on-demand. Make
sure your host has enough RAM and is not
swapping. As I mentioned before there are
various ways to make your models smaller
(http://www.statmt.org/moses/?n=Advanced.RuleTables),
which can make a big difference to speed
depending on your setup.
cheers - Barry
On 21/07/15 09:30, Oren wrote:
Hi Barry,
Thanks for the quick response.
I added the switch "-threads 18" to the command
to raise moses server. The slowness issue
persists but in a different form. Most requests
return right away, even under heavy load, but
some requests (about 5%) take far longer -
about 15-20seconds.
Perhaps there are other relevant switches?
Thanks again.
On Monday, July 20, 2015, Barry Haddow
<[email protected]> wrote:
Hi Oren
The threading model is different. In v1,
the server created a new thread for every
request, v3 uses a thread pool. Try
increasing the number of threads.
Also, make sure you use the compact phrase
table and KenLM as they are normally
faster, and pre-pruning your phrase table
can help,
cheers - Barry
On 20/07/15 12:01, Oren wrote:
Hi all,
We are in the process of migrating from
Moses 1 to Moses 3. We have noticed a
significant slowdown when sending many
requests at once to Moses Server. The
first request will actually finish about
25% faster that a single request using
Moses 1, but as more requests accumulate
there is a marked slowdown, until requests
take 5 times longer or more.
Is this a known issue? Is it specific to
Moses Server? What can we do about it?
Thanks!
Oren.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
The University of Edinburgh is a charitable body,
registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
