On 07/05/2015 20:43, Alla Rozovskaya wrote:
Hi Hieu,
Please see below.
(1) Thank you for your help. I am using a standard LM training for
now, without the N-gram corpus. Could you please let me know how I can
train a translation model without any reordering model (I did not find
anything in the manual about it).
what's the exact command you're using to create the translation model
and reordering model?
$mosesDir/scripts/training/train-model.perl -root-dir
$outputDir/mosesFiles/ -corpus $corpusDir/$trainFile.clean -f
$source -e $target -alignment grow-diag-final-and -reordering
msd-bidirectional-fe -lm 0:$order:$lm1 -external-bin-dir
/scratch/rozovska/mosesdecoder/tools/ -cores 2 --max-phrase-length 7
I don't know how to prevent the creation of the re-ordering model.
However, if you don't want to use it in decoding, then just delete the
LexicalReordering and Distortion model entries in the moses.ini file
BEFORE you tune.
I tried removing Distortion and LexicalReordering from the moses.ini
file but I still found these cases in the translated file.
(2) Also, can I use multiple language models when training a
translation model, or is it only possible to specify multiple LMs in
the moses.ini file?
the LM isn't used when creating the translation model. They're only
used in decoding
What is the -lm parameter in train-model.perl used for?
the -lm parameter is just used to add the LM model entry to the
moses.ini file.
In fact, if you don't like the moses.ini that is created automatically
for you, create your own that has just the models you need and use that
for tuning and decoding
Thanks a lot for your help,
Alla
On Wed, May 6, 2015 at 12:12 AM, Hieu Hoang <[email protected]
<mailto:[email protected]>> wrote:
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
On 6 May 2015 at 01:36, Alla Rozovskaya <[email protected]
<mailto:[email protected]>> wrote:
Hi Hieu,
Thank you for your help. I am using a standard LM training for
now, without the N-gram corpus. Could you please let me know
how I can train a translation model without any reordering
model (I did not find anything in the manual about it).
what's the exact command you're using to create the translation
model and reordering model?
Also, can I use multiple language models when training a
translation model, or is it only possible to specify multiple
LMs in the moses.ini file?
the LM isn't used when creating the translation model. They're
only used in decoding
Thanks again,
Alla
On Tue, Apr 28, 2015 at 10:06 AM, Hieu Hoang
<[email protected] <mailto:[email protected]>> wrote:
I spoke to Ken about using KenLM to train a standard
backoff LM with the n-gram corpus. It's not supported yet
or recommended.
I'm not sure whether the moses' SRILM wrapper will support
the count-based LM. And how much memory it will consume.
Try it and please let us know.
People have also been using the Common Crawl corpus to
build huge backoff LM. They're very difficult to use as it
consumes a lot of memory
On 25/04/2015 20:24, Alla Rozovskaya wrote:
Hello,
I have built an interpolated count-based LM on the Google
Web N-gram corpus using SRILM toolkit, as specified here:
http://www.speech.sri.com/projects/srilm/manpages/srilm-faq.7.html
Is it possible to use it in moses? In particular, since
this model uses count files and a file specifying
weights, what is the right way to specify the path in
moses.ini?
Thank you,
Alla
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
