HI Amir
You should be able to do the building of the LM with irstlm, and
binarising and decoding with KenLM, all via EMS.
Anyway, for the decoding, is Moses still running? Is it using CPU? It
could be swapping. If it's not running, did it crash? You could try
running again with a small stack size (the -s parameter in the
decoder-settings)
cheers - Barry
On 15/02/14 09:51, amir haghighi wrote:
Hello Barry
Thank you for your help.
I commneted out the KenLm in the configuration file and used
buil-binary to binarize lm file.( it doesn't create arpa file, should
I create it manually? )
It didn't get any error but it still takes too long to finish the
testing process.
I have 5000 sentences in my test set. I checked test.output.1 file and
noticed that it contains only the translation of 4690 sentences. it
seems that it gets into a loop for translating the 4691'th sentence.
I am afraid that I made a mistake in setting the parameters in my
configuration file. I will be thankful if you could take a look at my
file.
Thank you again
amir
On Fri, Feb 14, 2014 at 5:28 AM, Barry Haddow
<[email protected] <mailto:[email protected]>> wrote:
Hi Amir
Even if you use IRSTLM to build the language model, you can still
use KenLM for decoding. Make sure you create an arpa file with
IRSTLM, then use build_binary to binarise it so that it loads
quickly with KenLM. Then you can use multi-threaded decoding,
cheers - Barry
On 14/02/14 13:01, amir haghighi wrote:
Thank you Barry,
I use IRSTLM to build the language model. Can I use multi-thread
for decoder-setting?
I get "IRST LM is not threadsafe" error.
I want to use IRSTLM, is there any other way to speed up the
tuning step ?
Regrads
On Fri, Feb 14, 2014 at 1:53 AM, Barry Haddow
<[email protected] <mailto:[email protected]>>
wrote:
Hi Amir
You can add
decoder-settings = "-threads 4"
to your TUNING stanza.
Also try
filter-settings = "-MinScore 2:0.0001"
for more aggressive filtering.
Running tuning on a laptop though is always going to be slow,
cheers - Barry
On 14/02/14 09:26, amir haghighi wrote:
Thank you arezki and yohit
I don't know how can I change multi-thread setting in ems
config file.
On Fri, Feb 14, 2014 at 12:36 AM, Arezki Sadoune
<[email protected] <mailto:[email protected]>> wrote:
Hello Amir
I think your tuning process will go faster if you use a
multi-threaded Mert.
/home/mert-moses.pl <http://mert-moses.pl> --threads 4
you have of course tu indicate 8 instead of 4 if your
laptop is equipped with eight cores
Best regards
Le Vendredi 14 février 2014 8h27, amir haghighi
<[email protected]
<mailto:[email protected]>> a écrit :
Hello
I have a corpus with 400'000 sentences for training,
1000 sentences for tuning and 100'000 sentences for
test. I couldn't run ems on my corpus, after 3 days,
with my old laptop.
I have bought a new laptop (core i7, cpu 2.40 , 8G Ram)
but I can't still run ems! it is 3 days that it is in
the tuning step and it is not finished yet.
Is it possible that it gets in an endless loop?
How can I check it's process?
regards
Amir
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
