ah, you're decomposing the translation into factors. You have to be
careful that the number of translation rules it create doesn't explode.
Try these things
1. Set table-limit for each phrase-table to a small number, eg.
PhraseDictionaryCompact .... table-limit=5
2. Prune the generation table so that it keeps only the best 5
output factor for each input factor.
Once you know that your experiments can run in the time and memory you
need, you can increase these limits
On 09/06/2015 20:24, Bc. Robert ADAMEC wrote:
> Hello,
>
> this is the exact command:
>
> /opt/moses-3.0/scripts/training/mert-moses.pl
> /root/faktor_6/corpus/heldout.src /root/faktor_6/corpus/heldout.ref \
> /opt/moses-3.0/bin/moses /root/faktor_6/experiment/model/moses.ini \
> --working-dir /root/faktor_6/tmp \
> --decoder-flags "-threads 6 -v 100" \
> --rootdir /opt/moses-3.0/scripts -mertdir /opt/moses-3.0/bin \
> --maximum-iterations 10
>
> The moses.ini file and the log file are attached.
>
> It seems that the tool maybe is not stuck but proceeds so slowly that it is
> useless anyhow, strange thing is that it occurs on a certain point....until
> the hypothesis on the row 4844 it runs fast and then it slows heavily, the
> remaining hypotheses seen in the log from that row have been processed in
> about 16 hours while the hypotheses before that row were processed in like 5
> minutes, maybe even less.
>
> Still do not get why is that happening and it frustrates me because it is
> part of my diploma thesis.
>
> Best regards,
>
> Robert Adamec
>
>
>
> Úterý, 9 Červen, 2015 08:27 CEST, Hieu Hoang <[email protected]> napsal:
>
>> so u binarised the phrase table and re-ordering model. Did you change the
>> moses.ini file to tell the decoder to use the binarised models?
>>
>> to avoid confusion, please send me the exact command you used and the exact
>> moses.ini file you used
>>
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabi
>> http://www.hoang.co.uk/hieu
>>
>> On 9 June 2015 at 10:06, Bc. Robert ADAMEC <[email protected]> wrote:
>>
>>> Hello,
>>>
>>> the execution command is described in the first e-mail that I have send
>>> you. (the first e-mail in this e-mail chain)
>>>
>>> I have further observed the behavior of the tool and it seems to move
>>> through the hypothesis, but at very very slow rate. It runs normally for a
>>> while, then it hits the hypothesis, which can be seen in the log from
>>> previous e-mails, and starts to run very slowly, like 3 hypothesis per 10
>>> hours. I do not know if it would ever complete the decoding stage in the
>>> for cycle that I have posted previously but nevertheless I do not have time
>>> to let it run for weeks if it moves somewhere.
>>>
>>> By debug messages you mean the messages added by me or normal debug
>>> messages which Moses outputs with -v 3? You can see the normal debug output
>>> in the log from the first e-mail.
>>>
>>> Best regards and thank you for your time,
>>>
>>> Robert Adamec
>>>
>>>
>>> On Monday, June 8, 2015 18:27 CEST, Hieu Hoang <[email protected]>
>>> wrote:
>>>
>>>> What is the exact command you're executing? Can u please send me the
>>> debug
>>>> messages that Moses output
>>>> On 8 Jun 2015 19:54, "Bc. Robert ADAMEC" <[email protected]>
>>> wrote:
>>>>> Hello,
>>>>>
>>>>> I made the compact versions of the phrase tables and reordering models,
>>>>> the language models were already binarised. However it did not help,
>>> the
>>>>> tool still "freezes" on the same spot and stays there.
>>>>>
>>>>> I have let it run for almost 24 hours and it moved nowhere. I really do
>>>>> not think that it would change with time, the experiments that I used
>>>>> earlier run about 24 hours total, for this experiment I added factored
>>>>> corpus for both sides, previously it was only for the source language,
>>> and
>>>>> I do not think that it should impact the performance so heavily.
>>>>>
>>>>> I have tried to put some debug messages into moses itself, namely into
>>> the
>>>>> DecodeStepGeneration class around the lines 149 - 170 (the for (size_t
>>>>> currIter = 0 ; currIter < numIteration ; currIter++) cycle) and I
>>> found out
>>>>> that the tool freezes because it gets stuck in this cycle forever
>>> after a
>>>>> while (for the hypothesis that you can see as last in the log from the
>>>>> previous e-mail) and never leaves it.
>>>>>
>>>>> I do not know what is the reason and cannot find it out.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Robert Adamec
>>>>>
>>>>>
>>>>>
>>>>> Úterý, 2 Červen, 2015 14:05 CEST, Hieu Hoang <[email protected]>
>>>>> napsal:
>>>>>
>>>>>> please subscribe to the mailing list before posting to it. You can
>>>>>> subscribe here:
>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>
>>>>>> To answer your question - how do you know it is in an inifinite loop,
>>>>>> rather than just taking a very long time? From the moses.ini file, it
>>>>>> seems the phrase tables are not binarized. This will cause loading to
>>>>>> take a very long time and use a lot of memory. You should binarize
>>> your
>>>>>> phrase-table, reordering model and LM.
>>>>>>
>>>>>> Also, if you are new to moses, please work through the tutorial here
>>> to
>>>>>> get a better understanding of what moses is doing
>>>>>> http://www.statmt.org/moses/?n=Moses.Baseline
>>>>>>
>>>>>>
>>>>>> On 29/05/2015 14:35, Bc. Robert ADAMEC wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have encountered some trouble while working with Moses 3.0
>>>>> optimization.
>>>>>>> When I use the configuration which you can see in the attached
>>>>> moses.ini the Moses tool runs infinitely long in the run 1 and is
>>> stuck on
>>>>> the Loading table into memory line. Do you know what could be the
>>> problem?
>>>>> For the "-v 100" log see the atached log.txt
>>>>>>> When I tweak some of the config options in the moses.ini file I get
>>>>> segfault, for example if I change the GenerationModel from current 1
>>> -> 0
>>>>> to 0,1 -> 0.
>>>>>>> The source corpus has the "word|factor" format for both used
>>> languages
>>>>> so there should be no problems, the factor is an int number.
>>>>>>> Can you imagine any reason why it could be behaving this way and
>>> what
>>>>> could have gone wrong?
>>>>>>> I launch the optimization via this command:
>>>>>>>
>>>>>>> /opt/moses-3.0/scripts/training/mert-moses.pl
>>>>> /root/faktor_6/corpus/heldout.src /root/faktor_6/corpus/heldout.ref \
>>>>>>> /opt/moses-3.0/bin/moses
>>> /root/faktor_6/experiment/model/moses.ini
>>>>> \
>>>>>>> --working-dir /root/faktor_6/tmp \
>>>>>>> --decoder-flags "-threads 6 -v 0" \
>>>>>>> --rootdir /opt/moses-3.0/scripts -mertdir /opt/moses-3.0/bin \
>>>>>>> --maximum-iterations 10
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Robert Adamec
>>>>>> --
>>>>>> Hieu Hoang
>>>>>> Researcher
>>>>>> New York University, Abu Dhabi
>>>>>> http://www.hoang.co.uk/hieu
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
>
>
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
