Hi.
Thank you very much for the support. I made the scripts work.
I had used train-model.perl instead of EMS (even changing GIZA++ to point to
the incremental version and with the --final-alignment-model option) but didn't
realize that train-model.perl has a built-in version of plain2snt.out instead
of using the one of your GIZA++ installation. Therefore there was
a mismatch when I added the new corpus manually with the actual plain2snt.out,
and the following steps didn't run properly.
I'll let you know if I find something else.
Bye.
--
Carlos A. Henríquez Q.
________________________________
De: Hieu Hoang <[email protected]>
Para: moses-support <[email protected]>
Enviado: Viernes 15 de junio de 2012 0:12
Asunto: [Moses-support] Fwd: Re: problem using plain2snt.out for incremental
training
fyi
-------- Original Message --------
Subject: Re: [Moses-support] problem using plain2snt.out for incremental
training
Date: Thu, 14 Jun 2012 23:08:26 +0100
From: Abby Levenberg <[email protected]>
To: Hieu Hoang <[email protected]>
In answer to the question regarding the vocabs basically the answer
is that
the standard giza-pp and inc-giza-pp output files are not compatible. so if you
try to use a giza-pp vocab file with inc-giza-pp it will fail. The same is true
for
all the output files.
Abby
On Thu, Jun 14, 2012 at 11:06 PM, Abby Levenberg <[email protected]> wrote:
Yes. The whole point of the online EM is to update the model with
>the statistics from the new sentence pairs.
>
>Abby
>
>
>
>On Thu, Jun 14, 2012 at 2:16 PM, Hieu Hoang <[email protected]> wrote:
>
>Sure I asked u this before but does your Giza update the alignment model on
>new data, or does it just align new sentences using the old model?
>>
>>Hieu
>>Sent from my flying horse
>>
>>Begin forwarded message:
>>
>>
>>From: Hieu Hoang <[email protected]>
>>>Date: 13 June 2012 04:06:31 PM GMT+01:00
>>>To: Carlos Henriquez <[email protected]>
>>>Cc: marco turchi <[email protected]>, moses-support
>>><[email protected]>
>>>Subject: Re: [Moses-support] problem using plain2snt.out for incremental
>>>training
>>>
>>>
>>I believe mgiza uses the old model to align new sentences. However, you
>>should ask Qin about that.
>>>
>>>
>>>Does abby's incremental Giza update the model?
>>>
>>>Hieu
>>>Sent from my flying horse
>>>
>>>On 13 Jun 2012, at 02:51 PM, Carlos
Henriquez <[email protected]> wrote:
>>>
>>>
>>>Hi Marco.
>>>>
>>>>
>>>>I'm facing the same problem right now and I found out that the code has an
>>>>invalid assert. The IDs will never match. I try commenting that line and
>>>>keep going but then I found out that the resulting vocabulary starts with
>>>>an ID=2 instead of ID=1 like the original vocabulary.
>>>>
>>>>
>>>>Again, I changed that to keep going but then the snt2cooc threw a
>>>>segmentation fault when I ran it in the inverse direction and even GIZA++
>>>>complained later when I tried to align in the src-trg direction.
>>>>
>>>>
>>>>Basically, I haven't been able to follow the steps described on the web.
>>>>Did you solve it?
>>>>
>>>>
>>>>Now to Hieu.
>>>>
>>>>With mgiza I haven't had any problems.
>>>>
>>>>
>>>>That force alignment from mgiza, is really incremental? or it only uses the
>>>>old models to align new sentences.
>>>>
>>>>
>>>>
>>>>________________________________
>>>> De: Hieu Hoang <[email protected]>
>>>>Para: marco turchi <[email protected]>
>>>>CC: moses-support <[email protected]>
>>>>Enviado: Miércoles 22 de febrero de 2012 8:48
>>>>Asunto: Re: [Moses-support] problem using plain2snt.out for incremental
>>>>training
>>>>
>>>>
>>>>Hi marco
>>>>
>>>>
>>>>Mgiza also offers incremental training (it's called forced alignment
>>>>there).
>>>>
>>>>
>>>>Not sure how it differs from abby levenbergs Giza version
>>>>
>>>>Hieu
>>>>Sent from my flying horse
>>>>
>>>>On 21 Feb 2012, at 04:40 PM,
marco turchi
<[email protected]> wrote:
>>>>
>>>>
>>>>Hi
>>>>>I'm following the
incremental training steps
from the Moses web site
and I'm running the
plain2snt command:
>>>>>plain2snt.out newData.lower.1.fr newData.lower.1.en -txt1-vocab
>>>>>../training/prepared.1/fr.vcb -txt2-vocab ../training/prepared.1/en.vcb
>>>>>
>>>>>but I get this error
>>>>>plain2snt.out:
plain2snt.cpp:28: int
loadVocab(): Assertion
`iid1.size()-1 == ID'
failed.
>>>>>
>>>>>and it is not clear to me
what is going wrong... any
ideas?
>>>>>
>>>>>I have trained the model
using the last version of
moses and I have
downloaded the incremental
training scripts from the
web site.
>>>>>
>>>>>Thanks a lot
>>>>>Marco
>>>>>
>>>>_______________________________________________
>>>>>Moses-support mailing list
>>>>>[email protected]
>>>>>http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>_______________________________________________
>>>>Moses-support mailing list
>>>>[email protected]
>>>>http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
