fyi
-------- Original Message --------
Subject: Re: [Moses-support] problem using plain2snt.out for
incremental training
Date: Thu, 14 Jun 2012 23:08:26 +0100
From: Abby Levenberg <[email protected]>
To: Hieu Hoang <[email protected]>
In answer to the question regarding the vocabs basically the answer is that
the standard giza-pp and inc-giza-pp output files are not compatible. so
if you
try to use a giza-pp vocab file with inc-giza-pp it will fail. The same
is true for
all the output files.
Abby
On Thu, Jun 14, 2012 at 11:06 PM, Abby Levenberg <[email protected]
<mailto:[email protected]>> wrote:
Yes. The whole point of the online EM is to update the model with
the statistics from the new sentence pairs.
Abby
On Thu, Jun 14, 2012 at 2:16 PM, Hieu Hoang
<[email protected] <mailto:[email protected]>> wrote:
Sure I asked u this before but does your Giza update the
alignment model on new data, or does it just align new sentences
using the old model?
Hieu
Sent from my flying horse
Begin forwarded message:
*From:* Hieu Hoang <[email protected]
<mailto:[email protected]>>
*Date:* 13 June 2012 04:06:31 PM GMT+01:00
*To:* Carlos Henriquez <[email protected]
<mailto:[email protected]>>
*Cc:* marco turchi <[email protected]
<mailto:[email protected]>>, moses-support
<[email protected] <mailto:[email protected]>>
*Subject:* *Re: [Moses-support] problem using plain2snt.out
for incremental training*
I believe mgiza uses the old model to align new sentences.
However, you should ask Qin about that.
Does abby's incremental Giza update the model?
Hieu
Sent from my flying horse
On 13 Jun 2012, at 02:51 PM, Carlos Henriquez
<[email protected]
<mailto:[email protected]>> wrote:
Hi Marco.
I'm facing the same problem right now and I found out that
the code has an invalid assert. The IDs will never match. I
try commenting that line and keep going but then I found out
that the resulting vocabulary starts with an ID=2 instead of
ID=1 like the original vocabulary.
Again, I changed that to keep going but then the snt2cooc
threw a segmentation fault when I ran it in the inverse
direction and even GIZA++ complained later when I tried to
align in the src-trg direction.
Basically, I haven't been able to follow the steps described
on the web. Did you solve it?
Now to Hieu.
With mgiza I haven't had any problems.
That force alignment from mgiza, is really incremental? or it
only uses the old models to align new sentences.
------------------------------------------------------------------------
*De:* Hieu Hoang <[email protected]
<mailto:[email protected]>>
*Para:* marco turchi <[email protected]
<mailto:[email protected]>>
*CC:* moses-support <[email protected]
<mailto:[email protected]>>
*Enviado:* Miércoles 22 de febrero de 2012 8:48
*Asunto:* Re: [Moses-support] problem using plain2snt.out for
incremental training
Hi marco
Mgiza also offers incremental training (it's called forced
alignment there).
Not sure how it differs from abby levenbergs Giza version
Hieu
Sent from my flying horse
On 21 Feb 2012, at 04:40 PM, marco turchi
<[email protected] <mailto:[email protected]>> wrote:
Hi
I'm following the incremental training steps from the Moses
web site and I'm running the plain2snt command:
plain2snt.out newData.lower.1.fr <http://newData.lower.1.fr>
newData.lower.1.en -txt1-vocab ../training/prepared.1/fr.vcb
-txt2-vocab ../training/prepared.1/en.vcb
but I get this error
plain2snt.out: plain2snt.cpp:28: int loadVocab(): Assertion
`iid1.size()-1 == ID' failed.
and it is not clear to me what is going wrong... any ideas?
I have trained the model using the last version of moses and
I have downloaded the incremental training scripts from the
web site.
Thanks a lot
Marco
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
