Hey Sanne- For some reason the GIZA code dumps the expected counts, not the model parameters.-- see line 115 of model2.cpp. If you'd like to see the parameters, you can change the line to:
aTable.rintTable(afile.c_str()); On Wed, Jul 16, 2008 at 5:24 AM, Sanne Korzec <[EMAIL PROTECTED]> wrote: > Hi, > > I've been playing around with the alignment model in IBM model II from GIZA. > > If have found that the individual values, of a certain j,l,m combinations in > a(i|j,l,m) do not sum up to one in GIZA. Even worse, some values exceed one. > > > Is anyone familiar why this is? I would expect that we need to normalize > after each iteration. > > An example of giza de-en/a2.1 > > I j l m value > > 0 1 5 4 0.353848 > 1 1 5 4 0.353848 > 2 1 5 4 0.292304 > 3 1 5 4 0.353848 > 4 1 5 4 0.292304 > 3 1 5 4 0.353848 > > This is more than one obviously... > > Regards, > Sanne > > > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Dyer > Sent: woensdag 9 juli 2008 18:40 > To: [EMAIL PROTECTED] > Subject: Re: [Moses-support] giza question > > I don't do much with model 2, so I don't know exactly how this is > implemented in GIZA, but there is an option: > > parameters modifying the models: > -------------------------------- > compactadtable = 1 (1: only 3-dimensional alignment table for IBM-2 and > IBM-3) > > which reduces the parameter space by one variable. My guess is that > for most corpora sizes estimating a large (IxJxMxL) parameter space is > difficult and subject to overfitting, so they cut off one of the > variables. If you add -compactadtable 0 to your command line, you'll > get all four variables. > > Chris > > On Wed, Jul 9, 2008 at 3:39 AM, Sanne Korzec <[EMAIL PROTECTED]> wrote: >> Thanks a lot, I'm not sure what went wrong, but I've got them now... >> >> One last question: the table produces 4 variables. >> >> I am assuming this is i,j,m,l from a(i|j,m,l) and then the probability. >> >> But one variable, l in my interpretation is fixed at 100 instead of the > real >> sentence length, why is this? >> >> Thanks, >> Sanne >> >> >> >> -----Original Message----- >> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Dyer >> Sent: woensdag 9 juli 2008 1:12 >> To: [EMAIL PROTECTED] >> Subject: Re: [Moses-support] giza question >> >>> >>> However, when I set model2dumpfrequency to 1, the translation table is >>> outputted, but not the alignment table. >> >> That's surprising. There isn't a 2.a.* file that's being generated? >> The relevant code is model2.cpp, line 115. I'm using the following >> command and getting model 2 a-tables at each iteration: >> >> ../GIZA++-v2/GIZA++ -m1 3 -m2 3 -model2dumpfrequency 1 >> -model1dumpfrequency 0 -m3 0 -m4 0 -hmmiterations 0 -s >> ./corpus/de.vcb -t ./corpus/en.vcb -CoocurrenceFile en-de.cooc -c >> corpus/en-de-int-train.snt >> >> >> * >>> >>> You say the alignment table is uniformly distributed, I have reason to >>> believe it might not be. Are you sure, you do not mean the translation >>> table? >> >> I think so. Fist, if you look at the perplexities between the last >> iteration of model 1 and the first iteration of model 2 (when the >> alignments are still uniform, as they are in model 1), you see they're >> virtually the same: >> >> Model1: (3) TRAIN CROSS-ENTROPY 1.94339 PERPLEXITY 3.84608 >> Model2: (1) TRAIN CROSS-ENTROPY 1.91221 PERPLEXITY 3.76385 >> >> Second, the code that initializes the distribution looks correct (see >> line 48-60 of model2.cpp) for uniform distribution. >> >> However, there was a bug in some code that was corrected in the SVN >> version of the code a while ago that was never released, I've >> recreated the package with the fixes which you can download from >> http://code.google.com/p/giza-pp/ . This might be causing the >> non-uniform initialization in your version. Try downloading the new >> version and see if this fixes it. >> >>> >>> As I understand it, IBM model II is different from IBM model I as it does >>> not only keep track of the translation between words but also the >> alignments >>> between positions of words, this means it uses two distributions and >> tables. >>> I'm having trouble getting GIZA to output the alignment table. >>> >>> Thanks, >>> Sanne >>> >>> -----Original Message----- >>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris > Dyer >>> Sent: dinsdag 8 juli 2008 21:16 >>> To: [EMAIL PROTECTED] >>> Subject: Re: [Moses-support] giza question >>> >>> On Tue, Jul 8, 2008 at 12:38 PM, Sanne Korzec <[EMAIL PROTECTED]> wrote: >>>> Hi mailing, >>>> >>>> >>>> >>>> I am training a word based model with IBM Model II in GIZA. I have two >>>> questions regarding this: >>>> >>>> >>>> >>>> 1) Does anybody know how the alignment distribution a(i|j,m,l) is >>>> initialized in GIZA? >>> It's initialized uniformly. >>> >>>> >>>> 2) Is there a way to output this table in between iterations, just >>>> like modelxdumpfrequency does for the translation distribution? >>>> >>> set model2dumpfrequency to 1. >>>> >>>> >>>> Thanks, >>>> >>>> Sanne >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>>> >>> >>> >> >> > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
