They way that Giza's built-in dictionary option works is to take the dictionary into account *only* during the first round of Model 1 training, with the idea that it will be used for better initialization in EM. I believe that it's generally more effective to simply append the dictionary to the end of your parallel corpus (perhaps multiple times like Philipp suggests), so that it influences EM at every iteration for every Model.
--Chris On Feb 29, 2008, at 7:13 AM, Philipp Koehn wrote: > Hi, > > I do not know how the GIZA++ option of using a dictionary works, > I have never tried it. But a very common way of using a dictionary > is to include it as additional parallel training data. You could even > weight it more strongly by adding it multiple times (or fiddle with > the sentence count value in the .snt file, as you mention). > > -phi > > On Thu, Feb 28, 2008 at 2:45 AM, aditya sarpotdar > <[EMAIL PROTECTED]> wrote: >> I am trying to use dictionary for alignment using GIZA++. >> I have observed that even thought I ask GIZA to do so (using a >> command >> line), the output does not change. >> The model parameters without using the dictionary and after using the >> dictionary are same. >> When I looked into the source code, I found a flag indicating GIZA >> to use >> dictionary which is NEVER set to TRUE. >> When I changed the code to set the flag, I found GIZA using the >> dictionary >> in the exact opposite way. The score of the words which are in the >> dictionary went lower! >> Did anyone face a similar problem? >> Am I supposed to set occurance of sentence in .snt file (first line >> in every >> sentence pair) to -1? >> I appreciate if someone could shed some light on this. >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
