Hi, Thanks for the previous replies.
I am re-estimating the phrase pair table and enriching it with new phrases. The newly added phrases need values for prob, lw, inverse prob and inverse lw. Sometimes phrases are added in my systems whose lexical weights are unknown. For some the lw can be calculated, but for some it can not. For reasons I won't explain. I need to make a decision what to do with these unknown values. I have considered setting them to a fixed number: e.g. 0.1 or 0.01 or even 0.0001. I have however no clue what the impact of these values are. I was hoping someone could point me in the right direction. I would like to make an educated guess on what this value should be, but I do not have enough experience with MT to do this. I know from the previous replies that the values from the score vector are all multiplied together, after applying an exponential weight. But it would also help if someone could give me or point me towards the exact formula. Thanks in advance, Sanne -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Philipp Koehn Sent: vrijdag 8 mei 2009 15:29 To: [email protected] Subject: Re: [Moses-support] lexical weighting and inverse probabilities Hi, there should not be any zeros in this table, because that will, as you write, lead to an overall zero probability. -phi On Fri, May 8, 2009 at 11:54 AM, Sanne Korzec <[email protected]> wrote: > Ok thanks. > > Does this mean that if one of these values is zero in the table, one can > leave the entry out? Multiplication gives a result of zero. Or does the > exponential weight compensate for this? > > Sanne > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of Philipp > Koehn > Sent: donderdag 7 mei 2009 20:07 > To: [email protected] > Cc: [email protected] > Subject: Re: [Moses-support] lexical weighting and inverse probabilities > > Hi, > > they are all multiplied together, after applying an exponential weight. > > -phi > > On Thu, May 7, 2009 at 4:51 PM, Sanne Korzec <[email protected]> wrote: >> Hi, >> >> >> >> The final phrase pair table usually has a score vector of length 5: >> >> >> >> The components are: probability, lexical weights, inverse probability, >> inverde lex. Weights and a constant. >> >> >> >> How and why are the lexical weights, the inverse probabilities and the >> inverse lexical weighting exactly used during decoding? >> >> >> >> Sanne >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
