The first weight in the lattice format is called "transition probability", but it can be anything you want. It just becomes a feature in the system's log-linear model. The weight used to bias this feature is weight-i.
Chris On Fri, Mar 26, 2010 at 1:17 AM, Suzy Howlett <[email protected]> wrote: > Thanks, that sounds like a good thing to try. But where would you specify > the feature value? The numbers in the lattice format (as far as I know) are > transition probability and distance to next node, so unless you can extend > the list of numbers, I'm still not clear on how you incorporate the feature. > Also, what weight is used, weight-i? > > Suzy > > On 26/03/2010, at 12:30 PM, Chris Dyer wrote: > >> That sounds reasonable. And, I don't think you'll need to add an >> extra feature to moses to do this. The lattice input format lets you >> have a feature associated with a transition (in fact, I think you can >> have an arbitrary number of features), so you can use that to encoded >> whether the path your on corresponds to the reordered variant or not. >> -Chris >> >> On Thu, Mar 25, 2010 at 8:51 PM, Suzy Howlett >> <[email protected]> wrote: >>> >>> Hi Chris, >>> >>> The preprocessing I referred to is a reordering of the words of the >>> source >>> sentence before translation. The overall idea would be to have a single >>> Moses model that can handle both reordered and non-reordered sentences. >>> The >>> only way I've thought of to do this is to combine the sentence-level >>> feature >>> I mentioned with two phrase translation tables and a lattice input >>> combining >>> the reordered and non-reordered versions of a single sentence. Then we >>> could >>> have a number of other features that would influence the system's choice >>> of >>> which version to use. There are obviously a number of points at which >>> this >>> scheme could break down, and I have no idea if any of it will work, but I >>> figured the only way to find out would be to try. I appreciate any >>> suggestions you have. >>> >>> Suzy >>> >>> On 26/03/2010, at 11:32 AM, Chris Dyer wrote: >>> >>>> Moses uses features to discriminate between alternative translations >>>> of individual sentences, so if the value is constant for all possible >>>> translations (for example, because it is a function of the input), the >>>> model won't be able to take advantage of it. It sounds like you might >>>> be proposing something like this. What are you trying to do? >>>> >>>> -Chris >>>> >>>> On Thu, Mar 25, 2010 at 8:14 PM, Suzy Howlett >>>> <[email protected]> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I am just starting my foray into the world of adding features to Moses >>>>> and haven't quite got my head around it yet. Could someone please >>>>> check I'm on the right track, or tell me if I've overlooked an easier >>>>> alternative? >>>>> >>>>> The feature that I want to add is essentially a sentence-level flag to >>>>> say whether a given input sentence has undergone a particular kind of >>>>> preprocessing before being passed to Moses. My best guess is that I >>>>> need to create a file containing a look-up table to indicate which >>>>> sentences have been preprocessed, e.g. >>>>> >>>>> <Sentence 1> ||| 0 >>>>> <Sentence 2> ||| 0 >>>>> <Sentence 3> ||| 1 >>>>> <Sentence 4> ||| 0 >>>>> ... >>>>> >>>>> where 1 and 0 indicate that the sentence has and has not been >>>>> preprocessed, respectively. Is this the best way to do it? Does anyone >>>>> know of anyone doing something similar before? >>>>> >>>>> I imagine I will need a StatelessFeatureFunction that will open the >>>>> file and read off the value for the input sentence, and two parameters >>>>> added with AddParam (one for the weight and one to specify the file >>>>> containing the table above). Does that sound right so far? If anyone >>>>> has any pointers for getting started implementing this feature, I'd >>>>> appreciate them. >>>>> >>>>> Thanks, >>>>> Suzy >>>>> >>>>> _______________________________________________ >>>>> Moses-support mailing list >>>>> [email protected] >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>> >>> >>> > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
