Thanks, Chris, I'll give it a shot. I'll be back if I have trouble getting the lattice input to work as expected.
Suzy On 27/03/2010, at 12:45 AM, Chris Dyer wrote: > The first weight in the lattice format is called "transition > probability", but it can be anything you want. It just becomes a > feature in the system's log-linear model. The weight used to bias > this feature is weight-i. > > Chris > > On Fri, Mar 26, 2010 at 1:17 AM, Suzy Howlett > <[email protected]> wrote: >> Thanks, that sounds like a good thing to try. But where would you >> specify >> the feature value? The numbers in the lattice format (as far as I >> know) are >> transition probability and distance to next node, so unless you can >> extend >> the list of numbers, I'm still not clear on how you incorporate the >> feature. >> Also, what weight is used, weight-i? >> >> Suzy >> >> On 26/03/2010, at 12:30 PM, Chris Dyer wrote: >> >>> That sounds reasonable. And, I don't think you'll need to add an >>> extra feature to moses to do this. The lattice input format lets >>> you >>> have a feature associated with a transition (in fact, I think you >>> can >>> have an arbitrary number of features), so you can use that to >>> encoded >>> whether the path your on corresponds to the reordered variant or >>> not. >>> -Chris >>> >>> On Thu, Mar 25, 2010 at 8:51 PM, Suzy Howlett >>> <[email protected]> wrote: >>>> >>>> Hi Chris, >>>> >>>> The preprocessing I referred to is a reordering of the words of the >>>> source >>>> sentence before translation. The overall idea would be to have a >>>> single >>>> Moses model that can handle both reordered and non-reordered >>>> sentences. >>>> The >>>> only way I've thought of to do this is to combine the sentence- >>>> level >>>> feature >>>> I mentioned with two phrase translation tables and a lattice input >>>> combining >>>> the reordered and non-reordered versions of a single sentence. >>>> Then we >>>> could >>>> have a number of other features that would influence the system's >>>> choice >>>> of >>>> which version to use. There are obviously a number of points at >>>> which >>>> this >>>> scheme could break down, and I have no idea if any of it will >>>> work, but I >>>> figured the only way to find out would be to try. I appreciate any >>>> suggestions you have. >>>> >>>> Suzy >>>> >>>> On 26/03/2010, at 11:32 AM, Chris Dyer wrote: >>>> >>>>> Moses uses features to discriminate between alternative >>>>> translations >>>>> of individual sentences, so if the value is constant for all >>>>> possible >>>>> translations (for example, because it is a function of the >>>>> input), the >>>>> model won't be able to take advantage of it. It sounds like you >>>>> might >>>>> be proposing something like this. What are you trying to do? >>>>> >>>>> -Chris >>>>> >>>>> On Thu, Mar 25, 2010 at 8:14 PM, Suzy Howlett >>>>> <[email protected]> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I am just starting my foray into the world of adding features >>>>>> to Moses >>>>>> and haven't quite got my head around it yet. Could someone please >>>>>> check I'm on the right track, or tell me if I've overlooked an >>>>>> easier >>>>>> alternative? >>>>>> >>>>>> The feature that I want to add is essentially a sentence-level >>>>>> flag to >>>>>> say whether a given input sentence has undergone a particular >>>>>> kind of >>>>>> preprocessing before being passed to Moses. My best guess is >>>>>> that I >>>>>> need to create a file containing a look-up table to indicate >>>>>> which >>>>>> sentences have been preprocessed, e.g. >>>>>> >>>>>> <Sentence 1> ||| 0 >>>>>> <Sentence 2> ||| 0 >>>>>> <Sentence 3> ||| 1 >>>>>> <Sentence 4> ||| 0 >>>>>> ... >>>>>> >>>>>> where 1 and 0 indicate that the sentence has and has not been >>>>>> preprocessed, respectively. Is this the best way to do it? Does >>>>>> anyone >>>>>> know of anyone doing something similar before? >>>>>> >>>>>> I imagine I will need a StatelessFeatureFunction that will open >>>>>> the >>>>>> file and read off the value for the input sentence, and two >>>>>> parameters >>>>>> added with AddParam (one for the weight and one to specify the >>>>>> file >>>>>> containing the table above). Does that sound right so far? If >>>>>> anyone >>>>>> has any pointers for getting started implementing this feature, >>>>>> I'd >>>>>> appreciate them. >>>>>> >>>>>> Thanks, >>>>>> Suzy >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>> >>>> >> >> _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
