I think I've fixed it. Please git pull and recompile https://github.com/moses-smt/mosesdecoder/commit/44a0e52e3052371a850f7ee463279f4cc0522ea5
In case you wanted to know: The problem stems from the scores from the input confusion network is hacked into the decoder as a fake phrase-table score. Occasionally, this is forgotten by someone changing the code, hence the issue you bumped into. I'm refactoring the code to hopefully get rid of this problem. On 4 April 2013 23:56, Rob Voigt <[email protected]> wrote: > Thanks very much Hieu! > > Rob > > ----- Original Message ----- > From: "Hieu Hoang" <[email protected]> > To: "Rob Voigt" <[email protected]> > Sent: Thursday, April 4, 2013 5:21:42 AM > Subject: Re: [Moses-support] Running MERT with confusion network input > > > I'll take a look at it over the next few days. The confusion network isn't > often used. Hopefully, it hasn't been broken... > > > > > On 4 April 2013 10:18, Rob Voigt < [email protected] > wrote: > > > Dear Hieu, > > I put together a tiny sample model here: > http://www-nlp.stanford.edu/robvoigt/for_hieu.tar.gz > > Hopefully this will allow you to replicate the issue I'm having very > easily, as follows: > > $ tar -zvxf for_hieu.tar.gz > $ nohup nice ~/mosesdecoder/scripts/training/ mert-moses.pl > dev.zh.CNtinydev.en ~/mosesdecoder/bin/moses binarised-model/moses.ini > --decoder-flags="-threads 8 -inputtype 1 -drop-unknown" > --no-filter-phrase-table --working-dir tiny_mert-work --inputtype 1 > --mertdir ~/mosesdecoder/bin/ >& tiny_mert.out & > > Beyond those mentioned in my first e-mail, another alternative I tried was > setting -link-param-count 0 in the --decoder-flags. This seemed to allow > MERT to run, but it appeared to interpret the probabilities in the > confusion network as separate word entries, like so: > 0 -- (我 , ) (1.0 , ) > 1 -- (因为 , ) (1.0 , ) > 2 -- (*EPS* , ) (0.14 , ) (她 , ) (0.02 , ) (我们 , ) (0.01 , ) (他们 , ) (0.01 > , ) (你 , ) (0.00 , ) (他 , ) (0.01 , ) (我 , ) (0.21 , ) > I'm not sure why this is happening, but this too seems to clearly not be > the right solution. > > Any insights you might have would be sincerely appreciated. Thank you > again for your time! > > Best, > Rob > > > > > ----- Original Message ----- > From: "Rob Voigt" < [email protected] > > To: "Hieu Hoang" < [email protected] > > Sent: Wednesday, April 3, 2013 9:13:07 PM > Subject: Re: [Moses-support] Running MERT with confusion network input > > Hi Hieu, > > Thank you so much for your response. I'm sorry it took me a while to get > back to you - it's been a busy day! I still do not have mert working > with confusion networks. I'll make up a sample model and send it to you > tonight if you are still willing to look at it. Many thanks for your > assistance, > > Rob > > > > On 04/03/2013 05:25 PM, Hieu Hoang wrote: > > Hi > > > > Did you manage to get mert working with confusion networks? > > > > Hieu > > Sent while bumping into things > > > > On 3 Apr 2013, at 10:33 AM, Hieu Hoang < [email protected] > wrote: > > > >> Hi Rob > >> > >> I haven't used confusion network input but I do know about the > implementation of the input weights (it's added as an extra weight to the > weight table). > >> > >> If you can't figure out how to fix it, please send me a small sample > model to replicate the problem and I'll take a look at it. > >> > >> On 02/04/2013 20:42, Rob Voigt wrote: > >>> Hi all, > >>> > >>> I'm trying to use confusion networks with Moses - I can decode with > them successfully, manually inputting an insertion weight (-weight-i), but > am running into trouble doing MERT on a model that uses confusion networks. > >>> > >>> Running this command, with weight-i specified in the moses.ini file: > >>> nohup nice ~/mosesdecoder/scripts/training/ mert-moses.pl tune.zh > tune.en ~/mosesdecoder/bin/moses moses.ini --decoder-flags="-threads 8 > -inputtype 1 -drop-unknown" --inputtype 1 --mertdir ~/mosesdecoder/bin/ >& > CN_mert.out > >>> > >>> I get through one iteration of training, but then get the following > error during decoding on the second iteration of MERT: > >>> ERROR:You specified 6 input weights (weight-i), but you specified 1 > link parameters (link-param-count)! > >>> > >>> Looking at the run1.moses.ini file, it looks like five additional > weight-i values have been added. > >>> > >>> Things I've tried to fix this include: > >>> > >>> - Specifying 6 link parameters as a decoder flag to match - this dies > on the first iteration of MERT with this error: > >>> ERROR:You specified 1 input weights (weight-i), but you specified 6 > link parameters (link-param-count)! > >>> > >>> - Putting 6 weight-i parameters in the ini file and specifying 6 link > parameters as a decoder flag - this dies on the second iteration, with this > error: > >>> ERROR:You specified 11 input weights (weight-i), but you specified 6 > link parameters (link-param-count)! > >>> > >>> So it seems like the script is inserting 5 additional weight-i > parameters after the first iteration, that make it out of sync with the > link-param-count. > >>> > >>> How does one use MERT if the input is a confusion network? Is it > possible? > >>> > >>> Thanks very much for your time! > >>> > >>> -Rob > >>> _______________________________________________ > >>> Moses-support mailing list > >>> [email protected] > >>> http://mailman.mit.edu/mailman/listinfo/moses-support > > > > -- > Hieu Hoang > Research Associate > University of Edinburgh > http://www.hoang.co.uk/hieu > > > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
