I also have a problem with tuning a tree-based system.
I use a parallel text without syntactic annotation to train the
tree-based system. Then the extracted rule-table has the only
non-terminal on both source and target side. (in Japanese-to-English)
$ gzip -dc <path-to-model>/rule-table.gz | sort -k8 -t'|' -nr | head -n5
[X] [X] ||| 高 開口 数 による 高 解像度 の ||| with high resolution by means of a
high numerical aperture ||| ||| 1 2.07446e-05 1 2.2417e-08 2.718 ||| 0.0232558
0.0232558
[X] [X] ||| 高 開口 数 による 高 [X] ||| with high [X] high numerical aperture
||| 5-2 ||| 1 0.000225055 0.5 0.000282326 2.718 ||| 0.0232558 0.0465116
[X] [X] ||| 高 開口 数 による 高 [X] ||| with high [X] a high numerical
aperture ||| 5-2 ||| 1 0.000225055 0.5 1.64576e-05 2.718 ||| 0.0232558 0.0465116
[X] [X] ||| 高 開口 数 [X] 解像度 の ||| [X] resolution by means of a high
numerical aperture ||| 3-0 ||| 1 0.00589926 1 5.15592e-07 2.718 ||| 0.0232558
0.0232558
[X] [X] ||| 高 開口 数 [X] の ||| [X] of a high numerical aperture ||| 3-0
||| 1 0.0503172 1 0.000170465 2.718 ||| 0.0232558 0.0232558
Then passing "moses_chart" to "mert-moses-new" gives something wrong.
The first step of decoding is filtering of translation-table. The output
message of filtering scripts shows
filtering <path-to-model>/rule-table.gz ->
<path-to-tuning>/filtered/phrase-table.0-0.1...
0 of 550276 phrases pairs used (0.00%) - note: max length 10
filtering <path-to-model>/glue-grammar ->
<path-to-tuning>/filtered/phrase-table.0-0.2...
0 of 3 phrases pairs used (0.00%) - note: max length 10
It seems wrong. So how can I filter correctly? (or just do not filter?)
> From: Philipp Koehn <pko...@...>
> Subject: Re: tuning tree-based models
> Newsgroups: gmane.comp.nlp.moses.user
> Date: 2009-12-28 23:56:11 GMT (12 hours and 2 minutes ago)
>
> Hi,
>
> this should work... Does the moses process generate a proper n-best list file?
> There may be something wrong with running the decoder.
>
> Regarding the section "non-terminals" in the moses.ini file - don't worry,
> this
> is just a list of special non-terminals that are used for unknown words etc.
>
> -phi
>
> On Mon, Dec 28, 2009 at 2:37 PM, John Morgan
> <johnjosephmor...@...> wrote:
> > Hello All,
> > I want to tune a tree-based trained system. I pass the moses_chart
> > decoder to the mert-moses-new.perl script. The script seems to make 1
> > decoding run producing features.dat and scores.dat files, but it
> > writes an empty extract.out file. Then the script stops.
> > I also noticed that the train-moses.perl script only writes out a "X"
> > under [non-terminals] in the moses.ini file. I have several
> > non-terminals on the input side.
> > Thanks,
> >
> >
> > --
> > Regards,
> > John J Morgan
> > _______________________________________________
> > Moses-support mailing list
> > moses-supp...@...
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
--
Hwidong Na <[email protected]>
POSTECH
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
