I'm doing some large-scale testing and something odd has come up with placeholders.
I boiled down the various issues and they seem to all be related to these sentences (which differ only in the final token): foobar foobar <ne translation="@punctv@" entity="$$">@punctv@</ne> <ne translation="@numv@" entity="88">@numv@</ne> foobar and foobar foobar <ne translation="@punctv@" entity="$$">@punctv@</ne> <ne translation="@numv@" entity="88">@numv@</ne> . The first translates fine as foobar foobar foobar 88 $$ The second gives the error in the subject line. Moses2 output: [mike@c7test ~]$ /tmp/moses2 -f /opt/anlp/data/stelae/xlate.dut-eng/decoder.ini --input-file es2.txt Starting... Defined parameters (per moses.ini or switch): config: /opt/anlp/data/stelae/xlate.dut-eng/decoder.ini distortion-limit: 6 feature: UnknownWordPenalty WordPenalty PhrasePenalty ProbingPT name=TranslationModel0 num-features=4 path=/opt/anlp/data/stelae/xlate.dut-eng/PT2 input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 property-index=0 Distortion KENLM name=LM0 factor=0 path=/opt/anlp/data/stelae/xlate.dut-eng/languagemodel-en order=5 input-factors: 0 input-file: es2.txt mapping: 0 T 0 placeholder-factor: 1 threads: 8 weight: LexicalReordering0= 0.0417506 -0.00435432 0.0446905 0.037199 0.0339841 0.092545 Distortion0= 0.0506356 LM0= 0.147571 WordPenalty0= -0.232768 PhrasePenalty0= 0.0846606 TranslationModel0= 0.0786121 0.0609806 0.0258581 0.0643909 UnknownWordPenalty0= 1 xml-input: exclusive START featureFunctions.Load() Loading WordPenalty0 Finished loading WordPenalty0 Loading PhrasePenalty0 Finished loading PhrasePenalty0 Loading LexicalReordering0 Finished loading LexicalReordering0 Loading Distortion0 Finished loading Distortion0 Loading LM0 Finished loading LM0 Loading UnknownWordPenalty0 Finished loading UnknownWordPenalty0 Loading TranslationModel0 Initialized successfully! Finished loading TranslationModel0 START LoadMappings() END LoadMappings() END LoadDecodeGraphBackoff() Loaded : [2.4113] seconds RUN BATCH foobar foobar foobar 88 $$ terminate called after throwing an instance of 'util::Exception' what(): moses2/PhraseBased/../TargetPhrase.h:127 in std::map<long unsigned int, const Moses2::Factor*> Moses2::TargetPhrase<WORD>::GetPlaceholders(const Moses2::System&, const Moses2::Phrase<WORD>&) const [with WORD = Moses2::Word] threw util::Exception because `targetPos.size() != 1'. Placeholder should be aligned to 1, and only 1, word:2!=1 Aborted (core dumped) Any thoughts? This test case doesn't trigger failures in other language pairs.
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
