Hi Lang Why do you say it looks weird?
>From what i see: 1. For both pb and hierarchical, when the system is tuned on the same data as the test set, the test set performs best. This is the expected result. 2. Test scores for nist 2 are significantly higher than nist3 or nist5, and the only test set where hiero beats phrase-based. Is there anything different about nist2? 3. from what people say, hiero should beat pb for zh-en, but from your results, they are much lower. This is a bit strange, but it may be due to the size of your training data hieu ps. I'll take a look @ your prev email about max-length & get back to you On Thu, Dec 29, 2011 at 4:57 PM, Bill_Lang(Gmail) <[email protected]>wrote: > Hi Moses Friends, > These days, I am running phrase based and hierarchical based moses. > Here my training corpus is FBIS (240k sentence pairs) for Chinese to > English translation. My moses version is updated on Dec 19, 2011. After > training I used NIST02, NIST03, NIST05 for tunning, respectively. Here I > got weird BLEUs as follows. > > Phrase Based Tunning on NIST02: NIST02 0.3176, NIST03 0.2827, NIST05 0.2761 > Phrase Based Tunning on NIST03: NIST02 0.3141, NIST03 0.2861, NIST05 0.2746 > Phrase Based Tunning on NIST05: NIST02 0.3109, NIST03 0.2831, NIST05 0.2822 > > Hierarchical Tunning on NIST02: NIST02 0.3403, NIST03 0.1620, NIST05 0.1577 > Hierarchical Tunning on NIST03: NIST02 0.3259, NIST03 0.1732, NIST05 0.1669 > Hierarchical Tunning on NIST05: NIST02 0.3286, NIST03 0.1689, NIST05 0.1678 > > I feel it is usual on Phrase based training, running, and testing. > Meanwhile, on Hierarchical, it is usual on NIST02. But it is so weird on > NIST03 and NIST05. The most strange thing, I think, is that running on > NIST03 or NIST05, the NIST02 BLEU is also usual. > > For phrase based and hierarchical based training, tunning, testing, I used > the same NIST datasets. > > Here, for Hierarchical training, I used following scripts. > > ------------------------------------------------ > $SCRIPTS_ROOTDIR/training/train-model.perl \ > -scripts-root-dir $SCRIPTS_ROOTDIR \ > -root-dir $WORKING_DIR/$TASK_NAME/training \ > -corpus $WORKING_DIR/$TASK_NAME/corpus/$TASK_NAME -f ch -e en \ > -alignment grow-diag-final-and \ > -lm 0:5:$2 \ > -max-phrase-length 8 \ > -hierarchical -glue-grammar \ > -parallel \ > >& $WORKING_DIR/$TASK_NAME/training/training.out > ------------------------------------------------ > > After carefully checking, max-phrase-length is useless for hierarchical > training. I have explained it in my last moses-support email. > > Here, is there any friend ever encountered the same weird BLEUs for > hierarchical as me? Maybe I has missed some training parameters. Can > someone help me to solve this problem? > > > Happy New Year~! > -Lang Jun > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
