Dear Moses community, Is it correct that when using --discount_fallback, if discount is computable from Kneyser-Ney, the fallback will not be used?
Regards, Liling For context: $ cat test.zh 一 二 三 四 ~$ time ~/mosesdecoder/bin/lmplz -o 3 --discount_fallback=0 < test.zh > test.zh.arpa === 1/5 Counting and sorting n-grams === Reading /home/ltan/test.zh ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 **************************************************************************************************** Unigram tokens 4 types 7 === 2/5 Calculating and sorting adjusted counts === Chain sizes: 1:84 2:75238187008 3:141071597568 Substituting fallback discounts for order 0: D1=0 D2=0 D3+=0 Substituting fallback discounts for order 1: D1=0 D2=0 D3+=0 Substituting fallback discounts for order 2: D1=0 D2=0 D3+=0 Statistics: 1 7 D1=0 D2=0 D3+=0 2 5 D1=0 D2=0 D3+=0 3 4 D1=0 D2=0 D3+=0 Memory estimate for binary LM: type B probing 376 assuming -p 1.5 probing 436 assuming -r models -p 1.5 trie 298 without quantization trie 3328 assuming -q 8 -b 8 quantization trie 321 assuming -a 22 array pointer compression trie 3351 assuming -a 22 -q 8 -b 8 array pointer compression and quantization === 3/5 Calculating and sorting initial probabilities === Chain sizes: 1:84 2:80 3:80 ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 #################################################################################################### === 4/5 Calculating and writing order-interpolated probabilities === Chain sizes: 1:84 2:80 3:80 ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 #################################################################################################### === 5/5 Writing ARPA model === Name:lmplz VmPeak:211512300 kB VmRSS:4740 kB RSSMax:48754048 kB user:2.64 sys:159.492 CPU:162.132 real:162.148 real 2m42.152s user 0m2.644s sys 2m39.492s
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
