Hello, I have problems with phrase table binarization. I'm aware that phrase table should be sorted before processing. So I tried different sorting options that have previously been offered by mail list or forums
administra...@ubuntu:/media/disk/deneme/model$ cat phrase-table | sort | processPhraseTable \ > -ttable 0 0 - -nscores 5 -out phrase-table processing ptree for stdin ERROR: xsource phrase already inserted (B)! line(245): '0.048 per cent ||| yüzde 0,048'lik ||| (1) (0) () ||| (1) (0) ||| 0.5 0.000257626 0.5 0.843105 2.718' f: 3 61 49 57 Aborted These sorting options also did not help in my case: sort -t "|" -k1,1 phrase-table > phrase-table2 LC_ALL=C sort -t "|" -k1,1 phrase-table > phrase-table2 cat phrase-table | LC_ALL=C sort > phrase-table2 When I work with above mentioned sort methods, I face with an error during binarization. Created files are blank (phrase-table2.binphr.srctree and phrase-table2.binphr.tgtdata) processPhraseTable: PhraseDictionaryTree.cpp:482: int Moses::PhraseDictionaryTree::Create(std::istream&, const std::string&): Assertion `numElement == 3 || numElement == 5' failed. Aborted I'm using a standart command for phrase table processing, such as: processPhraseTable -ttable 0 0 - -nscores 5 -out phrase-table2 I would really appreciate your help on this issue Regards
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
