hi
I have problem when converting phrase-table.gz to hard disk binary image. i got
the following error:
[m...@ibb]# gzip -cd work/20100914/model/phrase-table.gz | LC_ALL=C sort |
nlp/moses/misc/processPhraseTable -ttable 0 0 - -nscores 5 -out
work/20100914/binary/model/phrase-table
processing ptree for stdin
..................................................[phrase:500000]
..........................distinct source phrases: 762319 distinct first words
of source phrases: 11727 number of phrase pairs (line count): 3639432
WARNING: there are src voc entries with no phrase translation: count 1156
There exists phrase translations for 10571 entries
i checked the line by the following command, and it seems to be ok.
[m...@ibb]# gzip -cd work/20100914/model/phrase-table.gz | sed -n '1150,1160p'
Then i removed 10 lines from 1150-1160, and the problem still exist
[m...@ibb]# LC_ALL=C sort | nlp/moses/misc/processPhraseTable -ttable 0 0 -
-nscores 5 -out work/20100914/binary/model/phrase-table <
work/20100914/model/phrase-table.cleaned
processing ptree for stdin
..................................................[phrase:500000]
..........................distinct source phrases: 762319 distinct first words
of source phrases: 11727 number of phrase pairs (line count): 3639423
WARNING: there are src voc entries with no phrase translation: count 1156
There exists phrase translations for 10571 entries
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
