Hello again,
I ran compile-lm against the model.lm.es.gz and got the output
model.lm.es.blm but when using 8 0 5 in my moses.ini it crashes with this:
unk 0
not blank File: model.es.lm.blm
Aborted
Is this down to bad artefacts in the language model?
Kind regards,
Lee Ball
Infrastructure
That looks like an IRSTLM-format binary file. You might need the --text
yes flag for compile-lm.
You can confirm that the file is in the right format by opening it with a
text viewer such as less. The ARPA format is defined here:
Hi guys,
I just tried using KenLM out of interest for the first time with an old set
of models that were using IRSTLM.
Based on this: http://kheafield.com/code/kenlm/moses/ I changed the settings
from:
[lmodel-file]
1 0 5 /home/moses/languagemodels/model.es.lm.gz
to
[lmodel-file]
8 0 5
Hi,
Which ASCII character sequence represents newline in your file? Try
converting to UNIX newlines. Also can you send me the output of
zcat /home/moses/languagemodels/model.es.lm.gz |head -n 10 |gzip send.gz
(I'm asking you to rezip so that your mail client doesn't convert the
enter
