Got an exception creating a PT2 with yesterday's master:
Binarize phrase and reordering model in probing table format:
/home/mike/stelae5/mosesdecoder/scripts/generic/binarize4moses2.perl
--phrase-table=/home/mike/stelae-projects/de-en/phrasemodel/model/phrase-table.gz
--lex-ro=/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz
--output-dir=/home/mike/stelae-projects/de-en/phrasemodel/PT2
--num-lex-scores=6
Executing: gzip -dc
/home/mike/stelae-projects/de-en/phrasemodel/model/phrase-table.gz |
/home/mike/stelae5/mosesdecoder/scripts/generic/../../contrib/sigtest-filter/filter-pt
-n 0 | gzip -c >
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.gz
sh:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../contrib/sigtest-filter/filter-pt:
No such file or directory
Executing:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/processLexicalTableMin
-in
/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz
-out /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro -T .
-threads all
Used options:
Text reordering table will be read from:
/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz
Output reordering table will be written to:
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr
Step size for source landmark phrases: 2^10=1024
Phrase fingerprint size: 16 bits / P(fp)=1.52588e-05
Single Huffman code set for score components: no
Using score quantization: no
Running with 24 threads
Pass 1/2: Creating phrase index + Counting scores
..................................................[5000000]
..................................................[10000000]
..................................................[15000000]
..................................................[20000000]
..................................................[25000000]
..................................................[30000000]
..................................................[35000000]
..................................................[40000000]
..................................................[45000000]
..................................................[50000000]
..................................................[55000000]
..................................................[60000000]
..................................................[65000000]
..................................................[70000000]
..................................................[75000000]
....................
Intermezzo: Calculating Huffman code sets
Creating Huffman codes for 32003 scores
Creating Huffman codes for 16732 scores
Creating Huffman codes for 31335 scores
Creating Huffman codes for 32076 scores
Creating Huffman codes for 15096 scores
Creating Huffman codes for 31659 scores
Pass 2/2: Compressing scores
..................................................[5000000]
..................................................[10000000]
..................................................[15000000]
..................................................[20000000]
..................................................[25000000]
..................................................[30000000]
..................................................[35000000]
..................................................[40000000]
..................................................[45000000]
..................................................[50000000]
..................................................[55000000]
..................................................[60000000]
..................................................[65000000]
..................................................[70000000]
..................................................[75000000]
....................
Saving to
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr
Done
Executing:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/addLexROtoPT
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.gz
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr |
gzip -c >
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.withLexRO.gz
Executing: ln -s pt.withLexRO.gz
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz
Executing:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/CreateProbingPT2
--num-scores 4 --log-prob --input-pt
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz
--output-dir /home/mike/stelae-projects/de-en/phrasemodel/PT2
--num-lex-scores 6
Starting...
Reading phrase table finished, writing remaining files to disk.
terminate called after throwing an instance of 'util::ProbingSizeException'
what(): ./util/probing_hash_table.hh:150 in
util::ProbingHashTable<EntryT, HashT, EqualT, ModT>::Entry*
util::ProbingHashTable<EntryT, HashT, EqualT, ModT>::Insert(const T&) [with
T = probingpt::Entry; EntryT = probingpt::Entry; HashT = boost::hash<long
unsigned int>; EqualT = std::equal_to<long unsigned int>; ModT =
util::DivMod; util::ProbingHashTable<EntryT, HashT, EqualT,
ModT>::MutableIterator = probingpt::Entry*; util::ProbingHashTable<EntryT,
HashT, EqualT, ModT>::Entry = probingpt::Entry] threw ProbingSizeException
because `++entries_ >= buckets_'.
Hash table with 1 buckets is full.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
