Hi Christian, We've had some problems with segfaults running SRILM-trained language models with KenLM settings and we're still troubleshooting the cause. This is the first report of a segfault using IRSTLM without KenLM. DoMY defaults to: [input-factors] 1 and [distortion-file] 0-0 wbe-msd-bidirectional-fe-allff..... and [ttable-limit] 20
These settings in your .ini file have been manually changed as there's no command line option to change them. Also, your [lmodel-file] setting shows an LM order=2. Is that right? This setting is configurable via the DoMY command line, but please make sure it matches the content of your LM. Did you compile mosesdecoder 4076 using the IRSTLM 5.60.03 built by DoMY or did you download/compile your own copy? Tom Precision Translation Tools. On Fri, 15 Jul 2011 16:30:29 +0200, Christian Wirth <[email protected]> wrote: > Dear Moses Users, > > i encountered a segmentation fault when using Moses (using DoMY-CE) > Ofc, i have read the according Mailing-List entries but my irstlm > version is already up-to-date (5.60.03) and the model was trained > with > the same moses version (4036) i'm using to run it. > I also manually installed the latest mosesdecoder 4076 (with irstlm > support) from SVN and tried to run my model with it, but that didnt > change anything. > > last output: > > Loading lexical distortion models...have 1 models > Creating lexical reordering... > weights: 1.300 1.300 1.300 1.300 1.300 1.300 > binary file loaded, default OFF_T: -1 > Start loading LanguageModel > /MTWorkspace/working-dir/lm-de/europarl.lm : > [1.000] seconds > In LanguageModelIRST::Load: nGramOrder = 2 > IRSTLM Language Model Type of > /MTWorkspace/working-dir/lm-de/europarl.lm > is 1 > Loading LM file (no MAP) > \data\ > loadtxt() > 1-grams: reading 49480 entries > 2-grams: reading 397470 entries > 3-grams: reading 100393 entries > 4-grams: reading 56471 entries > 5-grams: reading 28678 entries > done > starting to use OOV words [<unk>] > OOV code is 49480 > OOV code is 49480 > IRST: m_unknownId=49480 > creating cache for storing prob, state and statesize of ngrams > Finished loading LanguageModels : [2.000] seconds > Start loading PhraseTable > /MTWorkspace/working-dir/model-en-de/phrase-table : [2.000] seconds > filePath: /MTWorkspace/working-dir/model-en-de/phrase-table > Finished loading phrase tables : [2.000] seconds > Speicherzugriffsfehler > > > Moses.ini: > ######################### > ### MOSES CONFIG FILE ### > ######################### > > # input factors > [input-factors] > 1 > > # mapping steps > [mapping] > 0 T 0 > > # translation tables: table type (hierarchical(1), textual (0), > binary > (1)), source-factors, target-factors, number of scores, file > # OLD FORMAT is still handled for back-compatibility > # OLD FORMAT translation tables: source-factors, target-factors, > number > of scores, file > # OLD FORMAT a binary table type (1) is assumed > [ttable-file] > 1 0 0 5 /MTWorkspace/working-dir/model-en-de/phrase-table > > # no generation models, no generation-file section > > # language models: type(srilm/irstlm), factors, order, file > [lmodel-file] > 1 0 2 /MTWorkspace/working-dir/lm-de/europarl.lm > > > # limit on how many phrase translations e for each phrase f are > loaded > # 1 = all elements loaded > [ttable-limit] > 21 > > # distortion (reordering) files > [distortion-file] > 1-0 wbe-msd-bidirectional-fe-allff 6 > /MTWorkspace/working-dir/model-en-de/reordering-table.binary > > # distortion (reordering) weight > [weight-d] > 1.3 > 1.3 > 1.3 > 1.3 > 1.3 > 1.3 > 1.3 > > # language model weights > [weight-l] > 1.5000 > > > # translation model weights > [weight-t] > 1.20 > 1.20 > 1.20 > 1.20 > 1.20 > > # no generation models, no weight-generation section > > # word penalty > [weight-w] > -1 > > [distortion-limit] > 6 > > Training commands: > > train-model.perl -root-dir working-dir -corpus > working-dir/corpus/europarl.lowercased.$pair -f $LS -e $LT -alignment > grow-diag-final-and -reordering msd-bidirectional-fe -mgiza > -mgiza-cpus > 2 -lm 0:5:$WS/working-dir/lm-$LT/europarl.lm:0 > > cat phrase-table | sort | processPhraseTable -ttable 0 0 - -nscores > $NG > -alignment-info -out phrase-table > > processLexicalTable -in reordering-table.wbe-msd-bidirectional-fe > -out > reordering-table.binary _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
