hi pratyush
that seems a bit strange. I'm not sure what it could be. You may want to
upgrade irstlm, it's now on version 5.60.01
On 16/03/2011 01:43, Pratyush Banerjee wrote:
Hi All,
I have been using Moses Revision 3739 (from subversion trunk) for
some time(about 3 months) now along with irstlm-5.50.01.
So far I never had any issues with the decoder. However yesterday
onwards am encountering a strange error during MERT. The decoder
simply dies due to a segmentation fault.
This is the error shown in mert-err.log
Loading lexical distortion models...have 1 models
Creating lexical reordering...
weights: 0.065 0.065 0.065 0.065 0.065 0.065
Loading table into memory...done.
Start loading LanguageModel
/home/pbanerjee/forum-translation/de2en/consumer/l_models/lm.irstlm.gz
: [1.000] seconds
In LanguageModelIRST::Load: nGramOrder = 5
Loading LM file (no MAP)
blmt
loadbin()
loading 252472 1-grams
loading 2695266 2-grams
loading 1998504 3-grams
loading 3828996 4-grams
loading 4506338 5-grams
done
OOV code is 252471
IRST: m_unknownId=252471
creating cache for storing prob, state and statesize of ngrams
Finished loading LanguageModels : [3.000] seconds
Start loading PhraseTable
/home/pbanerjee/forum-translation/de2en/consumer/mert-work/filtered/phrase-table.0-0.1.1
: [3.000] seconds
filePath:
/home/pbanerjee/forum-translation/de2en/consumer/mert-work/filtered/phrase-table.0-0.1.1
Finished loading phrase tables : [3.000] seconds
Start loading phrase table from
/home/pbanerjee/forum-translation/de2en/consumer/mert-work/filtered/phrase-table.0-0.1.1
: [3.000] seconds
sh: line 1: 18257 Segmentation fault
/home/pbanerjee/software/mosesdecoder-R3739/moses-cmd/src/moses -s 500
-config filtered/moses.ini -inputtype 0 -w -0.217391 -lm 0.108696 -d
0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 -tm
0.043478 0.043478 0.043478 0.043478 0.043478 -n-best-list
run1.best200.out 200 -i
/home/pbanerjee/forum-translation/de2en/consumer/orig/devset.de > run1.out
Exit code: 139
The decoder died. CONFIG WAS -w -0.217391 -lm 0.108696 -d 0.065217
0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 -tm 0.043478
0.043478 0.043478 0.043478 0.043478
This has been seen over multiple models, multiple language pairs even
binarized language models.
I also ran moses with GDB and this was the output:
Loading lexical distortion models...have 1 models
Creating lexical reordering...
weights: 0.300 0.300 0.300 0.300 0.300 0.300
Loading table into memory...done.
Start loading LanguageModel /data/Symantec/fr2en/l_models/lm.irstlm.gz
: [0.000] seconds
In LanguageModelIRST::Load: nGramOrder = 5
Loading LM file (no MAP)
blmt
loadbin()
loading 486152 1-grams
loading 4050539 2-grams
loading 10809711 3-grams
loading 15872277 4-grams
loading 17651596 5-grams
done
OOV code is 486151
IRST: m_unknownId=486151
creating cache for storing prob, state and statesize of ngrams
Finished loading LanguageModels : [6.000] seconds
Start loading PhraseTable
/data/Symantec/fr2en/mert-work/filtered/phrase-table.0-0.1.1 : [6.000]
seconds
filePath: /data/Symantec/fr2en/mert-work/filtered/phrase-table.0-0.1.1
Finished loading phrase tables : [6.000] seconds
Start loading phrase table from
/data/Symantec/fr2en/mert-work/filtered/phrase-table.0-0.1.1 : [6.000]
seconds
Program received signal SIGSEGV, Segmentation fault.
lmtable::mybsearch (size=<value optimized out>, offs=<value optimized
out>, value=<value optimized out>, ptr=<value optimized out>,
this=<value optimized out>) at lmtable.cpp:1109
1109 lmtable.cpp: No such file or directory.
in lmtable.cpp
(gdb) bt
#0 0x00000000004f9a73 in lmtable::mybsearch (size=<value optimized
out>, offs=<value optimized out>, value=<value optimized out>,
ptr=<value optimized out>, this=<value optimized out>)
#1 0x00000000004f9b72 in lmtable::search(int, table_entry_pos_t,
table_entry_pos_t, int, int *, <anonymous enum>, char **)
(this=0x77baa0, lev=<value optimized out>, offs=<value optimized out>,
n=78,
sz=78, ngp=<value optimized out>, action=LMT_FIND,
found=0x7fffffffb2b0) at lmtable.cpp:1062
#2 0x00000000004fbde1 in lmtable::get (this=0x77baa0, ng=...,
n=<value optimized out>, lev=<value optimized out>) at lmtable.cpp:1740
#3 0x00000000004fc51d in lmtable::lprob (this=0x77baa0, ong=DWARF-2
expression error: DW_OP_reg operations must be used either alone or in
conjuction with DW_OP_piece.
) at lmtable.cpp:2046
#4 0x00000000004fcc42 in lmtable::clprob (this=0x77baa0, codes=<value
optimized out>, sz=<value optimized out>, bow=0x0, bol=0x0,
state=0x7fffffffb940, statesize=0x7fffffffb948) at lmtable.cpp:2181
#5 0x00000000004ca4e6 in Moses::LanguageModelIRST::GetValue
(this=0x779dd0, contextFactor=<value optimized out>, finalState=<value
optimized out>, len=0x7fffffffb94c) at LanguageModelIRST.cpp:212
#6 0x0000000000492806 in Moses::LanguageModel::CalcScore
(this=0x1f43130, phrase=..., fullScore=<value optimized out>,
ngramScore=<value optimized out>) at LanguageModel.cpp:121
#7 0x0000000000465931 in Moses::TargetPhrase::SetScore
(this=0x7fffffffbfb0, translationScoreProducer=<value optimized out>,
scoreVector=<value optimized out>, weightT=<value optimized out>,
weightWP=<value optimized out>, languageModels=...) at
TargetPhrase.cpp:170
#8 0x00000000004b2378 in Moses::PhraseDictionaryMemory::Load
(this=<value optimized out>, input=<value optimized out>,
output=<value optimized out>, filePath=<value optimized out>,
weight=<value optimized out>, tableLimit=<value optimized out>,
languageModels=..., weightWP=<value optimized out>) at
PhraseDictionaryMemory.cpp:125
#9 0x00000000004af486 in
Moses::PhraseDictionaryFeature::LoadPhraseTable (this=0x87e9f60,
system=<value optimized out>) at PhraseDictionary.cpp:96
#10 0x00000000004afdfb in
Moses::PhraseDictionaryFeature::InitDictionary (this=0x87e9f60,
system=0x87ea268) at PhraseDictionary.cpp:194
#11 0x000000000047161f in Moses::TranslationSystem::ConfigDictionaries
(this=0x87ea268) at TranslationSystem.cpp:99
#12 0x000000000045f8e8 in Moses::StaticData::LoadData (this=0x76e420,
parameter=<value optimized out>) at StaticData.cpp:521
#13 0x0000000000407bcd in LoadDataStatic (parameter=<value optimized
out>) at ../../moses/src/StaticData.h:250
#14 main (parameter=<value optimized out>) at Main.cpp:334
However this still could not provide me any details of the error. I
saw a similar error being pointed out in the Moses forums but that was
attributed to memory issues. I am using considerable RAM machines
(16Gb) and still this fails.
Could any one shed any light on this ? Is there anything wrong with
the version of the moses I am using ?
Thanks and regards,
Pratyush
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
