Yes, using factors (in this case word sense disambiguation).
The LM lines are:
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
9 0 3 europarl.cleaned.en.kblm.mmap
9 5 3 europarl.cleaned.en.wsd.kblm.mmap
Language models are created using:
% ngram-count -order 3 -interpolate -text IN -lm OUT
% build_binary IN OUT
As you suspected, an error also occurs when using SRI:
moses: LanguageModelSRI.cpp:150: virtual float
Moses::LanguageModelSRI::GetValue(const std::vector<const Moses::Word*,
std::allocator<const Moses::Word*> >&, const void**, unsigned int*) const:
Assertion `(*contextFactor[count-1])[factorType] != __null' failed.
I am not quite sure what is causing this.
Could it be related to the use of binarized phrase tables?
On Feb 10, 2011, at 4:00 PM, Kenneth Heafield wrote:
> Weird. It's already checked that contextFactor is non-empty. This
> could be a bad or NULL Word * object or factor set incorrectly.
>
> Are you using factors? What are your LM lines from moses.ini?
>
> On 02/10/11 04:39, Christian Rishøj Jensen wrote:
>>
>> I am seeing a segmentation fault in KenLM this morning:
>>
>> reading bin ttable
>> size of OFF_T 8
>> binary phrasefile loaded, default OFF_T: -1
>> reading bin ttable
>> size of OFF_T 8
>> binary phrasefile loaded, default OFF_T: -1
>> Collecting options took 0.700 seconds
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> GetValueGivenState (this=0x14dd8c0, contextFactor=<value optimized out>,
>> state=..., len=0x7fffffffd72c) at LanguageModelKen.cpp:179
>> 179 std::size_t factor =
>> contextFactor.back()->GetFactor(GetFactorType())->GetId();
>> (gdb) where
>> #0 GetValueGivenState (this=0x14dd8c0, contextFactor=<value optimized out>,
>> state=..., len=0x7fffffffd72c) at LanguageModelKen.cpp:179
>> #1 0x00000000004927c8 in Moses::LanguageModel::Evaluate (this=0x14e3ee0,
>> hypo=..., ps=<value optimized out>, out=<value optimized out>)
>> at LanguageModel.cpp:227
>> #2 0x0000000000426d5b in Moses::Hypothesis::CalcScore (this=0x5c02010,
>> futureScore=<value optimized out>) at Hypothesis.cpp:298
>> #3 0x000000000044cc9a in Moses::SearchNormal::ExpandHypothesis
>> (this=0x22bed80, hypothesis=..., transOpt=<value optimized out>,
>> expectedScore=<value optimized out>) at SearchNormal.cpp:308
>> #4 0x000000000044ceb9 in Moses::SearchNormal::ExpandAllHypotheses
>> (this=0x22bed80, hypothesis=..., startPos=<value optimized out>,
>> endPos=<value optimized out>) at SearchNormal.cpp:281
>> #5 0x000000000044d23b in Moses::SearchNormal::ProcessOneHypothesis
>> (this=0x22bed80, hypothesis=<value optimized out>) at SearchNormal.cpp:247
>> #6 0x000000000044e5a0 in Moses::SearchNormal::ProcessSentence
>> (this=0x22bed80) at SearchNormal.cpp:95
>> #7 0x000000000043081c in Moses::Manager::ProcessSentence
>> (this=0x7fffffffdfc0) at Manager.cpp:100
>> #8 0x000000000040a518 in TranslationTask::Run (this=0x22bd830) at
>> Main.cpp:87
>> #9 0x00000000004086cf in main (argc=<value optimized out>, argv=<value
>> optimized out>) at Main.cpp:392
>>
>> Is it obvious to anyone what might be the cause of this?
>>
>> I am using binarized, memory mapped language models.
>>
>> best
>> Christian
>> _______________________________________________
>> Moses-support mailing list
>> [email protected]
>> http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
