Hi Huyen,
Look at GetValueGivenState() in LM/Implementation.h, and you see that
"finalState" is a language model state (actually both input state before
and output state after the scoring of a word). An LM state represents
the limited n-gram history of words at a specific Hypothesis some way
through the sentence. The class LanguageModelSRI does not use the input
state, but only produces an output state.
The state eventually ends up being FFState, which is used to compare
(FFState::operator==) and recombine Hypotheses with the same
(limited-size: (n-gram order) - 1) history of words, which makes search
faster.
For SRILM, the state is PointerState created via
LanguageModelSingleFactor::NewState(), and thus seems to work by SRILM
returning the same pointer every time you call Ngram::contextID() with
the same words passed to it. Disclaimer: I do not know SRILM in detail.
The LM scores are log (base e) probabilities.
Hope this helps,
David
On 05.12.2015 04:32, Vu Thuong Huyen wrote:
>
> Hi all,
>
> I’m integrating my LM in Mosesdecoder. I followed the SRI.cpp and
> NeuralLMWrapper.cpp. I don’t know how to assign value for *finalState
> variable. Could you explain for me how does it affect when decoding? I
> logged out the context when the system call “GetValue” function in my
> LM and in SRILM LM, the number of contexts were different.
>
>
>
> LMResult LanguageModelSRI::GetValue(VocabIndex wordId, VocabIndex
> *context) const
>
> {
>
> LMResult ret;
>
> ret.score = FloorScore(TransformLMScore(m_srilmModel->wordProb(
> wordId, context)));
>
> ret.unknown = (wordId == m_unknownId);
>
> return ret;
>
> }
>
>
>
>
>
> Which is value of *ret.score*? Log Probability or Probability?
>
>
>
> Best Regards,
>
> Huyen.
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
