it's been a while since anyone looked at the SRILM code. It should still
work but I can't remember exactly whats going on
On 05/11/2015 14:04, koormoosh wrote:
I am integrating my LM in mosesdecoder. I started by looking into the
Skeleton files, and the SRI code. Things are clear except for these
lines in SRI.cpp which I cannot wrap my head around them:
ngram[count] = Vocab_None;
....
if (finalState) {
ngram[0] = lmId;
unsigned int dummy;
*finalState = m_srilmModel-
assuming that lmId is the id of the last word of the sequence, I don't
understand the functionality of finalState (probably because I lack
the MT background). I wonder if you could kindly clarify these things
if you are familiar with LM integration, or SRI integration in specific:
*1) why are we adding ngram[count] = Vocab_None*
*i guess this is initialising the array element. eg. if your LM is a
trigram but you only want to calculate the score for a unigram, then set
the 1st element in the array to the word, and the other elements to
Vocab_None.
This is how SRILM does it, you don't have to follow the same design in
your LM
*
*2) what is being checked in the if-condition if(finalState),
*
*Don't know*
*3) what is happenning in:
*finalState = m_srilmModel->contextID(ngram, dummy);*
*This is state information required by the decoder to decide whether to
recombine the hypothesis with another hypothesis. For the language
model, if the trigram is
a b c
The state information is a unique id for the BIGRAM 'b c'. This could be
the hash of the bigram, the memory address of the node where is bigram
is stored, or the string itself. As long as it is different from 'b d,
'd e' etc.
This is the basic description - there are some optimization you can do,
but its important you understand this 1st.
I recommend looking at Philipp Koehn's book. This paper describe a
similar thing, but for syntactic MT
https://kheafield.com/professional/edinburgh/left_paper.pdf
*
Thank you!
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
