Hi,
Moses server wants to keep one copy of a language model even if it has
different feature weights. The problem is that StatefulFeatureFunction
knows its own weight (a design I disagree with), so it can't just keep
one StatefulFeatureFunction.
LanguageModel is a child of StatefulFeatureFunction. All language
models are child classes of LanguageModel.
For KenLM, the object hierarchy looks like:
ScoreProducer < FeatureFunction < StatefulFeatureFunction <
LanguageModel < LanguageModelKen
and LanguageModelKen has boost::shared_ptr<lm::ngram::Model> m_ngram.
There is no LanguageModelImplementation intermediary any more but
m_ngram provides similar functionality with a different interface.
For all other language models, the object hierarchy is:
ScoreProducer < FeatureFunction < StatefulFeatureFunction <
LanguageModel < LMRefCount
LMRefCount's sole purpose is to delegate to its member
boost::shared_ptr<LanguageModelImplementation> m_impl
thereby passing calls to the LanguageModelImplementation class.
LanguageModelImplementation implements CalcScore, Evaluate, and
EvaluateChart. These call virtual methods. SRILM's hierarchy is
LanguageModelImplementation < LanguageModelSingleFactor <
LanguageModelPointerState < LanguageModelSRI
The interaction between these is somewhat messy and there's some old
code in there. I have simple wrappers for SRILM, IRSTLM, and RandLM
that implement the same interface as lm::ngram::Model (except for
ExtendLeft) and was considering making these wrappers template arguments
to LanguageModelKen (which would then be renamed). But it also makes
people suspect foul play if I break anything in the process.
Kenneth
On 02/07/2012 09:03 AM, Barry Haddow wrote:
> Hi Sylvain
>
> I've updated the doxygen docs, if that helps,
>
> cheers - Barry
>
> On Tuesday 07 February 2012 12:24:07 Sylvain Raybaud wrote:
>> Hi, List
>>
>> I've been digging into this but I'm still clueless... I really need to get
>> the LanguageModelImplementation object, but nowhere in Base.h/cpp or in
>> specific implementations like Ken.h/cpp could I find where it refers to
>> LanguageModelImplementation. Is it still used at all?
>>
>> By the way, I think mosesserver has got the same problem since its
>> implementation uses the same methods I used.
>>
>> regards,
>>
>> Sylvain
>>
>> On Monday 06 February 2012 13:20:07 Sylvain Raybaud wrote:
>>> Hello
>>>
>>> I'm trying to update a software that makes use of Moses API. It uses
>>> (among others) Moses' LanguageModel and LanguageModelImplementation
>>> classes. It used to call method GetLMImplementation of an object of type
>>> LanguageModel, in order to use methods NewState and GetValueForgotState
>>> of class LanguageModelImplementation. That was in august. Unfortunatelly,
>>> now I've pulled a new version of Moses from git, and method
>>> GetLMImplementation seems to have vanished. Methods NewState and
>>> GetValueForgotState seems to still be implemented in class
>>> LanguageModelImplementation. Now I can get the LM using:
>>>
>>> TranslationSystem ts =
>>> StaticData::Instance().GetTranslationSystem(std::string("default"));
>>> LMList list = ts.GetLanguageModels();
>>> LanguageModel *lm = *(list.begin());
>>>
>>> But I could not find a way to get the LanguageModelImplementation object
>>> in order to call aforementioned method. Do you have an idea of what I
>>> should change in the code?
>>>
>>> By the way, documentation on
>>> http://www.statmt.org/moses/html/hierarchy.html seems to be obsolete, as
>>> it seems to be lacking many files, classes and functions.
>>>
>>> regards,
>>>
>>> Sylvain
>>> _______________________________________________
>>> Moses-support mailing list
>>> [email protected]
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
> --
> Barry Haddow
> University of Edinburgh
> +44 (0) 131 651 3173
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
