Hi, guys,
I heard from some guy that we do not need to modify the moses
decoder
for adding some new features to moses.
As I currently dig into the source code, I found some places we
should pay attentions to:
1) phrase-extract/score.cpp specially, PhraseAlignment::createand
processPhrasePairs
2) train-factored-phrase-model.perl as mentioned by Jason!
I'm trying these stuff now, not totally success yet.
Does my understanding right? Anything missing?
The key puzzle is that need we modify the Decoder code?
Can't the Moses decoder dynamically recognize how many features we
used when we
add the new features in the phrase table for each rule?
Best Regards,
-Hongfei Jiang
2008/4/21 Jason Katz-Brown <[email protected]>
> Hi all,
>
> After snuffing out what I think is all of the steps required to add a
> new feature function to the Moses decoder and training system, I made
> a list of them. Perhaps it is helpful for others. Moses could
> certainly use more score producers to increase the count from anemic 5
> :-)
>
> --Jason
>
> HOWTO: add a new feature function to Moses
>
> ==MOSES decoder libary (moses/src/)==
> * Write ScoreProducer subclass, say MyScoreProducer. For example,
> implement these methods:
> - MyScoreProducer(ScoreIndexManager &scoreIndexManager);
> + Calls scoreIndexManager.AddScoreProducer(this).
> - std::vector<float> CalcMyScore(Hypothesis* hypothesis) const;
> + Calculates the value of the feature function.
> - size_t GetNumScoreComponents() const;
> - std::string GetScoreProducerDescription() const;
>
> * Add MyScoreProducer private member m_myScoreProducer to StaticData
> and GetMyScoreProducer() method.
> * Add m_weightMyScore float private member to StaticData and
> GetWeightMyScore() method.
> * In Hypothesis::CalcScore(), calculate feature function score using
> MyScoreProducer::CalcMyScore() and incorporate it with
> m_scoreBreakdown.PlusEquals().
> * In Parameter::Parameter(), AddParam("weight-m", "m", "weight for my
> score") where m is some letter.
> * In StaticData::LoadData(), load m_weightMyScore with
> m_parameter->GetParam().
> * In StaticData::LoadData(), create MyScoreProducer, passing in
> m_scoreIndexManager, and add its weights to m_allWeights. These two
> steps must take place exactly right after each other.
> * In StaticData::~StaticData(), delete m_myScoreProducer.
>
> ==MOSES decoder command-line program (moses-cmd/src/)==
> * In moses-cmd/src/IOStream::OutputNBestList(), output "m: " followed
> by your weights, separated by spaces.
>
> ==MOSES training scripts (scripts/training/)==
> * In train-factored-phrase-model.perl, change create_ini() to output
> default weights for weight-m.
> * Add in mert-moses.pl something like
> "m" => [ [ 1.0, 0.0, 2.0 ] ], # my score
> to _one_ of $default_triples or $additional_triples.
> * Add in mert-moses.pl "m=weight-m" to $ABBR_FULL_MAP.
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
