thanks!
On 12/01/15 04:49, Jun-ya NORIMATSU wrote:
Hi,
> In the Moses master branch I found one other feature function that
> requires modifications:
>
> moses/LM/DALMWrapper.cpp
>
> This feature is currently not covered by a regression test, and I don't
> have any setup with this feature myself. I would not be able to test any
> modifications in that code and therefore would like to request that the
> authors apply the necessary updates themselves.
I've just finished to check and update the code.
https://github.com/moses-smt/mosesdecoder/commit/39799188a0478eda822167202fb7d404b35fbaad
https://github.com/moses-smt/mosesdecoder/commit/832b725c595d586f8802be12588fce9b495d36b8
Please let me know if you find some problems.
By the way, I'd like to add the regression test in near future.
Would you mind my asking the place(directories or repositories...) to
add the test code?
You can download the regression tests and the data needed from
https://github.com/moses-smt/moses-regression-tests
Add your test to the 'tests' directory. Add your LM to the 'LM' directory.
I would make a copy of an existing test, eg. phrase.basic-surface-only,
and name it
phrase.basic-surface-only.withDALM
and change it to use DALM instead.
You can test only your new test by running
./bjam --with-DALM=... --with-regtest=....
*phrase.basic-surface-only.withDALM.passed*
Thanks,
--
Jun-ya NORIMATSU
件名: [Moses-support] Feature score deltas in the chart decoder
日付: 2015-01-08 04:17
発信者: Matthias Huck <[email protected]>
宛先: Moses-support <[email protected]>
Hi,
I've just pushed a commit to Moses that brings about a slight change
wrt. the way the chart decoder deals with feature scores.
The chart decoder now stores deltas of individual feature scores instead
of constantly summing everything up. This behaviour is similar to what
we have been doing in the phrase-based decoder since a long time
already. The main purpose of this modification is to improve efficiency
with sparse features a bit.
https://github.com/moses-smt/mosesdecoder/commit/465b47566424efb707bdc063d0bff52b0650eb0a
The modification may however break existing feature function
implementations.
As a rule of thumb, any feature function that calls
ScoreComponentCollection::Assign()
in
EvaluateWhenApplied(const ChartHypothesis&, ...)
is affected and needs to be adapted to the new behaviour.
Basically, the ScoreComponentCollection variable passed to
EvaluateWhenApplied() now accumulates the delta score of the current
rule application only, whereas it was previously accumulating the
overall score of the partial hypothesis.
I.e., calling Assign() in EvaluateWhenApplied() now does not replace the
overall score any more, but has the same effect as calling PlusEquals().
If you are the author of a feature function that implements
EvaluateWhenApplied(const ChartHypothesis&, ...) and calls Assign()
within that method, or if you are using such a feature function in your
experiments, please update your implementation. The feature function
should call PlusEquals() instead and add a score delta.
I've already updated moses/LM/Ken.cpp and moses/LM/Implementation.cpp
and Rico has updated moses/LM/BilingualLM.cpp .
In the Moses master branch I found one other feature function that
requires modifications:
moses/LM/DALMWrapper.cpp
This feature is currently not covered by a regression test, and I don't
have any setup with this feature myself. I would not be able to test any
modifications in that code and therefore would like to request that the
authors apply the necessary updates themselves.
Please let me know in case you notice any issues or if you need any
further information or advice regarding this modification.
Cheers,
Matthias
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
