i saw your checkins, is it working for you now?
Your test data doesn't seem to run for me, the lexical reordering file
seems to be corrupt.
(if you wanna share test data, can you do it via dropbox/google drive
rather than the Moses github)
On 03/09/2015 05:13, Michael Denkowski wrote:
Sounds good. I added a small test model to my branch:
https://github.com/moses-smt/mosesdecoder/raw/mjdenkowski/mmsapt-factor-test.tar.gz.
This translates a sample of fr-en news with a Mmsapt, surface LM, and
400-class LM.
--Michael
On Wed, Sep 2, 2015 at 2:56 AM, Hieu Hoang <[email protected]
<mailto:[email protected]>> wrote:
It should work. The function
EvaluateInIsolation()
in the LM is for optimisation reason. eg. if the target phrase is
'a b c d' and the LM is a trigram, the trigrams 'a b c' and 'b c
d' can be precalculated in EvaluateInIsolation().
Implementing a pt for factors requires setting up some variables,
which may not have happen yet in mmsapt. if you can send me a
small example model, i'll see what i can do
On 01/09/2015 02:11, Ulrich Germann wrote:
Hi Michael,
I have no experience with factored models, so I'm speculating
here to some degree. The reason the phrase table calls
EvaluateInIsolation is because all "isolated" phrase scores are
considered when pruning. In my opinion pruning should not happen
within the phrase tables (for exactly the reason that it does not
allow feature functions to be agnostic about other feature
functions) but by whatever object calls all the phrase tables and
does the generation. However, for software legacy reasons, that's
the way it is right now, and I'm not likely to address this issue
any time soon myself. The most reasonable fix for this in my
opinion is to move pruning where it belongs --- post all the
factor generation stuff.
Hieu is probably still the person with the best understanding of
how factored phrase table entry generation works, so maybe he can
chime in on this ...
Cheers - Uli
On Mon, Aug 31, 2015 at 11:29 PM, Michael Denkowski
<[email protected]
<mailto:[email protected]>> wrote:
Hi Ulrich,
I was looking into using a class-based LM with your dynamic
phrase table via generation models. I translate factor 0 to
0 with the Mmsapt, then generate target factor 1 (word class)
with a GM. The class-based LM operates on factor 1.
I'm hitting a segfault on what appears to be an
order-of-operations issue with the PT and LM. In
mmsapt.cpp:578, Mmsapt::mkTPhrase makes a call to
tp->EvaluateInIsolation. This calls all of the models,
including the LMs. The class LM tries to score factor 1,
which doesn't exist yet (since generation happens after
translation), and it dies. By nature, other phrase tables
don't have this issue since they can just pull up
pre-computed scores.
Is scoring with all of the models here a strategic choice to
get better performance or would it be sufficient to just
score with the PT features? Thanks!
--Michael
--
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh
_______________________________________________
Moses-developers mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-developers
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
