I'm trying to decode using two factors and two Generation Models. This used
to work (using a moses repository from approximately Feb of 2015), but no
longer does. It generates blank translations.
Debugging its workings, I've found that the first GM works fine, but the
second one, it seems to try to look up the word plus the first factor
(instead of just the input factor), and fails:
Generation Model (./models/lc.classes.gm): Word = announcement found of size 1
Generation Model (./models/lc.classes.gm): Word = 92 found of size 1
Generation Model (./models/lc.classes.gm): Word = an found of size 1
Generation Model (./models/lc.classes.gm): Word = see found of size 1
Generation Model (./models/lc.classes.gm): Word = field found of size 1
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word =
announcement|67 NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word = 92|55
NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word = an|18
NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word = see|58
NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word =
field|67 NOT FOUND!
I ran these same debug prints through the old code, and while the word also
had factors appended to it, it successfully found the match. So I think this
has to do with the underlying collection used by the GenerationDictionary:
Current version:
typedef boost::unordered_map<const Word* , OutputWordCollection,
UnorderedComparer<Word>, UnorderedComparer<Word> > Collection;
Old version:
typedef std::map<const Word* , OutputWordCollection, WordComparer>
Collection;
Perhaps the older one only compared on the first factor. (It is unclear if
it made use of the 'input-factor' to GenerationModel, or if it just assumed
factor 0...)
I will probably hack around this issue for now in my own local repository. I
am timid to make changes since I am not sure what is the correct 'moses' way
to fix this problem.
Thanks all!
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
