Yes. In combination both methods can achieve a size reduction of two
orders of magnitude compared to the standard binary unfiltered phrase
table.
W dniu 27.11.2012 11:39, Germán Sanchis Trilles pisze:
There are also some works addressing phrase-table size reduction. With
very limited loss in translation quality (if any), it has been shown
that the phrase-table can be pruned by up to 97%. See for instance:
http://www.mt-archive.info/EAMT-2011-Sanchis-Trilles.pdf
http://aclweb.org/anthology-new/N/N07/N07-2006.pdf
http://aclweb.org/anthology-new/D/D07/D07-1103.pdf
Cheers,
Germán
Phrase table shrinking is done :) See Compact Phrase Table in the
advanced section.
W dniu 27.11.2012 10:54, Miles Osborne pisze:
For a long time now I've wanted to see Moses on a small device.
Apart from
all of the extra functionality that isn't needed, one would
also need to
work on shrinking the phrase table and perhaps also the search
graph.
KenLM / RandLM already deal with making the language model
smaller.
An interesting research question would be as follows: can we frame
decoding on a
small device in terms of a budget and optimise that budget? We
normally don't
bother thinking this way and instead focus entirely on quality. But
it might be
possible to instead have a better connection with the amount of space
/ search
done and quality than we have already. I'm not sure if this is just
a matter of
fiddling with the beam size etc. Evince seems to suggest that this
doesn't
always give the expected behaviour (ie the relationship between BLEU
and beam
size isn't linear).
Miles
--
The University of Edinburgh is a charitable body, registered in
Scotland, with
registration number SC005336.
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
