Hello,
I haven't used suffix arrays lately, but in the past they were only used
for the forward translation probabilities, and not for the backards ones
(due to complexity issues). This can result in a loss of 1 point BLEU.
Holger
On 02/15/2013 11:32 AM, Mirkin, Shachar wrote:
Hi,
I've been trying for a while to use incremental training, but I'm
running into quite a few issues.
The updates seem to be working fine, as evident from updates
containing OOVs, but the performance when using the dynamic suffix
array is much inferior (several BLEU points) in comparison to using
the regular Moses server on the same dataset and with the same model.
In both cases I used a model trained with incremental GIZA, so this
seems like an effect of the suffix array rather than a different
alignment model.
I was not making any updates to the server in these experiments.
Could this be a result of a limit on the memory that is used when the
suffix array is loaded (my corpus contains 1M sentence pairs)? Any
other ideas for the cause of the decrease in performance?
Concerning updates, is there a way to change the incremental GIZA
parameters, such as the interpolation parameter gamma?
Lastly, when my ini file for loading the server in the suffix array
mode contains the complete reordering table of the trained model
(rather than the filtered one for the test test), it takes forever to
load. Any suggestions?
Thanks a lot,
Shachar
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
