Hi Lane Ideally, moses should return the same output for the same input data. However, as Christian already noted, there are points in the code where pointer comparison is used to determine the order in which the search graph is explored, and this could potentially change the nbest lists.
As far as I recall, the pointer comparison happens because we want to use feature state for ordering hypotheses, and for some of the language model features (eg SRI) the state is just a pointer to some implementation specific data. It looks to me as though kenlm implements a proper FFState object, as opposed to using PointerState, so you could try changing language model to see if you get deterministic behaviour. Of course, the problem you're observing could be caused by some other source of non-determinism. You should be able to observe the search graphs diverging by running with -v 3. hope that helps - cheers - Barry On Thursday 24 March 2011 13:26, Lane Schwartz wrote: > While re-running an experiment (using the exact same configuration, same > models, translating the same data) I noticed that occasionally, I get a > slightly different 1-best list. > > Upon further examination, running Moses with the same config multiple times > often (but not always) produces slightly different n-best lists. This is a > bit worrisome from the perspective of being able to re-run an experiment to > reproduce results. > > Is this a known issue? > > I've examined the n-best lists, and it seems there are at least a couple of > interesting cases. In the simplest case, several translations of a given > sentence produce the exact same score, and these tied translations appear > in different order during different runs. This is a bit odd, but terribly > worrisome. The stranger case is when there are two different decoding runs, > and for a given sentence, there are translations that appear only in run A, > and different translations that only appear in run B. > > I have these n-best lists available for review - I'll post them in a > separate email in this thread. > > Cheers, > Lane -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
