Philipp & Barry, Thanks for the info. At the moment, I'm treating my feature as only quasi-stateful.
To calculate the feature function score, I need to have the previous state and the current word, which allows me to calculate the current state, from which I get the current feature function score. However, the domain of the state space is extremely large, rendering it extremely unlikely that any two states will be identical. For this reason, I'm trying to treat the feature as stateless for the purposes of hypothesis recombination. I'm not sure if I've correctly implemented this. In my feature's FFState implementation, I am currently always returning zero from the function virtual int Compare(const FFState& other) const; Is that the correct way to ensure that for hypothesis recombination, this feature will not be taken into account? Thanks, Lane On 30 Mar 2010, at 4:48 PM, Philipp Koehn wrote: > Hi Lane, > > multi-threading in Moses works the same: different sentences > are distributed to different threads. > > Adding stateful feature functions has two costs: one is the > calculation of the function and one is the additional state splitting > which hurts recombination is the beam search. The first cost > is easy to measure, but the second one is more tricky - a different > trade-off between search errors and speed has to be found. > > -phi > > On Tue, Mar 30, 2010 at 8:36 PM, Lane Schwartz <[email protected]> wrote: >> Barry, >> >> Hieu may have mentioned that I'm working on a (more or less) stateful new >> feature that uses an incremental parser as a syntactic language model. >> >> In this feature, calculating the next state from a word and the previous >> state is not ridiculously expensive, but it certainly takes longer than most >> feature calculations. >> >> I'm wondering what parts of the decoding process are sped up when >> multithreading is enabled. In Joshua, we handle this by simply giving each >> thread a different sentence to translate. >> >> Thanks, >> Lane >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
