It's useful to have epsilons since it simplifies the creation of lattices in some cases. Yes, you can convert them to a deterministic equivalent, but that involves implementing FSA determinatization (or using a tool like https://pypi.python.org/pypi/pyfst), which may not be convenient.
Btw, I've also noticed that memory usage with lattices/CNs explodes with non-binarized phrase tables (maybe also with binarized PTs?). This is independent of the size of the phrase table and only seems to be a function of the lattice structure. I'm not sure what's going on (the code has changed substantially since I last looked at it). But, you should always match paths in the lattice with paths in the phrase table trie- maybe moses is now trying to extract all possible paths in the lattice up to max-phrase-size or something? On Fri, Oct 4, 2013 at 11:22 AM, Nicola Bertoldi <[email protected]> wrote: > I don't see any reason why a lattice should contain an EPSILON edge. > > In a confusion network, EPSILON are needed to allow the translation of input > of different lengths. > The sausage structure of the CN imposes the same amount of source words, > and the EPSILONs overcome this constraint. > > This is not the case for lattice, because you can have any number of > edges/words in a complete source path. > > > cheers, > Nicola > > > > On Oct 4, 2013, at 2:52 PM, Hieu Hoang wrote: > > I'm just looking at the lattices decoding, as implemented in moses. > > for confusion networks, it's fair to have EPSILON words (that represent blank > words). However, I don't see the point of them in lattices. > > Anyone have an opinion? How is it implemented in cdec & joshua? > > -- > Hieu Hoang > Research Associate > University of Edinburgh > http://www.hoang.co.uk/hieu > > _______________________________________________ > Moses-support mailing list > [email protected]<mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
