The Moses common vocabulary is moses/FactorCollection.h. Common practice in core Moses code is to pass around a const Factor * (which can be resolved to a StringPiece or a consecutive ID).
If a feature/phrase table has its own ids because e.g. it's baked into the binary file, then there's a std::vector to map from Moses ID to feature function ID. See moses/LM/Ken.h:99 for an example. std::string (or even StringPiece) conversion at decode time is a bug. A sadly common one. On 10/09/2015 10:22 PM, Lane Schwartz wrote: > Seriously? That sounds inefficient. > > I've found code in KenLM that maps from strings to integers, but not the > other way around. > > Marcin, do you know, for example, where any Moses code is for doing the > mapping for any data structure? > > > On Fri, Oct 9, 2015 at 4:14 PM, Marcin Junczys-Dowmunt > <[email protected] <mailto:[email protected]>> wrote: > > Hi, > This would only be a simple thing if there was a common framework > for that, but there isn't. Each datastructure implements its own > vocabularies and look-up tables. There is no common set of integers. > Best, > Marcin > > W dniu 09.10.2015 o 23:11, Lane Schwartz pisze: >> Hey, >> >> I know this should be a simple thing to find, but what code in >> Moses is responsible for mapping back and forth between strings >> and integers? >> >> Thanks, >> Lane >> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] <mailto:[email protected]> >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] <mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > -- > When a place gets crowded enough to require ID's, social collapse is not > far away. It is time to go elsewhere. The best thing about space travel > is that it made it possible to go elsewhere. > -- R.A. Heinlein, "Time Enough For Love" > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
