BTW, what's the rationale for using StringPiece instead of std::string? I thought the main reason for using StringPiece was for implicit conversion from char *
On Fri, Oct 9, 2015 at 5:15 PM, Kenneth Heafield <[email protected]> wrote: > The Moses common vocabulary is moses/FactorCollection.h. Common > practice in core Moses code is to pass around a const Factor * (which > can be resolved to a StringPiece or a consecutive ID). > > If a feature/phrase table has its own ids because e.g. it's baked into > the binary file, then there's a std::vector to map from Moses ID to > feature function ID. See moses/LM/Ken.h:99 for an example. > > std::string (or even StringPiece) conversion at decode time is a bug. A > sadly common one. > > On 10/09/2015 10:22 PM, Lane Schwartz wrote: > > Seriously? That sounds inefficient. > > > > I've found code in KenLM that maps from strings to integers, but not the > > other way around. > > > > Marcin, do you know, for example, where any Moses code is for doing the > > mapping for any data structure? > > > > > > On Fri, Oct 9, 2015 at 4:14 PM, Marcin Junczys-Dowmunt > > <[email protected] <mailto:[email protected]>> wrote: > > > > Hi, > > This would only be a simple thing if there was a common framework > > for that, but there isn't. Each datastructure implements its own > > vocabularies and look-up tables. There is no common set of integers. > > Best, > > Marcin > > > > W dniu 09.10.2015 o 23:11, Lane Schwartz pisze: > >> Hey, > >> > >> I know this should be a simple thing to find, but what code in > >> Moses is responsible for mapping back and forth between strings > >> and integers? > >> > >> Thanks, > >> Lane > >> > >> > >> > >> _______________________________________________ > >> Moses-support mailing list > >> [email protected] <mailto:[email protected]> > >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] <mailto:[email protected]> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > > > -- > > When a place gets crowded enough to require ID's, social collapse is not > > far away. It is time to go elsewhere. The best thing about space travel > > is that it made it possible to go elsewhere. > > -- R.A. Heinlein, "Time Enough For Love" > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- When a place gets crowded enough to require ID's, social collapse is not far away. It is time to go elsewhere. The best thing about space travel is that it made it possible to go elsewhere. -- R.A. Heinlein, "Time Enough For Love"
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
