Wouldn't factor->GetId() be the unique integer ID of the string? On Fri, Oct 9, 2015 at 5:54 PM, Hieu Hoang <[email protected]> wrote:
> const Factor* is the vocab id. It's guaranteed to be unique for each > unique string. You can map directly to the string using > factor->GetString() > > > > On 09/10/2015 22:55, Lane Schwartz wrote: > > Thanks, Marcin. > > So when the various components of Moses pass words back and forth, what do > they send each other? std::string? StringPiece? > > On Fri, Oct 9, 2015 at 4:28 PM, Marcin Junczys-Dowmunt <[email protected] > > wrote: > >> For instance in my phrase table that would be >> >> mosesdecoder/moses/TranslationModel/CompactPT/PhraseDecoder.h >> >> StringVector<unsigned char, unsigned, std::allocator> >> m_sourceSymbols; >> StringVector<unsigned char, unsigned, std::allocator> m_targetSymbols; >> >> That's a memory-mapped vector of strings. >> >> W dniu 09.10.2015 o 23:22, Lane Schwartz pisze: >> >> Seriously? That sounds inefficient. >> >> I've found code in KenLM that maps from strings to integers, but not the >> other way around. >> >> Marcin, do you know, for example, where any Moses code is for doing the >> mapping for any data structure? >> >> >> On Fri, Oct 9, 2015 at 4:14 PM, Marcin Junczys-Dowmunt < >> <[email protected]>[email protected]> wrote: >> >>> Hi, >>> This would only be a simple thing if there was a common framework for >>> that, but there isn't. Each datastructure implements its own vocabularies >>> and look-up tables. There is no common set of integers. >>> Best, >>> Marcin >>> >>> W dniu 09.10.2015 o 23:11, Lane Schwartz pisze: >>> >>> Hey, >>> >>> I know this should be a simple thing to find, but what code in Moses is >>> responsible for mapping back and forth between strings and integers? >>> >>> Thanks, >>> Lane >>> >>> >>> >>> _______________________________________________ >>> Moses-support mailing >>> [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> >> >> -- >> When a place gets crowded enough to require ID's, social collapse is not >> far away. It is time to go elsewhere. The best thing about space travel >> is that it made it possible to go elsewhere. >> -- R.A. Heinlein, "Time Enough For Love" >> >> >> > > > -- > When a place gets crowded enough to require ID's, social collapse is not > far away. It is time to go elsewhere. The best thing about space travel > is that it made it possible to go elsewhere. > -- R.A. Heinlein, "Time Enough For Love" > > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > > -- > Hieu Hoanghttp://www.hoang.co.uk/hieu > > -- When a place gets crowded enough to require ID's, social collapse is not far away. It is time to go elsewhere. The best thing about space travel is that it made it possible to go elsewhere. -- R.A. Heinlein, "Time Enough For Love"
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
