const Factor* is the vocab id. It's guaranteed to be unique for each
unique string. You can map directly to the string using
factor->GetString()
On 09/10/2015 22:55, Lane Schwartz wrote:
Thanks, Marcin.
So when the various components of Moses pass words back and forth,
what do they send each other? std::string? StringPiece?
On Fri, Oct 9, 2015 at 4:28 PM, Marcin Junczys-Dowmunt
<[email protected] <mailto:[email protected]>> wrote:
For instance in my phrase table that would be
mosesdecoder/moses/TranslationModel/CompactPT/PhraseDecoder.h
StringVector<unsigned char, unsigned, std::allocator>
m_sourceSymbols;
StringVector<unsigned char, unsigned, std::allocator>
m_targetSymbols;
That's a memory-mapped vector of strings.
W dniu 09.10.2015 o 23:22, Lane Schwartz pisze:
Seriously? That sounds inefficient.
I've found code in KenLM that maps from strings to integers, but
not the other way around.
Marcin, do you know, for example, where any Moses code is for
doing the mapping for any data structure?
On Fri, Oct 9, 2015 at 4:14 PM, Marcin Junczys-Dowmunt
<[email protected] <mailto:[email protected]>> wrote:
Hi,
This would only be a simple thing if there was a common
framework for that, but there isn't. Each datastructure
implements its own vocabularies and look-up tables. There is
no common set of integers.
Best,
Marcin
W dniu 09.10.2015 o 23:11, Lane Schwartz pisze:
Hey,
I know this should be a simple thing to find, but what code
in Moses is responsible for mapping back and forth between
strings and integers?
Thanks,
Lane
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected] <mailto:[email protected]>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
When a place gets crowded enough to require ID's, social collapse
is not
far away. It is time to go elsewhere. The best thing about
space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
