For instance in my phrase table that would be
mosesdecoder/moses/TranslationModel/CompactPT/PhraseDecoder.h
StringVector<unsigned char, unsigned, std::allocator> m_sourceSymbols;
StringVector<unsigned char, unsigned, std::allocator> m_targetSymbols;
That's a memory-mapped vector of strings.
W dniu 09.10.2015 o 23:22, Lane Schwartz pisze:
Seriously? That sounds inefficient.
I've found code in KenLM that maps from strings to integers, but not
the other way around.
Marcin, do you know, for example, where any Moses code is for doing
the mapping for any data structure?
On Fri, Oct 9, 2015 at 4:14 PM, Marcin Junczys-Dowmunt
<junc...@amu.edu.pl <mailto:junc...@amu.edu.pl>> wrote:
Hi,
This would only be a simple thing if there was a common framework
for that, but there isn't. Each datastructure implements its own
vocabularies and look-up tables. There is no common set of integers.
Best,
Marcin
W dniu 09.10.2015 o 23:11, Lane Schwartz pisze:
Hey,
I know this should be a simple thing to find, but what code in
Moses is responsible for mapping back and forth between strings
and integers?
Thanks,
Lane
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support