Hopefully StringPiece if it's newer code.
In my own code I wasn't yet using StringPiece and I did not rewrite it
after Moses switched mostly to StringPiece. Something to fix.
W dniu 09.10.2015 o 23:55, Lane Schwartz pisze:
Thanks, Marcin.
So when the various components of Moses pass words back and forth,
what do they send each other? std::string? StringPiece?
On Fri, Oct 9, 2015 at 4:28 PM, Marcin Junczys-Dowmunt
<junc...@amu.edu.pl <mailto:junc...@amu.edu.pl>> wrote:
For instance in my phrase table that would be
mosesdecoder/moses/TranslationModel/CompactPT/PhraseDecoder.h
StringVector<unsigned char, unsigned, std::allocator>
m_sourceSymbols;
StringVector<unsigned char, unsigned, std::allocator>
m_targetSymbols;
That's a memory-mapped vector of strings.
W dniu 09.10.2015 o 23:22, Lane Schwartz pisze:
Seriously? That sounds inefficient.
I've found code in KenLM that maps from strings to integers, but
not the other way around.
Marcin, do you know, for example, where any Moses code is for
doing the mapping for any data structure?
On Fri, Oct 9, 2015 at 4:14 PM, Marcin Junczys-Dowmunt
<junc...@amu.edu.pl <mailto:junc...@amu.edu.pl>> wrote:
Hi,
This would only be a simple thing if there was a common
framework for that, but there isn't. Each datastructure
implements its own vocabularies and look-up tables. There is
no common set of integers.
Best,
Marcin
W dniu 09.10.2015 o 23:11, Lane Schwartz pisze:
Hey,
I know this should be a simple thing to find, but what code
in Moses is responsible for mapping back and forth between
strings and integers?
Thanks,
Lane
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu <mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
When a place gets crowded enough to require ID's, social collapse
is not
far away. It is time to go elsewhere. The best thing about
space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
