On Mon, Aug 03, 2009 at 06:09:57PM -0700, K. John Wu wrote:
> The current design is based on C convention of string and user can
> input strings through an ASCII file (which can not contain nil value
> anyway).
> [..]
> The current API in ibis::tablex uses std::string for the character
> arrays. It is common to assume nil character terminates the string,
> even though it is actually possible to interpret nil character
> differently.
If users would *not* use an ASCII file as input source but rather the
ibis::tablex interface which supports std::string, would FastBit
internally invoke c_str() or only use the characters up to the first NUL
byte? Or is the entire std::string object (with NULs) copied into the
table?
> One possibility of taking advantage of this feature would be to have a
> different string type column, say type opaque, to explicitly take the
> length information instead of relying on the string terminators.
I like the idea. But I would even go further and create a light-weight
buffer data structure to abstract it. This buffer class could then be an
possible argument to an append function. In particular, I like the
buffer model used by Boost Asio [1]:
A buffer object represents a contiguous region of memory as a
2-tuple consisting of a pointer and size in bytes. A tuple of the
form {void*, size_t} specifies a mutable region of memory.
Here are some examples from [1] that should give an intuition about how
to use buffers.
Creating a buffer:
char d1[128];
boost::asio::buffer b1(d1);
std::vector<char> d2(128);
boost::asio::buffer b2(d2);
boost::array<char, 128> d3;
boost::asio::buffer b3(d3);
Accessing the buffer:
unsigned char* p1 = boost::asio::buffer_cast<unsigned char*>(b1);
Buffer arithmetic:
boost::array<char, 6> a = { 'a', 'b', 'c', 'd', 'e' };
b1 = boost::asio::buffer(a);
b2 = boost::asio::buffer(a, 3); // { 'a', 'b', 'c' }
b3 = b1 + 2; // { 'c', 'd', 'e' }
b4 = boost::asio::buffer(b1 + 1, 3); // { 'b', 'c', 'd' }
What do you think about this model?
Matthias
[1]
http://www.boost.org/doc/libs/1_39_0/doc/html/boost_asio/reference/buffer.html
--
Matthias Vallentin
[email protected]
http://www.icir.org/matthias
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
