Hi, Matthias, There is a new release of FastBit with some minimal support for blobs. The main functions are defined in src/category.h as class ibis::blob. It only supports reading and writing -- no querying and no participations in group-by operations either.
If you have the chance please give a try and let us know what additional functions might be useful. Thanks. John PS: The latest tar ball is located at <https://codeforge.lbl.gov/frs/download.php/90/fastbit-ibis1.1.3.tar.gz>. You can find some information about the release at <https://codeforge.lbl.gov/frs/shownotes.php?release_id=91> On 8/5/2009 9:28 AM, Matthias Vallentin wrote: > On Tue, Aug 04, 2009 at 05:32:02PM -0700, K. John Wu wrote: >> I would prefer to stay away from boost library because it introduces >> dependencies for FastBit only to make use of something relatively >> minor.. > > The reason why I references this buffer model was only to give feel how > such a raw byte buffer could be implemented in FastBit (I did not mean > to propose the inclusion of Boost in FastBit). This abstraction has > several benefits such as > > * performance. Data is not copied but stored in its original place: > while std::string::c_str performs a deep copy up to the first NUL > byte, the construction of a buffer would only point into the memory > location. > > * decoupling from the storage interface. The user can decide how to > store the raw data (e.g., in a std::string, std::vector or simple > uchar[] array), and multiple overloaded buffer constructors perform > the appropriate conversion with the raw data as argument. > > A downside is that the buffer access presupposes that the pointee is > alive. > >> Another alternative might be to input values of the new data type >> with a pair of arrays (one for const char*, another for lengths of >> the buffers). Since this is different from other data types, guess >> we will also need to make a new column type to avoid messing with >> existing functions. > > What is the benefit of introducing a separate array for the length? > I could image that it has a negative impact on performance due to > locality reasons. > >> This will affect how the values are outputted too.. > > What about printing raw data in hex? > >> Anyway, looks like we will need to introduce a new column type one >> way of another. Using std::string might be a little cleaner as far >> as I can imagine right now. What do you think? > > I think std::string forms a possible interface. However, unlike data in > a std::vector, the current ISO C++ standard does not guarantee that a > std::string is allocated in a contiguous chunk of memory, that's why > c_str() exists to create a string copy. Yet in practice, the majority of > STL implementations store string data contiguously [1]. So using > std::string string should not yield any weird side effects and &str[0] > or str.data() both return a NUL-terminated contiguous string. > > Matthias > > [1] > http://herbsutter.wordpress.com/2008/04/07/cringe-not-vectors-are-guaranteed-to-be-contiguous/#comment-483 _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
