Dear John: Thanks for your detailed information very much. I think my problem is the uncompressed bitmaps do not fit into the memory. I will try the data set with low cardinality and modify the source. Thanks again.
Reagards, Mian > Dear Mian Lu, > > Thanks for your interest in FastBit work. I have just checked the > source code that implements the option to decompess bitmaps, it is > still there. > > It sounds like you have the indexes already in the data directory. To > force the indexes to be rebuilt, you might want to add -z following -b > option. You shouldn't need that, but it explicitly forces the > existing indexes to be removed. > > Another thing to keep in mind is that each uncompressed index must fit > in the memory. If you have high-cardinality data, this can be a > challenge. FastBit builds the compressed indexes first and then > decompress the bitmaps on request. If it fails to decompress them, it > will keep on using the compressed version -- there should be some > warning message about this sort of thing if the verboseness level is a > positive number. > > If you plan to measure the query processing time using the > uncompressed index, you might find that it is not what you expect. In > the current version of the software, the raw data is assumed to be > present. Therefore, to evaluate any query, there are at least two > options, using the index and using the raw data (which is a vertical > projection only, not the whole relation). The decision on which one > to use is based on which option will access less bytes. If the index > is uncompressed, then it is much more likely that using the index will > read more bytes than using the vertical projection. The result is > that you will not see how the uncompressed index behaves. To modify > this behavior, you need to edit the function > ibis::column::evaluateRange. if you only want to test the performance > of some simple range conditions, the version of this function you want > starts at line 4911 in src/column.cpp. > > Hope this helps. > > John > > > > > On 10/11/2009 8:02 AM, Mian Lu wrote: >> Dear Dr. Wu: >> >> I am a third year Ph.D. student from Hong Kong University of Science and >> Technology. Currently, I am very interested in your FastBit project. >> However, for research purpose, I want to get the equality encoding >> bitmap >> index without any compressions. I have tried the command line in your >> document, like this >> >> examples/ibis -v -d tmp -b "<binning none/><encoding >> equality/><compressing uncompressAll/>" >> >> However, it looks like no effect. The index should be still built with >> compressions (the size is not changed). I would like to ask whether such >> options have been removed, please? I really appreciate if you can still >> provide the bitmap without any compressions. Thanks very much. >> >> Regards, >> Mian Lu >> > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
