Hi, Roman, Thanks for your interest in FastBit.
Inside FastBit, strings are converted into integers through the ibis::dictionary class. This class uses a hash function from STL to do most of the work for converting strings to integers. In terms of string length, it matters a little bit because every character of a long string has to be processed in order to generate a hash and every character has to process in order to compare two strings. However, the time spent on these operations should be very small and not worth the trouble for you to convert your long strings into shorter ones. Regarding your second question, we will find some time to look into it, most likely next weekend;-) Good luck and feel free to let us know if you have more questions. John On 5/16/15 6:58 AM, Shuangxin Yu wrote: > > Hi John, > > I am a fresh man in this group . I have two questions when I used > fastbit to do my work. > > 1. To generate fastbit index, I follow two steps. First, ardea command > to generate fastbit data. Then ibis command to generate fastbit index. > My question is that whether I can reduce time which costs on > generating fastbit data by decrease string length for the text type > column. > > 2. I used to generate fastbit index by the version 1.3.8 before. And > now , I upgrade my fastbit version to 2.0.2, then generating fastbit > index does not work well for the following parameters（fig1）. It > seems that the parameter “none” does not work. I compared two version > of source code about index.cpp, and located the piece of code which > spec variable compare to “none”,”noindex”. In my opinion the code is > right, I am confused about the result that building all index for columns. > > *$fastbithome/bin/ibis -v -d $indexdir \ > -b 6 \ > -b "a:none" \ > -b "b:none" \ > -b "c:none" \ > -b 'd:keywords delimiters=";"' \ > -b 'e:keywords delimiters=";"' \ > -b 'f:keywords delimiters=";"' \ > -b "*:<binning none/><encoding equality/>" -z* > > fig1 > > Thanks, > Roman > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
