Hi John,
When I tried to build the indexes one at a time with FastBit.build_index(), the
index of the category type column is not built, and the metadata of the
category column is lost in -part.txt. Am I doing something wrong? I posted my
code here:
public static void hybridIndexTest() throws FastBitStringWriterException
{
String[] strArr = {"0", "1", "2", "3"};
long[] intArr = {4, 5, 6, 7};
String partName = "/path/to/dir";
FastBit fb = new FastBit(null);
FastBitStringWriter writer = new FastBitStringWriter();
fb.add_longs("long", intArr);
fb.write_buffer(partName);
writer.addCategories(partName, "str", strArr, "UTF-8");
writer.addStringColumnToMetadata(partName, "str", "key");
fb.build_index(partName, "long", "basic");
fb.build_index(partName, "str", "keywords");
}
Cheers,
Bailu
________________________________________
From: K. John Wu [[email protected]]
Sent: Wednesday, May 29, 2013 1:56 PM
To: Bailu Ding
Cc: FastBit Users
Subject: Re: [FastBit-users] Bug in build indexes with FastBit JNI?
Hi, Bailu,
There is a function called build_index that works with one column at a
time which allows you to specific indexing options.
By the way, the column type category only supports one type of index -
this index type can not be changed. Indexing option to this column
type is ignored.
John
On 5/29/13 1:34 PM, Bailu Ding wrote:
> Thanks John! When I change the type of the text columns to category, add the
> indexing type for each column in the -part.txt, and then run
> FastBit.build_indexes(), it works.
>
> Aside from setting indexing types in the -part.txt and leaving the index
> option blank in FastBit.build_indexes(), can I specify the indexing type for
> each column with the JNI?
>
> Cheers,
> Bailu
>
>
> ________________________________________
> From: K. John Wu [[email protected]]
> Sent: Wednesday, May 29, 2013 1:13 PM
> To: FastBit Users
> Cc: Bailu Ding
> Subject: Re: [FastBit-users] Bug in build indexes with FastBit JNI?
>
> Hi, Bailu,
>
> by default FastBit does not index TEXT columns. Furthermore, a TEXT
> column can only support index type 'keywords' and you have to
> explicitly specify the index option in order for FastBit to build a
> keyword index on the column.
>
> If you intent to treat each row as an atomic unit, then it is better
> for you to use the function addCategories instead of addText.
>
> John
>
>
> On 5/29/13 12:59 PM, Bailu Ding wrote:
>> Hi,
>>
>> I am using FastBit JNI to build indexes. I have a problem when I try
>> to build indexes of both integer and text columns. Here is the process
>> to reproduce the problem:
>>
>> = Load an integral column with FastBit.add_longs() and
>> FastBit.write_buffer().
>> = Load a text column with FastBitStringWriter.addText() and
>> FastBitStringWriter.addStringColumnToMetadata()
>> = At this time, the -part.txt is correctly constructed for the two
>> columns.
>> = Then I run FastBit.build_indexes(). Only the intergral column has an
>> index. The -part.txt file now only contains the metadata for the
>> intergral column.
>>
>> Is this the right way to build indexes on a mix of integral and text
>> columns?
>>
>> I paste the code here:
>>
>> import gov.lbl.fastbit.FastBit;
>> import gov.lbl.fastbit.FastBitStringWriter;
>> import gov.lbl.fastbit.FastBitStringWriterException;
>>
>> public class FastBitTest {
>>
>> public static void main(String args[]) throws
>> FastBitStringWriterException
>> {
>> String[] strArr = {"0", "1", "2", "3"};
>> long[] intArr = {4, 5, 6, 7};
>> String partName = "/path/to/the/data/folder";
>> FastBit fb = new FastBit(null);
>> FastBitStringWriter writer = new FastBitStringWriter();
>>
>> fb.add_longs("long", intArr);
>> fb.write_buffer(partName);
>>
>> writer.addText(partName, "str", strArr, "UTF-8");
>> writer.addStringColumnToMetadata(partName, "str", "text");
>>
>> fb.build_indexes(partName, "basic");
>> }
>> }
>>
>>
>> Cheers,
>> Bailu
>>
>>
>>
>> _______________________________________________
>> FastBit-users mailing list
>> [email protected]
>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
