Hi, Bailu,
Would you mind take a look at FastBit SVN revision 638 and see if it
fixes your problems?
Thanks.
John
On 5/30/13 9:15 AM, Bailu Ding wrote:
> Hi John,
>
> While isolating the piece of the code, I found one of the problems is that
> the query result from text (or category) column is not correct.
> It is an equality query, but the result returned is 0 (which shouldn't be the
> case). I post the code below:
>
> public static void queryTest() throws FastBitStringWriterException
> {
> int size = 1024;
> Random rand = new Random();
> int resCnt = 0;
>
> String[] strArr = new String[size];
> long[] longArr = new long[size];
>
> for (int i = 0; i < size; ++i) {
> longArr[i] = rand.nextLong() % 1024;
> if (longArr[i] < key) {
> ++resCnt;
> }
> strArr[i] = "0123456789";
> }
>
> String partName = "/path/to/dir";
>
> FastBit fb = new FastBit(null);
> FastBitStringWriter writer = new FastBitStringWriter();
>
> fb.add_longs("long", longArr);
> fb.write_buffer(partName);
>
> writer.addText(partName, "str", strArr, "UTF-8");
> writer.addStringColumnToMetadata(partName, "str", "text");
>
> String where = "str = '" + strArr[0] + "'";
> System.out.println(where);
> QueryHandle handle = fb.build_query("", partName, where);
> System.out.println(fb.get_result_size(handle) + " tuples
> selected , expect " + resCnt + " ...");
>
> }
>
> Cheers,
> Bailu
>
>
> ________________________________________
> From: K. John Wu [[email protected]]
> Sent: Wednesday, May 29, 2013 2:17 PM
> To: FastBit Users
> Cc: Bailu Ding
> Subject: Re: [FastBit-users] incorrect query results using JNI for text
> columns
>
> Hi, Bailu,
>
> Looks like there is an off-by-1 problem somewhere. Would you be able
> to share your test code with me? I should be able to find time to
> look into the problem in the next few days.
>
> John
>
>
>
> On 5/29/13 2:12 PM, Bailu Ding wrote:
>> There is no empty text in the data. Indeed each text column is a
>> fixed-size string.
>>
>> Cheers,
>> Bailu
>>
>> ----------------------------------------------------------------------
>> *From:* [email protected]
>> [[email protected]] on behalf of Enns, Steven
>> [[email protected]]
>> *Sent:* Wednesday, May 29, 2013 1:59 PM
>> *To:* FastBit Users
>> *Subject:* Re: [FastBit-users] incorrect query results using JNI for
>> text columns
>>
>> We may have seen something similar where rows with empty values for
>> text columns were omitted. Did you try replacing any empty text
>> values with 'NULL'?
>>
>> From: Bailu Ding <[email protected] <mailto:[email protected]>>
>> Reply-To: FastBit Users <[email protected]
>> <mailto:[email protected]>>
>> Date: Wednesday, May 29, 2013 1:56 PM
>> To: "[email protected]
>> <mailto:[email protected]>" <[email protected]
>> <mailto:[email protected]>>
>> Subject: [FastBit-users] incorrect query results using JNI for text
>> columns
>>
>> Hi,
>>
>> After I load the data and issue queries to FastBit.build_query(), I
>> often get a warning like the following:
>>
>> Warning -- text[test.col1]::stringSearchdata file
>> "/path/to/file/folder/test/col1" contains 18721 strings, but expected
>> 18722
>>
>> When I load the data into FastBit with FastBitStringWriter.addText(),
>> the number of columns is 18722. The metadata of the column in
>> -part.txt also says 18722.
>>
>> I double-check the result from what is returned from MySQL, the result
>> is incorrect when I have this kind of warnings.
>>
>> There is no index built on the text column, the query is equality,
>> i.e. where col1 = some_text_string. Have anyone ever had this problem
>> before?
>>
>> Cheers,
>> Bailu
>>
>>
>>
>> _______________________________________________
>> FastBit-users mailing list
>> [email protected]
>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
