Hi John,
While isolating the piece of the code, I found one of the problems is that the
query result from text (or category) column is not correct.
It is an equality query, but the result returned is 0 (which shouldn't be the
case). I post the code below:
public static void queryTest() throws FastBitStringWriterException
{
int size = 1024;
Random rand = new Random();
int resCnt = 0;
String[] strArr = new String[size];
long[] longArr = new long[size];
for (int i = 0; i < size; ++i) {
longArr[i] = rand.nextLong() % 1024;
if (longArr[i] < key) {
++resCnt;
}
strArr[i] = "0123456789";
}
String partName = "/path/to/dir";
FastBit fb = new FastBit(null);
FastBitStringWriter writer = new FastBitStringWriter();
fb.add_longs("long", longArr);
fb.write_buffer(partName);
writer.addText(partName, "str", strArr, "UTF-8");
writer.addStringColumnToMetadata(partName, "str", "text");
String where = "str = '" + strArr[0] + "'";
System.out.println(where);
QueryHandle handle = fb.build_query("", partName, where);
System.out.println(fb.get_result_size(handle) + " tuples
selected , expect " + resCnt + " ...");
}
Cheers,
Bailu
________________________________________
From: K. John Wu [[email protected]]
Sent: Wednesday, May 29, 2013 2:17 PM
To: FastBit Users
Cc: Bailu Ding
Subject: Re: [FastBit-users] incorrect query results using JNI for text columns
Hi, Bailu,
Looks like there is an off-by-1 problem somewhere. Would you be able
to share your test code with me? I should be able to find time to
look into the problem in the next few days.
John
On 5/29/13 2:12 PM, Bailu Ding wrote:
> There is no empty text in the data. Indeed each text column is a
> fixed-size string.
>
> Cheers,
> Bailu
>
> ----------------------------------------------------------------------
> *From:* [email protected]
> [[email protected]] on behalf of Enns, Steven
> [[email protected]]
> *Sent:* Wednesday, May 29, 2013 1:59 PM
> *To:* FastBit Users
> *Subject:* Re: [FastBit-users] incorrect query results using JNI for
> text columns
>
> We may have seen something similar where rows with empty values for
> text columns were omitted. Did you try replacing any empty text
> values with 'NULL'?
>
> From: Bailu Ding <[email protected] <mailto:[email protected]>>
> Reply-To: FastBit Users <[email protected]
> <mailto:[email protected]>>
> Date: Wednesday, May 29, 2013 1:56 PM
> To: "[email protected]
> <mailto:[email protected]>" <[email protected]
> <mailto:[email protected]>>
> Subject: [FastBit-users] incorrect query results using JNI for text
> columns
>
> Hi,
>
> After I load the data and issue queries to FastBit.build_query(), I
> often get a warning like the following:
>
> Warning -- text[test.col1]::stringSearchdata file
> "/path/to/file/folder/test/col1" contains 18721 strings, but expected
> 18722
>
> When I load the data into FastBit with FastBitStringWriter.addText(),
> the number of columns is 18722. The metadata of the column in
> -part.txt also says 18722.
>
> I double-check the result from what is returned from MySQL, the result
> is incorrect when I have this kind of warnings.
>
> There is no index built on the text column, the query is equality,
> i.e. where col1 = some_text_string. Have anyone ever had this problem
> before?
>
> Cheers,
> Bailu
>
>
>
> _______________________________________________
> FastBit-users mailing list
> [email protected]
> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
