Hi John, The warnings are gone and the results are correct now. Thanks for your work!
Cheers, Bailu ________________________________________ From: K. John Wu [[email protected]] Sent: Sunday, June 02, 2013 7:58 PM To: Bailu Ding Cc: FastBit Users Subject: Re: [FastBit-users] incorrect query results using JNI for text columns Hi, Bailu, Would you mind take a look at FastBit SVN revision 638 and see if it fixes your problems? Thanks. John On 5/30/13 9:15 AM, Bailu Ding wrote: > Hi John, > > While isolating the piece of the code, I found one of the problems is that > the query result from text (or category) column is not correct. > It is an equality query, but the result returned is 0 (which shouldn't be the > case). I post the code below: > > public static void queryTest() throws FastBitStringWriterException > { > int size = 1024; > Random rand = new Random(); > int resCnt = 0; > > String[] strArr = new String[size]; > long[] longArr = new long[size]; > > for (int i = 0; i < size; ++i) { > longArr[i] = rand.nextLong() % 1024; > if (longArr[i] < key) { > ++resCnt; > } > strArr[i] = "0123456789"; > } > > String partName = "/path/to/dir"; > > FastBit fb = new FastBit(null); > FastBitStringWriter writer = new FastBitStringWriter(); > > fb.add_longs("long", longArr); > fb.write_buffer(partName); > > writer.addText(partName, "str", strArr, "UTF-8"); > writer.addStringColumnToMetadata(partName, "str", "text"); > > String where = "str = '" + strArr[0] + "'"; > System.out.println(where); > QueryHandle handle = fb.build_query("", partName, where); > System.out.println(fb.get_result_size(handle) + " tuples > selected , expect " + resCnt + " ..."); > > } > > Cheers, > Bailu > > > ________________________________________ > From: K. John Wu [[email protected]] > Sent: Wednesday, May 29, 2013 2:17 PM > To: FastBit Users > Cc: Bailu Ding > Subject: Re: [FastBit-users] incorrect query results using JNI for text > columns > > Hi, Bailu, > > Looks like there is an off-by-1 problem somewhere. Would you be able > to share your test code with me? I should be able to find time to > look into the problem in the next few days. > > John > > > > On 5/29/13 2:12 PM, Bailu Ding wrote: >> There is no empty text in the data. Indeed each text column is a >> fixed-size string. >> >> Cheers, >> Bailu >> >> ---------------------------------------------------------------------- >> *From:* [email protected] >> [[email protected]] on behalf of Enns, Steven >> [[email protected]] >> *Sent:* Wednesday, May 29, 2013 1:59 PM >> *To:* FastBit Users >> *Subject:* Re: [FastBit-users] incorrect query results using JNI for >> text columns >> >> We may have seen something similar where rows with empty values for >> text columns were omitted. Did you try replacing any empty text >> values with 'NULL'? >> >> From: Bailu Ding <[email protected] <mailto:[email protected]>> >> Reply-To: FastBit Users <[email protected] >> <mailto:[email protected]>> >> Date: Wednesday, May 29, 2013 1:56 PM >> To: "[email protected] >> <mailto:[email protected]>" <[email protected] >> <mailto:[email protected]>> >> Subject: [FastBit-users] incorrect query results using JNI for text >> columns >> >> Hi, >> >> After I load the data and issue queries to FastBit.build_query(), I >> often get a warning like the following: >> >> Warning -- text[test.col1]::stringSearchdata file >> "/path/to/file/folder/test/col1" contains 18721 strings, but expected >> 18722 >> >> When I load the data into FastBit with FastBitStringWriter.addText(), >> the number of columns is 18722. The metadata of the column in >> -part.txt also says 18722. >> >> I double-check the result from what is returned from MySQL, the result >> is incorrect when I have this kind of warnings. >> >> There is no index built on the text column, the query is equality, >> i.e. where col1 = some_text_string. Have anyone ever had this problem >> before? >> >> Cheers, >> Bailu >> >> >> >> _______________________________________________ >> FastBit-users mailing list >> [email protected] >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >> _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
