Hi, Julian,
Thanks for spotting the problem with buffer size computation in
ikeywords.cpp.
I am not sure I totally sure the issue with item 2, so will need some
time to study the issue. The truth is that I have not touched with
Java API for a while...
The C API and the Java API currently do not support aggregation
operations. It might be high time for us to add this support.
John
On 2/4/13 1:43 PM, Suszko, Julian wrote:
> Hi there,
>
> I've been trying to create a partition, build an index, and query it
> via the provided Java API. I've run into a few issues which may or may
> not be bugs. I'm hoping that somebody can either point out what I'm
> doing wrong or acknowledge that there's some unintended behaviour
> occurring.
>
> *1. * Building keyword indexes for large columns containing strings
> longer than 2048 characters fails, but I think I've tracked down the
> issue.
>
> Consider ikeywords.cpp.411 (/parseTextFile/):
> /buf.resize((sz+2047)/2048); /
>
> I don't think the buffer is resizing properly. Changing the line
> to buf/./resize/(((sz+/2047/)///2048) * 2048) /appears to fix the
> issue./ /
>
> *2.* I don't think the in-memory metadata is being updated correctly.
>
> Consider the following code snippet:
> /// Write out an integer column.
>
> /
> /fb.add_ints("i", iArray);/
> /fb.write_buffer(partDir);/
> /
> /
> /// Write out a string column./
> /fbStringWriter.addText(partDir, "s", sArray, "UTF-8");/
> /fbStringWriter.addStringColumnToMetadata(partDir, "s", "text");/
>
> /// Build the indexes./
> /fb.build_index(partDir, "i", "basic");/
> /fb.build_index(partDir, "s", "keywords");/
>
> When attempting to build the keywords index this produces an error
> indicating that column "s" cannot be found (message:
> /fastbit_build_index -- can not find column "s"in data directory
> "/home/suszko/fbIndex/part0"/); however, column "i" can be found and
> the index is built without issue. If I comment out all the integer
> column code, the keyword index for "s" builds without issue.
>
> Now consider capi.cpp.259 (/fastbit_build_index/):
> /t = _capi_tlist/->find(dir/);/
> /
> /
> Now add the following code:
> /// Force the partition to update its data./
> /t->updateData();/
> /
> /
> Now both indexes build without issue; for some reason, the in-memory
> metadata isn't consistent with what's been written to –part.txt.
>
> *3.* SQL aggregate functions do not appear to work via the Java API.
>
> Consider this snippet:
> /QueryHandle query = fb.build_query("SUM(i)", partDir, "i > 0");/
> /int[] imps = fb.get_qualified_ints(query, "i");/
>
> This just returns a list of all qualified integers in column "i"; not
> the sum. Are aggregate functions not support via the Java API, am I
> using the API incorrectly, or is this unintended behaviour?
>
> Thank you,
> Julian Suszko
> A9.com SDE Intern
> University of Waterloo – Bachelor's of Computer Science candidate (3rd
> year)
>
>
> _______________________________________________
> FastBit-users mailing list
> [email protected]
> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
>
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
