Hi there,
I've been trying to create a partition, build an index, and query it via the
provided Java API. I've run into a few issues which may or may not be bugs. I'm
hoping that somebody can either point out what I'm doing wrong or acknowledge
that there's some unintended behaviour occurring.
1. Building keyword indexes for large columns containing strings longer than
2048 characters fails, but I think I've tracked down the issue.
Consider ikeywords.cpp.411 (parseTextFile):
buf.resize((sz+2047)/2048);
I don't think the buffer is resizing properly. Changing the line to
buf.resize(((sz+2047)/2048) * 2048) appears to fix the issue.
2. I don't think the in-memory metadata is being updated correctly.
Consider the following code snippet:
// Write out an integer column.
fb.add_ints("i", iArray);
fb.write_buffer(partDir);
// Write out a string column.
fbStringWriter.addText(partDir, "s", sArray, "UTF-8");
fbStringWriter.addStringColumnToMetadata(partDir, "s", "text");
// Build the indexes.
fb.build_index(partDir, "i", "basic");
fb.build_index(partDir, "s", "keywords");
When attempting to build the keywords index this produces an error indicating
that column "s" cannot be found (message: fastbit_build_index -- can not find
column "s"in data directory "/home/suszko/fbIndex/part0"); however, column "i"
can be found and the index is built without issue. If I comment out all the
integer column code, the keyword index for "s" builds without issue.
Now consider capi.cpp.259 (fastbit_build_index):
t = _capi_tlist->find(dir);
Now add the following code:
// Force the partition to update its data.
t->updateData();
Now both indexes build without issue; for some reason, the in-memory metadata
isn't consistent with what's been written to –part.txt.
3. SQL aggregate functions do not appear to work via the Java API.
Consider this snippet:
QueryHandle query = fb.build_query("SUM(i)", partDir, "i > 0");
int[] imps = fb.get_qualified_ints(query, "i");
This just returns a list of all qualified integers in column "i"; not the sum.
Are aggregate functions not support via the Java API, am I using the API
incorrectly, or is this unintended behaviour?
Thank you,
Julian Suszko
A9.com SDE Intern
University of Waterloo – Bachelor's of Computer Science candidate (3rd year)
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
