Hi, I'm sorry I was not able to explain my problem clearly. However, I found a close enough solution, so let me explain that and perhaps clarify it a bit.
So the idea was to save an index to the data based on a query, and reuse that index later. I want to save the result, because the filtering is done with pretty expensive "LIKE" queryies on text columns (and previously outside Fastbit). So I'm saving the bitvectors retrieved from an evaluated query-object. Then, I want to run aggregate queries over these results, which can be done mainly using the table interface. However, the normal interface does not allow use of bitvectors to limit the query to only those rows defined in the bitvector, because the "generic" table could be, for instance, using many partitions (I suppose). I found out that I can populate a table myself using directly the bord-class and its append-method, and bring in just the records needed. Then I can call groupby for those results. I did not find a straightforward way to reuse these bitvectors with the query interface, so at the moment I'm simply doing AND between the two bitvector results, which is fine when the following queries are not using text columns. One option could be to use the getRIDs(bitvector) method to convert the bitvector to RIDSet and then use that with setRIDs() method, byt I have not looked into that yet. Thanks! Patrik On Mon, Dec 10, 2012 at 5:26 PM, K. John Wu <[email protected]> wrote: > Hi, Patrik, > > If you would like to write a data table to screen or a file in CSV > format, use the function ibis::table::dump. If you plan to write a > data table out in binary format that can be used for further queries, > then use the function ibis::table::backup. > > John > > > On 12/10/12 4:29 AM, Patrik Nisen wrote: >> Hi, >> >> yes, it would. Would it then be possible to do that and save the >> intermediary results into a file? >> >> Thank you. >> >> Patrik >> >> >> On 12/08/12 at 12:56pm, K. John Wu wrote: >>> Hi, Patrik, >>> >>> Would it be possible for to issue two queries with the same where >>> clause, but different select clauses? >>> >>> John >>> >>> >>> On 12/7/12 4:13 AM, Patrik Nisen wrote: >>>> Hi, >>>> >>>> and thank you for your great work! >>>> >>>> I am currently looking into performing operations for pre-filtered >>>> sets of rows, and I would need some help to understand if this is >>>> possible at the moment to do with fastbit, or advice to implement it. >>>> >>>> I have a dataset saved into a single data partition and my goal is to >>>> perform filtering with varying conditions, save these results and use >>>> them as starting sets for later queries. If I have understood >>>> correctly, this is at the moment possible by retrieving the RIDSet from >>>> an evaluated query, saving that, and setting it (query::setRIDs) to the >>>> next query. However, I would like to use aggregate functions with the >>>> following queries, but I did not find a way to do similar things with >>>> the table interface. >>>> >>>> So my question is: how could I perform the described aggregation for a >>>> pre-filtered set of rows? In addition, as I'm only having one partition, >>>> I would prefer to save the filtering results as bitvectors >>>> (query::getHitVector) and reuse them later as masks due to their smaller >>>> size. There's a protected function query::doEvaluate having this >>>> functionality, and perhaps that could be opened. Would this make any >>>> sense? >>>> >>>> Thanks for your help! >>>> >>>> >>>> Patrik Nisen >>>> _______________________________________________ >>>> FastBit-users mailing list >>>> [email protected] >>>> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users >>>> _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
