Hi, Andrew,
Currently, there is no logic in the code to recognize that the two
parts of the data are sorted already. Therefore, the orderby call
will use a generic sorting procedure.
We are contemplating revamping the groupby and orderby operations, we
will keep this in mind when we redesign things.
In the mean time, hopefully, the sorting cost is not too much and
response time of the additional orderby is tolerable..
John
On 8/31/11 1:41 PM, Olson, Andrew wrote:
> Hi,
> I have a data set that is split into two partitions (A and B). Each
> partition has columns position and score. The -part.txt file has a metaTags
> entry corresponding to the partition (A or B). Usually I query only one
> partition at a time, but I would like to query across both partitions by
> pointing to the parent directory.
>
> tbl = ibis::table::create(datadir);
> res = tbl->select("position,score",qcnd);
>
> later on I retreive the positions and scores as follows:
>
> uint64_t ierr = res->getColumnAsUInts("position", positions);
> uint64_t ierr = res->getColumnAsUInts("score", scores);
>
> The code that follows expects the positions to be sorted (they are sorted in
> each partition - and -part.txt has "sorted = true").
> I suspect I need to do this before populating positions[] and scores[]:
> sorted_results = res->orderby("position");
>
> So my question is, since each partition is already sorted by position, is
> there logic in place that uses this info to do the orderby more quickly? On
> a related note, if I add the orderby() to my code, will it slow down when
> querying only one partition (already sorted by position)?
>
> Thanks,
> Andrew
> _______________________________________________
> FastBit-users mailing list
> [email protected]
> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
