Hi, Dominique, Thanks for pointing this one out. Obviously, we have not tested this part of the code thoroughly.
Regarding the efficiency of using query or table::select function, there really isn't much of a difference. The table::select function a newer addition, and you are likely to bump into more little bugs like you've encountered. At this point, they both share the bundle class to perform the underlying group-by operations. The behavior of table::select is more closely aligned with SQL standard than the query class. John On 1/16/12 12:45 PM, Dominique Prunier wrote: > Hmm, so far, chaning this single line seems to do the trick. I > successfully created a simple projection applying the following patch: > > > > diff --git a/src/bord.cpp b/src/bord.cpp > > index 1f2ffa5..a9c34ba 100755 > > --- a/src/bord.cpp > > +++ b/src/bord.cpp > > @@ -1859,7 +1859,7 @@ int ibis::bord::backup(const char* dir, const > char* tname, > > } > > > > ibis::bitvector msk0, msk1; > > - msk0.set(0, nEvents); > > + msk0.set(1, nEvents); > > md << "# meta data for data partition " << tname > > << " written by bord::backup on " << stamp << "\n\n" > > << "BEGIN HEADER\nName = " << tname << "\nDescription = " > > > > I’m really not sure of what i’m doing, but as far as i understood, > msk0 is the bitvector of the values we are supposed to write, so it > seems to make sense... > > John, feel free to comment/ammend this. > > > > Thanks, > > > > *From:*[email protected] > [mailto:[email protected]] *On Behalf Of *Dominique > Prunier > *Sent:* Monday, January 16, 2012 3:06 PM > *To:* FastBit Users > *Subject:* Re: [FastBit-users] Convert a bundle into a partition > > > > Hmm, maybe not... It seems that it always save NULL values, i’ll keep > investigating and keep you posted... > > > > Thanks, > > > > *From:*[email protected] > [mailto:[email protected]] *On Behalf Of *Dominique > Prunier > *Sent:* Monday, January 16, 2012 2:37 PM > *To:* FastBit Users > *Subject:* Re: [FastBit-users] Convert a bundle into a partition > > > > Hey, > > > > I found a pretty easy way to create an on-disk projection. > > > > ibis::table* tbl = ibis::table::*create*("<partition directory>"); > > ibis::table* sel = tbl->*select*("<cols>", "<where>"); > > ibis::table* dev = sel->*groupby*("<cols> "); > > > > dev->*backup*("<projection directory>", "<projection name>", > "<projection description>"); > > * * > > It seems to do pretty much what you’d expect. > > > > John, please tell us if this is a decently efficient way of doing it. > More specifically, i have no idea about the memory overhead of the > intermediate seland the devtables (does it only store a hit vector and > reference the tbltable ? i see that groupby uses a bundle, but what > about the dev table itself ?). > > > > Thanks, > > > > *From:*Jon Strabala [mailto:[email protected]] > *Sent:* Monday, January 16, 2012 1:35 PM > *To:* K. John Wu > *Cc:* Dominique Prunier > *Subject:* Re: [FastBit-users] Convert a bundle into a partition > > > > John, > > > > This might be a very usefull enhancement, e.g. imagine a system where > fasbit could a created directed permenate materialized views > (manged explicitly for say a DSS) or a alternatively a cache > of materialized views from the most recent queries. > > > > I can image several use cases for such functionality. > > > > Best Regards, > > > > Jon Strabala > > > > On Mon, Jan 16, 2012 at 7:59 AM, K. John Wu <[email protected] > <mailto:[email protected]>> wrote: > > Hi, Dominique, > > There is an in-memory data partition known as ibis::bord which can > take a select clause and a data partition as input argument. This > might be the closest to what you want. A little bit more work is > needed to get to where you need it.. I am out of office during the > day, might have time to look into this in the night.. > > John > > > > On 1/16/12 6:25 AM, Dominique Prunier wrote: >> Hi, >> >> >> >> I was wondering if there were a way to convert a a bundle into a >> partition. My goal here would be to create some projection of my >> partition grouped by a subset of columns. For ex, my main partition >> would be: >> >> >> >> Col1 Col2 Col3 >> >> 1 A B >> >> 2 C D >> >> 3 A B >> >> 4 A B >> >> >> >> And i’d like the following projection (Col2 and Col3), which is the >> result of “SELECT Col2,Col3 GROUP BY Col2,Col3”: >> >> >> >> Col2 Col3 >> >> A B >> >> C D >> >> >> >> Thanks, >> >> >> >> */Dominique Prunier/**//* >> >> APG Lead Developper >> >> Logo-W4N-100dpi >> >> 4388, rue Saint-Denis >> >> Bureau 309 >> >> Montreal (Quebec) H2J 2L1 >> >> Tel. +1 514-842-6767 x310 <tel:%2B1%20514-842-6767%20%20x310> >> >> Fax +1 514-842-3989 <tel:%2B1%20514-842-3989> >> >> [email protected] > <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>> >> >> www.watch4net.com <http://www.watch4net.com> > <http://www.watch4net.com/> >> >> / / >> >> /This message is for the designated recipient only and may contain >> privileged, proprietary, or otherwise private information. If you have >> received it in error, please notify the sender immediately and delete >> the original. Any other use of this electronic mail by you is > prohibited. >> >> //Ce message est pour le récipiendaire désigné seulement et peut >> contenir des informations privilégiées, propriétaires ou autrement >> privées. Si vous l'avez reçu par erreur, S.V.P. avisez l'expéditeur >> immédiatement et effacez l'original. Toute autre utilisation de ce >> courrier électronique par vous est prohibée./// >> >> >> >> >> >> _______________________________________________ >> FastBit-users mailing list >> [email protected] <mailto:[email protected]> >> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > _______________________________________________ > FastBit-users mailing list > [email protected] <mailto:[email protected]> > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users > > > > > > _______________________________________________ > FastBit-users mailing list > [email protected] > https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users _______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
