Hey,
I found a pretty easy way to create an on-disk projection.
ibis::table* tbl = ibis::table::create("<partition directory>");
ibis::table* sel = tbl->select("<cols>", "<where>");
ibis::table* dev = sel->groupby("<cols> ");
dev->backup("<projection directory>", "<projection name>", "<projection
description>");
It seems to do pretty much what you’d expect.
John, please tell us if this is a decently efficient way of doing it. More
specifically, i have no idea about the memory overhead of the intermediate sel
and the dev tables (does it only store a hit vector and reference the tbl table
? i see that groupby uses a bundle, but what about the dev table itself ?).
Thanks,
From: Jon Strabala [mailto:[email protected]]
Sent: Monday, January 16, 2012 1:35 PM
To: K. John Wu
Cc: Dominique Prunier
Subject: Re: [FastBit-users] Convert a bundle into a partition
John,
This might be a very usefull enhancement, e.g. imagine a system where fasbit
could a created directed permenate materialized views (manged explicitly for
say a DSS) or a alternatively a cache of materialized views from the most
recent queries.
I can image several use cases for such functionality.
Best Regards,
Jon Strabala
On Mon, Jan 16, 2012 at 7:59 AM, K. John Wu <[email protected]<mailto:[email protected]>>
wrote:
Hi, Dominique,
There is an in-memory data partition known as ibis::bord which can
take a select clause and a data partition as input argument. This
might be the closest to what you want. A little bit more work is
needed to get to where you need it.. I am out of office during the
day, might have time to look into this in the night..
John
On 1/16/12 6:25 AM, Dominique Prunier wrote:
> Hi,
>
>
>
> I was wondering if there were a way to convert a a bundle into a
> partition. My goal here would be to create some projection of my
> partition grouped by a subset of columns. For ex, my main partition
> would be:
>
>
>
> Col1 Col2 Col3
>
> 1 A B
>
> 2 C D
>
> 3 A B
>
> 4 A B
>
>
>
> And i’d like the following projection (Col2 and Col3), which is the
> result of “SELECT Col2,Col3 GROUP BY Col2,Col3”:
>
>
>
> Col2 Col3
>
> A B
>
> C D
>
>
>
> Thanks,
>
>
>
> */Dominique Prunier/**//*
>
> APG Lead Developper
>
> Logo-W4N-100dpi
>
> 4388, rue Saint-Denis
>
> Bureau 309
>
> Montreal (Quebec) H2J 2L1
>
> Tel. +1 514-842-6767 x310<tel:%2B1%20514-842-6767%20%20x310>
>
> Fax +1 514-842-3989<tel:%2B1%20514-842-3989>
>
> [email protected]<mailto:[email protected]>
> <mailto:[email protected]<mailto:[email protected]>>
>
> www.watch4net.com<http://www.watch4net.com> <http://www.watch4net.com/>
>
> / /
>
> /This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise private information. If you have
> received it in error, please notify the sender immediately and delete
> the original. Any other use of this electronic mail by you is prohibited.
>
> //Ce message est pour le récipiendaire désigné seulement et peut
> contenir des informations privilégiées, propriétaires ou autrement
> privées. Si vous l'avez reçu par erreur, S.V.P. avisez l'expéditeur
> immédiatement et effacez l'original. Toute autre utilisation de ce
> courrier électronique par vous est prohibée.///
>
>
>
>
>
> _______________________________________________
> FastBit-users mailing list
> [email protected]<mailto:[email protected]>
> https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
_______________________________________________
FastBit-users mailing list
[email protected]<mailto:[email protected]>
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
_______________________________________________
FastBit-users mailing list
[email protected]
https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
