The link that you've given is fastbit and not for hdf5- fastquery. I guess fastquery wont be out that soon.
Sent from my iPhone On Aug 14, 2010, at 6:26 PM, "Werner Benger" <[email protected]> wrote: > https://codeforge.lbl.gov/frs/?group_id=44 > > should do? > > > Werner > > > On Sat, 14 Aug 2010 07:00:57 +0200, Roy Mendelssohn > <[email protected]> wrote: > >> On the internet there is a reference to HDF5-FastQuery that uses FastBits >> within HDF5 files. But finding the actual API has eluded me. Does anyone >> know how to actually get this API? >> >> Thanks, >> >> -Roy M. >> >> On Aug 13, 2010, at 9:01 PM, Rodel Piano wrote: >> >>> Hi, >>> >>> Thats glad to know. I have explored fastbit, but fastbit requires that the >>> file be converted from hdf5 to binary file. With that in mind, would it be >>> ideal to just create the file in binary in the first place to reduce >>> overhead. And use fastbit itself instead of hdf5 >>> >>> Sent from my iPhone >>> >>> On Aug 14, 2010, at 4:44 AM, Quincey Koziol <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> On Aug 13, 2010, at 12:57 PM, Werner Benger wrote: >>>> >>>>> Hi, >>>>> >>>>> HDF5 doesn't support searching within datasets. You might want to look >>>>> into something like Fastbit: >>>>> >>>>> http://highscalability.com/blog/2009/5/1/fastbit-an-efficient-compressed-bitmap-index-technology.html >>>>> http://crd.lbl.gov/~kewu/fastbit/doc/index.html >>>>> >>>>> This is an addon-library which builds indices on HDF5 datasets allowing >>>>> to select certain regions based on indexed properties. >>>> >>>> We (The HDF Group) have just had a proposal funded that will integrate >>>> FastBit [style] indexing more tightly with HDF5. The work is scheduled >>>> for a little while from now (year 2 or 3 of the proposal, I believe), but >>>> I just wanted to let people know that this functionality will be coming... >>>> >>>> Quincey >>>> >>>>> Werner >>>>> >>>>> >>>>> On Fri, 13 Aug 2010 19:40:12 +0200, Rodel Piano <[email protected]> >>>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> But that approach would be slow right? assuming that we are dealing >>>>> with the worst possible volume of data. >>>>> >>>>> From: Francesc Alted <[email protected]> >>>>> To: HDF Users Discussion List <[email protected]> >>>>> Sent: Fri, August 13, 2010 10:30:56 PM >>>>> Subject: Re: [Hdf-forum] Select query options >>>>> >>>>> 2010/8/13 Rodel Piano <[email protected]> >>>>> Thanks for thr reply Francesc. >>>>> >>>>> My codes are in C. Lets say my records are about 12million. I cant use >>>>> hyperslab coz i wont know which region to look for. What is the expected >>>>> result of my query is only 10 records out 12 million. >>>>> >>>>> If you don't know which region you should look, then you should look into >>>>> your complete table. Just use hyperslabs for reading your dataset in >>>>> small chunks (my suggestion is that these chunks should fit your CPU >>>>> cache, for improved performance). >>>>> >>>>> -- >>>>> Francesc Alted >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> ___________________________________________________________________________ >>>>> Dr. Werner Benger Visualization Research >>>>> Laboratory for Creative Arts and Technology (LCAT) >>>>> Center for Computation & Technology at Louisiana State University >>>>> (CCT/LSU) >>>>> 211 Johnston Hall, Baton Rouge, Louisiana 70803 >>>>> Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 >>>>> _______________________________________________ >>>>> Hdf-forum is for HDF software users discussion. >>>>> [email protected] >>>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >>>> >>> _______________________________________________ >>> Hdf-forum is for HDF software users discussion. >>> [email protected] >>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >> >> ********************** >> "The contents of this message do not reflect any position of the U.S. >> Government or NOAA." >> ********************** >> Roy Mendelssohn >> Supervisory Operations Research Analyst >> NOAA/NMFS >> Environmental Research Division >> Southwest Fisheries Science Center >> 1352 Lighthouse Avenue >> Pacific Grove, CA 93950-2097 >> >> e-mail: [email protected] (Note new e-mail address) >> voice: (831)-648-9029 >> fax: (831)-648-8440 >> www: http://www.pfeg.noaa.gov/ >> >> "Old age and treachery will overcome youth and skill." >> "From those who have been given much, much will be expected" >> >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >> >> > > > -- > ___________________________________________________________________________ > Dr. Werner Benger Visualization Research > Laboratory for Creative Arts and Technology (LCAT) > Center for Computation & Technology at Louisiana State University (CCT/LSU) > 211 Johnston Hall, Baton Rouge, Louisiana 70803 > Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
