You and me both :) Sent from my iPhone
On Aug 18, 2010, at 12:35 PM, elisa sibarani <[email protected]> wrote: > hi, > > this morning I'm curious about HDF5-Fast Query and try to search if there is > a free trial for that API, and when I look into this thread, it is fit to > what I'm looking for. Hope that someone got the information about this API. > > thanks > > > --- On Sat, 8/14/10, Roy Mendelssohn <[email protected]> wrote: > > From: Roy Mendelssohn <[email protected]> > Subject: Re: [Hdf-forum] Select query options > To: "Rodel Piano" <[email protected]> > Cc: "HDF Users Discussion List" <[email protected]> > Date: Saturday, August 14, 2010, 11:16 AM > > Yes, that was my reaction also. I can find fastbits but not hdf5-fastquery. > If the FastBits API can be used as is to create HDF5-FastQuery it is unclear > to me. > > -Roy > > On Aug 14, 2010, at 7:33 AM, Rodel Piano wrote: > > > The link that you've given is fastbit and not for hdf5- fastquery. I guess > > fastquery wont be out that soon. > > > > Sent from my iPhone > > > > On Aug 14, 2010, at 6:26 PM, "Werner Benger" <[email protected]> wrote: > > > >> https://codeforge.lbl.gov/frs/?group_id=44 > >> > >> should do? > >> > >> > >> Werner > >> > >> > >> On Sat, 14 Aug 2010 07:00:57 +0200, Roy Mendelssohn > >> <[email protected]> wrote: > >> > >>> On the internet there is a reference to HDF5-FastQuery that uses FastBits > >>> within HDF5 files. But finding the actual API has eluded me. Does > >>> anyone know how to actually get this API? > >>> > >>> Thanks, > >>> > >>> -Roy M. > >>> > >>> On Aug 13, 2010, at 9:01 PM, Rodel Piano wrote: > >>> > >>>> Hi, > >>>> > >>>> Thats glad to know. I have explored fastbit, but fastbit requires that > >>>> the file be converted from hdf5 to binary file. With that in mind, would > >>>> it be ideal to just create the file in binary in the first place to > >>>> reduce overhead. And use fastbit itself instead of hdf5 > >>>> > >>>> Sent from my iPhone > >>>> > >>>> On Aug 14, 2010, at 4:44 AM, Quincey Koziol <[email protected]> wrote: > >>>> > >>>>> Hi all, > >>>>> > >>>>> On Aug 13, 2010, at 12:57 PM, Werner Benger wrote: > >>>>> > >>>>>> Hi, > >>>>>> > >>>>>> HDF5 doesn't support searching within datasets. You might want to look > >>>>>> into something like Fastbit: > >>>>>> > >>>>>> http://highscalability.com/blog/2009/5/1/fastbit-an-efficient-compressed-bitmap-index-technology.html > >>>>>> http://crd.lbl.gov/~kewu/fastbit/doc/index.html > >>>>>> > >>>>>> This is an addon-library which builds indices on HDF5 datasets > >>>>>> allowing to select certain regions based on indexed properties. > >>>>> > >>>>> We (The HDF Group) have just had a proposal funded that will integrate > >>>>> FastBit [style] indexing more tightly with HDF5. The work is scheduled > >>>>> for a little while from now (year 2 or 3 of the proposal, I believe), > >>>>> but I just wanted to let people know that this functionality will be > >>>>> coming... > >>>>> > >>>>> Quincey > >>>>> > >>>>>> Werner > >>>>>> > >>>>>> > >>>>>> On Fri, 13 Aug 2010 19:40:12 +0200, Rodel Piano > >>>>>> <[email protected]> wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> But that approach would be slow right? assuming that we are dealing > >>>>>> with the worst possible volume of data. > >>>>>> > >>>>>> From: Francesc Alted <[email protected]> > >>>>>> To: HDF Users Discussion List <[email protected]> > >>>>>> Sent: Fri, August 13, 2010 10:30:56 PM > >>>>>> Subject: Re: [Hdf-forum] Select query options > >>>>>> > >>>>>> 2010/8/13 Rodel Piano <[email protected]> > >>>>>> Thanks for thr reply Francesc. > >>>>>> > >>>>>> My codes are in C. Lets say my records are about 12million. I cant use > >>>>>> hyperslab coz i wont know which region to look for. What is the > >>>>>> expected result of my query is only 10 records out 12 million. > >>>>>> > >>>>>> If you don't know which region you should look, then you should look > >>>>>> into your complete table. Just use hyperslabs for reading your > >>>>>> dataset in small chunks (my suggestion is that these chunks should fit > >>>>>> your CPU cache, for improved performance). > >>>>>> > >>>>>> -- > >>>>>> Francesc Alted > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> ___________________________________________________________________________ > >>>>>> Dr. Werner Benger Visualization Research > >>>>>> Laboratory for Creative Arts and Technology (LCAT) > >>>>>> Center for Computation & Technology at Louisiana State University > >>>>>> (CCT/LSU) > >>>>>> 211 Johnston Hall, Baton Rouge, Louisiana 70803 > >>>>>> Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 > >>>>>> _______________________________________________ > >>>>>> Hdf-forum is for HDF software users discussion. > >>>>>> [email protected] > >>>>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > >>>>> > >>>> _______________________________________________ > >>>> Hdf-forum is for HDF software users discussion. > >>>> [email protected] > >>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > >>> > >>> ********************** > >>> "The contents of this message do not reflect any position of the U.S. > >>> Government or NOAA." > >>> ********************** > >>> Roy Mendelssohn > >>> Supervisory Operations Research Analyst > >>> NOAA/NMFS > >>> Environmental Research Division > >>> Southwest Fisheries Science Center > >>> 1352 Lighthouse Avenue > >>> Pacific Grove, CA 93950-2097 > >>> > >>> e-mail: [email protected] (Note new e-mail address) > >>> voice: (831)-648-9029 > >>> fax: (831)-648-8440 > >>> www: http://www.pfeg.noaa.gov/ > >>> > >>> "Old age and treachery will overcome youth and skill." > >>> "From those who have been given much, much will be expected" > >>> > >>> > >>> _______________________________________________ > >>> Hdf-forum is for HDF software users discussion. > >>> [email protected] > >>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > >>> > >>> > >> > >> > >> -- > >> ___________________________________________________________________________ > >> Dr. Werner Benger Visualization Research > >> Laboratory for Creative Arts and Technology (LCAT) > >> Center for Computation & Technology at Louisiana State University (CCT/LSU) > >> 211 Johnston Hall, Baton Rouge, Louisiana 70803 > >> Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 > >> > >> _______________________________________________ > >> Hdf-forum is for HDF software users discussion. > >> [email protected] > >> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > ********************** > "The contents of this message do not reflect any position of the U.S. > Government or NOAA." > ********************** > Roy Mendelssohn > Supervisory Operations Research Analyst > NOAA/NMFS > Environmental Research Division > Southwest Fisheries Science Center > 1352 Lighthouse Avenue > Pacific Grove, CA 93950-2097 > > e-mail: [email protected] (Note new e-mail address) > voice: (831)-648-9029 > fax: (831)-648-8440 > www: http://www.pfeg.noaa.gov/ > > "Old age and treachery will overcome youth and skill." > "From those who have been given much, much will be expected" > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
