hi, this morning I'm curious about HDF5-Fast Query and try to search if there is a free trial for that API, and when I look into this thread, it is fit to what I'm looking for. Hope that someone got the information about this API.
thanks --- On Sat, 8/14/10, Roy Mendelssohn <[email protected]> wrote: From: Roy Mendelssohn <[email protected]> Subject: Re: [Hdf-forum] Select query options To: "Rodel Piano" <[email protected]> Cc: "HDF Users Discussion List" <[email protected]> Date: Saturday, August 14, 2010, 11:16 AM Yes, that was my reaction also. I can find fastbits but not hdf5-fastquery. If the FastBits API can be used as is to create HDF5-FastQuery it is unclear to me. -Roy On Aug 14, 2010, at 7:33 AM, Rodel Piano wrote: > The link that you've given is fastbit and not for hdf5- fastquery. I guess > fastquery wont be out that soon. > > Sent from my iPhone > > On Aug 14, 2010, at 6:26 PM, "Werner Benger" <[email protected]> wrote: > >> https://codeforge.lbl.gov/frs/?group_id=44 >> >> should do? >> >> >> Werner >> >> >> On Sat, 14 Aug 2010 07:00:57 +0200, Roy Mendelssohn >> <[email protected]> wrote: >> >>> On the internet there is a reference to HDF5-FastQuery that uses FastBits >>> within HDF5 files. But finding the actual API has eluded me. Does anyone >>> know how to actually get this API? >>> >>> Thanks, >>> >>> -Roy M. >>> >>> On Aug 13, 2010, at 9:01 PM, Rodel Piano wrote: >>> >>>> Hi, >>>> >>>> Thats glad to know. I have explored fastbit, but fastbit requires that the >>>> file be converted from hdf5 to binary file. With that in mind, would it be >>>> ideal to just create the file in binary in the first place to reduce >>>> overhead. And use fastbit itself instead of hdf5 >>>> >>>> Sent from my iPhone >>>> >>>> On Aug 14, 2010, at 4:44 AM, Quincey Koziol <[email protected]> wrote: >>>> >>>>> Hi all, >>>>> >>>>> On Aug 13, 2010, at 12:57 PM, Werner Benger wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> HDF5 doesn't support searching within datasets. You might want to look >>>>>> into something like Fastbit: >>>>>> >>>>>> http://highscalability.com/blog/2009/5/1/fastbit-an-efficient-compressed-bitmap-index-technology.html >>>>>> http://crd.lbl.gov/~kewu/fastbit/doc/index.html >>>>>> >>>>>> This is an addon-library which builds indices on HDF5 datasets allowing >>>>>> to select certain regions based on indexed properties. >>>>> >>>>> We (The HDF Group) have just had a proposal funded that will integrate >>>>>FastBit [style] indexing more tightly with HDF5. The work is scheduled >>>>>for a little while from now (year 2 or 3 of the proposal, I believe), but >>>>>I just wanted to let people know that this functionality will be coming... >>>>> >>>>> Quincey >>>>> >>>>>> Werner >>>>>> >>>>>> >>>>>> On Fri, 13 Aug 2010 19:40:12 +0200, Rodel Piano <[email protected]> >>>>>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> But that approach would be slow right? assuming that we are dealing >>>>>>with the worst possible volume of data. >>>>>> >>>>>> From: Francesc Alted <[email protected]> >>>>>> To: HDF Users Discussion List <[email protected]> >>>>>> Sent: Fri, August 13, 2010 10:30:56 PM >>>>>> Subject: Re: [Hdf-forum] Select query options >>>>>> >>>>>> 2010/8/13 Rodel Piano <[email protected]> >>>>>> Thanks for thr reply Francesc. >>>>>> >>>>>> My codes are in C. Lets say my records are about 12million. I cant use >>>>>> hyperslab coz i wont know which region to look for. What is the expected >>>>>> result of my query is only 10 records out 12 million. >>>>>> >>>>>> If you don't know which region you should look, then you should look >>>>>> into your complete table. Just use hyperslabs for reading your dataset >>>>>> in small chunks (my suggestion is that these chunks should fit your CPU >>>>>> cache, for improved performance). >>>>>> >>>>>> -- >>>>>> Francesc Alted >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> ___________________________________________________________________________ >>>>>> Dr. Werner Benger Visualization Research >>>>>> Laboratory for Creative Arts and Technology (LCAT) >>>>>> Center for Computation & Technology at Louisiana State University >>>>>> (CCT/LSU) >>>>>> 211 Johnston Hall, Baton Rouge, Louisiana 70803 >>>>>> Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 >>>>>> _______________________________________________ >>>>>> Hdf-forum is for HDF software users discussion. >>>>>> [email protected] >>>>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >>>>> >>>> _______________________________________________ >>>> Hdf-forum is for HDF software users discussion. >>>> [email protected] >>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >>> >>> ********************** >>> "The contents of this message do not reflect any position of the U.S. >>> Government or NOAA." >>> ********************** >>> Roy Mendelssohn >>> Supervisory Operations Research Analyst >>> NOAA/NMFS >>> Environmental Research Division >>> Southwest Fisheries Science Center >>> 1352 Lighthouse Avenue >>> Pacific Grove, CA 93950-2097 >>> >>> e-mail: [email protected] (Note new e-mail address) >>> voice: (831)-648-9029 >>> fax: (831)-648-8440 >>> www: http://www.pfeg.noaa.gov/ >>> >>> "Old age and treachery will overcome youth and skill." >>> "From those who have been given much, much will be expected" >>> >>> >>> _______________________________________________ >>> Hdf-forum is for HDF software users discussion. >>> [email protected] >>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org >>> >>> >> >> >> -- >> ___________________________________________________________________________ >> Dr. Werner Benger Visualization Research >> Laboratory for Creative Arts and Technology (LCAT) >> Center for Computation & Technology at Louisiana State University (CCT/LSU) >> 211 Johnston Hall, Baton Rouge, Louisiana 70803 >> Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org ********************** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ********************** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 e-mail: [email protected] (Note new e-mail address) voice: (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
