So close. Exactly in reverse. Something along the lines of DirectChunkRead. I would like to read the chunk directly from file into memory for decompression later.
Thanks. Jebb -- Jebb Stewart NOAA's Earth System Research Laboratory (ESRL) Global Systems Division 325 Broadway, R/GSD6, Boulder, CO 80305 303-497-6724 On Tue, Apr 29, 2014 at 11:14 AM, Dana Robinson <[email protected]>wrote: > Are you looking to do something like this? > > > > > http://www.hdfgroup.org/HDF5/doc/Advanced/DirectChunkWrite/UsingDirectChunkWrite.pdf > > > > Dana > > > > *From:* Hdf-forum [mailto:[email protected]] *On > Behalf Of *Jebb Stewart - NOAA Affiliate > *Sent:* Tuesday, April 29, 2014 12:45 PM > *To:* [email protected]; HDF Users Discussion List > *Subject:* Re: [Hdf-forum] Direct access to compressed block of data? > > > > Dan, > > > > Thanks for the feedback. This isn't quite what I am looking for. While I > can read into a memory file, data is still decompressed when read a > particular hyperslab. I would like to extract the raw bytes of the > compressed hyperslab. My data are 3 dimensional float arrays stored in > compressed (deflate) slabs by vertical level. Right now I extract the > float values for a particular vertical level and compress them before > sending across the wire to a remote location. I would like to directly > read the compressed slab stored within the file and send it, skipping the > extra decompress/compress step. My hyperslab dimensions match my specified > chunk dimensions and in theory the decompression wouldn't be required since > I am not extracting a subset of my chunk dimensions. I was hoping to get > direct access to the deflated hyperslab as a byte array. I don't see a way > to do this with the in memory file. Did I miss something? > > > > Jebb > > > > > -- > Jebb Stewart > NOAA's Earth System Research Laboratory (ESRL) > Global Systems Division > 325 Broadway, R/GSD6, Boulder, CO 80305 > 303-497-6724 > > > > On Tue, Apr 29, 2014 at 7:33 AM, Daniel Kahn <[email protected]> > wrote: > > Jebb, > > Would it not work to H5OCopy your array into a memory file? See > http://www.hdfgroup.org/HDF5/doc1.6/UG/08_TheFile.html, section 8.8 on > the CORE driver. > > --dan > > > > > > On 04/25/14 19:02, Jebb Q Stewart wrote: > > > I am looking for a way to read in a chunk of data from a HDF5 file without > the decompression step. This would be similar to how H5OCopy works when > copying between two hdf5 files, except I want the data to stay resident in > memory. Is it possible to read the compressed bytes of a chunk into a byte > array? > > For additional background, data is stored using the deflate filter. I > define my read dimensions to match my original chunking parameters to read > in one exact chunk at a time. > > Thanks for the help. > > Jebb > > > > _______________________________________________ > > Hdf-forum is for HDF software users discussion. > > [email protected] > > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > > > > -- > > Daniel Kahn > > Science Systems and Applications Inc. > > 301-867-2162 > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
