Hi Andrew,
Excellent, I'm glad it's something that'll help h5py. BTW, we're also
planning a new API routine to perform in-memory scatter/gather operations
between two memory buffers. (But you might not need that if you don't have to
work around this issue :-)
Quincey
On Sep 27, 2012, at 5:16 PM, Andrew Collette wrote:
> Hi Jason & Quincey,
>
>>> The part that is currently problematic is that the buffer passed to the
>>> conversion routine does not have a vector of hvl_ts but rather a vector of
>>> some sort of internal hdf5 types.
>
> I'm the original person who asked about this (I'm the main author of
> h5py). We have to convert from HDF5 vlen strings to an opaque object
> (a Python string). Since h5py has to deal with this behavior in
> released versions of HDF5 we implemented a workaround, which I briefly
> described in that thread:
>
> 1. Read from the dataset selection into a contiguous conversion buffer
> with exactly the same type as the dataset
> 2. Call H5Tconvert to go from the dataset type to your destination
> type. The correct data is supplied to the custom converter when you
> do this (for some reason)
> 3. Scatter the converted points from the buffer to your memory destination.
>
> This is kind of annoying because the gather/scatter process is
> time-intensive, and getting everything correct w.r.t backing buffers,
> etc. is a real headache. You can see our implementation here (in
> Cython):
>
> https://code.google.com/p/h5py/source/browse/h5py/_proxy.pyx (starts
> at line 102)
>
> I agree it would be great if this were fixed (in HDF5 1.10?). Along
> with identifier recycling, this is one of the biggest sources of pain
> in the h5py codebase.
>
> Parenthetically, are you aware of h5labview
> (http://sourceforge.net/p/h5labview)? Maybe you and Martijn could
> join forces.
>
> Andrew
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
