Hi He,
On Nov 13, 2012, at 6:17 PM, [email protected] wrote:
> Hi Quincey
> The problem is the image is not allocated by me, it is from another
> library, for some hardware reason, the image data always add some padding
> data, and so it can be processed faster, for example, the opengl has 1 or ,2,
> 4 bytes alignment, and the intel ipp library has its own malloc fuction, in
> those image class/structure, there always is a step member which is the
> length in bytes to show the scanline length. For most case, we can assume it
> will align perfectly, but there may have some special case, the padding bits
> is not integer multiply element size. so I want to know if hdf5 api directly
> support the image scanline length, as most image processing library support
> it. But It seems I have to use a buffer to strip out it myself.
> HDF5 have been used a lot in remote sensing and satellite image processing
> application. the tiff format should have this kind of problem. Hope in the
> future it will support those steps.
> Thanks for answer my problem.
Hmm, I think you may have misunderstood my previous response - you can
make multiple H5Dwrite calls to perform this I/O, by setting the memory buffer
pointer to H5Dwrite to different locations within the awkwardly structured
buffer you receive from the other library, and selecting a small number of
elements to write each time. It'll just take a larger number of H5Dwrite calls
than otherwise... :-)
We'll definitely keep the use case in mind, in the future.
Quincey
> He Yang
>
>
>
> On Tue, Nov 13, 2012 at 5:26 PM, Quincey Koziol <[email protected]> wrote:
>
> On Nov 13, 2012, at 5:08 PM, [email protected] wrote:
>
>> Hi Quincey
>> I have tried this, the problem is the padding part is only partial of the
>> element size, so the stride can only be set to be interger, which means
>> if i have scan line like 00000000, 11111111, 00 here the element size is
>> 8bit, but the padding bits is 2bit, if i select hyperslab, the stride can
>> be only the interger, which means the padding bits at least 8bit. So it
>> will cause problem.
>
>
> *blah* What an awkward data layout :-) I suppose you could create the
> packed, matrix-like dataset in the file and then write small pieces of your
> buffer in memory to regions of the dataset in the file, with multiple I/O
> calls. Not the best performance possibly, but if you use chunked dataset
> stored for the dataset in the file, the chunk caching in the HDF5 library
> should mitigate that performance issue significantly.
>
> Quincey
>
>> Thanks
>> He Yang
>>
>>
>>
>> On Tue, Nov 13, 2012 at 4:45 PM, Quincey Koziol <[email protected]> wrote:
>> Hi He,
>> You can build up this sort of selection with the H5Sselect_hyperslab()
>> routine by using the H5S_SELECT_OR operator (or another convenient operator).
>>
>> Quincey
>>
>>
>> On Nov 13, 2012, at 1:06 PM, [email protected] wrote:
>>
>>> Hi
>>> The problem is the hyperslab's strid is meaused by the number of pixels,
>>> not bytes, so I i have a 8 16bit pixels, and a 4 padding bits,
>>> the stride will not work. as if the stride is 1 it will be 16bit not 4bits.
>>>
>>> He Yang
>>>
>>>
>>>
>>> On Tue, Nov 13, 2012 at 12:24 PM, Quincey Koziol <[email protected]>
>>> wrote:
>>>
>>> On Nov 12, 2012, at 8:52 AM, [email protected] wrote:
>>>
>>> > Hi All
>>> > I am trying to save image data to the dataset, the image data is
>>> > allocated by other library, so the image's scanline length is not equal
>>> > to the number
>>> > of pixels, and will have the padding bytes. I want save it into hdf5 as
>>> > a matrix, which don't have those padding bytes, as the image can be
>>> > float, unsigned short, and come from other data, the scanline length(/
>>> > step ) between each row may not be the multiples of the pixels. for
>>> > example, the
>>> > image has 9 unsigned short pixels, and the sacn line length is defined by
>>> > allocate 9*(16/2)+4=76 bytes, the stride in hdf5 can only be the interger
>>> > times the elements length. so it can not work. And I don't want use a
>>> > temporary buffer to change the memory layout. I also want all other hdf5
>>> > application can directly read the image out as matrix, so I don't want
>>> > use raw data format. Is there a way which can directly use hdf5's api
>>> > and it will automatically strip the padding bytes for image data.
>>>
>>> You should be able to use hyperslab or point selections to achieve
>>> what you want.
>>>
>>> Quincey
>>>
>>>
>>>
>>> _______________________________________________
>>> Hdf-forum is for HDF software users discussion.
>>> [email protected]
>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>>
>>> _______________________________________________
>>> Hdf-forum is for HDF software users discussion.
>>> [email protected]
>>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [email protected]
>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [email protected]
>> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
