Just after I sent my mail I found the information at

    https://support.hdfgroup.org/hdf5-quest.html#tsafe

So in short, the HDF5 library needs to be built with --thread-safe in
order to be thread-safe, and will then effectively serialize all
library calls. This happens to be OK in my case, since I'm not
primarily after performance, I'm just using threading to keep the UI
responsive (would be nice if truly parallel calls were possible
though...).

I then looked at the Ubuntu package, and saw that it is built with
--thread-safe, which explains why it works there.

Looking at the Arch package I'm using [1], it seems it only passes
--with-pthreads=/usr/lib/, not --thread-safe. Correct me if I'm wrong,
but this means the library is not thread-safe? If I look at
configure.ac, I get the feeling that --with-pthreads has no effect
unless --thread-safe is also passed? If so, it's probably simply a
mistake by the Arch package maintainer and I'll ask if he can add
--thread-safe as well.

Elvis

[1] 
https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/hdf5-cpp-fortran


2016-09-22 18:42 GMT+02:00 Elvis Stansvik <elvis.stans...@orexplore.com>:
> Hi all,
>
> I'm using the C++ API to read HDF5 files from separate threads (no
> writing). None of my threads read the same file, but they do execute
> simultaneously. The reason I'm using threading is not to speed things
> up or get better throughput, but simply to not block the UI (it's Qt
> application) while reading. So this is not about "Parallel HDF5" or
> anything, just simply using the serial library "from scratch" from
> multiple threads.
>
> This has been working fine when testing on Ubuntu 16.04 (our target
> OS), which has HDF5 1.8.16.
>
> I recently tested on my personal Arch Linux machine though, which has
> HDF5 1.10.0, and got this segmentation fault:
>
> (gdb) bt
> #0  0x00007ffff67c57d9 in H5SL_search () from /usr/lib/libhdf5.so.100
> #1  0x00007ffff678dd19 in H5P_copy_plist () from /usr/lib/libhdf5.so.100
> #2  0x00007ffff66a7fc0 in H5F_new () from /usr/lib/libhdf5.so.100
> #3  0x00007ffff66a8f55 in H5F_open () from /usr/lib/libhdf5.so.100
> #4  0x00007ffff66a155d in H5Fopen () from /usr/lib/libhdf5.so.100
> #5  0x00007ffff6b79546 in H5::H5File::p_get_file(char const*, unsigned
> int, H5::FileCreatPropList const&, H5::FileAccPropList const&) () from
> /usr/lib/libhdf5_cpp.so.100
> #6  0x00007ffff6b79750 in H5::H5File::H5File(char const*, unsigned
> int, H5::FileCreatPropList const&, H5::FileAccPropList const&) () from
> /usr/lib/libhdf5_cpp.so.100
> #7  0x000000000041f00e in HDF5ImageReader::RequestInformation
> (this=0x7fffbc002de0, request=0x7fffbc010da0, inputVector=0x0,
> outputVector=0x7fffbc0039d0) at
> /home/estan/Projekt/orexplore/dev/src/insight/src/model/HDF5ImageReader.cpp:91
> #8  0x00007fffee8200d0 in vtkExecutive::CallAlgorithm(vtkInformation*,
> int, vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #9  0x00007fffee837fa9 in
> vtkStreamingDemandDrivenPipeline::ExecuteInformation(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #10 0x00007fffee81ce05 in
> vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #11 0x00007fffee835c55 in
> vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #12 0x00007fffee816e1a in
> vtkCompositeDataPipeline::ForwardUpstream(vtkInformation*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #13 0x00007fffee81ccb5 in
> vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #14 0x00007fffee835c55 in
> vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #15 0x00007fffee816e1a in
> vtkCompositeDataPipeline::ForwardUpstream(vtkInformation*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #16 0x00007fffee81ccb5 in
> vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #17 0x00007fffee835c55 in
> vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #18 0x00007fffee816e1a in
> vtkCompositeDataPipeline::ForwardUpstream(vtkInformation*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #19 0x00007fffee81ccb5 in
> vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #20 0x00007fffee835c55 in
> vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*,
> vtkInformationVector**, vtkInformationVector*) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #21 0x00007fffee836482 in
> vtkStreamingDemandDrivenPipeline::Update(int) () from
> /usr/lib/libvtkCommonExecutionModel.so.1
> #22 0x00007ffff1289a76 in vtkAbstractVolumeMapper::GetBounds() () from
> /usr/lib/libvtkRenderingCore.so.1
> #23 0x00007ffff13459f9 in vtkVolume::GetBounds() () from
> /usr/lib/libvtkRenderingCore.so.1
> #24 0x000000000043f235 in createVolume (image=..., from=0,
> to=2.7803999378532183, opacityFunction=..., colorFunction=...) at
> /home/estan/Projekt/orexplore/dev/src/insight/src/view/Pipeline.cpp:123
> #25 0x00000000004295c4 in CreateVolume::operator() (this=0x829248,
> image=...) at 
> /home/estan/Projekt/orexplore/dev/src/insight/src/view/Pipeline.h:45
> #26 0x000000000042bc7a in
> QtConcurrent::MappedEachKernel<QList<Image>::const_iterator,
> CreateVolume>::runIteration (this=0x829210, it=...,
> result=0x7fffbc002da8) at
> /usr/include/qt/QtConcurrent/qtconcurrentmapkernel.h:176
> #27 0x000000000042bd5d in
> QtConcurrent::MappedEachKernel<QList<Image>::const_iterator,
> CreateVolume>::runIterations (this=0x829210,
> sequenceBeginIterator=..., begin=1, end=2, results=0x7fffbc002da8) at
> /usr/include/qt/QtConcurrent/qtconcurrentmapkernel.h:186
> #28 0x000000000042c4e1 in
> QtConcurrent::IterateKernel<QList<Image>::const_iterator,
> vtkSmartPointer<vtkVolume> >::forThreadFunction (this=0x829210) at
> /usr/include/qt/QtConcurrent/qtconcurrentiteratekernel.h:256
> #29 0x000000000042bedc in
> QtConcurrent::IterateKernel<QList<Image>::const_iterator,
> vtkSmartPointer<vtkVolume> >::threadFunction (this=0x829210) at
> /usr/include/qt/QtConcurrent/qtconcurrentiteratekernel.h:218
> #30 0x00007ffff7bd5cfd in QtConcurrent::ThreadEngineBase::run() ()
> from /usr/lib/libQt5Concurrent.so.5
> #31 0x00007ffff489a01f in ?? () from /usr/lib/libQt5Core.so.5
> #32 0x00007ffff489dd78 in ?? () from /usr/lib/libQt5Core.so.5
> #33 0x00007fffeb3f5454 in start_thread () from /usr/lib/libpthread.so.0
> #34 0x00007fffec5f07df in clone () from /usr/lib/libc.so.6
> (gdb)
>
> Before I start digging into what is happening here I'd just like to
> ask: Do I have to do something special when using the HDF5 library
> from two different threads? I'm not reading the same files or
> anything, it's simply two completely separate usages of the library in
> threads that run in parallel.
>
> Does the library have any global structures or something that must be
> initialized before spawning any threads that use it?
>
> The reason I'm a little worried is that perhaps I've just been lucky
> when running under Ubuntu / HDF5 1.8.16.
>
> My usage in each thread basically looks like:
>
> 1) Create a H5::H5File
> 2) Open a dataset using file.openDataset
> 3) Get the dataspace for the dataset and select a hyperslab
> 4) Create a memory dataspace
> 5) Perform a single read(..) operation from the dataset dataspace to
> the memory dataspace
>
> And it's always different files that the threads work with.
>
> Is there some step 0 I'm missing?
>
> Thanks in advance for any advice.
>
> Elvis

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Reply via email to