Drew Parsons a écrit le 05/04/2020 à 18:57 : > On 2020-04-05 22:34, Gilles Filippini wrote: >> I suspect a mismatch between mpi and serial build-depdencies: >> >> python3-h5py-serial + libhdf5-dev => OK >> python3-h5py-serial + libhdf5-openmpi-dev => KO >> python3-h5py-mpi + libhdf5-dev => KO >> python3-h5py-mpi + libhdf5-openmpi-dev => KO(*) >> >> (*) because it fails to import h5py submodules when python3-h5py-serial >> is not installed: >> ImportError: cannot import name 'h5f' from 'h5py' (unknown location) >> >> With the 2.10.0-5 layout it seems impossible to build against the 'mpi' >> flavour, while building against 'serial' flavour requires serial flavour >> of libhdf5 -dev package as well. > > > I've set up h5py so that the mpi build is only invoked when actually run > with mpirun. In that sense the two builds are complementary rather than > replacements (there was complaining that module loading was too slow > when I had h5py built only against hdf5-mpi). It seemed to me that's > desirable to allow both serial and mpi builds of h5py to be installable > at the same time, just as libhdf_*.so is, so I didn't want > python3-h5py-serial and python3-h5py-mpi to Replace and Conflicts with > each other. > > Probably this is what's going on. bitshuffle is trying to make an mpi > build, but running the build single processor so h5py accesses hdf5-serial. > > In that case, running the bitshuffle mpi build in mpi might help, as in > "mpirun -n 1 python3 setup.py". Not sure if that would be easy to do, > e.g. if it's as "simple" as > > override_dh_auto_build: > > mpirun -n 1 dh_auto_build > > Another option is to create an environment variable to force h5py to > load the mpi version even when run in a serial environment without > mpirun. Easy enough to set up, though I'm interested to see if "mpirun > -n 1 dh_auto_build" or a variation of that is viable. Maybe > %: > mpirun -n 1 dh $@ --with python3 --buildsystem=pybuild
This, way the test cases run against python3.7 is OK, but it fails
against python3.8 with:
I: pybuild base:217: cd
/build/bitshuffle-z2ZvpN/bitshuffle-0.3.5/.pybuild/cpython3_3.8_bitshuffle/build;
python3.8 -m unittest discover -v
[pinibrem15:43725] OPAL ERROR: Unreachable in file ext3x_client.c at
line 112
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[pinibrem15:43725] Local abort before MPI_INIT completed completed
successfully, but am not able to aggregate error messages, and not able
to guarantee that all other processes were killed!
E: pybuild pybuild:352: test: plugin distutils failed with: exit code=1:
cd
/build/bitshuffle-z2ZvpN/bitshuffle-0.3.5/.pybuild/cpython3_3.8_bitshuffle/build;
python3.8 -m unittest discover -v
dh_auto_test: error: pybuild --test -i python{version} -p "3.7 3.8"
returned exit code 13
But the HDF5 error is no more present with python3.7. So it seems a good
point.
_g.
signature.asc
Description: OpenPGP digital signature
