Your message dated Fri, 27 Mar 2020 19:00:11 +0000 with message-id <[email protected]> and subject line Bug#944617: fixed in h5py 2.10.0-3 has caused the Debian Bug report #944617, regarding python3-h5py import performance severely degraded in 2.10.0 release (due to OpenMPI?) to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact [email protected] immediately.) -- 944617: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944617 Debian Bug Tracking System Contact [email protected] with problems
--- Begin Message ---Package: python3-h5py Version: 2.10.0-2 Severity: normal This was mentioned in a different report (#944192) but I want to give it it's own report because I think it's serious enough. I'm finding multiple problems with the latest 2.10.0 release in unstable, which I think can all be traced to this new release being compiled against the OpenMPI library. I'm no longer sure it's good to compile against this library by default, since it's degrading the basic performance too much. Other than the stderr output on import described in #944192 (and seen below), the import performance is now x7 slower. This is a pretty big drag for any application using this library: servo:~ 0$ python3 -c 'import h5py; print(h5py.__version__)' -------------------------------------------------------------------------- [[4525,1],0]: A high-performance Open MPI point-to-point messaging module was unable to find any relevant network interfaces: Module: OpenFabrics (openib) Host: servo Another transport will be used instead, although this may result in lower performance. NOTE: You can disable this warning by setting the MCA parameter btl_base_warn_component_unused to 0. -------------------------------------------------------------------------- 2.10.0 servo:~ 0$ multitime -qq -n 10 python3 -c 'import h5py' ===> multitime results 1: -qq python3 -c "import h5py" Mean Std.Dev. Min Median Max real 0.973 0.027 0.932 0.967 1.022 user 0.200 0.017 0.159 0.201 0.221 sys 0.068 0.014 0.034 0.069 0.093 servo:~ 0$ servo:~ 0$ python3 -c 'import h5py; print(h5py.__version__)' 2.8.0 servo:~ 0$ multitime -qq -n 10 python3 -c 'import h5py' ===> multitime results 1: -qq python3 -c "import h5py" Mean Std.Dev. Min Median Max real 0.146 0.012 0.118 0.151 0.156 user 0.119 0.010 0.103 0.121 0.134 sys 0.027 0.008 0.016 0.027 0.044 servo:~ 0$ Most users of h5py are not using the OpenMPI functionality, so I don't see how it's a win to degrade performance for most users in order to increase performance for the few. When I mentioned this issue upstream, their recommendation is that debian should *not* be building against OpenMPI by default: https://github.com/h5py/h5py/issues/1433#issuecomment-552640391 I agree with this recommendation. I think there should be separate packages providing OpenMPI support (e.g. python3-h5py-openmpi). Thanks. jamie. -- System Information: Debian Release: bullseye/sid APT prefers testing APT policy: (600, 'testing'), (500, 'stable'), (200, 'unstable'), (101, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 5.2.0-3-amd64 (SMP w/8 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages python3-h5py depends on: ii libc6 2.29-3 ii libhdf5-openmpi-103 1.10.4+repack-10 ii python3 3.7.5-1 ii python3-mpi4py 3.0.2-13 ii python3-numpy [python3-numpy-abi9] 1:1.16.5-1 ii python3-six 1.12.0-2 python3-h5py recommends no packages. Versions of packages python3-h5py suggests: pn python-h5py-doc <none> -- no debconf information
--- End Message ---
--- Begin Message ---Source: h5py Source-Version: 2.10.0-3 Done: Drew Parsons <[email protected]> We believe that the bug you reported is fixed in the latest version of h5py, which is due to be installed in the Debian FTP archive. A summary of the changes between this version and the previous one is attached. Thank you for reporting the bug, which will now be closed. If you have further comments please address them to [email protected], and the maintainer will reopen the bug report if appropriate. Debian distribution maintenance software pp. Drew Parsons <[email protected]> (supplier of updated h5py package) (This message was generated automatically at their request; if you believe that there is a problem with it please contact the archive administrators by mailing [email protected]) -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Format: 1.8 Date: Thu, 05 Mar 2020 19:22:27 +0800 Source: h5py Binary: python-h5py-doc python3-h5py python3-h5py-mpi python3-h5py-mpi-dbg python3-h5py-serial python3-h5py-serial-dbg Architecture: source all amd64 Version: 2.10.0-3 Distribution: unstable Urgency: medium Maintainer: Debian Science Maintainers <[email protected]> Changed-By: Drew Parsons <[email protected]> Description: python-h5py-doc - documentation for h5py python3-h5py - general-purpose Python interface to hdf5 python3-h5py-mpi - general-purpose Python interface to hdf5 (Python 3) python3-h5py-mpi-dbg - debug extensions for h5py (Python 3) python3-h5py-serial - general-purpose Python interface to hdf5 (Python 3) python3-h5py-serial-dbg - debug extensions for h5py (Python 3) Closes: 944617 944769 946986 Changes: h5py (2.10.0-3) unstable; urgency=medium . * Team upload. . [ Debian Janitor ] * Use secure URI in Homepage field. . [ Drew Parsons ] * provide separate serial and MPI builds in python3-h5py-serial and python3-h5py-mpi. python3-h5py-serial provides a h5py.py loader file to only import h5py-mpi for multiprocess jobs, otherwise import h5py-serial. python3-h5py is a dummy package depending on python3-h5py-serial or python3-h5py-mpi. - provides generic h5py/__init__.py which imports h5py_serial or h5py_mpi as required - needs debian patches: - hdf5_pkgconfig.patch allows pkg-config entry for HDF5 to be specified (e.g. hdf5-serial) - build_flavour.patch allows build flavour (_serial or _mpi) to be specified for module path - stop_circular_dep.patch stops circular dependency in _objects - drop_deprecated_dtype_6a77b91.patch, remove_deprecated_highlevel_module_2f41c78.patch, file_default_read_5e71c49.patch for upstream commits 6a77b91, 2f41c78 and 5e71c49 and drop_deprecation_tests.patch removes long-deprecated dtypes, highlevel.py and tests/test_deprecation.py and use of H5pyDeprecationWarning in tests (dropped upstream) - tests_as_local_build.patch launches tests relative to the given h5py build (i.e. don't hardcode "h5py") - Closes: #944769, #944617, #946986. * Standards-Version: 4.5.0 * Build-Depends: mpi-default-dev (need to know which mpi since there is currently no hdf5-mpi.pc to invoke, only specific mpi versions) Checksums-Sha1: 862bf844c0fc5344e1c7ac85000625a4a57ede23 2764 h5py_2.10.0-3.dsc ffaf2bd0d6692df4ca833cb89a950b919014d346 18160 h5py_2.10.0-3.debian.tar.xz 5169f8b1d00963c950fd9a4c6a72b3d6e63e64ad 12329 h5py_2.10.0-3_amd64.buildinfo 54788192048d8b630a2e2a806e8e0d660494dbbc 188112 python-h5py-doc_2.10.0-3_all.deb 5c6ead607b9966f9f72bacc4f4429b7b3dca3858 14578756 python3-h5py-mpi-dbg_2.10.0-3_amd64.deb 9c708247396c2c26b4fe84145f29b3a949542b2c 1390592 python3-h5py-mpi_2.10.0-3_amd64.deb e5b5c79436fcc3c7deafab02e5474346e9e88dbd 14457992 python3-h5py-serial-dbg_2.10.0-3_amd64.deb 1a4db82cab5e4ebb61955439024a34c3e3fb57ea 1365896 python3-h5py-serial_2.10.0-3_amd64.deb 582f6c6b6eb8a169d27d7ac5f5f239ff241f8f46 9676 python3-h5py_2.10.0-3_amd64.deb Checksums-Sha256: a3c440a9ac09b61ef72a7a7b3eef1db0efec7e71cf53231a96b1251eb2f7520c 2764 h5py_2.10.0-3.dsc 34991bc03a7473c7831421bd022604a8a84e5f746be674b3759a132a18f6bf89 18160 h5py_2.10.0-3.debian.tar.xz 18b0de7e6a9a245483541d6da5f3e321a07da6f4f5e102e01a0e31822f3b8c02 12329 h5py_2.10.0-3_amd64.buildinfo c7bf9302937c9d1a792d72a86d92811f805b3b59fb2c1144e841d4fa4a02bc38 188112 python-h5py-doc_2.10.0-3_all.deb 1299cebcfc5cab13fa9e91e0565cd128f3317e248250d8038d2fa26082f0c814 14578756 python3-h5py-mpi-dbg_2.10.0-3_amd64.deb ed549ab1c8ea2f0028c88b394044cc8125d9c7123a6bd188dcafe93268b9522b 1390592 python3-h5py-mpi_2.10.0-3_amd64.deb 64f803d8578cfbc39253f7a4904ba9c8af24e1764ccf0d23f03a8f748f92ef99 14457992 python3-h5py-serial-dbg_2.10.0-3_amd64.deb 60c37468188a8f93df79aaf2442964fbc50bb689042818d466a8368fbe98aa0a 1365896 python3-h5py-serial_2.10.0-3_amd64.deb d56e264d327180f77ec7574c2f786eaeaa1f0c0601ae3d9c709761412f3198dc 9676 python3-h5py_2.10.0-3_amd64.deb Files: a52f09b8bd6c4b1cbfd6b683c995e04f 2764 python optional h5py_2.10.0-3.dsc 267956979ad2731d5562cc55ab3842f9 18160 python optional h5py_2.10.0-3.debian.tar.xz da6ef17626f6e5dd59d79605d0d60497 12329 python optional h5py_2.10.0-3_amd64.buildinfo 0517f8561ba5bda1f1d1b05347f5c091 188112 doc optional python-h5py-doc_2.10.0-3_all.deb 40452cd0c21d03b462de3f12cdf09cf7 14578756 debug optional python3-h5py-mpi-dbg_2.10.0-3_amd64.deb 375ae5eb59f79457ce5b6794bf03509a 1390592 python optional python3-h5py-mpi_2.10.0-3_amd64.deb 6203e61d31e3b245f96c867b0ba64811 14457992 debug optional python3-h5py-serial-dbg_2.10.0-3_amd64.deb cddba92ac21e123ed95ce82852c2748f 1365896 python optional python3-h5py-serial_2.10.0-3_amd64.deb 4e52b0834b8ee6cf4b8b204bca2dc3c2 9676 python optional python3-h5py_2.10.0-3_amd64.deb -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEI8mpPlhYGekSbQo2Vz7x5L1aAfoFAl5g9AwACgkQVz7x5L1a AfpYxRAAgMVRLXReUOyPS3WEbkUaSUuOTvt6T/ylXh1Y7LIOBcTGXuZV3+e7Lmex 4rIugb1dYgRgs1pLbc0UOIsH0n88lfYojF4iJDen2Mvne9D75Q3WnxEHJoIhiHFM DNlu507XSrWn2u84dcr4ChffuRjyVPKzqGjvpVqkNfx3eL206fD/VEpzQxK+c5t1 Hy1+59w6PsIz3behyH/oAjjhqyIJjHEQ65o0x8B03oWOP6XmnIQsCx6Pm2GCEUUh pDfAidwS5Anlcs/5yWnzY0ogMNnxgTf5448BqG17hgWxzF2xHuFhE8FCip6FKCyX zYjuzckI68ROhlMputLPGgAZN263k2JiliaZkf9/3uS3BsCA5GZLZzEALoKlCXo1 g8IGNMuIO6kBM81yjssWMbSu0YNWwAIUNmlYT6ldgN2w+RYb/0/Ef3f5lT1iPJFl 1vHluRvt4H6oIHJFLDY5zbW3+if/I5Mq7pktPXptxUM5QQgOI5WMTQZ/mIbjxv4U hnKyBslggRztPJ5TdSeWYFk8mKvpiKZkvcpUIKzXnv95zhU7aRzUuyWwEKYkjAau GI5/yFyID7Sko1hEBuFP6SGXZ9FrImxxIVx8s4PMnJAJk/U+jj2ahl33l1qzCiaB 0n4CfyhiwGDR5jBjk92NySk/I2yDbtRaU5KoSAepZINDp+jH0Cs= =jAk/ -----END PGP SIGNATURE-----
--- End Message ---
-- debian-science-maintainers mailing list [email protected] https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-science-maintainers
