New submission from Rafael Laboissière <raf...@laboissiere.net>: * Preamble:
The problem reported hereafter possibly comes from the h5py module, which is not part of Python per se. This problem has been already reported to the h5py developers: https://github.com/h5py/h5py/issues/1467 and also against the Debian package python3-h5py: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=946986 I apologize if this issue report is considered abusive. Please, feel free to close it, if it is the case. * The problem: The combination of "import h5py", "subprocess.Popen", and "mpirun" is yielding a weird result. Consider these two scripts: ############################################################# ### File name: bugtest-without-h5py.py import subprocess simulationProc = subprocess.Popen("mpirun", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = simulationProc.communicate() returnCode = simulationProc.wait() print("stdout = ", stdout) print("stderr = ", stderr) print("return code = ", returnCode) ############################################################# ############################################################# ### File name: bugtest-with-h5py.py import subprocess import h5py simulationProc = subprocess.Popen("mpirun", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (stdout, stderr) = simulationProc.communicate() returnCode = simulationProc.wait() print("stdout = ", stdout) print("stderr = ", stderr) print("return code = ", returnCode) ############################################################# The only difference between them is the line containing "import h5py" in the second. Here is the result when running the first script: $ python3 bugtest-without-h5py.py stdout = b'' stderr = b'--------------------------------------------------------------------------\nmpirun could not find anything to do.\n\nIt is possible that you forgot to specify how many processes to run\nvia the "-np" argument.\n--------------------------------------------------------------------------\n' return code = 1 and here is the result for the second script: $ python3 bugtest-with-h5py.py stdout = b'' stderr = b'' return code = 1 It seems that, when h5py is imported, the mpirun command is not even launched by subprocess.Popen, even though there is noting in this call that is related to h5py. When "mpirun" is replaced by other commands (e.g. "date"), then the output is identical for both scripts, as it should be. As I wrote in the preamble, this is possibly a problem with h5py. I am reporting it here because the developers of the subprocess module may have an idea about the origin of the problem or give me a hint on how to debug the it. The tests were done on a Debian bullseye system with the following versions: h5py 2.10.0 (compiled with MPI support) Python 3.7.6 ---------- components: Library (Lib) messages: 360638 nosy: Rafael Laboissière priority: normal severity: normal status: open title: h5py not playing nicely with subprocess and mpirun type: behavior versions: Python 3.7 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue39445> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
