[issue10635] Calling subprocess.Popen with preexec_fn=signal.pause blocks forever
New submission from joseph.h.garvin joseph.h.gar...@gmail.com: The following code will cause the interpreter to hang: import subprocess import signal subprocess.Popen(/bin/echo, preexec_fn=signal.pause) Replace /bin/echo with any valid program on your box, it's just the simplest Linux example. It's expected for signal.pause to block, but the documentation for Popen says that preexec_fn will be run in the /child/ process, so it shouldn't cause the interpreter running this code to block. I'm going to guess Popen is using signals for some reason internally and signal.pause interferes with it. It's probably worth making sure that signals related functions work with preexec_fn in general. -- components: Library (Lib) messages: 123444 nosy: joseph.h.garvin priority: normal severity: normal status: open title: Calling subprocess.Popen with preexec_fn=signal.pause blocks forever type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10635 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10636] subprocess module has race condition with SIGCHLD handlers
New submission from joseph.h.garvin joseph.h.gar...@gmail.com: The following code will result in a traceback 99% of the time, though it may take two runs (sometimes the first run won't trigger it, I think due to the changing in timing from genrating the .pyc file). It spawns an instance of /bin/echo, chosen because it's a very quick to finish program. Any program that executes faster than subprocess.Popen can return will work as a substitute though: import signal import subprocess to_launch = None def sig_chld_handler(signum, frame): global to_launch # Crashes here. # 'NoneType' object has no attribute 'poll' to_launch.poll() print to_launch.returncode signal.signal(signal.SIGCHLD, sig_chld_handler) to_launch = subprocess.Popen(/bin/echo) And the traceback: Traceback (most recent call last): File /tmp/sigchld.py, line 15, in module to_launch = subprocess.Popen(/bin/echo) File /usr/lib/python2.6/subprocess.py, line 623, in __init__ errread, errwrite) File /usr/lib/python2.6/subprocess.py, line 1130, in _execute_child data = _eintr_retry_call(os.read, errpipe_read, 1048576) File /usr/lib/python2.6/subprocess.py, line 455, in _eintr_retry_call return func(*args) File /tmp/sigchld.py, line 9, in sig_chld_handler to_launch.poll() AttributeError: 'NoneType' object has no attribute 'poll' I believe the problem is that the process completes before Popen can return, which means the assignment of to_launch hasn't happened yet, so it's not defined when we get into sig_chld_handler. I tried to work around this issue by setting preexec_fn to signal.pause and sending the child process a signal after the assignment, but then ran into another bug: http://bugs.python.org/issue10635 If when it caught SIGCHLD python pushed an event onto its internal event loop to execute the handler, I think that would make sure it's deferred until after the assignment. There might be other consequences of that, but I'm not familiar with the interpreter internals. Alternatively it could be fixed with an API change -- let Popen return an object before it actually launches the process, and have a separate start() method. -- components: Library (Lib) messages: 123445 nosy: joseph.h.garvin priority: normal severity: normal status: open title: subprocess module has race condition with SIGCHLD handlers versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10636] subprocess module has race condition with SIGCHLD handlers
joseph.h.garvin joseph.h.gar...@gmail.com added the comment: Sorry I wasn't trying to make a request, just suggesting one potential 'fix' (I agree that it isn't really though) to make things more intutive. Unless the app is delayed from launching until after the assignment finishes though I think a workaround is required. When it's deferred until after the assignment you at least have the ability to get it stored into a global. When it's not deferred I think you're required to something like disable SIGCHLD like you say, spawn the child, enable, and then manually poll each of the subprocess popen objects you've opened before to see if they died while the signal was down. I realize now this isn't really a bug. Principle of least surprise is probably too much to hope for with signals anyway. Feel free to close unless you want to leave it open as a bug to get sigblock/sigsetmask in. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10636 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8125] shutil.copytree behavior is inconsistent with copyfile
New submission from joseph.h.garvin joseph.h.gar...@gmail.com: shutil.copyfile's behavior is to replace the dst file if it already exists. shutil.copytree requires that the destination not already exist, and throws an OSError if it does. I see 3 problems with this behavior: 1. It's inconsistent with copyfile 2. It's inconsistent with 'cp', which is what users would be using if they were writing shell script. Given shutil's namesake I assume it's supposed to help make python a viable shell script replacement. 3. It makes it difficult to use copytree with tempfile.mkdtemp(). If I want to make a temporary directory and copy a folder into it, I need to copytree to a nonexist subfolder then move all the files down a level or resort to other unpythonic hacks. In my project I copy unit test resources to a temp folder because they're manipulated throughout the test and I want to keep the originals. I imagine that this is a common use case. That said, if anyone depended on shutil.copytree failing to test whether or not a folder exists, changing this would break compatibility, so if it can't be changed, maybe a shutil.mergetree would be appropriate. -- components: Library (Lib) messages: 100943 nosy: joseph.h.garvin severity: normal status: open title: shutil.copytree behavior is inconsistent with copyfile versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8125 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8125] shutil.copytree behavior is inconsistent with copyfile
Changes by joseph.h.garvin joseph.h.gar...@gmail.com: -- type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8125 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8125] shutil.copytree behavior is inconsistent with copyfile
joseph.h.garvin joseph.h.gar...@gmail.com added the comment: Sorry I wasn't terribly clear explaining the problem with mkdtemp. Say you create your temporary folder, /tmp/foo. Now you have a regular folder bar laid out like so: bar/ blarg1.ext blarg2.ext subdir/ blarg3.ext blarg4.ext I'd like this to be the result of copying: /tmp /foo blarg1.ext blarg2.ext subdir/ blarg3.ext blarg4.ext Basically, I'd like to copy bar/ to /tmp/foo such that the paths of the files relative to bar/ are the same as the paths of the copied files relative to /tmp/foo. AFAICT there's no easy way to do this with copytree's current behavior. Because tempfile.mkdtemp() creates /tmp/foo to start, you can't use copytree to do this. But now I realize copy tree wants to copy the source directory as a directory, so you always get /tmp/foo/bar. What I wanted was copy(recursive_glob(src, *), dst). That would give you the effect you normally get from a utility like 'cp'. It doesn't mean the semantics have to be exactly the same. Actually, many Python users are under Windows where semantics will be different. I believe 'copy' has the same cp-like semantics I desire under Windows, but it's been quite some time since I used it. So, not a bug, but sticks out to me as missing. Might make sense someday to have a keyword arg to copytree that would make it behave this way. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8125 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5228] multiprocessing not compatible with functools.partial
joseph.h.garvin joseph.h.gar...@gmail.com added the comment: I think this bug still exists in Python 2.6.4, and I haven't tested 2.6.5, but since 2.6.4 was released 6 months after this bug was closed I assume it's still an issue. Running ndbecker's test program as is produces the following output on Solaris 10: Process PoolWorker-1: Process PoolWorker-2: Traceback (most recent call last): Traceback (most recent call last): File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/process.py, line 232, in _bootstrap File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/process.py, line 232, in _bootstrap Process PoolWorker-3: Traceback (most recent call last): File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/process.py, line 232, in _bootstrap self.run() self.run() File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/process.py, line 88, in run File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/process.py, line 88, in run self._target(*self._args, **self._kwargs) self._target(*self._args, **self._kwargs) File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/pool.py, line 57, in worker File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/pool.py, line 57, in worker self.run() task = get() task = get() File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/queues.py, line 352, in get File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/process.py, line 88, in run File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/queues.py, line 352, in get self._target(*self._args, **self._kwargs) File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/pool.py, line 57, in worker return recv() return recv() TypeError: type 'partial' takes at least one argument TypeError: type 'partial' takes at least one argument task = get() File /opt/app/g++lib6/python-2.6/lib/python2.6/multiprocessing/queues.py, line 352, in get return recv() TypeError: type 'partial' takes at least one argument -- nosy: +joseph.h.garvin versions: +Python 2.6 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5228 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
