Re: [Python-Dev] Add an optional timeout to lock operations
Antoine Pitrou wrote: Guido van Rossum guido at python.org writes: Will locks be interruptible with ^C? That is an oft-requested feature which also wasn't supported at that time; what's the situation nowadays? They still aren't interruptible. From what I can read it may be possible to make them interruptible in the POSIX semaphore-based implementation, not in the POSIX condition variable-based implementation (which is used as a fallback when POSIX semaphores are not available, but I don't know whether this fallback is still useful). I'm pretty sure at least some variants of *BSD still don't have OS level semaphores - their lack is the reason multiprocessing doesn't necessarily work everywhere that the threading module works (since mp needs semaphores in order to work its magic). Jesse would probably know the gory details. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
On Nov 18, 2009, at 5:38 AM, Nick Coghlan ncogh...@gmail.com wrote: Antoine Pitrou wrote: Guido van Rossum guido at python.org writes: Will locks be interruptible with ^C? That is an oft-requested feature which also wasn't supported at that time; what's the situation nowadays? They still aren't interruptible. From what I can read it may be possible to make them interruptible in the POSIX semaphore-based implementation, not in the POSIX condition variable-based implementation (which is used as a fallback when POSIX semaphores are not available, but I don't know whether this fallback is still useful). I'm pretty sure at least some variants of *BSD still don't have OS level semaphores - their lack is the reason multiprocessing doesn't necessarily work everywhere that the threading module works (since mp needs semaphores in order to work its magic). Jesse would probably know the gory details. Cheers, Nick. Nick is right, many of the BSDs and FreeBSD up until fairly recently did not have named shared semaphore support. Still yet, the behavior is broken on some OSes such as OS X which you have to work around. I wouldn't recommend using them for threading right now, there's an assumption that threading just works unlike multiprocessing, which people understand has caveats. Jesse ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
Jesse Noller jnoller at gmail.com writes: Nick is right, many of the BSDs and FreeBSD up until fairly recently did not have named shared semaphore support. Still yet, the behavior is broken on some OSes such as OS X which you have to work around. The core locking support only uses anonymous semaphores (they don't need to be shared accross processes obviously). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
On Wed, Nov 18, 2009 at 8:50 AM, Antoine Pitrou solip...@pitrou.net wrote: Jesse Noller jnoller at gmail.com writes: Nick is right, many of the BSDs and FreeBSD up until fairly recently did not have named shared semaphore support. Still yet, the behavior is broken on some OSes such as OS X which you have to work around. The core locking support only uses anonymous semaphores (they don't need to be shared accross processes obviously). Regards Antoine. That's what I get for replying from an iphone over morning coffee. See also http://semanchuk.com/philip/sysv_ipc/ for Philip Semanchuk semaphore/IPC work as well. Given both multiprocessing and Philip's work are concerned about the shared aspects, I think you're right - FBSD and others support the posix semaphores for non-ipc stuff ok AFAIK (recent benchmark http://www.ioremap.net/node/216). jesse ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
Antoine Pitrou solip...@pitrou.net writes: I've submitted a patch (*) to add an optional timeout to locking operations (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to know if there was any good reason for not doing it. I always assumed it was because as a least-common-denominator set of functionality, some platforms didn't have the necessary support. Providing the discussion on this ends up with the an implementation being accepted, I'd absolutely love to see this then leveraged by threading.Condition.wait() rather rather than the current poll-with-timed-sleep approach. The overhead (CPU, but most notably latency) of that approach (which also directly impacts Queues) has historically been my top reason in various applications on Windows to have to implement my own Queue or Condition structures using native Windows calls. -- David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
Hello, I've submitted a patch (*) to add an optional timeout to locking operations (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to know if there was any good reason for not doing it. I always assumed it was because as a least-common-denominator set of functionality, some platforms didn't have the necessary support. Guido's answer says so indeed. Now py3k only needs to support POSIX and Windows (and, provided Andrew MacIntyre maintains the port, OS/2), which both have standard support for waiting-with-timeout. Providing the discussion on this ends up with the an implementation being accepted, I'd absolutely love to see this then leveraged by threading.Condition.wait() rather rather than the current poll-with-timed-sleep approach. Agreed. The current patch (as proposed on http://bugs.python.org/issue7316) includes exactly that. Poll-with-timed-sleep is especially sub-optimal on laptops where short but frequent wakeups can cause a significant decrease in battery life. (the Linux community has been chasing this using powertop: http://www.lesswatts.org/projects/powertop/) Feel free to test or review if you're interested. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Add an optional timeout to lock operations
Hello, I've submitted a patch (*) to add an optional timeout to locking operations (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to know if there was any good reason for not doing it. (*) http://bugs.python.org/issue7316 Thank you Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
I think I can answer the why question: thread.c is *very* old code, in fact it predates the posix threads standard. When we (actually Sjoerd Mullender) wrote it, we had a number of OS-specific locking APIs to work with and the API was designed to fit all of them. I don't even recall the initial set, but I think it included SGI Irix and and old version of SunOS. So then over time new platforms were added, but adding a new method or parameter to the API was nearly impossible because someone would have to find out how to implement the new feature on all supported platforms. I think the number of platforms has dwindled to two or three (Posix, Windows, and maybe one minority OS?), so now's the time to do it. (IOW I think the idea of the patch is fine.) Will locks be interruptible with ^C? That is an oft-requested feature which also wasn't supported at that time; what's the situation nowadays? --Guido On Tue, Nov 17, 2009 at 3:46 AM, Antoine Pitrou solip...@pitrou.net wrote: Hello, I've submitted a patch (*) to add an optional timeout to locking operations (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to know if there was any good reason for not doing it. (*) http://bugs.python.org/issue7316 Thank you Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add an optional timeout to lock operations
Guido van Rossum guido at python.org writes: I think the number of platforms has dwindled to two or three (Posix, Windows, and maybe one minority OS?), so now's the time to do it. (IOW I think the idea of the patch is fine.) Thanks. (the minority OS would be OS/2, I think) Will locks be interruptible with ^C? That is an oft-requested feature which also wasn't supported at that time; what's the situation nowadays? They still aren't interruptible. From what I can read it may be possible to make them interruptible in the POSIX semaphore-based implementation, not in the POSIX condition variable-based implementation (which is used as a fallback when POSIX semaphores are not available, but I don't know whether this fallback is still useful). As for Windows I have absolutely no idea. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
