Re: [Python-Dev] Software Transactional Memory for Python

2011-08-31 Thread Armin Rigo
Hi, On Tue, Aug 30, 2011 at 11:33 PM, Yury Selivanov yselivanov...@gmail.com wrote: Maybe it'd be better to put 'atomic' in the threading module? 'threading' is pure Python. But anyway the consensus is to not have 'atomic' at all in the stdlib, which means it is in its own 3rd-party extension

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-30 Thread Armin Rigo
Re-hi, 2011/8/29 Armin Rigo ar...@tunes.org: The problem is that many locks are actually acquired implicitely. For example, `print` to a buffered stream will acquire the fileobject's mutex. Indeed. (...) I suspect that I need to do a more thorough review of the stdlib (...) I found a

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-30 Thread Yury Selivanov
Maybe it'd be better to put 'atomic' in the threading module? On 2011-08-30, at 4:02 PM, Armin Rigo wrote: Re-hi, 2011/8/29 Armin Rigo ar...@tunes.org: The problem is that many locks are actually acquired implicitely. For example, `print` to a buffered stream will acquire the fileobject's

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-29 Thread Armin Rigo
Hi Guido, On Sun, Aug 28, 2011 at 6:43 PM, Guido van Rossum gu...@python.org wrote: This sounds like a very interesting idea to pursue, even if it's late, and even if it's experimental, and even if it's possible to cause deadlocks (no news there). I propose that we offer a C API in Python 3.3

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-29 Thread Antoine Pitrou
On Sun, 28 Aug 2011 09:43:33 -0700 Guido van Rossum gu...@python.org wrote: This sounds like a very interesting idea to pursue, even if it's late, and even if it's experimental, and even if it's possible to cause deadlocks (no news there). I propose that we offer a C API in Python 3.3 as

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-29 Thread Gregory P. Smith
On Mon, Aug 29, 2011 at 5:20 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 28 Aug 2011 09:43:33 -0700 Guido van Rossum gu...@python.org wrote: This sounds like a very interesting idea to pursue, even if it's late, and even if it's experimental, and even if it's possible to cause

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-29 Thread Armin Rigo
Hi Charles-François, 2011/8/27 Charles-François Natali neolo...@free.fr: The problem is that many locks are actually acquired implicitely. For example, `print` to a buffered stream will acquire the fileobject's mutex. Indeed. After looking more at the kind of locks used throughout the stdlib,

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-28 Thread Guido van Rossum
On Sat, Aug 27, 2011 at 6:08 AM, Armin Rigo ar...@tunes.org wrote: Hi Nick, On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan ncogh...@gmail.com wrote: 1. How does the patch interact with C code that explicitly releases the GIL? (e.g. IO commands inside a with atomic: block) As implemented, any

[Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Armin Rigo
Hi all, About multithreading models: I recently made an observation which might be obvious to some, but not to me, and as far as I know not to most of us either. I think that it's worth being pointed out :-) http://mail.python.org/pipermail/pypy-dev/2011-August/008153.html A bientôt, Armin.

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Nick Coghlan
On Sat, Aug 27, 2011 at 8:45 PM, Armin Rigo ar...@tunes.org wrote: Hi all, About multithreading models: I recently made an observation which might be obvious to some, but not to me, and as far as I know not to most of us either.  I think that it's worth being pointed out :-)

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Armin Rigo
Hi Nick, On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan ncogh...@gmail.com wrote: 1. How does the patch interact with C code that explicitly releases the GIL? (e.g. IO commands inside a with atomic: block) As implemented, any code in a with atomic is prevented from explicitly releasing and

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Antoine Pitrou
On Sat, 27 Aug 2011 15:08:36 +0200 Armin Rigo ar...@tunes.org wrote: Hi Nick, On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan ncogh...@gmail.com wrote: 1. How does the patch interact with C code that explicitly releases the GIL? (e.g. IO commands inside a with atomic: block) As

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Armin Rigo
Hi Antoine, You then risk deadlocks. Say: (...) Yes, it is indeed not a solution that co-operates transparently and deadlock-freely with regular locks. You risk the same kind of deadlocks as you would when using only locks. The reason is similar to threads that try to acquire two locks in

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Charles-François Natali
Hi Armin, This is basically dangerous, because it corresponds to taking lock GIL and lock L, in that order, whereas the thread B takes lock L and plays around with lock GIL in the opposite order.  I think a reasonable solution to avoid deadlocks is simply not to use explicit locks inside