Re: is +=1 thread safe

2008-05-04 Thread Carl Banks
On May 3, 7:44 pm, Gary Herron [EMAIL PROTECTED] wrote: Alexander Schmolck wrote: AlFire [EMAIL PROTECTED] writes: The threading module already has a function to return the number of Thread objects currently alive. I have threads within threads - so it does not suit me :-(. How about

Re: is +=1 thread safe

2008-05-04 Thread Gary Herron
Carl Banks wrote: On May 3, 7:44 pm, Gary Herron [EMAIL PROTECTED] wrote: Alexander Schmolck wrote: AlFire [EMAIL PROTECTED] writes: The threading module already has a function to return the number of Thread objects currently alive. I have threads within threads -

Re: is +=1 thread safe

2008-05-04 Thread Bjoern Schliessmann
Gary Herron wrote: No NO NO! The only way to increment a variable in memory is through a three step process: Load a register from a memory location Increment the register Store the value back into memory. I suggest you read Intel 64 and IA-32 Architectures Software Developer's

Re: is +=1 thread safe

2008-05-04 Thread Alexander Schmolck
Gary Herron [EMAIL PROTECTED] writes: But... It's not! A simple test shows that. I've attached a tiny test program that shows this extremely clearly. Please run it and watch it fail. In [7]: run ~/tmp/t.py final count: 200 should be: 200 (I took the liberty to correct your

Re: is +=1 thread safe

2008-05-04 Thread Marc Christiansen
Alexander Schmolck [EMAIL PROTECTED] wrote: Gary Herron [EMAIL PROTECTED] writes: But... It's not! A simple test shows that. I've attached a tiny test program that shows this extremely clearly. Please run it and watch it fail. In [7]: run ~/tmp/t.py final count: 200 should

Re: is +=1 thread safe

2008-05-04 Thread Gary Herron
Alexander Schmolck wrote: Gary Herron [EMAIL PROTECTED] writes: But... It's not! A simple test shows that. I've attached a tiny test program that shows this extremely clearly. Please run it and watch it fail. In [7]: run ~/tmp/t.py final count: 200 should be: 200 (I

Re: is +=1 thread safe

2008-05-04 Thread Carl Banks
On May 4, 12:03 pm, Gary Herron [EMAIL PROTECTED] wrote: Alexander Schmolck wrote: Gary Herron [EMAIL PROTECTED] writes: But... It's not! A simple test shows that. I've attached a tiny test program that shows this extremely clearly. Please run it and watch it fail. In [7]: run

Re: is +=1 thread safe

2008-05-04 Thread Carl Banks
On May 4, 2:13 pm, Carl Banks [EMAIL PROTECTED] wrote: However, what I said was not wholly untrue: code in C extensions is protected by the GIL and thus not interruptable, unless it either releases the GIL, or calls back into Python code (which is apparently what numpy scalars do). And,

Re: is +=1 thread safe

2008-05-04 Thread Gary Herron
Carl Banks wrote: On May 4, 12:03 pm, Gary Herron [EMAIL PROTECTED] wrote: Alexander Schmolck wrote: Gary Herron [EMAIL PROTECTED] writes: But... It's not! A simple test shows that. I've attached a tiny test program that shows this extremely clearly. Please run it

Re: is +=1 thread safe

2008-05-04 Thread Aahz
In article [EMAIL PROTECTED], Gary Herron [EMAIL PROTECTED] wrote: However, the upshot of all of this is that one must maintain extreme skepticism. Unless you know both your Python code and any extension modules you call, and you know them at a level necessary to find such details, you must

Re: is +=1 thread safe

2008-05-04 Thread Alexander Schmolck
Gary Herron [EMAIL PROTECTED] writes: The test was meant to simulate the OP's problem, but even with your suggestion of using numpy, it *still* fails! Well, although I haven't tested it extensively, it doesn't appear to fail for me, with numpy 1.02 and an AMD Athlon(tm) XP 2800+ under linux

Re: is +=1 thread safe

2008-05-03 Thread Marc 'BlackJack' Rintsch
On Fri, 02 May 2008 19:23:54 +0100, Arnaud Delobelle wrote: Marc 'BlackJack' Rintsch [EMAIL PROTECTED] writes: There are no modern processors with an opcode for incrementing a memory location!? At least my C64 can do that. ;-) Indeed! I remember a simple use was to make the border

Re: is +=1 thread safe

2008-05-03 Thread Arnaud Delobelle
Marc 'BlackJack' Rintsch [EMAIL PROTECTED] writes: On Fri, 02 May 2008 19:23:54 +0100, Arnaud Delobelle wrote: Marc 'BlackJack' Rintsch [EMAIL PROTECTED] writes: There are no modern processors with an opcode for incrementing a memory location!? At least my C64 can do that. ;-)

Re: is +=1 thread safe

2008-05-03 Thread Alexander Schmolck
AlFire [EMAIL PROTECTED] writes: The threading module already has a function to return the number of Thread objects currently alive. I have threads within threads - so it does not suit me :-(. How about using a scalar numpy array? They are mutable, so I assume that x += 1 should be atomic.

Re: is +=1 thread safe

2008-05-03 Thread AlFire
Alexander Schmolck wrote: AlFire [EMAIL PROTECTED] writes: The threading module already has a function to return the number of Thread objects currently alive. I have threads within threads - so it does not suit me :-(. How about using a scalar numpy array? They are mutable, so I assume that

Re: is +=1 thread safe

2008-05-03 Thread Gary Herron
Alexander Schmolck wrote: AlFire [EMAIL PROTECTED] writes: The threading module already has a function to return the number of Thread objects currently alive. I have threads within threads - so it does not suit me :-(. How about using a scalar numpy array? They are mutable,

Re: is +=1 thread safe

2008-05-02 Thread Marc 'BlackJack' Rintsch
On Thu, 01 May 2008 15:33:09 -0700, Gary Herron wrote: Of course it's not thread safe. For the same reason and more basic, even the expression i++ is not thread safe in C++. Any such calculation, on modern processors, requires three operations: retrieve value of i into a register,

Re: is +=1 thread safe

2008-05-02 Thread Paul Rubin
AlFire [EMAIL PROTECTED] writes: But I still can not believe that +=1 is not a thread safe operation. In CPython I believe it is thread safe, because of the global interpreter lock. Thread switches can happen only between bytecode executions, not in the middle of a bytecode, even though

Re: is +=1 thread safe

2008-05-02 Thread Duncan Booth
Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote: On Thu, 01 May 2008 15:33:09 -0700, Gary Herron wrote: Of course it's not thread safe. For the same reason and more basic, even the expression i++ is not thread safe in C++. Any such calculation, on modern processors, requires three

Re: is +=1 thread safe

2008-05-02 Thread Hrvoje Niksic
Paul Rubin http://[EMAIL PROTECTED] writes: AlFire [EMAIL PROTECTED] writes: But I still can not believe that +=1 is not a thread safe operation. In CPython I believe it is thread safe, because of the global interpreter lock. Thread switches can happen only between bytecode executions, not

Re: is +=1 thread safe

2008-05-02 Thread Diez B. Roggisch
Duncan Booth schrieb: Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote: On Thu, 01 May 2008 15:33:09 -0700, Gary Herron wrote: Of course it's not thread safe. For the same reason and more basic, even the expression i++ is not thread safe in C++. Any such calculation, on modern

Re: is +=1 thread safe

2008-05-02 Thread Arnaud Delobelle
Marc 'BlackJack' Rintsch [EMAIL PROTECTED] writes: There are no modern processors with an opcode for incrementing a memory location!? At least my C64 can do that. ;-) Indeed! I remember a simple use was to make the border change colour very fast, a v. cool effect when you're 12!

Re: is +=1 thread safe

2008-05-02 Thread Paul Rubin
Hrvoje Niksic [EMAIL PROTECTED] writes: Since the += operator is not compiled into a single bytecode instruction, it needs the lock. Aha, you are right. What I was remembering is that xrange.next is atomic in CPython, i.e. you can say something like counter = xrange(1) and then

Re: is +=1 thread safe

2008-05-02 Thread AlFire
Duncan Booth wrote: [...] is equivalent to: x = x.__iadd__(1) thx all for answers and hints ... Generating hundreds of threads is, BTW, a very good way to get poor performance on any system. Don't do that. Create a few threads and put the actions for those threads into a Queue. If

is +=1 thread safe

2008-05-01 Thread AlFire
Hi, I have a piece of software which uses threads in very massive way - like hundreds of them generated every second. there is also a piece of code which maintains the number of outstanding threads, simply counter+=1 is executed when before starting the thread and counter-=1 after it

Re: is +=1 thread safe

2008-05-01 Thread Jeff
If no other threads will be accessing the counter, there will be no problem. -- http://mail.python.org/mailman/listinfo/python-list

Re: is +=1 thread safe

2008-05-01 Thread Duncan Booth
AlFire [EMAIL PROTECTED] wrote: But I still can not believe that +=1 is not a thread safe operation. Any clue? The statement: x+=1 is equivalent to: x = x.__iadd__(1) i.e. a function call followed by an assignment. If the object is mutable then this *may* be safe so long

Re: is +=1 thread safe

2008-05-01 Thread John Nagle
AlFire wrote: Hi, all is very simple and by the end of the program life I expect the counter to zero out. however I am getting values -1, -2, 1 ,2 ,3 and quite often 0 as expected. I guarded those statement with Lock.{acquire,release} and now it always returns 0. But I still can not

Re: is +=1 thread safe

2008-05-01 Thread Diez B. Roggisch
AlFire schrieb: Hi, I have a piece of software which uses threads in very massive way - like hundreds of them generated every second. there is also a piece of code which maintains the number of outstanding threads, simply counter+=1 is executed when before starting the thread and

Re: is +=1 thread safe

2008-05-01 Thread Gary Herron
AlFire wrote: Hi, I have a piece of software which uses threads in very massive way - like hundreds of them generated every second. there is also a piece of code which maintains the number of outstanding threads, simply counter+=1 is executed when before starting the thread and