Re: [Python-Dev] deja-vu .. python locking

2006-09-20 Thread Martin Devera
Greg Ewing wrote: Martin Devera wrote: Greg, what change do you have in mind regarding that 3 instruction addition to refcounting ? I don't have any change in mind. If even an atomic inc is too expensive, it seems there's no hope for us. Just from curiosity, would be a big problem

Re: [Python-Dev] deja-vu .. python locking

2006-09-20 Thread Martin v. Löwis
Martin Devera schrieb: Just from curiosity, would be a big problem removing refcounting and live with garbage collection only ? I'm not sure if some parts of py code depends on exact refcnt behaviour (I guess it should not). Now, this gives a true deja-vu. Python applications often rely on

Re: [Python-Dev] deja-vu .. python locking

2006-09-20 Thread Josiah Carlson
Martin Devera [EMAIL PROTECTED] wrote: [snip] Even if you can do fast atomic inc/dec, it forces cacheline with refcounter to ping-pong between caches of referencing cpus (for read only class dicts for example) so that you can probably never get good SMP scalability. That's ok. Why? Because

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin Devera
Greg Ewing wrote: Martin Devera wrote: As I've written in Big reader lock paragraph of the original proposal, these objects could be handled by not blocking in read path But as was just pointed out, because of refcounting, there's really no such thing as read-only access to an object.

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin Devera
Ah, I think I understand now. First the minor critique: I believe the locking algorithm isn't thread-safe: while (ob-owner_thread != self_thread()) { unlock_mutex(thread_mutex[self_thread()]) // wait for owning thread to go to quiscent state

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Greg Ewing
Martin Devera wrote: Greg, what change do you have in mind regarding that 3 instruction addition to refcounting ? I don't have any change in mind. If even an atomic inc is too expensive, it seems there's no hope for us. -- Greg ___ Python-Dev

[Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin Devera
Hello, as someone has written in FAQ, sometimes someone starts a thread about finer grained locking in Python. Ok here is one. I don't want to start a flamewar. I only seek suggestions and constructive critic. I have some ideas whose are new in this context (I believe) and I only wanted to make

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin v. Löwis
Martin Devera schrieb: RCU like locking Solution I have in mind is similar to RCU. In Python we have quiscent state - when a thread returns to main loop of interpreter. There might be a terminology problem here. RCU is read-copy-update, right? I fail to see the copy (copy data structure to

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin Devera
Martin v. Löwis wrote: Martin Devera schrieb: RCU like locking Solution I have in mind is similar to RCU. In Python we have quiscent state - when a thread returns to main loop of interpreter. There might be a terminology problem here. RCU is read-copy-update, right? I fail to see the

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Jean-Paul Calderone
On Mon, 18 Sep 2006 17:06:47 +0200, Martin Devera [EMAIL PROTECTED] wrote: Martin v. Löwis wrote: Martin Devera schrieb: RCU like locking Solution I have in mind is similar to RCU. In Python we have quiscent state - when a thread returns to main loop of interpreter. There might be a

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin Devera
So that you are right. It is not RCU. It only uses similar technique as RCU uses for free-ing old copy of data. It is based on assumption that an object is typicaly used by single thread. Which thread owns builtins? Or module dictionaries? If two threads are running the same function

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Phillip J. Eby
At 07:08 PM 9/18/2006 +0200, Martin Devera wrote: So that you are right. It is not RCU. It only uses similar technique as RCU uses for free-ing old copy of data. It is based on assumption that an object is typicaly used by single thread. Which thread owns builtins? Or module

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin v. Löwis
Martin Devera schrieb: It is based on assumption that an object is typicaly used by single thread. You must lock it anyway just for case if another thread steps on it. The idea is that each object is owned by a thread. Owner can use its objects without locking. If a thread wants to use

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Greg Ewing
Martin Devera wrote: Regarding implementation, I wanted to look for some opinions before starting to implement something as big as this patch. Probably someone can look and say, hey it is stupit, you forgot that FILL_IN ... ;-) If I understand correctly, your suggestion for avoiding

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Greg Ewing
Martin Devera wrote: As I've written in Big reader lock paragraph of the original proposal, these objects could be handled by not blocking in read path But as was just pointed out, because of refcounting, there's really no such thing as read-only access to an object. What *looks* like

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Greg Ewing
Phillip J. Eby wrote: So, I think for your plan to work, you would have to eliminate reference counting, in order to bring the lock overhead down to a manageable level. There's a possibility it wouldn't be atrociously bad. Seems like it would only add the 3 instructions or whatever overhead