Re: [Python-Dev] deja-vu .. python locking

2006-09-20 Thread Josiah Carlson
Martin Devera <[EMAIL PROTECTED]> wrote: [snip] > Even if you can do fast atomic inc/dec, it forces cacheline with > refcounter to ping-pong between caches of referencing cpus (for read only > class dicts for example) so that you can probably never get good SMP > scalability. That's ok. Why? Be

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin v. Löwis
Martin Devera schrieb: > Just from curiosity, would be a big problem removing refcounting and live > with garbage collection only ? I'm not sure if some parts of py code > depends on exact refcnt behaviour (I guess it should not). Now, this gives a true deja-vu. Python applications often rely on r

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin Devera
Greg Ewing wrote: > Martin Devera wrote: > >> Greg, what change do you have in mind regarding that "3 instruction >> addition" to refcounting ? > > I don't have any change in mind. If even an atomic inc > is too expensive, it seems there's no hope for us. Just from curiosity, would be a big prob

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Greg Ewing
Martin Devera wrote: > Greg, what change do you have in mind regarding that "3 instruction > addition" to refcounting ? I don't have any change in mind. If even an atomic inc is too expensive, it seems there's no hope for us. -- Greg ___ Python-Dev mai

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin v. Löwis
Martin Devera schrieb: > I measured it. Lock op in futex based linux locking is of the same > speed as windows critical section and it is about 30 cycles on my > P4 1.8GHz in uncontented case. > As explained in already mentioned http://www.linuxjournal.com/article/6993 > it seems due to pipeline fl

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin Devera
> Ah, I think I understand now. First the minor critique: I believe > the locking algorithm isn't thread-safe: > > while (ob->owner_thread != self_thread()) { >unlock_mutex(thread_mutex[self_thread()]) > // wait for owning thread to go to quiscent state >lock_mutex(thread_m

Re: [Python-Dev] deja-vu .. python locking

2006-09-19 Thread Martin Devera
Greg Ewing wrote: > Martin Devera wrote: > >> As I've written in "Big reader lock" paragraph of the original >> proposal, these >> objects could be handled by not blocking in read path > > But as was just pointed out, because of refcounting, > there's really no such thing as read-only access to

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Greg Ewing
Phillip J. Eby wrote: > So, I think for your plan to work, you would have to eliminate reference > counting, in order to bring the lock overhead down to a manageable level. There's a possibility it wouldn't be atrociously bad. Seems like it would only add the 3 instructions or whatever overhead

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Greg Ewing
Martin Devera wrote: > As I've written in "Big reader lock" paragraph of the original proposal, these > objects could be handled by not blocking in read path But as was just pointed out, because of refcounting, there's really no such thing as read-only access to an object. What *looks* like read-

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Greg Ewing
Martin Devera wrote: > Regarding implementation, I wanted to look for some opinions before starting > to > implement something as big as this patch. Probably someone can look and say, > hey > it is stupit, you forgot that FILL_IN ... ;-) If I understand correctly, your suggestion for avoidi

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin v. Löwis
Martin Devera schrieb: > It is based on assumption that an object is typicaly used by single > thread. You must lock it anyway just for case if another thread steps > on it. The idea is that each object is "owned" by a thread. Owner can > use its objects without locking. If a thread wants to use f

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Phillip J. Eby
At 07:08 PM 9/18/2006 +0200, Martin Devera wrote: > >> So that you are right. It is not RCU. It only uses similar technique > as RCU > >> uses for free-ing old copy of data. > >> > >> It is based on assumption that an object is typicaly used by single > thread. > > > > Which thread owns builtins?

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin Devera
>> So that you are right. It is not RCU. It only uses similar technique as RCU >> uses for free-ing old copy of data. >> >> It is based on assumption that an object is typicaly used by single thread. > > Which thread owns builtins? Or module dictionaries? If two threads are > running the same f

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Jean-Paul Calderone
On Mon, 18 Sep 2006 17:06:47 +0200, Martin Devera <[EMAIL PROTECTED]> wrote: >Martin v. Löwis wrote: >> Martin Devera schrieb: >>> RCU like locking >>> Solution I have in mind is similar to RCU. In Python we have quiscent >>> state - when a thread returns to main loop of interpreter. >> >> Ther

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin Devera
Martin v. Löwis wrote: > Martin Devera schrieb: >> RCU like locking >> Solution I have in mind is similar to RCU. In Python we have quiscent >> state - when a thread returns to main loop of interpreter. > > There might be a terminology problem here. RCU is read-copy-update, > right? I fail to

Re: [Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin v. Löwis
Martin Devera schrieb: > RCU like locking > Solution I have in mind is similar to RCU. In Python we have quiscent > state - when a thread returns to main loop of interpreter. There might be a terminology problem here. RCU is read-copy-update, right? I fail to see the copy (copy data structure

[Python-Dev] deja-vu .. python locking

2006-09-18 Thread Martin Devera
Hello, as someone has written in FAQ, sometimes someone starts a thread about finer grained locking in Python. Ok here is one. I don't want to start a flamewar. I only seek suggestions and constructive critic. I have some ideas whose are new in this context (I believe) and I only wanted to make t