[issue15139] Speed up threading.Condition wakeup

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

In our 2.7 branches, this approach has been superseded with a natively 
impolemented _Condition class.  This is even more efficient.  It is available 
if the underlying Lock implementation is based on pthread locks (not 
semaphores).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Catalin Patulea]

Changes by Catalin Patulea :


--
nosy: +Catalin.Patulea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Nope.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Matt Joiner]

Matt Joiner  added the comment:

Did this make it into 3.3?

--
nosy: +anacrolix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Thank you Richard.
A new patch is included.  Now the processing of "timeout" is done in 
_acquire_condition().  None is infinite, and negative timeouts are clipped to 
zero.

Do you feel that it is unnecessary to be able to support other locks than 
Lock() and RLock() as the outer lock?  If so, then we can drop the 
"_acquire_restore()" as suggested by Martin.

--
Added file: http://bugs.python.org/file26243/condition.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Richard Oudkerk]

Richard Oudkerk  added the comment:

Currently negative timeouts are treated as zero timeouts by Condition.wait().  
The patches turns them into errors.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Well, the rationale for keeping the try_finally is that the Condition should 
continue to work with other Locks, as long as they provide _release_save and 
_acquire_restore methods.  Lock._acquire_condtion onnly knows Lock and RLock 
intimately enough to automatically reaquire them with teh GIL released.

If I remove the _acuire_restore, we have prohibited the Condition class from 
using customized locks, or subclasses of locks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Made lock_acquire_condition() more robust, always acquiring the outer lock if 
possible.
For Lock objects (not RLocks), set the "locked" variable that was recently 
added.

--
Added file: http://bugs.python.org/file26117/condition.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Martin v . Löwis]

Martin v. Löwis  added the comment:

If the claim is that the _acquire_restore call becomes unnecessary, the entire 
try-finally should go. If that causes _acquire_restore to become unused, it 
should also go.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

I'll fix the patch so that the lock is _always_ re-aqcuired.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

_acquire_condition() should _always_ re-aquire the outer lock, even for a 
timeout or an interrupt.

I see however that it doesn't do so if there is an argument error, but that 
should not be a problem for an internal method, since it would mean that 
threading.Condition() were incorrectly implemented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Martin v . Löwis]

Martin v. Löwis  added the comment:

I believe the patch is incorrect. It changes self._acquire_restore into a 
no-op, claiming that lock_acquire_condition will correctly restore the lock's 
state.

However, lock_acquire_condition may fail (e.g. if the timeout is not strictly 
positive), in which case the lock's case isn't properly restored.

--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Georg Brandl]

Georg Brandl  added the comment:

Antoine is much more of an expert here, and I defer to his judgment that it is 
better to wait.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

hm, still one more day.  It will be a pity to wait two more years, perhaps 
Georg is feeling lucky :)

--
nosy: +georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> aw, that's too bad.  Since this is not a feature implementat, we have
> the entire beta cycle to shake out any possible bugs.

We treat performance improvements as new features, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

aw, that's too bad.  Since this is not a feature implementat, we have the 
entire beta cycle to shake out any possible bugs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I think this should wait for 3.4. It's a bit too late now.

--
nosy: +pitrou, sbt
stage:  -> patch review
versions: +Python 3.4 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Jesús Cea Avión]

Changes by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

Changes by Kristján Valur Jónsson :


Added file: http://bugs.python.org/file26086/queuetest.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15139] Speed up threading.Condition wakeup

[Kristján Valur Jónsson]

New submission from Kristján Valur Jónsson :

This patch is based on work previously done locally in python 2.7, see 
http://blog.ccpgames.com/kristjan/2012/05/25/optimizing-python-condition-variables-with-telemetry/

The idea is to acquire the signaling Lock() of the threading.Condition object 
_and_ the outer lock, without releasing and re-aquiring the GIL in between that 
is, as an atomic operation from the program's point of view.

_thread.LockType objects, used by the Condition object, now have an 
_acquire_condition method which knows how to do it for LockType and RLock 
objects.

Both lock types grow _release_save and _acquire_restore methods, the latter of 
which is a no-op.

_acquire_coindition() will ignore any other kinds of locks, including 
subclasses (if such things exist).

The result of this is a very marked improvement in signaling time on a 
contested machine.  A file, queuetest.py is provided which measures the wakeup 
delay.

Typical before the change:
best:totaltime: 0.167006, avg delay: 0.004766, delay_stddev: 0.002058
typical: totaltime: 0.154271, avg delay: 0.027621, delay_stddev: 0.007479

after the change
typical: totaltime: 0.169217, avg delay: 0.001410, delay_stddev: 0.000722

--
components: Interpreter Core, Library (Lib)
files: condition.patch
keywords: patch
messages: 163423
nosy: kristjan.jonsson
priority: normal
severity: normal
status: open
title: Speed up threading.Condition wakeup
type: performance
versions: Python 3.3
Added file: http://bugs.python.org/file26085/condition.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com