[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

STINNER Victor added the comment:

Hum, my commit was not reported in the issue:

commit da6d305b6fcd49ba1224b1fd2131d7648a5be6b9
Author: Victor Stinner 
Date:   Thu Jun 22 02:09:40 2017 +0200

bpo-8799: Reduce timing sensitivity of condition test by explicitly (#2320)
delaying the main thread so that it doesn't race ahead of the workers.
(cherry picked from commit 020af2a2bc4708215360a3793b5a1790e15d05dd)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

STINNER Victor added the comment:

> In Python 3 the test was failed if decrease the sleep time in _wait() to 
> 0.001. Patched test no longer fail for any small sleep intervals.

I used sleep(1e-9), 1 nanosecond, for my tests :-)

I confirm that I'm unable to reproduce the bug on 2.7 and master
branches anymore. Well done Serhiy!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In Python 3 the test was failed if decrease the sleep time in _wait() to 0.001. 
Patched test no longer fail for any small sleep intervals.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 7709b4d57b433ef027a2e7e63b4cab3fc9ad910d by Serhiy Storchaka in 
branch '2.7':
[2.7] bpo-30727: Fix a race condition in test_threading. (GH-2334). (#2353)
https://github.com/python/cpython/commit/7709b4d57b433ef027a2e7e63b4cab3fc9ad910d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset e2aec8e691d8acb08373889d9af48a5b1d03b689 by Serhiy Storchaka in 
branch '3.5':
[3.5] bpo-30727: Fix a race condition in test_threading. (GH-2334) (#2352)
https://github.com/python/cpython/commit/e2aec8e691d8acb08373889d9af48a5b1d03b689


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset c1d5345679eaa5fccd719b1c130140eecc8ba4c8 by Serhiy Storchaka in 
branch '3.6':
[3.6] bpo-30727: Fix a race condition in test_threading. (GH-2334) (#2351)
https://github.com/python/cpython/commit/c1d5345679eaa5fccd719b1c130140eecc8ba4c8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +2399

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +2398

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +2397

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 32cb968a2edd482891c33b6f2ebae10f1d305424 by Serhiy Storchaka in 
branch 'master':
bpo-30727: Fix a race condition in test_threading. (#2334)
https://github.com/python/cpython/commit/32cb968a2edd482891c33b6f2ebae10f1d305424


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Actually this approach already is used in this test. PR 2334 uses it to ensure 
all workers into cond.wait().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +2378

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I still didn't see a code, but perhaps we could use a spin-lock. Every child 
thread increases the global counter, and the main thread waits until the 
counter equal to the number of child threads (sleeping for short time in a 
loop) before calling notify().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Antoine Pitrou]

Antoine Pitrou added the comment:

Yes, the whole reason I wrote the tests like this is to avoid depending on 
Python synchronization primitives inside the tests for Python synchronization 
primitives.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[Kristján Valur Jónsson]

Kristján Valur Jónsson added the comment:

My favorite topic :)

You could use threading.Barrier() which is designed to synchronize N threads 
doing this kind of lock-step processing.  The problem is that the Barrier() is 
implemented using Condition variables, so for unit-testing, Condition 
Variables, this presents a conundrum...

--
nosy: +kristjan.jonsson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

STINNER Victor added the comment:

It's getting late here, so I chose to first backport the commit 
020af2a2bc4708215360a3793b5a1790e15d05dd to Python 2.7, to reduce the likehood 
of this bug on 2.7 buildbots.

--
pull_requests: +2372

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

STINNER Victor added the comment:

The bug occurs when notify(n) is called while less than n threads are waiting 
in cond.wait(): when some threads didn't reach cond.wait() yet. The 
synchronization code is very fragile and depends on time. The issue was worked 
again in the master banch by adding more _wait() calls...

To really fix the race condition, reliable synchronization primitives should be 
used, like threading.Event.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

STINNER Victor added the comment:

See also bpo-8799 "Hang in lib/test/test_threading.py" and the commit 
020af2a2bc4708215360a3793b5a1790e15d05dd which added two _wait() calls to 
ConditionTests._check_notify().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

STINNER Victor added the comment:

notify_bug.py: script close totest.test_threading.ConditionTests.test_notify() 
but different:

* _wait() sleeps 1 nanosecond rather than 10 ms
* log many messages into stdout (file descriptor 1) using os.write()

It's quite easy to reproduce the bug with this script.

--
Added file: http://bugs.python.org/file46966/notify_bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30727] [2.7] test_threading.ConditionTests.test_notify() hangs randomly on Python 2.7

[STINNER Victor]

Changes by STINNER Victor :


--
title: [2.7] test_threading.ConditionTests.test_notify() hangs randomly on 
FreeBSD on Python 2.7 -> [2.7] test_threading.ConditionTests.test_notify() 
hangs randomly on Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com