[issue27585] asyncio.Lock deadlock after cancellation

[Yury Selivanov]

Changes by Yury Selivanov :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.5, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Yury Selivanov]

Changes by Yury Selivanov :


--
pull_requests: +2103

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Yury Selivanov]

Changes by Yury Selivanov :


--
pull_requests: +2102

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Yury Selivanov]

Yury Selivanov added the comment:


New changeset 7be651c7aad8e4d46012205811b58ef127b08e0e by Yury Selivanov in 
branch 'master':
bpo-27585: Add a NEWS entry for #1031 (#2036)
https://github.com/python/cpython/commit/7be651c7aad8e4d46012205811b58ef127b08e0e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Yury Selivanov]

Changes by Yury Selivanov :


--
pull_requests: +2101

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Guido van Rossum]

Guido van Rossum added the comment:

I'm really sorry, but I personally don't have time to look into this. If you 
have a fix in mind you can send a PR to GitHub mentioning bpi-27585.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Alexey Popravka]

Alexey Popravka added the comment:

Also there is the same problem with asyncio.Condition wait() / notify() couple 
as it repeats the same Lock.acquire()/release() logic

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Alexey Popravka]

Alexey Popravka added the comment:

Hi guys! Any update on this?
I've just hit this issue.
Cancelled futures in _waiters is not a problem (any more).

There is still a problem when release() wakes up next waiter
but task waiting for it gets cancelled and so all the rest waiters
are in pending state forever.
I've attach one more test to reproduce this issue.

--
nosy: +Alexey Popravka
Added file: http://bugs.python.org/file46876/deadlock2.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Mathieu Sornay]

Changes by Mathieu Sornay :


--
pull_requests: +1188

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Mathieu Sornay]

Mathieu Sornay added the comment:

Fix attempt : https://github.com/python/asyncio/pull/467

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Guido van Rossum]

Guido van Rossum added the comment:

Looks like create_waiter() entered the with-statement but didn't leave it
properly due to the cancellation, right? @1st1 any idea? async with bug or
asyncio bug?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Mathieu Sornay]

Mathieu Sornay added the comment:

I might have found another pathological case not fixed by 
https://github.com/python/asyncio/pull/393

Tested in 3.6.0b3

The deadlock.py file prints :

DEADLOCK HERE
_locked:  False
_waiters: deque([])

--
nosy: +msornay
versions: +Python 3.6 -Python 3.5
Added file: http://bugs.python.org/file45489/deadlock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Guido van Rossum]

Guido van Rossum added the comment:

OK, merged upstream. It'll eventually come down to Python 3.5.3 and 3.6.

On Thu, Aug 4, 2016 at 1:28 PM, sss  wrote:
>
> sss added the comment:
>
> Thank you. This looks good to me
>
> --
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[sss]

sss added the comment:

Thank you. This looks good to me

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Guido van Rossum]

Guido van Rossum added the comment:

Tentative fix is in https://github.com/python/asyncio/pull/393

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[sss]

sss added the comment:

Yes I'm running python-3.5.2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[Guido van Rossum]

Guido van Rossum added the comment:

Does this repro with the latest Python 3.5.2? IIRC the lock code was
overhauled after 3.5.0 or 3.5.1.

(PS. These are tasks, not threads.)

--Guido (mobile)

On Aug 4, 2016 2:33 AM, "sss"  wrote:

sss added the comment:

Yes, await asyncio.sleep(0) does fix it, but it would be weird if I'm
supposed to put those all over my code. I made a new reproducer now that
uses 2 threads in case it's not allowed to take the same lock twice from
the same thread.

--
Added file: http://bugs.python.org/file44007/lock2.py

___
Python tracker 

___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[sss]

sss added the comment:

Yes, await asyncio.sleep(0) does fix it, but it would be weird if I'm supposed 
to put those all over my code. I made a new reproducer now that uses 2 threads 
in case it's not allowed to take the same lock twice from the same thread.

--
Added file: http://bugs.python.org/file44007/lock2.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[R. David Murray]

R. David Murray added the comment:

If you give the loop a chance to run (an await asyncio.sleep(0)) before doing 
the acquire) it does not deadlock.  I don't understand enough of the asyncio 
internals to know if this is expected or a bug.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27585] asyncio.Lock deadlock after cancellation

[sss]

New submission from sss:

The lock.py file prints

DEADLOCK HERE
_locked:  False
_waiters: deque([])

I think the problem is that acquire() will block because of the cancelled 
future in _waiters, but release() has already ran so no one will wake it up.

--
components: asyncio
files: lock.py
messages: 270944
nosy: gvanrossum, haypo, sss, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.Lock deadlock after cancellation
versions: Python 3.5
Added file: http://bugs.python.org/file43820/lock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com