[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Maximilian Hils]

Maximilian Hils  added the comment:

Thank you for the super quick turnaround.
My heart goes out to you and everyone else in Ukraine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks!

--
resolution:  -> fixed
status:  -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[miss-islington]

miss-islington  added the comment:


New changeset 64a68c39cb508b016e5a4486ebb4052f6e65fca0 by Miss Islington (bot) 
in branch '3.9':
bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892)
https://github.com/python/cpython/commit/64a68c39cb508b016e5a4486ebb4052f6e65fca0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[miss-islington]

miss-islington  added the comment:


New changeset 88c243fd8d5a43282ef06bd0872e3b88c68bb856 by Miss Islington (bot) 
in branch '3.10':
bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892)
https://github.com/python/cpython/commit/88c243fd8d5a43282ef06bd0872e3b88c68bb856


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[miss-islington]

Change by miss-islington :


--
pull_requests: +3
pull_request: https://github.com/python/cpython/pull/31906

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 10.0 -> 11.0
pull_requests: +2
pull_request: https://github.com/python/cpython/pull/31905

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 70155412f1543f100d4aa309b8691cbcabd3e0e1 by Maximilian Hils in 
branch 'main':
bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892)
https://github.com/python/cpython/commit/70155412f1543f100d4aa309b8691cbcabd3e0e1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Andrew Svetlov]

Change by Andrew Svetlov :


--
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Maximilian Hils]

Change by Maximilian Hils :


--
pull_requests: +29990
pull_request: https://github.com/python/cpython/pull/31892

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Maximilian, thanks for the investigation.

A check for 'fileno != -1' seems correct to me.
Would you prepare a pull request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Maximilian Hils]

Maximilian Hils  added the comment:

asvetlov: Sorry if I articulated myself badly, but I do think this is a valid 
bug. It's unfortunately hard to provide a better repro (I tried), but we are 
hitting this regularly when mitmproxy is accepting connections under heavy 
load. We're just calling `asyncio.start_server(handler, "127.0.0.1", 8080)` in 
mitmproxy and never interact with the underlying socket object.

Here are some observations that are true for all crashes:

- The socket fileno is -1 when it crashes.
- `_call_connection_lost` is called by `_ProactorBasePipeTransport.close`, 
which is called by `_ProactorBasePipeTransport.__del__` [1]
- There are no previous calls to `_call_connection_lost`.
- Windows only, loopback connections in our case.
- Wireshark shows that client and server are first happily exchanging packets. 
At some point the client sends a FIN, which the Python server ACKs immediately. 
A few seconds later the Python server sends a FIN back.


An obvious fix without understanding the root cause would be to check fileno in 
https://github.com/python/cpython/blob/d929aa70e2a324ea48fed221c3257f929be05115/Lib/asyncio/proactor_events.py#L161.
 I'm not too familar with proactor to assess if that is a good idea. Sorry for 
not being able to provide more details.


[1] 
https://github.com/python/cpython/blob/d929aa70e2a324ea48fed221c3257f929be05115/Lib/asyncio/proactor_events.py#L102-L116

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution: not a bug -> 
status: closed -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Maximilian Hils]

Change by Maximilian Hils :


--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Maximilian Hils]

Maximilian Hils  added the comment:

We are hitting the same traceback with mitmproxy on Windows without ever 
passing a socket object to open_connection. In other words, this can be 
triggered without performing any action on 'sock' due to some race conditions 
in the Windows network stack. I unfortunately don't have a better repro for 
that race other than what Daniel provided.

--
nosy: +mhils

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Please consider passing 'sock' argument as the ownership transfer.

You should not perform any action on 'sock' object directly anymore.
This is true for all asyncio API.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Adhika Setya Pramudita]

Change by Adhika Setya Pramudita :


--
nosy: +adhika.setyap

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Emmanuel Arias]

Change by Emmanuel Arias :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Daniel Engel]

Daniel Engel  added the comment:

It reproduced on a windows machine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Emmanuel Arias]

Emmanuel Arias  added the comment:

I cannot reproduce on a Debian machine. Seems to be a Windows component issue?

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43253] asyncio open_connection fails when a socket is explicitly closed

[Daniel Engel]

New submission from Daniel Engel :

Python 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit 
(AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import socket
   ...: s1, s2 = socket.socketpair()
   ...: import asyncio
   ...: async def test():
   ...: r1, w1 = await asyncio.open_connection(sock=s1)
   ...: r2, w2 = await asyncio.open_connection(sock=s2)
   ...: s1.close()
   ...: asyncio.run(test())
Exception in callback 
_ProactorBasePipeTransport._call_connection_lost(ConnectionAbo...e, 1236, None))
handle: 
Traceback (most recent call last):
  File "c:\python39\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
  File "c:\python39\lib\asyncio\proactor_events.py", line 162, in 
_call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
OSError: [WinError 10038] An operation was attempted on something that is not a 
socket

--
components: asyncio
messages: 387228
nosy: asvetlov, danielen1337, yselivanov
priority: normal
severity: normal
status: open
title: asyncio open_connection fails when a socket is explicitly closed
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com