[issue43253] asyncio open_connection fails when a socket is explicitly closed
Maximilian Hils added the comment: Thank you for the super quick turnaround. My heart goes out to you and everyone else in Ukraine. -- ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43253] asyncio open_connection fails when a socket is explicitly closed
Change by Maximilian Hils : -- pull_requests: +29990 pull_request: https://github.com/python/cpython/pull/31892 ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47014] ProactorEventLoop ignores Ctrl+C after closing unrelated loop
New submission from Maximilian Hils : When a (second) ProactorEventLoop is garbage-collected, the current ProactorEventLoop starts to ignore Ctrl+C on Windows until it is woken up. The attached repro shows a minimal example. Uncommenting the `create_task` call or not using a second event loop fixes the behavior. I couldn't find any shared/global objects in asyncio/windows_events.py, so I'm not sure what is causing this, or why the wakeup task helps. FWIW, the second event loop in our code base originated from a stray `get_event_loop()` call, one more example that speaks in favor of `get_running_loop()`! -- components: asyncio files: proactor-repro.py messages: 415169 nosy: asvetlov, mhils, yselivanov priority: normal severity: normal status: open title: ProactorEventLoop ignores Ctrl+C after closing unrelated loop type: behavior versions: Python 3.10 Added file: https://bugs.python.org/file50675/proactor-repro.py ___ Python tracker <https://bugs.python.org/issue47014> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43253] asyncio open_connection fails when a socket is explicitly closed
Maximilian Hils added the comment: asvetlov: Sorry if I articulated myself badly, but I do think this is a valid bug. It's unfortunately hard to provide a better repro (I tried), but we are hitting this regularly when mitmproxy is accepting connections under heavy load. We're just calling `asyncio.start_server(handler, "127.0.0.1", 8080)` in mitmproxy and never interact with the underlying socket object. Here are some observations that are true for all crashes: - The socket fileno is -1 when it crashes. - `_call_connection_lost` is called by `_ProactorBasePipeTransport.close`, which is called by `_ProactorBasePipeTransport.__del__` [1] - There are no previous calls to `_call_connection_lost`. - Windows only, loopback connections in our case. - Wireshark shows that client and server are first happily exchanging packets. At some point the client sends a FIN, which the Python server ACKs immediately. A few seconds later the Python server sends a FIN back. An obvious fix without understanding the root cause would be to check fileno in https://github.com/python/cpython/blob/d929aa70e2a324ea48fed221c3257f929be05115/Lib/asyncio/proactor_events.py#L161. I'm not too familar with proactor to assess if that is a good idea. Sorry for not being able to provide more details. [1] https://github.com/python/cpython/blob/d929aa70e2a324ea48fed221c3257f929be05115/Lib/asyncio/proactor_events.py#L102-L116 -- ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43253] asyncio open_connection fails when a socket is explicitly closed
Change by Maximilian Hils : -- versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43253] asyncio open_connection fails when a socket is explicitly closed
Maximilian Hils added the comment: We are hitting the same traceback with mitmproxy on Windows without ever passing a socket object to open_connection. In other words, this can be triggered without performing any action on 'sock' due to some race conditions in the Windows network stack. I unfortunately don't have a better repro for that race other than what Daniel provided. -- nosy: +mhils ___ Python tracker <https://bugs.python.org/issue43253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40124] Clearer assertion error
Maximilian Hils added the comment: I'm pretty sure this issue is a duplicate of bpo-issue29930. -- nosy: +mhils versions: +Python 3.10 ___ Python tracker <https://bugs.python.org/issue40124> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44926] typing.get_type_hints() raises for type aliases with forward references
Change by Maximilian Hils : -- keywords: +patch pull_requests: +26317 stage: -> patch review pull_request: https://github.com/python/cpython/pull/27859 ___ Python tracker <https://bugs.python.org/issue44926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44926] typing.get_type_hints() raises for type aliases with forward references
Maximilian Hils added the comment: Thanks Guido! I agree on not pursuing the List["Foo"] case for the reasons you mentioned. Let me know if you think it'd be useful to mention this limitation briefly in one of the relevant PEPs or somewhere else. I'm not sure if it meets the bar for notability, you probably have a better gut feeling for this. Other than that I'd propose we close this here as wontfix. Thank you again for the very useful feedback! :) -- ___ Python tracker <https://bugs.python.org/issue44926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44926] typing.get_type_hints() raises for type aliases with forward references
Maximilian Hils added the comment: Thank you Ken and Guido for the super quick feedback! > It's not fine if it were to look in the wrong namespace That's a really good point. Here's another terrible example: foo.py: ``` import typing FooData: typing.TypeAlias = "Data" class Data: pass ``` bar.py: ``` import typing import foo BarData: typing.TypeAlias = "Data" def func1(x: foo.FooData, y: BarData): pass class Data: pass print(typing.get_type_hints(func1)) # incorrectly uses bar.Data twice. ``` I don't see how we could distinguish FooData from BarData without static analysis. Changing get_type_hints to not pick the unrelated class with the same name would essentially mean not using func.__globals__, which breaks almost all ForwardRef evaluation. This doesn't seem viable. Potential alternatives: 1. Accept the current state: get_type_hints does not work well with type aliases that use forward references. 2. Fix it at least for cases where a ForwardRef is constructed immediately (typing.List["Foo"], but not "Foo" or list["Foo"]), similar to how it's been done for #41249. I have made a very basic proof-of-concept at https://github.com/mhils/cpython/commit/4adbcf088d2857166b579f7dd2954ff9981fc7db, but it's a mess (see commit comments). 3. Deprecate/discourage use of forward references in type aliases and/or change the syntax for [forward references in] type aliases so that their origin can be tracked. In summary, it seems like there are no really good solutions here. I'm fine if this ends up as a pragmatic wontfix, maybe with a comment added somewhere in the docs or in PEP 613. -- ___ Python tracker <https://bugs.python.org/issue44926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44587] argparse BooleanOptionalAction displays default=SUPPRESS unlike other action types
Change by Maximilian Hils : -- keywords: +patch nosy: +mhils nosy_count: 2.0 -> 3.0 pull_requests: +26259 stage: -> patch review pull_request: https://github.com/python/cpython/pull/27672 ___ Python tracker <https://bugs.python.org/issue44587> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44926] typing.get_type_hints() raises for type aliases with forward references
Maximilian Hils added the comment: @Guido van Rossum: Yes, GH-27017 is the same as #41249 in the initial post. There are also some cases where we don't even have a ForwardRef though: foo.py: ``` import typing FooType: typing.TypeAlias = "Foo" class Foo: pass ``` bar.py: ``` import typing import foo def func3(x: foo.FooType): pass print(typing.get_type_hints(func3)) # NameError: name 'Foo' is not defined. ``` In this example, `FooType` is just a regular str (with unknown origin). That's a case where get_type_hints just needs to give up? -- ___ Python tracker <https://bugs.python.org/issue44926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44926] typing.get_type_hints() raises for type aliases with forward references
Maximilian Hils added the comment: > For your specific use case (where the user is using Python 3.6), you could > pass in globalns and localns to get_type_hints as a temporary workaround. Off > the top of my head: get_type_hints(func2, globalns=foo.__dict__) might work. Would that work for your library? I guess the hard part is knowing that the type annotation comes from `foo`. In the example here we can of course hardcode it, but that doesn't work in the general case or for pdoc, the documentation generator I'm working on (https://pdoc.dev). I have experimented quite a bit with walking the AST to figure out where type aliases are imported from to then re-executing ForwardRefs with that globalns. Long story short, trying to reverse-engineer __forward_module__ quickly becomes a tangled hot mess where you need to adjust for import aliases, reimports, and so on. -- ___ Python tracker <https://bugs.python.org/issue44926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44926] typing.get_type_hints() raises for type aliases with forward references
New submission from Maximilian Hils :
Someone reported this rather interesting issue where typing.get_type_hints
crashes on type aliases with forward references. The original report is at
https://github.com/mitmproxy/pdoc/issues/290. Here's an extended minimal
example:
foo.py:
```
import typing
FooList1: typing.TypeAlias = list["Foo"]
FooList2: typing.TypeAlias = typing.List["Foo"]
class Foo:
pass
```
bar.py:
```
import typing
import foo
def func1(x: foo.FooList1):
pass
def func2(x: foo.FooList2):
pass
print(typing.get_type_hints(func1)) # {'x': list['Foo']}
print(typing.get_type_hints(func2)) # NameError: name 'Foo' is not defined.
```
Observations:
1. func1 doesn't crash, but also doesn't resolve the forward reference. I am
not sure if this expected behavior.
If it isn't, this should eventually run in the same problem as func2.
2. func2 crashes because "Foo" is evaluated in the context of bar.py (where
class Foo is unknown) and not in the context of foo.py. ForwardRef._evaluate
would
somehow need to know in which context it was defined. #41249 (TypedDict
inheritance doesn't work with get_type_hints) introduced
ForwardRef.__forward_module__, which would be a logical place for that
information. I'm not sure if it is a good idea
to use __forward_module__ more widely.
3. This may end up as quite a bit of complexity for an edge case, I'm fine if
it is considered wontfix.
The reason I'm bringing it up is that PEP 613 (Explicit Type Aliases)
decidedly allows forward references in type aliases.
For the record, PEP 563 (postponed evaluations) does not change the outcome
here. However, postponed evaluations often make it possible to avoid the
forward references by declaring the aliases last.
--
components: Library (Lib)
messages: 399660
nosy: mhils
priority: normal
severity: normal
status: open
title: typing.get_type_hints() raises for type aliases with forward references
type: crash
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue44926>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38956] argparse.BooleanOptionalAction should not add the default value to the help string by default
Change by Maximilian Hils : -- nosy: +mhils nosy_count: 5.0 -> 6.0 pull_requests: +26160 pull_request: https://github.com/python/cpython/pull/27672 ___ Python tracker <https://bugs.python.org/issue38956> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6926] socket module missing IPPROTO_IPV6, IPPROTO_IPV4
Maximilian Hils added the comment: Thanks for the insanely quick feedback. I still think this is a bug, so I filed https://bugs.python.org/issue29515 with additional details. Let's continue there. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue6926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29515] socket module missing IPPROTO_IPV6, IPPROTO_IPV4 on Windows
New submission from Maximilian Hils: The latest Windows builds for Python 3.5.3/3.6.0 do not export socket.IPPROTO_IPV6, even though e.g. socket.IPV6_V6ONLY is exported. This seems to be wrong to me as IPV6_V6ONLY requires the corresponding socket option level IPPROTO_IPV6 to be actually useful. The same issue at least also applies to IPPROTO_IPV4. christian.heimes mentioned that this is intended behaviour in https://bugs.python.org/issue6926 as Windows would not define the constants. Now I am all but an expert on this, but it seems to me that IPPROTO_IPV6 is defined in Windows: https://msdn.microsoft.com/en-us/library/windows/hardware/ff543746(v=vs.85).aspx. Apologies if I'm missing something here. Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)] on win32 >>> import socket; socket.IPPROTO_IPV6 Traceback (most recent call last): File "", line 1, in AttributeError: module 'socket' has no attribute 'IPPROTO_IPV6' As a workaround, IPPROTO_IPV6 can be substituted with the hardcoded constant 41. -- components: Windows messages: 287449 nosy: christian.heimes, mhils, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: socket module missing IPPROTO_IPV6, IPPROTO_IPV4 on Windows type: behavior versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6926] socket module missing IPPROTO_IPV6, IPPROTO_IPV4
Maximilian Hils added the comment: This still seems to be an issue with the offical Python 3.6 builds on Windows 10: Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)] on win32 >>> import socket >>> socket.IPPROTO_IPV6 Traceback (most recent call last): File "", line 1, in AttributeError: module 'socket' has no attribute 'IPPROTO_IPV6' -- nosy: +mhils ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue6926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
