[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Guido van Rossum]

Guido van Rossum  added the comment:

Looks like there may be a new plan where we solve a smaller problem (overloads) 
in the context of typing only.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Jelle Zijlstra]

Change by Jelle Zijlstra :


--
keywords: +patch
pull_requests: +29832
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31716

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Alex Waygood]

Alex Waygood  added the comment:

The latest plan sounds good to me. I have some Thoughts on the proposed API, 
but it will be easier to express those as part of a PR review. Looking forward 
to seeing the PR!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

We could make my proposed overload registry more reusable by putting it in a 
different module, probably functools. (Another candidate is inspect, but 
inspect.py imports functools.py, so that would make it difficult to use the 
registry for functools.singledispatch.)

We could then bill it as a "variant registry", with an API like this:

def register_variant(key: str, variant: Callable) -> None: ...
def get_variants(key: str) -> list[Callable]: ...
def get_key_for_callable(callable: Callable) -> str | None: ...

@overload could then call register_variant() to register each overload, and 
code that wants a list of overloads (pydoc, inspect.signature, runtime type 
checkers) could call get_variants().

get_key_for_callable() essentially does 
f"{callable.__qualname__}.{callable.__name__}", but returns None for objects it 
can't handle. It will also support at least classmethods and staticmethods.

I will prepare a PR implementing this idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Alex Waygood]

Alex Waygood  added the comment:

I'd dearly like better introspection tools for functions decorated with 
@overload, but I'd rather have a solution where:

- inspect.signature doesn't have to import typing. That doesn't feel worth it 
for users who aren't using typing.overload, but inspect.signature would have to 
import typing whether or not @overload was being used, in order to *check* 
whether @overload was being used.
- The solution could be reused by, and generalised to, other kinds of functions 
that have multiple signatures.

If we create an __overloads__ dunder that stored the signatures of 
multi-signature functions, as Raymond suggests, inspect.signature could check 
that dunder to examine whether the function is a multi-dispatch signature, and 
change its representation of the function accordingly. This kind of solution 
could be easily reused by other parts of the stdlib, like 
@functools.singledispatch, and by third-party packages such as plum-dispatch, 
multipledispatch, and Nikita's dry-python/classes library.

So, while it would undoubtedly be more complex to implement, I much prefer 
Raymond's suggested solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +sobolevn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Spencer Brown]

Spencer Brown  added the comment:

Had a potential thought. Since the only situation we care about is overload 
being used on function definitions in lexical order, valid calls are only that 
on definitions with ascending co_firstlineno counts. Expanding on Jelle's 
solution, the overload() decorator could compare the current function's line 
number to the first in the list, and if it's <= clear out the list (we're 
re-defining). Then repeated re-definitions wouldn't duplicate overloads. 

The other change I'd suggest is to make get_overloads_for() first check 
__overloads__, then only if not present pop from the _overloads dict and assign 
to that attribute. That way if code calls get_overloads_for() at least once, 
the function will be referring to the actual overloads created at the same 
time. It'd also get garbage collected then when the function dies. It also 
means you could manually assign to add overloads to any callable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

I'm OK with not fully supporting overloads created in nested functions; that's 
a pretty marginal use case. But it's true that my proposed implementation would 
create a memory leak if someone does do that. I don't immediately see a way to 
fix that with weakrefs. Maybe we need to put something in the defining 
namespace, as Raymond suggested.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Spencer Brown]

Spencer Brown  added the comment:

I'm not sure a get_overloads() function potentially called after the fact would 
fully work - there's the tricky case of nested functions, where the overload 
list would need to be somehow cleared to ensure every instantiation doesn't 
endlessly append to the same list. It's probably also desirable to weakref it 
(or make it an attribute) so they can be decrefed if the function isn't being 
used.

--
nosy: +Spencer Brown

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Guido van Rossum]

Guido van Rossum  added the comment:

Sounds good to me. (I don’t care what happens at runtime but I want to
support the folks who do.)--
--Guido (mobile)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

I made a similar suggestion in issue46821 (thanks Alex for pointing me to this 
older issue):

Currently, the implementation of @overload 
(https://github.com/python/cpython/blob/59585d6b2ea50d7bc3a9b336da5bde61367f527c/Lib/typing.py#L2211)
 simply returns a dummy function and throws away the decorated function. This 
makes it virtually impossible for type checkers using the runtime function 
object to find overloads specified at runtime.

In pyanalyze, I worked around this by providing a custom @overload decorator, 
working something like this:

_overloads: dict[str, list[Callable]] = {}

def _get_key(func: Callable) -> str:
return f"{func.__module__}.{func.__qualname__}"

def overload(func):
key = _get_key(func)
_overloads.setdefault(key, []).append(func)
return _overload_dummy

def get_overloads_for(func):
key = _get_key(func)
return _overloads.get(key, [])

A full implementation will need more error handling.

I'd like to add something like this to typing.py so that other tools can also 
use this information.

---

With my suggested solution, help() would need to call 
typing.get_overloads_for() to get any overloads for the function. Unlike 
Raymond's suggestion, we would not need to change the function creation 
machinery.

--
nosy: +Jelle Zijlstra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Note, I'm not proposing a create_function() decorator.  That is just for the 
proof of concept.  The actual logic would go into normal function creation, the 
same place that __annotations__ gets added.

Also, there may be a better place than func.__globals__ to accumulate the 
overloads.  For the proof-of-concept, it was just the easiest way to go.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45100] Improve help() by making typing.overload() information accessible at runtime

[Raymond Hettinger]

Change by Raymond Hettinger :


--
title: Teach help about typing.overload() -> Improve help() by making 
typing.overload() information accessible at runtime
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com