Would a property or a copy be faster for existing and possible use cases? In practice, how frequently will __qual/name__ be called on partials?
- Copying __qual/name__ would definitely be a performance regression - There are probably as many use cases for partials as other methods of functional composition, - __qual/name__ support is not yet extant - it's faster to run e.g. a grid search *without* partials, due to function call overhead, due to scope allocation on the stack in stackful pythons [1] [1] Hyper Parameter Search > Scaling hyperparameter searches https://ml.dask.org/hyper-parameter-search.html#scaling-hyperparameter-searches [2] Pipeline caching in TPOT http://epistasislab.github.io/tpot/using/#pipeline-caching-in-tpot #parallel-training-with-dask ; TPOT generates actual python source code instead of an ensemble of partials On Tue, Aug 30, 2022, 12:07 PM Charles Machalow <csm10...@gmail.com> wrote: > We may be able to do __name__/__qualname__ as a property to make it > evaluate when called as opposed to computed once on creation. That way we > just work with .func upon call so no need for extra references, etc. > > As for documentation generation tools, it may be different at first, > though I believe the existing ispartial checks would catch partials still. > If they want to (in a new version) swap to using __name__/__qualname__ that > should be fine, but this likely wouldn't inherently break existing tools. > > - Charlie Scott Machalow > > > On Mon, Aug 29, 2022 at 11:08 PM Wes Turner <wes.tur...@gmail.com> wrote: > >> Is there a non-performance regressive way to proxy attr access to >> func.__name__ of the partial function (or method; Callable)? >> >> Would this affect documentation generation tools like e.g. sphinx-spidoc, >> which IIRC use __name__ and probably now __qualname__ for generating >> argspecs in RST for HTML and LaTeX? >> >> >> - https://docs.python.org/3/library/inspect.html >> - functions and methods have __name__ and __qualname__ >> - see: sphinx.utils.inspect >> >> - https://docs.python.org/3/library/functools.html#functools.partial >> - >> https://docs.python.org/3/library/functools.html#functools.partialmethod >> - https://docs.python.org/3/library/functools.html#partial-objects >> >> > partial Objects¶ >> > partial objects are callable objects created by partial(). They have >> three read-only attributes: >> > >> > partial.func >> > A callable object or function. Calls to the partial object will be >> forwarded to func with new arguments and keywords. >> > >> > partial.args >> > The leftmost positional arguments that will be prepended to the >> positional arguments provided to a partial object call. >> > >> > partial.keywords >> > The keyword arguments that will be supplied when the partial object is >> called. >> >> > partial objects are like function objects in that they are callable, >> weak referencable, and can have attributes. There are some important >> differences. For instance, the __name__ and __doc__ attributes are not >> created automatically. Also, partial objects defined in classes behave like >> static methods and do not transform into bound methods during instance >> attribute look-up. >> >> >> - https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html >> - https://www.sphinx-doc.org/en/master/_modules/sphinx/ext/autodoc.html >> : 18 references to __qualname__, >> >> >> https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L49-L66 >> : >> >> ```python >> def unwrap_all(obj: Any, *, stop: Optional[Callable] = None) -> Any: >> """ >> Get an original object from wrapped object (unwrapping partials, >> wrapped >> functions, and other decorators). >> """ >> while True: >> if stop and stop(obj): >> return obj >> elif ispartial(obj): >> obj = obj.func >> elif inspect.isroutine(obj) and hasattr(obj, '__wrapped__'): >> obj = obj.__wrapped__ # type: ignore >> elif isclassmethod(obj): >> obj = obj.__func__ >> elif isstaticmethod(obj): >> obj = obj.__func__ >> else: >> return obj >> ``` >> >> From >> https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/util/inspect.py#L173-L186 >> : >> >> ```python >> def unpartial(obj: Any) -> Any: >> """Get an original object from partial object. >> This returns given object itself if not partial. >> """ >> while ispartial(obj): >> obj = obj.func >> >> return obj >> >> >> def ispartial(obj: Any) -> bool: >> """Check if the object is partial.""" >> return isinstance(obj, (partial, partialmethod)) >> ``` >> >> - https://github.com/sphinx-doc/sphinx/issues/4826#issuecomment-808699254 >> >> >> https://docs.python.org/3/library/stdtypes.html#definition.__name__ >> >> >> https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy >> > Callable types > User defined functions does list both __name__ and >> __qualname__ >> >> Is there a non-performance regressive way to proxy attr access to >> __name__ of the partially curried (?) function? >> >> From "PEP 3155 – Qualified name for classes and functions" >> https://peps.python.org/pep-3155/#limitations : >> >> > ### Limitations >> > With nested functions (and classes defined inside functions), the >> dotted path will not be walkable programmatically as a function’s namespace >> is not available from the outside. It will still be more helpful to the >> human reader than the bare __name__. >> > >> > As the __name__ attribute, the __qualname__ attribute is computed >> statically and it will not automatically follow rebinding. >> >> From >> https://wrapt.readthedocs.io/en/latest/wrappers.html#proxy-object-attributes >> : >> >> > Proxy Object Attributes >> > When an attempt is made to access an attribute from the proxy, the same >> named attribute would in normal circumstances be accessed from the wrapped >> object. When updating an attributes value, or deleting the attribute, that >> change will also be reflected in the wrapped object. >> >> From https://docs.python.org/3/library/weakref.html#weakref.proxy : >> >> > weakref.proxy(object[, callback])¶ >> > Return a proxy to object which uses a weak reference. This supports use >> of the proxy in most contexts instead of requiring the explicit >> dereferencing used with weak reference objects. The returned object will >> have a type of either ProxyType or CallableProxyType, depending on whether >> object is callable. Proxy objects are not hashable regardless of the >> referent; this avoids a number of problems related to their fundamentally >> mutable nature, and prevent their use as dictionary keys. callback is the >> same as the parameter of the same name to the ref() function. >> >> On Tue, Aug 30, 2022, 1:14 AM Charles Machalow <csm10...@gmail.com> >> wrote: >> >>> 1: There are cases where one may need the __name__/__qualname__ of a >>> given callable. If someone uses partial to create a new callable, there is >>> no __name__/__qualname__ given. In my particular case, I'm logging what >>> callback function is passed to a different function... if someone uses >>> partial, there is no __name__/__qualname__ which leads to a current >>> traceback... of course i can work around it but still was an odd case to me. >>> >>> Per the docs on functools.partial: >>> "Return a new partial object which when called will behave like func >>> called with the positional arguments args and keyword arguments keywords" >>> ... which made me initially think that in order to behave like the >>> passed in function: it should have __name__ and __qualname__... like the >>> func did. >>> >>> 2: I would say have both __qualname__ and __name__. Both could be based >>> off of __name__/__qualname__ of the passed in func. >>> >>> 3: This would be more difficult since you would have to disassemble the >>> lambda to figure out the called method (or methods)... We can table the >>> lambda discussion for the purpose of this idea. I recall that typically it >>> is preferred to use partial over lambdas, so this could be an additional >>> functionality/benefit of using partial over lambda. >>> >>> Notes: >>> ... __name__ being something like partial(foo, "x") would be fine with >>> me... I just feel as though something should be there. >>> >>> - Charlie Scott Machalow >>> >>> >>> On Mon, Aug 29, 2022 at 9:56 PM Paul Bryan <pbr...@anode.ca> wrote: >>> >>>> +0 >>>> >>>> Questions: >>>> >>>> 1. What's the use case for partial having __name__? >>>> 2. Does this imply it should have __qualname__ as well? >>>> 3. What name would be given to (an inherently anonymous) lambda? >>>> >>>> Notes: >>>> >>>> 1. I would prefer __name__ to be more qualifying like its repr (e.g. >>>> partial(foo, "x") → "<partial foo>") >>>> >>>> >>>> On Mon, 2022-08-29 at 21:31 -0700, Charles Machalow wrote: >>>> >>>> Hey folks, >>>> >>>> I propose adding __name__ to functools.partial. >>>> >>>> >>> get_name = functools.partial(input, "name: ") >>>> >>> get_name() >>>> name: hi >>>> 'hi' >>>> >>> get_name.__name__ >>>> Traceback (most recent call last): >>>> File "<stdin>", line 1, in <module> >>>> AttributeError: 'functools.partial' object has no attribute '__name__' >>>> >>> get_name.func >>>> <built-in function input> >>>> >>> get_name.func.__name__ >>>> 'input' >>>> >>>> We could set __name__ based off of partial.func.__name__ or we could >>>> try to set it to something like 'partial calling func.__name__' >>>> >>>> If the callable doesn't have a name, we could fall back to a None >>>> __name__ or set it to something generic. >>>> >>>> Even lambdas have __name__ set: >>>> >>>> >>> l = lambda: input('name: ') >>>> >>> l.__name__ >>>> '<lambda>' >>>> >>>> This proposal makes __name__ on partial objects more useful than the >>>> current behavior of __name__ on lambda objects as well. We could port over >>>> similar functionality to lambda if we'd like. >>>> >>>> - Charlie Scott Machalow >>>> _______________________________________________ >>>> Python-ideas mailing list -- python-ideas@python.org >>>> To unsubscribe send an email to python-ideas-le...@python.org >>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>>> Message archived at >>>> https://mail.python.org/archives/list/python-ideas@python.org/message/WK3FO357ORPVAD3XRUBRH6IHIYSPS3G2/ >>>> Code of Conduct: http://python.org/psf/codeofconduct/ >>>> >>>> >>>> _______________________________________________ >>> Python-ideas mailing list -- python-ideas@python.org >>> To unsubscribe send an email to python-ideas-le...@python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/ONAEBCIEJ4DJNWUNWE2ESJ6SBB4O7O6W/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >>
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YGARINTBW6W32V7YELNEX4VPD7YMPFMR/ Code of Conduct: http://python.org/psf/codeofconduct/
