[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

Superseded by https://www.python.org/dev/peps/pep-0575/

--
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

See https://mail.python.org/pipermail/python-ideas/2018-March/049398.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

> So I expect that the case you 'care most about' already works.

Yes, it works. That's the most ironic part of this issue: getfullargspec(func) 
works but packages like Sphinx will not call getfullargspec(func) because they 
do not detect that "func" is actually a function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Terry J. Reedy]

Terry J. Reedy added the comment:

inspect.getargspec is deprecated in favor of .getfullargspec and .signature and 
is implemented in with .getfullargspec.  This, in turn, calls 
._signature_from_callable which ultimately looks for (perhaps after recursive 
unwrap calls) obj.__signature__.  So I expect that the case you 'care most 
about' already works.  True?

It appears that .signature is intended to work for cython functions via the 
following helper function. Its code is somewhat awkward and tests that the 
object has needed attributes with needed types.

def _signature_is_functionlike(obj):
"""Private helper to test if `obj` is a duck type of FunctionType.
A good example of such objects are functions compiled with
Cython, which have all attributes that a pure Python function
would have, but have their code statically compiled.
"""

That does leave cases like the inspect.getfile code you quoted.  It could be 
fixed with some fiddly code, but there would still be .getclosurevariables and 
a couple of other uses of isfunction to review.

I reviewed the function and code attributes listed in
https://docs.python.org/3/library/inspect.html#types-and-members
and I think the necessary differences a function compiled by CPython and 
anything else are limited to the code object.

Proposal: for a cleaner solution, define a 'mincode' base class that lacks, for 
instance, co_code, co_consts, co_flags, co_lnotab, and co_stacksize.  Make code 
a subclass of this.  Define 'minfunction' as a function whose __code__ is a 
mincode.  Make function a subclass of this.  Define 'isminfunction' and replace 
'isfunction' where a mincode is sufficient.  This might allow, for instance, 
_signature_is_functionlike to be removed.

Details should perhaps be specified in a relatively short PEP.  Discussion 
could maybe continue on python-ideas.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

For the record: the __code__ attribute of a Cython function is a real "code" 
object (the same type as the __code__ attribute of a Python function). Of 
course not all fields are relevant, for example co_code is empty.

So I think it's clear that Cython tries really hard to be compatible with 
Python functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

> As indicated above, perfect emulation seems impossible for Cython or any 
> other external compiler that does not use the same bytecode.

True, Cython functions are not implemented using Python bytecode, so perfect 
emulation is impossible. The use case I care most about is getargspec(), which 
is fully supported by Cython functions.

> If it were possible for Cython to makes its CythonFunction class a subclass 
> of FunctionType, the issue would be 'solved', though the incompatibilities 
> would remain.

That's an interesting idea. Currently, that is simply impossible because

>>> from types import FunctionType
>>> class X(FunctionType): pass
Traceback (most recent call last):
  File "", line 1, in 
TypeError: type 'function' is not an acceptable base type

Still, one could argue to change the implementation of FunctionType. If you do 
that, it would be best to define a BaseFunctionType and then have Cython 
functions and Python functions inherit from that. Personally, I think that's an 
even better but much more involved solution (I guess it would require a PEP).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Terry J. Reedy]

Terry J. Reedy added the comment:

inspect.isfunction(object) is documented to
Return true if the object is a Python function, which includes functions 
created by a lambda expression.

This is currently implemented as "isinstance(object, types.FunctionType)".

The docs usually regard a 'Python function' as the result of a def statement or 
lambda expression.  The inspect doc says that a function includes a particular 
set of attributes.  One of them is a code object with its own fairly extensive 
set of attributes.  Some of them are derived from the Python code.  But others, 
in particular co_code, are specific to the current CPython bytecode version.  
(And co_code is intentionally writable.)

To me, the main purpose of checking that something is a function, as opposed to 
just being callable, is to know whether one can dependably access the 
attributes.  Given that some are inherently CPython specific, including objects 
compiled by third-party software seems dubious.  (There is also the issue of 
not being able to test with 3rd party objects.)

The referenced cython doc says
"""While it is quite possible to emulate the interface of functions in Cython’s 
own function type, and recent Cython releases have seen several improvements 
here,"""

To me, this implies to me that Cython function (compiled from Cython's extended 
def statements) do not yet perfectly emulate (fulfill) 'Python functions'.  As 
indicated above, perfect emulation seems impossible for Cython or any other 
external compiler that does not use the same bytecode.

"""the “inspect” module does not consider a Cython implemented function a 
“function”, because it tests the object type explicitly instead of comparing an 
abstract interface or an abstract base class. This has a negative impact on 
code that uses inspect to inspect function objects, but would require a change 
to Python itself."""

Where the current situation would be annoying is if working code uses 
isfunction and then Cython is used to speed up the code.  But Cython could 
supply, if it does not now, expanded functions along with the list of 
cyfunction attributes and an indication of which are compatible with CPython 
function attributes.

Cython is not the only 3rd party compiler, and not the only one that might ever 
be linkable to CPython.  So any change to CPython should not be limited to 
Cython.

If it were possible for Cython to makes its CythonFunction class a subclass of 
FunctionType, the issue would be 'solved', though the incompatibilities would 
remain.

--
nosy: +terry.reedy
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

At the very least, the inspect module should use more duck-typing internally. 
For example, consider this code from "getfile":

if ismethod(object):
object = object.__func__
if isfunction(object):
object = object.__code__
if istraceback(object):
object = object.tb_frame
if isframe(object):
object = object.f_code
if iscode(object):
return object.co_filename

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

__code__ is not enough for quacking as a function. Different code can expect 
other function attributes (for example __name__, __qualname__ and __module__).

See also issue8488. inspect.isroutine() and inspect.ismethoddescriptor() return 
True for some descriptors, but they don't quack good enough for pydoc.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[R. David Murray]

R. David Murray added the comment:

The python standard library makes extensive use of duck typing.  Duck typing is 
a pretty fundamental part of the design of Python, IMO.  Even the ABC module 
does a bunch of duck typing, rather than requiring strict subclassing or 
registration.

I think the request is valid, and it is mostly a matter of agreeing on the best 
way to identify function ducks.  (I agree that Steven's example is 
intentionally trying to quack like a duck and so is not, IMO, a valid counter 
argument against using __code__).  I doubt we would make such a change in 
anything except a feature release, though. 

Let's see what other devs besides Steven and I think.

--
nosy: +r.david.murray
type:  -> enhancement
versions:  -Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

> If inspect reports something is a duck, it should be an actual duck, not just 
> something that quacks.

The problem is that some Python packages (Sphinx and IPython for example) 
really need to know whether it quacks. And the only tool they have is 
inspect.isfunction(), so they use that. It's silly that every single package 
using inspect.isfunction() should be fixed. Better just fix 
inspect.isfunction().

>>> from types import SimpleNamespace
>>> x = SimpleNamespace(__code__=1, spam=2)
>>> '__code__' in dir(x)

Of course, you can always break stuff. User code is not supposed to invent new 
__dunder__ special names.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Steven D'Aprano]

Steven D'Aprano added the comment:

Duck typing is not something that "Python" does, it is a style of programming 
done by Python programmers. You wouldn't expect isinstance() to try to "duck 
type", and likewise the inspect module should be precise about what it is 
inspecting. If inspect reports something is a duck, it should be an actual 
duck, not just something that quacks.

I'm not sure that the CPython inspect module should care about Cython objects. 
I don't think that Cython functions should count as Python functions, I think 
they are different kinds of callables.

But even if we decide that Cython function should be recognised by 
inspect.isfunction(), I don't think your patch is the right way to deal with 
it. Not every object with a __code__ attribute is a function.

py> from types import SimpleNamespace
py> x = SimpleNamespace(__code__=1, spam=2)
py> '__code__' in dir(x)
True

Your patch would wrongly detect x as a function when it isn't even callable.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

Changes by Jeroen Demeyer :


--
keywords: +patch
Added file: http://bugs.python.org/file46803/isfunction.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30071] Duck-typing inspect.isfunction()

[Jeroen Demeyer]

New submission from Jeroen Demeyer:

Python is supposed to encourage duck-typing, but the "inspect" module doesn't 
follow this advice. A particular problem is that Cython functions are not 
recognized by the inspect module to be functions: 
http://cython.readthedocs.io/en/latest/src/userguide/limitations.html#inspect-support

--
components: Library (Lib)
messages: 291647
nosy: jdemeyer, scoder
priority: normal
severity: normal
status: open
title: Duck-typing inspect.isfunction()
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com