[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Alright, now it makes sense. Thank you for writing a thoughtful response. -- ___ Python tracker

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: Ionel please give commenters the benefit of the doubt. In this case, Raymond is merely articulating our policy: if something is in pre-PEP stage we don't generally keep issues open in the tracker. We open issues for PEPs when they

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: It should be open for getting some visibility, as I need some help here - Raymond, I hope you can find a way to be hospitable here and stop with the kindergarten behavior. -- status: closed -> open

[issue23990] Callable builtin doesn't respect descriptors

Raymond Hettinger added the comment: Marking as closed. This can be reopened if a PEP is submitted and is favorably received. -- assignee: rhettinger -> status: open -> closed versions: +Python 3.7 -Python 3.5 ___

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Hello everyone, Is anyone still interested in fixing this bug and help with whatever PEP drafting was needed for convincing people? I would sketch up a draft but for me at least it's not clear what are the disadvantages of not

[issue23990] Callable builtin doesn't respect descriptors

Terry J. Reedy added the comment: Ionel, 8 core developers (an unusually large number for a tracker issue) have posted here and the closest thing to a consensus is that this is an enhancement request. Beyond that, opinions are scattered. No one is going to commit a patch in this

[issue23990] Callable builtin doesn't respect descriptors

Graham Dumpleton added the comment: If someone is going to try and do anything in the area of better proxy object support, I hope you will also look at all the work I have done on that before for wrapt (https://github.com/GrahamDumpleton/wrapt). Two related issue this has already found are

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: The BDFL would be the BDFL (Guido van Rossum ;) or whomever he asks to do the job. Don't worry too much about drafts. Pick one of the existing PEPs (409 is fairly short), and follow the same type of layouts. Make your arguments for what should be changed

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Okay ... is anyone interested in helping with this (reviewing drafts)? Who would be the BDFL? -- ___ Python tracker

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: I'm not against them, but there are many diverse opinions. A PEP will: - broaden discussion, so all (or at least most) pros and cons can be considered - highlight any other pieces of Python that would need to change to properly support proxies - get a

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, May 23, 2016 at 11:04 PM, Ethan Furman wrote: > `issubclass` doesn't "do the right thing", as evidenced by another dev. > Hmmm. Technically, in that case, the problem is in collections.abc.Iterable, not issubclass. I can

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: `issubclass` doesn't "do the right thing", as evidenced by another dev. Continued complaining to this issue is not going to get you what you want, but writing a PEP to address the issue might. -- ___ Python tracker

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, May 23, 2016 at 9:34 PM, Ethan Furman wrote: > "Add proxy support to builtins" and should address such things as > callable, issubclass, and whatever else is is appropriate. ​As previously stated this builtin is the only

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: To change this behavior at this point will require a PEP. The PEP should be something along the lines of "Add proxy support to builtins" and should address such things as callable, issubclass, and whatever else is is appropriate. As for working around your

[issue23990] Callable builtin doesn't respect descriptors

Changes by Ionel Cristian Mărieș : Added file: http://bugs.python.org/file42957/callable2.diff ___ Python tracker ___

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, Apr 20, 2015 at 5:08 PM, Eric Snow wrote: > FYI, I'll re-iterate something I said before, there is a different > approach you can take here if this is just an issue of proxying. Use two > different proxy types depending

[issue23990] Callable builtin doesn't respect descriptors

Changes by Graham Dumpleton : -- nosy: +grahamd ___ Python tracker ___ ___

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: The only 'consistency' fix that would make any sense, IMO, would be to disallow special methods to be descriptors. We can't do that for backward compatibility reasons, so that's pretty much case closed. Eric already mentioned one of the other 'capability'

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: My point was about consistency in descriptor handling, not consistency of fault (eg: broken everywhere). I don't understand why that's not clear here. The big idea here is to harmonize capability checking with descriptor handling. Justifying breakage in

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Also, descriptors are a core mechanism in new-style classes - you can't have methods without descriptors. Why would you even consider removing descriptors from the special method lookup if that's part of the object model design? On Monday, April 20,

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: I in case it wasn't clear, I closed this not because of my case closed statement, but because as Eric pointed out we *do* have consistency here: things which check *capabilities* (as opposed to actually *using* the special methods), like callable and

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, Apr 20, 2015 at 7:02 AM, Raymond Hettinger rep...@bugs.python.org wrote: AFAICT, there isn't a real problem here and the API for better-or-worse has proven to be usable in practice (the callable() API has been around practically forever and

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, Apr 20, 2015 at 4:50 AM, Eric Snow rep...@bugs.python.org wrote: We should simply leave callable alone (and consistent with the other helpers that inspect the special *capability* of objects). ​Which are the other helpers?​ Thanks, -- Ionel

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: Because special methods are special. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: FYI, I'll re-iterate something I said before, there is a different approach you can take here if this is just an issue of proxying. Use two different proxy types depending on if the proxied object is callable or not: class Proxy: # all the proxy stuff...

[issue23990] Callable builtin doesn't respect descriptors

Raymond Hettinger added the comment: My point was about consistency in descriptor handling, not consistency of fault (eg: broken everywhere). I don't understand why that's not clear here. That is clear but also isn't sufficient motivation. The proposed change is unnecessary and not rooted

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: Ionel Cristian Mărieș added the comment: Also, descriptors are a core mechanism in new-style classes - you can't have methods without descriptors. Why would you even consider removing descriptors from the special method lookup if that's part of the object model

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, Apr 20, 2015 at 5:42 PM, Raymond Hettinger rep...@bugs.python.org wrote: That is clear but also isn't sufficient motivation. The proposed change is unnecessary and not rooted in real use cases. It is a semantic change to a long-standing

[issue23990] Callable builtin doesn't respect descriptors

eryksun added the comment: To be consistent you'd have to do the attribute check in PyObject_Call as well, i.e. if an object isn't callable, then trying to call it should raise a TypeError. I think the cost can be mitigated by only checking heap types (i.e. tp_flags Py_TPFLAGS_HEAPTYPE). It

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: Just to be clear, I'm still -1 on any of this. On the one hand, there's a risk of backward-compatibility breakage (just as much a corner-case as the need expressed in this issue). On the other hand, I'd actually push for _PyObject_LookupSpecial to be fixed to

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: s/TypeError/RuntimeError/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: Ionel Cristian Mărieș added the comment: #1. It's specified/documented, therefore it's intended The first thing a maintainer does is check the docs. This is a sensible thing to do - as you cannot have all the details in your hear. The main question at that

[issue23990] Callable builtin doesn't respect descriptors

Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: - rhettinger nosy: +rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: What would be the right thing? My suggestion of using a metaclass is actually not effective here because __call__ has meaning for metaclasses. Otherwise you could have made __call__ more dynamic via a metaclass. Except that is another reason why my suggestion

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: I want to address the four main points of criticism in fixing this issue, just in case it's not clear why I think those lines of thought are wrong: #1. It's specified/documented, therefore it's intended The first thing a maintainer does is check the

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: Note that (in my mind, unfortunately) the pickle module looks up several dunder methods on instances. That isn't quite the same thing since the issue is about callable not triggering the descriptor protocol. However it is closely related. I point this out because

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: The right thing, using a meta-class, is to have the meta-class check if the proxied object is callable, and if so, put in the __call__ function in the class that is being created. -- ___ Python tracker

[issue23990] Callable builtin doesn't respect descriptors

Steven D'Aprano added the comment: This bug report seems to be completely based on a false premise. In the very first message of this issue, Ionel says: it return True even if __call__ is actually an descriptor that raise AttributeError (clearly not callable at all). but that is wrong. It

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Sun, Apr 19, 2015 at 9:01 PM, Eric Snow rep...@bugs.python.org wrote: Finally, instead of changing callable, why not use a metaclass that does the right thing? I believe MagicMock does something along those lines. ​What would be the right thing?

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Sun, Apr 19, 2015 at 10:01 PM, Ethan Furman rep...@bugs.python.org wrote: The right thing, using a meta-class, is to have the meta-class check if the proxied object is callable, and if so, put in the __call__ function in the class that is being

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: This is a different case from raising an AttributeError inside the __call__; class C(object): ... def __call__(self): ... raise AttributeError() ... hasattr(C(), '__call__') True class D(object): ... @property ... def __call__(self): ...

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Sun, Apr 19, 2015 at 10:29 PM, Steven D'Aprano rep...@bugs.python.org wrote: This bug report seems to be completely based on a false premise. In the very first message of this issue, Ionel says: it return True even if __call__ is actually an

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: Ionel Cristian Mărieș added the comment: What exactly are you proposing? Getting rid of AttributeError masking? That isn't really a practical thing to consider, so no. :) Instead I'm suggesting there isn't a lot of justification to change the behavior of

[issue23990] Callable builtin doesn't respect descriptors

Eric Snow added the comment: Ionel Cristian Mărieș added the comment: It's not. Did you see the example with iter()/__iter__? It does convert the AttributeError into a TypeError. callable and iter are not the same thing though. callable checks for a capability. iter invokes a capability.

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, Apr 20, 2015 at 2:59 AM, Eric Snow rep...@bugs.python.org wrote: However, that is consistent across Python and has been this way for a long time (so there are backward compatibility concerns that cannot be ignored). ​It's not. Did you see

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Mon, Apr 20, 2015 at 2:59 AM, Eric Snow rep...@bugs.python.org wrote: It not a problem currently for callable. It is one you are proposing to introduce. It is one which current users of callable don't have to worry about. Were do we draw the

[issue23990] Callable builtin doesn't respect descriptors

Raymond Hettinger added the comment: Unless there are some serious objections, I propose to close this on the basis of practicality beats purity (and as David Murray noted, there may not be a pure answer). Eric Snow's comments are dead-on. AFAICT, there isn't a real problem here and the API

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: This is exactly analogous to what you are seeing with __call__ and callable(). Your example is incorrect, __next__ is what makes an object iterable but not what makes an object have an iterator (what __iter__ does). This correctly characterises the

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Sat, Apr 18, 2015 at 7:23 PM, Ethan Furman rep...@bugs.python.org wrote: class GenericProxy: def __init__(self, proxied): self.proxied = proxied # in case proxied is an __iter__ iterator @property def

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: Perhaps callable() should be in the inspect module? ;) Speaking of which, how do all the is... functions there work with this descriptor implementation? -- ___ Python tracker rep...@bugs.python.org

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: They use isinstance, except for a couple that also check co_flags, and the ones that check if the object is a descriptor. I haven't thought this through fully, but I think this means that in general the descriptor protocol has been invoked or not by the

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Turns out I've replied through email, and code got mangled. This is the correct version: class GenericProxy: def __init__(self, proxied): self.proxied = proxied @property def __iter__(self): if not hasattr(self.proxied,

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: I am happy to be proven wrong. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list

[issue23990] Callable builtin doesn't respect descriptors

Christian Heimes added the comment: On 2015-04-18 19:43, Ionel Cristian Mărieș wrote: Ionel Cristian Mărieș added the comment: On Sat, Apr 18, 2015 at 8:35 PM, Christian Heimes rep...@bugs.python.org wrote: You also haven't shown that this behavior violates the documentation and

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: I understand Ionel's point, and it is indeed 'callable' that is the outlier here. It only looks for the *existence* of the attribute, rather than actually retrieving it through the descriptor protocol (and therefore getting the AttributeError from the

[issue23990] Callable builtin doesn't respect descriptors

Christian Heimes added the comment: All major Python implementation have a mutual agreement that callable() just checks for a __call__ member on the type. You also haven't shown that this behavior violates the documentation and language spec. The check for existence of __call__ on the type is

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: Your example shows /having/ an iterator, while mine is /being/ an iterator. A simple iterator: # iterator protocol class uc_iter(): def __init__(self, text): self.text = text self.index = 0 def __iter__(self): return

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: On Sat, Apr 18, 2015 at 8:35 PM, Christian Heimes rep...@bugs.python.org wrote: You also haven't shown that this behavior violates the documentation and language spec. ​How can I show it violates the spec when there's not such thing? :-) AFAIK,

[issue23990] Callable builtin doesn't respect descriptors

R. David Murray added the comment: Oops, I accidentally changed the bug status due to not refreshing before I posted. -- resolution: not a bug - stage: resolved - status: closed - open versions: +Python 3.5 -Python 2.7, Python 3.2, Python 3.3, Python 3.4

[issue23990] Callable builtin doesn't respect descriptors

New submission from Maries Ionel Cristian: It appears that callable doesn't really care for the descriptor protocol, so it return True even if __call__ is actually an descriptor that raise AttributeError (clearly not callable at all). Eg: Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015,

[issue23990] Callable builtin doesn't respect descriptors

Maries Ionel Cristian added the comment: For context this respects the descriptor protocol: a() Traceback (most recent call last): File stdin, line 1, in module File stdin, line 4, in __call__ AttributeError: go away Mind you, this is legal use: class B: ... @property ... def

[issue23990] Callable builtin doesn't respect descriptors

Changes by Joe Jevnik j...@quantopian.com: -- nosy: +ll ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Christian, it's not clear why you're closing this. You basically just described the current broken behaviour. That by itself is not a reason enough (it's a sort of circular argument ;-). Can you please explain what prevents this to be fixed or how we

[issue23990] Callable builtin doesn't respect descriptors

Christian Heimes added the comment: callable() just checks that an object can be called. It doesn't check if the actual call or even the access to the __call__() member succeeds. Magic methods are resolved on the class or type of an object, not the object itself. Therefore the descriptor

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: I am also confused by this; I would imagine that callable(obj) would respect the descriptor protocol. I have a proposed patch that would make this work with descriptors. -- keywords: +patch Added file: http://bugs.python.org/file39090/callable.diff

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: As Christian Heimes explained, this is not a bug. Please do not reopen it. -- nosy: +ethan.furman resolution: - not a bug status: open - closed ___ Python tracker rep...@bugs.python.org

[issue23990] Callable builtin doesn't respect descriptors

Changes by Claudiu Popa pcmantic...@gmail.com: -- nosy: +Claudiu.Popa ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list

[issue23990] Callable builtin doesn't respect descriptors

Christian Heimes added the comment: I have closed the issue because the code behaves according to the language specs and the language design. It is not broken at all. The callable test just checks for the attribute __call__ on the *type* of an object. The check is not performed on the

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: See: https://docs.python.org/3/reference/datamodel.html#special-method-names and https://docs.python.org/3/reference/datamodel.html#object.__getattribute__ and https://docs.python.org/3/reference/datamodel.html#special-lookup --

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Ethan, those sections you have linked to have nothing to do with special methods that are descriptors, or behaviour regarding descriptors. As seen in this example, descriptors are working even for special methods: class B: ... @property ... def

[issue23990] Callable builtin doesn't respect descriptors

Changes by Joe Jevnik j...@quantopian.com: Removed file: http://bugs.python.org/file39090/callable.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: Oops, I messed up the test case; here is a fixed version (the class name was wrong). Just a note: all the existing test cases passed AND the one proposed in this thread. I understand that it is currently working as intended; however, the argument is that the

[issue23990] Callable builtin doesn't respect descriptors

Alexander Belopolsky added the comment: From https://mail.python.org/pipermail/python-ideas/2015-April/033018.html: GvR I think you've found an unintended and undocumented backdoor. I admit I don't understand how this works in CPython. Overloaded operators like __add__ or __call__ should

[issue23990] Callable builtin doesn't respect descriptors

Alexander Belopolsky added the comment: But somehow defining them with @property works (I guess because @property is in the class). IMO, this is the bug and not the callable() behavior. -- ___ Python tracker rep...@bugs.python.org

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: I don't think that using a property to define a callable in the class is a bug; while it seems less ideal, I don't understand why it would be unsupported with callable when it executes correctly. -- ___ Python tracker

[issue23990] Callable builtin doesn't respect descriptors

James Edwards added the comment: It seems like this issue has morphed over time. At the beginning, it looked like you expected perfectly reasonable (but odd) definitions of __call__ attributes, where the logic inside raised an Exception, to be somehow determined to be uncallable. This does

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Actually it does address it, as AttributeError is very special: a.__call__ Traceback (most recent call last): File stdin, line 1, in module File stdin, line 4, in __call__ AttributeError: go away hasattr(a, '__call__') False --

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: Some more discussions happened here: https://mail.python.org/pipermail/python-ideas/2015-April/033027.html So for Christian or Ethan, can you reconsider this issue? Joe has already kindly provided a patch for this that just does one extra check. It

[issue23990] Callable builtin doesn't respect descriptors

Ionel Cristian Mărieș added the comment: But somehow defining them with @property works (I guess because @property is in the class). IMO, this is the bug and not the callable() behavior. By the same reasoning we shouldn't be able to use staticmethod or classmethod for special methods.

[issue23990] Callable builtin doesn't respect descriptors

Terry J. Reedy added the comment: To extend msg241386: A bug, for this tracker, is a discrepancy between specification and behavior. Wrapping an instance method with @property changes its behavior so that it no longer meets the specification for an 'instance method'. (PS to Joe: please

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: Python 3.4.0 (default, Apr 11 2014, 13:05:18) [GCC 4.8.2] on linux Type help, copyright, credits or license for more information. -- class NonIter: ...pass ... -- list(iter(NonIter())) Traceback (most recent call last): File stdin, line 1, in module

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: As ionelmc mentioned, it does address the issue proposed originally and in the patch this is added as another test case for the callable function -- ___ Python tracker rep...@bugs.python.org

[issue23990] Callable builtin doesn't respect descriptors

Terry J. Reedy added the comment: __call__ is a reserved name defined to be an instance method. I view wrapping *any* reserved-name instance method with @property to be a bug in that it redefines the name as a data attribute implemented with a 'hidden' get method. A function wrapped with

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: I don't see how it is a bug that you can make __call__ an arbitrary descriptor as long as it returns a valid callable. if n.__call__ is a valid callable, why should it matter that it was looked up as a descriptor or as an instancemethod? -- type:

[issue23990] Callable builtin doesn't respect descriptors

Ethan Furman added the comment: The purpose of callable is to report whether an instance is callable or not, and that information is available on the instance's class, via the presence of __call__. It is not up to callable() nor iter() nor ... to figure out that, even though the special

[issue23990] Callable builtin doesn't respect descriptors

Joe Jevnik added the comment: The purpose of callable is to report whether an instance is callable or not I am totally with you so far until you get to: and that information is available on the instance's class, via the presence of __call__. I don't understand why this assumption must be