[issue1615] descriptor protocol bug

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee:  -> rhettinger
priority: high -> normal
versions:  -Python 2.7, Python 3.2, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Ethan Furman]

Ethan Furman added the comment:

Benjamin Peterson added the comment:
>
> I consider this to be a feature. Properties can raise AttributeError to defer 
> to
> __getattr__.

Micah Friesen added the comment:
>
>I submit that this is not a feature - I can't imagine a real-world scenario 
>[...]

No need to imagine.  The new Enum class uses this ability to support having 
both protected properties on enum members and enum members of the same name:

--> from enum import Enum
--> class Field(Enum):
--> name = 1
... age = 2
... location = 3
... 
--> Field.name

--> Field.name.name
'name'

Enum's custom __getattr__ is located in the metaclass, however, not in the 
class.  For future reference, here is a short test script and it's output in 
3.4.0a2+:
=
class WithOut:
@property
def huh(self):
return self.not_here

class With:
@property
def huh(self):
return self.not_here
def __getattr__(self, name):
print('looking up %s' % name)
raise AttributeError('%s not in class %s' % (name, type(self)))

try:
WithOut().huh
except AttributeError as exc:
print(exc)

print()
try:
With().huh
except AttributeError as exc:
print(exc)

print()
import enum  # broken value property tries to access self.not_here
class TestEnum(enum.Enum):
one = 1

print(TestEnum.one.value)
=
'WithOut' object has no attribute 'not_here'

looking up not_here
looking up huh
huh not in class 

meta getattr with __new_member__
meta getattr with __new_member__
meta getattr with one
'TestEnum' object has no attribute 'not_here'
=

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Ethan Furman]

Changes by Ethan Furman :


--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Eric Snow]

Eric Snow added the comment:

The whole AttributeError gaining an attribute seems impractical, but the second 
approach still seems reasonable to me.  I've attached a rough patch to 
demonstrate.  If that looks okay, I can flesh it out.

--
keywords: +patch
Added file: http://bugs.python.org/file29167/getattr-descriptors.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Eric Snow]

Eric Snow added the comment:

Got bit by a variation of this today in 2.7:


class Spam(object):
def __getattr__(self, attr):
raise AttributeError(attr)
@property
def eggs(self):
return self.ham

s = Spam()
s.eggs
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in __getattr__
AttributeError: eggs


It took me a little while to figure out what was going on.  A real 
head-scratcher.  This is because the AttributeError was attributed to the 
property, but was actually caused by the __getattr__ call triggered by the 
property's code.  I would expect the AttributeError to reference "ham", not 
"eggs".  As already noted, if __getattr__() is not there, that's what happens.

Regardless, I'm just not seeing where the hurdle is to improving this behavior. 
 I certainly agree that this is not a feature.  It is the source of very 
mysterious failures.

I was surprised that AttributeError does not have an attribute to which the 
name would be bound.  If it did, then slot_tp_getattr_hook() could check 
against that:


if (res == NULL && PyErr_ExceptionMatches(PyExc_AttributeError)) {
PyObject *tp, *exc, *tb, *exc_attr;

PyErr_Fetch(&tp, &exc, &tb);
exc_attr = PyObject_GetAttrString(exc, "attribute");
PyErr_Restore(tp, exc, tb);

if (!exc_attr || exc_attr == name) { 
PyErr_Clear();
res = call_attribute(self, getattr, name);
}
Py_XDECREF(exc_attr);
}


Alternatively, when an AttributeError comes out of a getter in 
_PyObject_GenericSetAttrWithDict() (in either spot they're called), another 
exception (not AttributeError) could be raised with the original chained onto 
it.  Then slot_tp_getattr_hook() won't silently ignore it.  It would be 
something like this:


if (f != NULL && PyDescr_IsData(descr)) {
res = f(descr, obj, value);
if (!res && PyErr_ExceptionMatches(PyExc_AttributeError)) {
PyObject *msg = PyUnicode_FromFormat("getter failed for '%U'", 
name);
/* implicit chaining here */
PyErr_SetObject(PyExc_???Error, msg);
}
goto done;
}


Conceptually, it's as though locating the attribute and extracting the value 
are lumped together here.  Distinguishing the two would help make this failure 
situation much less confusing.

Additionally, it would be really helpful to have a brief mention of this 
behavior (AttributeErrors in getters falling back to __getattr__) in the 
language reference entry for __getattr__ and/or descriptors.

--
nosy: +eric.snow
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Éric Araujo]

Changes by Éric Araujo :


--
nosy: +eric.araujo
versions: +Python 3.2, Python 3.3 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Micah Friesen]

Micah Friesen  added the comment:

I ran into this recently, as well, and have lost probably a day's worth of time 
debugging it. I submit that this is not a feature - I can't imagine a 
real-world scenario where you actually want to write debuggable code where a 
descriptor defers to __getattr__ (except perhaps for exception handling, in 
which case some re-factoring is in order), particularly because descriptors are 
effectively mix-ins and can be used on multiple classes.

I worked around this by writing an ancestor descriptor that catches 
AttributeErrors and re-raises them as a user-defined exception.

--
nosy: +Micah.Friesen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Benjamin Peterson]

Benjamin Peterson  added the comment:

I consider this to be a feature. Properties can raise AttributeError to defer 
to __getattr__.

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Mark Lawrence]

Mark Lawrence  added the comment:

This is still an issue with the latest trunk.

--
nosy: +BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Daniel Urban]

Changes by Daniel Urban :


--
nosy: +durban

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
versions: +Python 2.7 -Python 2.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Thomas Lee]

Thomas Lee <[EMAIL PROTECTED]> added the comment:

Related reading from a few years back:

http://coding.derkeiler.com/Archive/Python/comp.lang.python/2005-05/msg03829.html

--
nosy: +thomas.lee

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[ganges master]

ganges master <[EMAIL PROTECTED]> added the comment:

here's a short example of the bug

>>> class Foo(object):
...   def __getattr__(self, name): 
... return 42
...   @property
...   def bacon(self): 
... return int.lalala
...   @property
...   def eggs(self): 
... return 17
... 
>>> f = Foo()
>>> f.bacon   # raises an AttributeError, and silently ignores it
42
>>> f.eggs
17
>>> 

are there any news in this front?

___
Python tracker <[EMAIL PROTECTED]>

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Christian Heimes]

Changes by Christian Heimes:


--
priority:  -> high

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Antoine Pitrou]

Antoine Pitrou added the comment:

PyObject_GenericGetAttr is invoked from slot_tp_getattr_hook in
typeobject.c via tp_getattro. The problem is that, when tp_getattro
returns with an AttributeError, there is no way for slot_tp_getattr_hook
to know whether the error was raised by PyObject_GenericGetAttr itself
or by subsequent invocation of user code. Perhaps by adding an attribute
to the raised AttributeError?

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[Antoine Pitrou]

Antoine Pitrou added the comment:

I can confirm that with SVN trunk, and it's actually even worse because
it can return unexpected results without raising an exception at all:

>>> class Foo(object):
...   def __getattr__(self, name): return 42
...   @property
...   def bacon(self): return int.lalala
... 
>>> f = Foo()
>>> f.bacon
42
>>> Foo.bacon.__get__(f)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in bacon
AttributeError: type object 'int' has no attribute 'lalala'

--
nosy: +pitrou
severity: normal -> major

__
Tracker <[EMAIL PROTECTED]>

__
___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1615] descriptor protocol bug

[ganges master]

New submission from ganges master:

it seems the code of PyObject_GenericGetAttr, which invokes the
descriptor protocol, silences any AttributeErrors raised by the
descriptor, for classes that also define __getattr__. it should
propagate up rather than being silently ignored.

the attached example is quite artificial, but it's a simplification of
real world code i had hard time debugging. turned out i misspelled an
attribute name inside the property getter function, which raised an
AttributeError as expected -- but the exception i got was quite
misleading, saying the instance has no attribute named so.

this bug only happens when the class defines a custom __getattr__. see
attached demo file for details.

--
components: Interpreter Core
files: demo.txt
messages: 58581
nosy: gangesmaster
severity: normal
status: open
title: descriptor protocol bug
type: behavior
versions: Python 2.5, Python 2.6
Added file: http://bugs.python.org/file8943/demo.txt

__
Tracker <[EMAIL PROTECTED]>

__only happens when the class defines a custom __getattr__:

>>> class Foo(object):
... def __getattr__(self, name):
... if name == "spam":
... return 17
... else:
... raise AttributeError("%s object has no attribute %r" %
... (self.__class__.__name__, name))
... @property
... def bacon(self):
... return int.lalala
...
>>>
>>> f = Foo()
>>> f.spam
17
>>> f.bacon
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 7, in __getattr__
AttributeError: Foo object has no attribute 'bacon'  <<-- expected 'int object 
has no attribute lalala'
>>>



without a custom __getattr__ works as expected

>>> class Bar(object):
... def __init__(self):
... self.x = 17
...
... @property
... def bacon(self):
... return int.lalala
...
>>> b = Bar()
>>> b.x
17
>>> b.bacon
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 7, in bacon
AttributeError: type object 'int' has no attribute 'lalala'
>>>___
Python-bugs-list mailing list 
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com