[issue29944] Argumentless super() calls do not work in classes constructed with type()

[Nick Coghlan]

Nick Coghlan added the comment:

Right, there's a very similar instance-method-reuse related problem described 
in http://bugs.python.org/issue29270, where ctypes re-uses a class namespace is 
define a swapped-endianness version of the originally defined class.

The easiest place to deflect conflicts is at the point where bound instance 
methods are created:


>>> bound_method = MyList().insert
>>> bound_method.__self__.__class__

>>> bound_method.__func__.__closure__[0].cell_contents

>>> bound_method.__self__.__class__ is 
>>> bound_method.__func__.__closure__[0].cell_contents
False

However, that method of detection only works for plain instance methods that 
only close over the `__class__` variable: as soon as you wrap them in a 
decorator, you may not have easy access to the `__closure__` attribute any 
more, and if the method has closure references to more than just `__class__`, 
then it's a bit more work to find the right closure index from outside the 
function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29944] Argumentless super() calls do not work in classes constructed with type()

[Josh Rosenberg]

Josh Rosenberg added the comment:

This is what I get for leaving a response up and working on it intermittently 
for three hours, and not rechecking: I found the same basic problem.

For the record, I also found a terribly hacky way of doing this sort of 
rebinding manually, should you actually need to do so under the current design:

def make_class_closure(__class__):
'''Makes a one-tuple closure with a cell for the provided class'''
return (lambda: super).__closure__

class MyList(list):
def insert(self, index, object):
super().insert(index, object)

class MyList2(list):
pass

MyList2.insert = types.FunctionType(MyList.insert.__code__, globals(), 
closure=make_class_closure(MyList2))

It would need further improvements to recognize when it needs to run, deal with 
existing closures, handle top-level functions without __class__ in their 
closure co_freevars, etc., and I won't even pretend that's a sane workaround 
for the problem, but it does demonstrate what is happening/what would need to 
be changed to make it work, if indeed that's desirable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29944] Argumentless super() calls do not work in classes constructed with type()

[Josh Rosenberg]

Josh Rosenberg added the comment:

It looks like this is a general problem caused by the fact that a function that 
is:

1. Defined in a class
2. References the name "super" (even if it's a local variable name, even if 
it's never called)

isn't "just" a plain function. Even though Python 3 officially removed the 
concept of unbound methods, there is still some unbound method-like behavior 
going on, which makes that sort of function intrinsically tied to where it 
executes.

Specifically, a function of that sort is effectively a closure, where the 
closure scope defines __class__ (insert.__code__.co_freevars will be non-empty 
with the string '__class__' in it); the value of that closure variable cell is 
set when the class finishes being defined (that is, the closure cell is empty 
when the def block finishes, but when the class block itself completes, the 
cell is populated with a reference to the class).

No argument super (specifically, super_init) relies on this assistance, as it 
looks up the __class__ cell in the caller's frame when not provided a class 
explicitly. In your repro, the problem is that __class__ is set to the original 
MyList (it was set at the moment the class MyList(list): block finished, before 
the decorator was invoked), but at call time, the self instance is of the new 
type you created (which has the same name, but the name isn't looked up, it has 
a cached reference to the class).

The same problem applies if you do:

class MyList(list):
def insert(self, index, object):
super().insert(index, object)

class MyOtherList(list):
insert = MyList.insert

because MyList.insert is permanently tied to MyList, it doesn't get 
"reclosured" as a result of MyOtherList using it. Similarly:

class MyList(list):
pass

def insert(self, index, object):
super().insert(index, object)

MyList.insert = insert

fails when insert is called with error: "RuntimeError: super(): __class__ cell 
not found", because functions def-ed outside a class block entirely don't have 
__class__ as "virtual closure" scope.

It's arguable whether this should be changed: If you're using super() without 
arguments, the implicit behavior is that it works with "whatever class I'm 
currently defining", so trying to reuse it with some other class is trying to 
extend that implicit behavior to "whatever class I was eventually assigned to". 
Making it work would mean that already "class bound" methods must be rebound 
when assigned to a new class, and non-bound methods must be bound (in both 
cases, you'd need to handle this when implicitly assigned in the class body, or 
explicitly assigned later as an attribute on the constructed class).

Doing so would break at least one pattern someone might be using already, where 
they have multiple inheritance, and while most methods should prioritize the 
first class in the MRO, they want a specific method to bypass one or more 
classes in the MRO without calling it. For example:

class Stuff:  # Top of diamond, just so super().stuff() valid in all children
def stuff(self):
print("Stuff")

class Spam:
def stuff(self):
print("Spam")
super().stuff()

class Eggs:
def stuff(self):
print("Eggs")
super().stuff()

class SpamAndEggsWithoutSpam(Spam, Eggs):
stuff = Eggs.stuff  # I'd like my stuff without Spam

Under the current design, Eggs.stuff is already bound to Eggs, so this works; 
you see:

Eggs
Stuff

printed, but not Spam. If rebinding were made a thing, you'd end up with:

Eggs
Spam
Eggs
Stuff

with the first Eggs being from the rebound method.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29944] Argumentless super() calls do not work in classes constructed with type()

[Nick Coghlan]

Nick Coghlan added the comment:

Interestingly, even `types.new_class` misbehaves in this case:

```
>>> def mydec(cls):
... return types.new_class(cls.__name__, cls.__bases__, exec_body=lambda 
ns: ns.update(cls.__dict__))
... 
>>> @mydec
... class MyList(list):
... def insert(self, idx, obj):
... super().insert(idx, obj)
... 
>>> MyList().insert(0, 1)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 4, in insert
TypeError: super(type, obj): obj must be an instance or subtype of type
```

The error message is confusing here because it's using 'type' as a metavariable 
to refer to the first argument of super(), *not* to the builtin called "type".

If we poke around in the MyList.insert closure, we can see the origin of the 
problem:

```
>>> MyList.insert.__closure__[0].cell_contents

>>> MyList.insert.__closure__[0].cell_contents is MyList
False
```

The class cell is still bound to the originally created class object, but the 
decorator threw that away and returned a completely different object. As a 
result, the "self" passed to the bound method is *not* an instance of the type 
stored in the "__class__" cell, and super() complains.

Given that super() doesn't take keyword arguments, perhaps we should propose on 
python-dev that "type" be changed to "starting_type" in the super() error 
messages and documentation?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29944] Argumentless super() calls do not work in classes constructed with type()

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +ncoghlan, xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29944] Argumentless super() calls do not work in classes constructed with type()

[assume_away]

New submission from assume_away:

The simplest example:

def mydec(cls):
return type(cls.__name__, cls.__bases__, dict(cls.__dict__))

@mydec
class MyList(list):

def extend(self, item):
super(MyList, self).extend(item)

def insert(self, index, object):
super().insert(index, object)

>>> lst = MyList()
>>> lst.extend([2,3])
>>> lst.insert(0, 1)
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> lst
[2, 3]

If this is intended behavior, at least the error message could be fixed.

--
messages: 290823
nosy: assume_away
priority: normal
severity: normal
status: open
title: Argumentless super() calls do not work in classes constructed with type()
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com