[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Vladimir Matveev]

Vladimir Matveev  added the comment:

Apologies for the delay in reply: in more concrete numbers for IG codebase 
enabling this optimization resulted in 0.2% CPU win.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Mark Shannon]

Mark Shannon  added the comment:

Numbers please.

What is "non-negligible cost of allocation/initialization" mean as a fraction 
of runtime?
What sort of speed up are you seeing on whole programs?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Guido van Rossum]

Guido van Rossum  added the comment:

This looks like a sensible idea to me. The safeguards to ensure that customized 
'super' still works seem reasonable to me. I have to admit that I sometimes 
refrain from using super() where I should because of the expense, so this would 
be welcome.

I do wonder -- is two-arg super() important enough to support it at all? Maybe 
the code would be somewhat simpler if the special opcodes were only generated 
for zero-arg super() calls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Vladimir Matveev]

Vladimir Matveev  added the comment:

>Currently, super() is decoupled from the core language.  It is just a builtin 
>that provides customized attribute lookup.  This PR makes super() more tightly 
>integrated with the core language, treating it as if it were a keyword and 
>part of the grammar.  Also note, users can currently create their own versions 
>of super(), shadowing the builtin super().

This is true however:
- this patch does not block people from introducing custom version of `super` 
so this scenario still work. The idea was to streamline the common case
- based on digging into Instagram codebase and its transitive dependencies 
(which is reasonably large amount of code) all spots where `super()` appear in 
sources assume `super` to be builtin and for a pretty common use-case its cost 
is noticeable in profiler.
- zero-argument `super()` still a bit magical since it requires compiler 
support to create cell for `__class__` and assumes certain shape of the frame 
object so this patch is a step forward with a better compiler support and 
removing runtime dependency on the frame

> Do you have any evidence that the overhead of super() is significant in real 
> programs

I do see the non-negligible cost of allocation/initialization of `super` object 
in IG profiling data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Currently, super() is decoupled from the core language.  It is just a builtin 
that provides customized attribute lookup.  This PR makes super() more tightly 
integrated with the core language, treating it as if it were a keyword and part 
of the grammar.  Also note, users can currently create their own versions of 
super(), shadowing the builtin super().

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Mark Shannon]

Mark Shannon  added the comment:

Why?

Do you have any evidence that the overhead of super() is significant in real 
programs, or that the proposed change actually speeds up anything beyond your 
micro-benchmark?

--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Vladimir Matveev]

Change by Vladimir Matveev :


--
keywords: +patch
pull_requests: +23696
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24936

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43563] Use dedicated opcodes to speed up calls/attribute lookups with super() as receiver

[Vladimir Matveev]

New submission from Vladimir Matveev :

Calling methods and lookup up attributes when receiver is `super()` has extra 
cost comparing to regular attribute lookup. It mainly comes from the need to 
allocate and initialize the instance of the `super` which for zero argument 
case also include peeking into frame/code object for the `__class__` cell and 
first argument. In addition because `PySuper_Type` has custom implementation of 
tp_getattro - `_PyObject_GetMethod` would always return bound method. 
```
import timeit

setup = """
class A:
def f(self): pass
class B(A):
def f(self): super().f()
def g(self): A.f(self)
b = B()
"""
print(timeit.timeit("b.f()", setup=setup, number=2000))
print(timeit.timeit("b.g()", setup=setup, number=2000))

7.329449548968114
3.892987059080042
```
 
One option to improve it could be to make compiler/interpreter aware of super 
calls so they can be treated specially. Attached patch introduces two new 
opcodes LOAD_METHOD_SUPER and LOAD_ATTR_SUPER that are intended to be 
counterparts for LOAD_METHOD and LOAD_ATTR for cases when receiver is super 
with either zero or two arguments.

Immediate argument for both LOAD_METHOD_SUPER and LOAD_ATTR_SUPER is a pair 
that consist of:
 0: index of method/attribute in co_names
 1: Py_True if super was originally called with 0 arguments and Py_False 
otherwise.

Both LOAD_METHOD_SUPER and LOAD_ATTR_SUPER expect 3 elements on the stack:
TOS3: global_super
TOS2: type
TOS1: self/cls

Result of LOAD_METHOD_SUPER is the same as LOAD_METHOD.
Result of LOAD_ATTR_SUPER is the same as LOAD_ATTR

In runtime both LOAD_METHOD_SUPER and LOAD_ATTR_SUPER will check if 
`global_super` is `PySuper_Type` to handle situations when `super` is patched. 
If `global_super` is `PySuper_Type` then it can use dedicated routine to 
perform the lookup for provided `__class__` and `cls/self` without allocating 
new `super` instance. If `global_super` is different from `PySuper_Type` then 
runtime will fallback to the original logic using `global_super` and original 
number of arguments that was captured in immediate.

Benchmark results with patch:
4.381768501014449
3.9492998640052974

--
components: Interpreter Core
messages: 389114
nosy: v2m
priority: normal
severity: normal
status: open
title: Use dedicated opcodes to speed up calls/attribute lookups with super() 
as receiver
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com