On 19/06/16 18:29, Brett Cannon wrote:
On Sat, 18 Jun 2016 at 21:49 Guido van Rossum <gu...@python.org
<mailto:gu...@python.org>> wrote:
Hi Brett,
I've got a few questions about the specific design. Probably you
know the answers, it would be nice to have them in the PEP.
Once you're happy with my answers I'll update the PEP.
First, why not have a global hook? What does a hook per interpreter
give you? Would even finer granularity buy anything?
We initially considered a per-code object hook, but we figured it was
unnecessary to have that level of control, especially since people like
Numba have gotten away with not needing it for this long (although I
suspect that's because they are a decorator so they can just return an
object that overrides __call__()). We didn't think that a global one was
appropriate as different workloads may call for different
JITs/debuggers/etc. and there is no guarantee that you are executing
every interpreter with the same workload. Plus we figured people might
simply import their JIT of choice and as a side-effect set the hook, and
since imports are a per-interpreter thing that seemed to suggest the
granularity of interpreters.
IOW it seemed to be more in line with sys.settrace() than some global
thing for the process.
Next, I'm a bit (but no more than a bit) concerned about the extra 8
bytes per code object, especially since for most people this is just
waste (assuming most people won't be using Pyjion or Numba). Could
it be a compile-time feature (requiring recompilation of CPython but
not extensions)?
Probably. It does water down potential usage thanks to needing a special
build. If the decision is "special build or not", I would simply pull
out this part of the proposal as I wouldn't want to add a flag that
influences what is or is not possible for an interpreter.
Could you figure out some other way to store per-code-object data?
It seems you considered this but decided that the co_extra field was
simpler and faster; I'm basically pushing a little harder on this.
Of course most of the PEP would disappear without this feature; the
extra interpreter field is fine.
Dino and I thought of two potential alternatives, neither of which we
have taken the time to implement and benchmark. One is to simply have a
hash table of memory addresses to JIT data that is kept on the JIT side
of things. Obviously it would be nice to avoid the overhead of a hash
table lookup on every function call. This also doesn't help minimize
memory when the code object gets GC'ed.
Hash lookups aren't that slow. If you combine it with the custom flags
suggested by MRAB, then you would only suffer the lookup penalty when
actually entering the special interpreter.
You can use a weakref callback to ensure things get GC'd properly.
Also, if there is a special extra field on code-object, then everyone
will want to use it. How do you handle clashes?
The other potential solution we came up with was to use weakrefs. I have
not looked into the details, but we were thinking that if we registered
the JIT data object as a weakref on the code object, couldn't we iterate
through the weakrefs attached to the code object to look for the JIT
data object, and then get the reference that way? It would let us avoid
a more expensive hash table lookup if we assume most code objects won't
have a weakref on it (assuming weakrefs are stored in a list), and it
gives us the proper cleanup semantics we want by getting the weakref
cleanup callback execution to make sure we decref the JIT data object
appropriately. But as I said, I have not looked into the feasibility of
this at all to know if I'm remembering the weakref implementation
details correctly.
Finally, there are some error messages from pep2html.py:
https://www.python.org/dev/peps/pep-0523/#copyright
All fixed in
https://github.com/python/peps/commit/6929f850a5af07e51d0163558a5fe8d6b85dccfe .
-Brett
--Guido
On Fri, Jun 17, 2016 at 7:58 PM, Brett Cannon <br...@python.org
<mailto:br...@python.org>> wrote:
I have taken PEP 523 for this:
https://github.com/python/peps/blob/master/pep-0523.txt .
I'm waiting until Guido gets back from vacation, at which point
I'll ask for a pronouncement or assignment of a BDFL delegate.
On Fri, 3 Jun 2016 at 14:37 Brett Cannon <br...@python.org
<mailto:br...@python.org>> wrote:
For those of you who follow python-ideas or were at the
PyCon US 2016 language summit, you have already seen/heard
about this PEP. For those of you who don't fall into either
of those categories, this PEP proposed a frame evaluation
API for CPython. The motivating example of this work has
been Pyjion, the experimental CPython JIT Dino Viehland and
I have been working on in our spare time at Microsoft. The
API also works for debugging, though, as already
demonstrated by Google having added a very similar API
internally for debugging purposes.
The PEP is pasted in below and also available in rendered
form at
https://github.com/Microsoft/Pyjion/blob/master/pep.rst (I
will assign myself a PEP # once discussion is finished as
it's easier to work in git for this for the rich rendering
of the in-progress PEP).
I should mention that the difference from python-ideas and
the language summit in the PEP are the listed support from
Google's use of a very similar API as well as clarifying the
co_extra field on code objects doesn't change their
immutability (at least from the view of the PEP).
----------
PEP: NNN
Title: Adding a frame evaluation API to CPython
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <br...@python.org
<mailto:br...@python.org>>,
Dino Viehland <di...@microsoft.com
<mailto:di...@microsoft.com>>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 16-May-2016
Post-History: 16-May-2016
03-Jun-2016
Abstract
========
This PEP proposes to expand CPython's C API [#c-api]_ to
allow for
the specification of a per-interpreter function pointer to
handle the
evaluation of frames [#pyeval_evalframeex]_. This proposal also
suggests adding a new field to code objects [#pycodeobject]_
to store
arbitrary data for use by the frame evaluation function.
Rationale
=========
One place where flexibility has been lacking in Python is in
the direct
execution of Python code. While CPython's C API [#c-api]_
allows for
constructing the data going into a frame object and then
evaluating it
via ``PyEval_EvalFrameEx()`` [#pyeval_evalframeex]_, control
over the
execution of Python code comes down to individual objects
instead of a
hollistic control of execution at the frame level.
While wanting to have influence over frame evaluation may
seem a bit
too low-level, it does open the possibility for things such as a
method-level JIT to be introduced into CPython without
CPython itself
having to provide one. By allowing external C code to
control frame
evaluation, a JIT can participate in the execution of Python
code at
the key point where evaluation occurs. This then allows for
a JIT to
conditionally recompile Python bytecode to machine code as
desired
while still allowing for executing regular CPython bytecode when
running the JIT is not desired. This can be accomplished by
allowing
interpreters to specify what function to call to evaluate a
frame. And
by placing the API at the frame evaluation level it allows for a
complete view of the execution environment of the code for
the JIT.
This ability to specify a frame evaluation function also
allows for
other use-cases beyond just opening CPython up to a JIT. For
instance,
it would not be difficult to implement a tracing or
profiling function
at the call level with this API. While CPython does provide the
ability to set a tracing or profiling function at the Python
level,
this would be able to match the data collection of the
profiler and
quite possibly be faster for tracing by simply skipping per-line
tracing support.
It also opens up the possibility of debugging where the frame
evaluation function only performs special debugging work when it
detects it is about to execute a specific code object. In that
instance the bytecode could be theoretically rewritten
in-place to
inject a breakpoint function call at the proper point for
help in
debugging while not having to do a heavy-handed approach as
required by ``sys.settrace()``.
To help facilitate these use-cases, we are also proposing
the adding
of a "scratch space" on code objects via a new field. This
will allow
per-code object data to be stored with the code object
itself for easy
retrieval by the frame evaluation function as necessary. The
field
itself will simply be a ``PyObject *`` type so that any data
stored in
the field will participate in normal object memory management.
Proposal
========
All proposed C API changes below will not be part of the
stable ABI.
Expanding ``PyCodeObject``
--------------------------
One field is to be added to the ``PyCodeObject`` struct
[#pycodeobject]_::
typedef struct {
...
PyObject *co_extra; /* "Scratch space" for the code
object. */
} PyCodeObject;
The ``co_extra`` will be ``NULL`` by default and will not be
used by
CPython itself. Third-party code is free to use the field as
desired.
Values stored in the field are expected to not be required
in order
for the code object to function, allowing the loss of the
data of the
field to be acceptable (this keeps the code object as
immutable from
a functionality point-of-view; this is slightly contentious
and so is
listed as an open issue in `Is co_extra needed?`_). The
field will be
freed like all other fields on ``PyCodeObject`` during
deallocation
using ``Py_XDECREF()``.
It is not recommended that multiple users attempt to use the
``co_extra`` simultaneously. While a dictionary could
theoretically be
set to the field and various users could use a key specific
to the
project, there is still the issue of key collisions as well as
performance degradation from using a dictionary lookup on
every frame
evaluation. Users are expected to do a type check to make
sure that
the field has not been previously set by someone else.
Expanding ``PyInterpreterState``
--------------------------------
The entrypoint for the frame evalution function is
per-interpreter::
// Same type signature as PyEval_EvalFrameEx().
typedef PyObject* (__stdcall
*PyFrameEvalFunction)(PyFrameObject*, int);
typedef struct {
...
PyFrameEvalFunction eval_frame;
} PyInterpreterState;
By default, the ``eval_frame`` field will be initialized to
a function
pointer that represents what ``PyEval_EvalFrameEx()``
currently is
(called ``PyEval_EvalFrameDefault()``, discussed later in
this PEP).
Third-party code may then set their own frame evaluation
function
instead to control the execution of Python code. A pointer
comparison
can be used to detect if the field is set to
``PyEval_EvalFrameDefault()`` and thus has not been mutated yet.
Changes to ``Python/ceval.c``
-----------------------------
``PyEval_EvalFrameEx()`` [#pyeval_evalframeex]_ as it
currently stands
will be renamed to ``PyEval_EvalFrameDefault()``. The new
``PyEval_EvalFrameEx()`` will then become::
PyObject *
PyEval_EvalFrameEx(PyFrameObject *frame, int throwflag)
{
PyThreadState *tstate = PyThreadState_GET();
return tstate->interp->eval_frame(frame, throwflag);
}
This allows third-party code to place themselves directly in
the path
of Python code execution while being backwards-compatible
with code
already using the pre-existing C API.
Updating ``python-gdb.py``
--------------------------
The generated ``python-gdb.py`` file used for Python support
in GDB
makes some hard-coded assumptions about
``PyEval_EvalFrameEx()``, e.g.
the names of local variables. It will need to be updated to
work with
the proposed changes.
Performance impact
==================
As this PEP is proposing an API to add pluggability, performance
impact is considered only in the case where no third-party
code has
made any changes.
Several runs of pybench [#pybench]_ consistently showed no
performance
cost from the API change alone.
A run of the Python benchmark suite [#py-benchmarks]_ showed no
measurable cost in performance.
In terms of memory impact, since there are typically not
many CPython
interpreters executing in a single process that means the
impact of
``co_extra`` being added to ``PyCodeObject`` is the only worry.
According to [#code-object-count]_, a run of the Python test
suite
results in about 72,395 code objects being created. On a 64-bit
CPU that would result in 579,160 bytes of extra memory being
used if
all code objects were alive at once and had nothing set in their
``co_extra`` fields.
Example Usage
=============
A JIT for CPython
-----------------
Pyjion
''''''
The Pyjion project [#pyjion]_ has used this proposed API to
implement
a JIT for CPython using the CoreCLR's JIT [#coreclr]_. Each code
object has its ``co_extra`` field set to a
``PyjionJittedCode`` object
which stores four pieces of information:
1. Execution count
2. A boolean representing whether a previous attempt to JIT
failed
3. A function pointer to a trampoline (which can be type
tracing or not)
4. A void pointer to any JIT-compiled machine code
The frame evaluation function has (roughly) the following
algorithm::
def eval_frame(frame, throw_flag):
pyjion_code = frame.code.co_extra
if not pyjion_code:
frame.code.co_extra = PyjionJittedCode()
elif not pyjion_code.jit_failed:
if not pyjion_code.jit_code:
return
pyjion_code.eval(pyjion_code.jit_code, frame)
elif pyjion_code.exec_count > 20_000:
if jit_compile(frame):
return
pyjion_code.eval(pyjion_code.jit_code, frame)
else:
pyjion_code.jit_failed = True
pyjion_code.exec_count += 1
return PyEval_EvalFrameDefault(frame, throw_flag)
The key point, though, is that all of this work and logic is
separate
from CPython and yet with the proposed API changes it is able to
provide a JIT that is compliant with Python semantics (as of
this
writing, performance is almost equivalent to CPython without
the new
API). This means there's nothing technically preventing
others from
implementing their own JITs for CPython by utilizing the
proposed API.
Other JITs
''''''''''
It should be mentioned that the Pyston team was consulted on an
earlier version of this PEP that was more JIT-specific and
they were
not interested in utilizing the changes proposed because
they want
control over memory layout they had no interest in directly
supporting
CPython itself. An informal discusion with a developer on
the PyPy
team led to a similar comment.
Numba [#numba]_, on the other hand, suggested that they would be
interested in the proposed change in a post-1.0 future for
themselves [#numba-interest]_.
The experimental Coconut JIT [#coconut]_ could have
benefitted from
this PEP. In private conversations with Coconut's creator we
were told
that our API was probably superior to the one they developed for
Coconut to add JIT support to CPython.
Debugging
---------
In conversations with the Python Tools for Visual Studio
team (PTVS)
[#ptvs]_, they thought they would find these API changes
useful for
implementing more performant debugging. As mentioned in the
Rationale_
section, this API would allow for switching on debugging
functionality
only in frames where it is needed. This could allow for either
skipping information that ``sys.settrace()`` normally
provides and
even go as far as to dynamically rewrite bytecode prior to
execution
to inject e.g. breakpoints in the bytecode.
It also turns out that Google has provided a very similar API
internally for years. It has been used for performant debugging
purposes.
Implementation
==============
A set of patches implementing the proposed API is available
through
the Pyjion project [#pyjion]_. In its current form it has more
changes to CPython than just this proposed API, but that is
for ease
of development instead of strict requirements to accomplish
its goals.
Open Issues
===========
Allow ``eval_frame`` to be ``NULL``
-----------------------------------
Currently the frame evaluation function is expected to
always be set.
It could very easily simply default to ``NULL`` instead
which would
signal to use ``PyEval_EvalFrameDefault()``. The current
proposal of
not special-casing the field seemed the most
straight-forward, but it
does require that the field not accidentally be cleared,
else a crash
may occur.
Is co_extra needed?
-------------------
While discussing this PEP at PyCon US 2016, some core developers
expressed their worry of the ``co_extra`` field making code
objects
mutable. The thinking seemed to be that having a field that was
mutated after the creation of the code object made the
object seem
mutable, even though no other aspect of code objects changed.
The view of this PEP is that the `co_extra` field doesn't
change the
fact that code objects are immutable. The field is specified
in this
PEP as to not contain information required to make the code
object
usable, making it more of a caching field. It could be viewed as
similar to the UTF-8 cache that string objects have internally;
strings are still considered immutable even though they have
a field
that is conditionally set.
The field is also not strictly necessary. While the field
greatly
simplifies attaching extra information to code objects,
other options
such as keeping a mapping of code object memory addresses to
what
would have been kept in ``co_extra`` or perhaps using a weak
reference
of the data on the code object and then iterating through
the weak
references until the attached data is found is possible. But
obviously
all of these solutions are not as simple or performant as
adding the
``co_extra`` field.
Rejected Ideas
==============
A JIT-specific C API
--------------------
Originally this PEP was going to propose a much larger API
change
which was more JIT-specific. After soliciting feedback from
the Numba
team [#numba]_, though, it became clear that the API was
unnecessarily
large. The realization was made that all that was truly
needed was the
opportunity to provide a trampoline function to handle
execution of
Python code that had been JIT-compiled and a way to attach that
compiled machine code along with other critical data to the
corresponding Python code object. Once it was shown that
there was no
loss in functionality or in performance while minimizing the API
changes required, the proposal was changed to its current form.
References
==========
.. [#pyjion] Pyjion project
(https://github.com/microsoft/pyjion)
.. [#c-api] CPython's C API
(https://docs.python.org/3/c-api/index.html)
.. [#pycodeobject] ``PyCodeObject``
(https://docs.python.org/3/c-api/code.html#c.PyCodeObject)
.. [#coreclr] .NET Core Runtime (CoreCLR)
(https://github.com/dotnet/coreclr)
.. [#pyeval_evalframeex] ``PyEval_EvalFrameEx()``
(https://docs.python.org/3/c-api/veryhigh.html?highlight=pyframeobject#c.PyEval_EvalFrameEx)
.. [#pycodeobject] ``PyCodeObject``
(https://docs.python.org/3/c-api/code.html#c.PyCodeObject)
.. [#numba] Numba
(http://numba.pydata.org/)
.. [#numba-interest] numba-users mailing list:
"Would the C API for a JIT entrypoint being proposed by
Pyjion help out Numba?"
(https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/yRl_0t8-m1g)
.. [#code-object-count] [Python-Dev] Opcode cache in ceval loop
(https://mail.python.org/pipermail/python-dev/2016-February/143025.html)
.. [#py-benchmarks] Python benchmark suite
(https://hg.python.org/benchmarks)
.. [#pyston] Pyston
(http://pyston.org)
.. [#pypy] PyPy
(http://pypy.org/)
.. [#ptvs] Python Tools for Visual Studio
(http://microsoft.github.io/PTVS/)
.. [#coconut] Coconut
(https://github.com/davidmalcolm/coconut)
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (python.org/~guido <http://python.org/~guido>)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
