Hi Brett, I've got a few questions about the specific design. Probably you know the answers, it would be nice to have them in the PEP.
First, why not have a global hook? What does a hook per interpreter give you? Would even finer granularity buy anything? Next, I'm a bit (but no more than a bit) concerned about the extra 8 bytes per code object, especially since for most people this is just waste (assuming most people won't be using Pyjion or Numba). Could it be a compile-time feature (requiring recompilation of CPython but not extensions)? Could you figure out some other way to store per-code-object data? It seems you considered this but decided that the co_extra field was simpler and faster; I'm basically pushing a little harder on this. Of course most of the PEP would disappear without this feature; the extra interpreter field is fine. Finally, there are some error messages from pep2html.py: https://www.python.org/dev/peps/pep-0523/#copyright --Guido On Fri, Jun 17, 2016 at 7:58 PM, Brett Cannon <br...@python.org> wrote: > I have taken PEP 523 for this: > https://github.com/python/peps/blob/master/pep-0523.txt . > > I'm waiting until Guido gets back from vacation, at which point I'll ask > for a pronouncement or assignment of a BDFL delegate. > > On Fri, 3 Jun 2016 at 14:37 Brett Cannon <br...@python.org> wrote: > >> For those of you who follow python-ideas or were at the PyCon US 2016 >> language summit, you have already seen/heard about this PEP. For those of >> you who don't fall into either of those categories, this PEP proposed a >> frame evaluation API for CPython. The motivating example of this work has >> been Pyjion, the experimental CPython JIT Dino Viehland and I have been >> working on in our spare time at Microsoft. The API also works for >> debugging, though, as already demonstrated by Google having added a very >> similar API internally for debugging purposes. >> >> The PEP is pasted in below and also available in rendered form at >> https://github.com/Microsoft/Pyjion/blob/master/pep.rst (I will assign >> myself a PEP # once discussion is finished as it's easier to work in git >> for this for the rich rendering of the in-progress PEP). >> >> I should mention that the difference from python-ideas and the language >> summit in the PEP are the listed support from Google's use of a very >> similar API as well as clarifying the co_extra field on code objects >> doesn't change their immutability (at least from the view of the PEP). >> >> ---------- >> PEP: NNN >> Title: Adding a frame evaluation API to CPython >> Version: $Revision$ >> Last-Modified: $Date$ >> Author: Brett Cannon <br...@python.org>, >> Dino Viehland <di...@microsoft.com> >> Status: Draft >> Type: Standards Track >> Content-Type: text/x-rst >> Created: 16-May-2016 >> Post-History: 16-May-2016 >> 03-Jun-2016 >> >> >> Abstract >> ======== >> >> This PEP proposes to expand CPython's C API [#c-api]_ to allow for >> the specification of a per-interpreter function pointer to handle the >> evaluation of frames [#pyeval_evalframeex]_. This proposal also >> suggests adding a new field to code objects [#pycodeobject]_ to store >> arbitrary data for use by the frame evaluation function. >> >> >> Rationale >> ========= >> >> One place where flexibility has been lacking in Python is in the direct >> execution of Python code. While CPython's C API [#c-api]_ allows for >> constructing the data going into a frame object and then evaluating it >> via ``PyEval_EvalFrameEx()`` [#pyeval_evalframeex]_, control over the >> execution of Python code comes down to individual objects instead of a >> hollistic control of execution at the frame level. >> >> While wanting to have influence over frame evaluation may seem a bit >> too low-level, it does open the possibility for things such as a >> method-level JIT to be introduced into CPython without CPython itself >> having to provide one. By allowing external C code to control frame >> evaluation, a JIT can participate in the execution of Python code at >> the key point where evaluation occurs. This then allows for a JIT to >> conditionally recompile Python bytecode to machine code as desired >> while still allowing for executing regular CPython bytecode when >> running the JIT is not desired. This can be accomplished by allowing >> interpreters to specify what function to call to evaluate a frame. And >> by placing the API at the frame evaluation level it allows for a >> complete view of the execution environment of the code for the JIT. >> >> This ability to specify a frame evaluation function also allows for >> other use-cases beyond just opening CPython up to a JIT. For instance, >> it would not be difficult to implement a tracing or profiling function >> at the call level with this API. While CPython does provide the >> ability to set a tracing or profiling function at the Python level, >> this would be able to match the data collection of the profiler and >> quite possibly be faster for tracing by simply skipping per-line >> tracing support. >> >> It also opens up the possibility of debugging where the frame >> evaluation function only performs special debugging work when it >> detects it is about to execute a specific code object. In that >> instance the bytecode could be theoretically rewritten in-place to >> inject a breakpoint function call at the proper point for help in >> debugging while not having to do a heavy-handed approach as >> required by ``sys.settrace()``. >> >> To help facilitate these use-cases, we are also proposing the adding >> of a "scratch space" on code objects via a new field. This will allow >> per-code object data to be stored with the code object itself for easy >> retrieval by the frame evaluation function as necessary. The field >> itself will simply be a ``PyObject *`` type so that any data stored in >> the field will participate in normal object memory management. >> >> >> Proposal >> ======== >> >> All proposed C API changes below will not be part of the stable ABI. >> >> >> Expanding ``PyCodeObject`` >> -------------------------- >> >> One field is to be added to the ``PyCodeObject`` struct >> [#pycodeobject]_:: >> >> typedef struct { >> ... >> PyObject *co_extra; /* "Scratch space" for the code object. */ >> } PyCodeObject; >> >> The ``co_extra`` will be ``NULL`` by default and will not be used by >> CPython itself. Third-party code is free to use the field as desired. >> Values stored in the field are expected to not be required in order >> for the code object to function, allowing the loss of the data of the >> field to be acceptable (this keeps the code object as immutable from >> a functionality point-of-view; this is slightly contentious and so is >> listed as an open issue in `Is co_extra needed?`_). The field will be >> freed like all other fields on ``PyCodeObject`` during deallocation >> using ``Py_XDECREF()``. >> >> It is not recommended that multiple users attempt to use the >> ``co_extra`` simultaneously. While a dictionary could theoretically be >> set to the field and various users could use a key specific to the >> project, there is still the issue of key collisions as well as >> performance degradation from using a dictionary lookup on every frame >> evaluation. Users are expected to do a type check to make sure that >> the field has not been previously set by someone else. >> >> >> Expanding ``PyInterpreterState`` >> -------------------------------- >> >> The entrypoint for the frame evalution function is per-interpreter:: >> >> // Same type signature as PyEval_EvalFrameEx(). >> typedef PyObject* (__stdcall *PyFrameEvalFunction)(PyFrameObject*, int); >> >> typedef struct { >> ... >> PyFrameEvalFunction eval_frame; >> } PyInterpreterState; >> >> By default, the ``eval_frame`` field will be initialized to a function >> pointer that represents what ``PyEval_EvalFrameEx()`` currently is >> (called ``PyEval_EvalFrameDefault()``, discussed later in this PEP). >> Third-party code may then set their own frame evaluation function >> instead to control the execution of Python code. A pointer comparison >> can be used to detect if the field is set to >> ``PyEval_EvalFrameDefault()`` and thus has not been mutated yet. >> >> >> Changes to ``Python/ceval.c`` >> ----------------------------- >> >> ``PyEval_EvalFrameEx()`` [#pyeval_evalframeex]_ as it currently stands >> will be renamed to ``PyEval_EvalFrameDefault()``. The new >> ``PyEval_EvalFrameEx()`` will then become:: >> >> PyObject * >> PyEval_EvalFrameEx(PyFrameObject *frame, int throwflag) >> { >> PyThreadState *tstate = PyThreadState_GET(); >> return tstate->interp->eval_frame(frame, throwflag); >> } >> >> This allows third-party code to place themselves directly in the path >> of Python code execution while being backwards-compatible with code >> already using the pre-existing C API. >> >> >> Updating ``python-gdb.py`` >> -------------------------- >> >> The generated ``python-gdb.py`` file used for Python support in GDB >> makes some hard-coded assumptions about ``PyEval_EvalFrameEx()``, e.g. >> the names of local variables. It will need to be updated to work with >> the proposed changes. >> >> >> Performance impact >> ================== >> >> As this PEP is proposing an API to add pluggability, performance >> impact is considered only in the case where no third-party code has >> made any changes. >> >> Several runs of pybench [#pybench]_ consistently showed no performance >> cost from the API change alone. >> >> A run of the Python benchmark suite [#py-benchmarks]_ showed no >> measurable cost in performance. >> >> In terms of memory impact, since there are typically not many CPython >> interpreters executing in a single process that means the impact of >> ``co_extra`` being added to ``PyCodeObject`` is the only worry. >> According to [#code-object-count]_, a run of the Python test suite >> results in about 72,395 code objects being created. On a 64-bit >> CPU that would result in 579,160 bytes of extra memory being used if >> all code objects were alive at once and had nothing set in their >> ``co_extra`` fields. >> >> >> Example Usage >> ============= >> >> A JIT for CPython >> ----------------- >> >> Pyjion >> '''''' >> >> The Pyjion project [#pyjion]_ has used this proposed API to implement >> a JIT for CPython using the CoreCLR's JIT [#coreclr]_. Each code >> object has its ``co_extra`` field set to a ``PyjionJittedCode`` object >> which stores four pieces of information: >> >> 1. Execution count >> 2. A boolean representing whether a previous attempt to JIT failed >> 3. A function pointer to a trampoline (which can be type tracing or not) >> 4. A void pointer to any JIT-compiled machine code >> >> The frame evaluation function has (roughly) the following algorithm:: >> >> def eval_frame(frame, throw_flag): >> pyjion_code = frame.code.co_extra >> if not pyjion_code: >> frame.code.co_extra = PyjionJittedCode() >> elif not pyjion_code.jit_failed: >> if not pyjion_code.jit_code: >> return pyjion_code.eval(pyjion_code.jit_code, frame) >> elif pyjion_code.exec_count > 20_000: >> if jit_compile(frame): >> return pyjion_code.eval(pyjion_code.jit_code, frame) >> else: >> pyjion_code.jit_failed = True >> pyjion_code.exec_count += 1 >> return PyEval_EvalFrameDefault(frame, throw_flag) >> >> The key point, though, is that all of this work and logic is separate >> from CPython and yet with the proposed API changes it is able to >> provide a JIT that is compliant with Python semantics (as of this >> writing, performance is almost equivalent to CPython without the new >> API). This means there's nothing technically preventing others from >> implementing their own JITs for CPython by utilizing the proposed API. >> >> >> Other JITs >> '''''''''' >> >> It should be mentioned that the Pyston team was consulted on an >> earlier version of this PEP that was more JIT-specific and they were >> not interested in utilizing the changes proposed because they want >> control over memory layout they had no interest in directly supporting >> CPython itself. An informal discusion with a developer on the PyPy >> team led to a similar comment. >> >> Numba [#numba]_, on the other hand, suggested that they would be >> interested in the proposed change in a post-1.0 future for >> themselves [#numba-interest]_. >> >> The experimental Coconut JIT [#coconut]_ could have benefitted from >> this PEP. In private conversations with Coconut's creator we were told >> that our API was probably superior to the one they developed for >> Coconut to add JIT support to CPython. >> >> >> Debugging >> --------- >> >> In conversations with the Python Tools for Visual Studio team (PTVS) >> [#ptvs]_, they thought they would find these API changes useful for >> implementing more performant debugging. As mentioned in the Rationale_ >> section, this API would allow for switching on debugging functionality >> only in frames where it is needed. This could allow for either >> skipping information that ``sys.settrace()`` normally provides and >> even go as far as to dynamically rewrite bytecode prior to execution >> to inject e.g. breakpoints in the bytecode. >> >> It also turns out that Google has provided a very similar API >> internally for years. It has been used for performant debugging >> purposes. >> >> >> Implementation >> ============== >> >> A set of patches implementing the proposed API is available through >> the Pyjion project [#pyjion]_. In its current form it has more >> changes to CPython than just this proposed API, but that is for ease >> of development instead of strict requirements to accomplish its goals. >> >> >> Open Issues >> =========== >> >> Allow ``eval_frame`` to be ``NULL`` >> ----------------------------------- >> >> Currently the frame evaluation function is expected to always be set. >> It could very easily simply default to ``NULL`` instead which would >> signal to use ``PyEval_EvalFrameDefault()``. The current proposal of >> not special-casing the field seemed the most straight-forward, but it >> does require that the field not accidentally be cleared, else a crash >> may occur. >> >> >> Is co_extra needed? >> ------------------- >> >> While discussing this PEP at PyCon US 2016, some core developers >> expressed their worry of the ``co_extra`` field making code objects >> mutable. The thinking seemed to be that having a field that was >> mutated after the creation of the code object made the object seem >> mutable, even though no other aspect of code objects changed. >> >> The view of this PEP is that the `co_extra` field doesn't change the >> fact that code objects are immutable. The field is specified in this >> PEP as to not contain information required to make the code object >> usable, making it more of a caching field. It could be viewed as >> similar to the UTF-8 cache that string objects have internally; >> strings are still considered immutable even though they have a field >> that is conditionally set. >> >> The field is also not strictly necessary. While the field greatly >> simplifies attaching extra information to code objects, other options >> such as keeping a mapping of code object memory addresses to what >> would have been kept in ``co_extra`` or perhaps using a weak reference >> of the data on the code object and then iterating through the weak >> references until the attached data is found is possible. But obviously >> all of these solutions are not as simple or performant as adding the >> ``co_extra`` field. >> >> >> Rejected Ideas >> ============== >> >> A JIT-specific C API >> -------------------- >> >> Originally this PEP was going to propose a much larger API change >> which was more JIT-specific. After soliciting feedback from the Numba >> team [#numba]_, though, it became clear that the API was unnecessarily >> large. The realization was made that all that was truly needed was the >> opportunity to provide a trampoline function to handle execution of >> Python code that had been JIT-compiled and a way to attach that >> compiled machine code along with other critical data to the >> corresponding Python code object. Once it was shown that there was no >> loss in functionality or in performance while minimizing the API >> changes required, the proposal was changed to its current form. >> >> >> References >> ========== >> >> .. [#pyjion] Pyjion project >> (https://github.com/microsoft/pyjion) >> >> .. [#c-api] CPython's C API >> (https://docs.python.org/3/c-api/index.html) >> >> .. [#pycodeobject] ``PyCodeObject`` >> (https://docs.python.org/3/c-api/code.html#c.PyCodeObject) >> >> .. [#coreclr] .NET Core Runtime (CoreCLR) >> (https://github.com/dotnet/coreclr) >> >> .. [#pyeval_evalframeex] ``PyEval_EvalFrameEx()`` >> ( >> https://docs.python.org/3/c-api/veryhigh.html?highlight=pyframeobject#c.PyEval_EvalFrameEx >> ) >> >> .. [#pycodeobject] ``PyCodeObject`` >> (https://docs.python.org/3/c-api/code.html#c.PyCodeObject) >> >> .. [#numba] Numba >> (http://numba.pydata.org/) >> >> .. [#numba-interest] numba-users mailing list: >> "Would the C API for a JIT entrypoint being proposed by Pyjion help >> out Numba?" >> ( >> https://groups.google.com/a/continuum.io/forum/#!topic/numba-users/yRl_0t8-m1g >> ) >> >> .. [#code-object-count] [Python-Dev] Opcode cache in ceval loop >> ( >> https://mail.python.org/pipermail/python-dev/2016-February/143025.html) >> >> .. [#py-benchmarks] Python benchmark suite >> (https://hg.python.org/benchmarks) >> >> .. [#pyston] Pyston >> (http://pyston.org) >> >> .. [#pypy] PyPy >> (http://pypy.org/) >> >> .. [#ptvs] Python Tools for Visual Studio >> (http://microsoft.github.io/PTVS/) >> >> .. [#coconut] Coconut >> (https://github.com/davidmalcolm/coconut) >> >> >> Copyright >> ========= >> >> This document has been placed in the public domain. >> >> >> .. >> Local Variables: >> mode: indented-text >> indent-tabs-mode: nil >> sentence-end-double-space: t >> fill-column: 70 >> coding: utf-8 >> End: >> >> > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
