Your email didn't compile. The compiler says that it's a naming conflict, but actually I think you forgot a semicolon! (Don't worry, it happens to all of us, whether we be happy or not :-)
--Koos On Sat, Nov 18, 2017 at 3:05 AM, Victor Stinner <victor.stin...@gmail.com> wrote: > Hi, > > tl; dr I propose to extend the existing "stable API" to make it almost > as complete as the current API. For example, add back > PyTuple_GET_ITEM() to be stable API, but it becomes a function call > rather than a macro. The final question is if it's not too late to > iterate on an implementation of this idea for Python 3.7? Knowing that > the stable API doesn't affect the "current API" at all, since the "new > C API" (extended stable API) would only be accessible using an > *opt-in* flag. > > > Since I failed to find time to write a proper PEP (sorry about that), > and the incoming deadline for Python 3.7 features is getting closer, I > decided to dump my dump into this email. Sorry about the raw > formatting. > > It's a third turn of feedback. The first one was done in python-ideas > last July, the second was at the CPython sprint last September at > Instagram. Both were positive. > > https://mail.python.org/pipermail/python-ideas/2017-July/046399.html > > The C API of CPython is amazing. It made CPython as powerful as we > know it today. Without the C API, Python wouldn't have numpy. Without > numpy, Python wouldn't be as popular as it is nowadays. Ok, enough for > the introduction. > > The C API is awful. I hate it so much! The C API is old, error prone, > too big, and expose every single CPython implementation detail... The > C API is likely the first reason why faster implementation of Python > (PyPy?) are not as popular as they should be. > > The fact that the C API is so widely used prevents many evolutions of > CPython. CPython is stuck by its own (public) C API. > > PyPy developers spent a lot of time to make cffi great and advertize > it all around the world. While it's a great project, the fact is that > the C API *is* still used and the motivation to rewrite old code > written with the C API to cffi, Cython or whateer else is too low. > Maybe because the carrot is not big enough. Let's make a bigger > carrot! (bigger than PyPy amazing performances? I'm not sure that it's > doable :-( we will see) > > Using Cython, it's simpler to write C extensions and Cython code is > more "portable" on different Python versions, since the C API evolved > the last 10 years. For example, Python 3 renamed PyString with PyBytes > and dropped the PyInt type. Writing a C extension working on Python 2 > and Python 3 requires to "pollute" the code with many #ifdef. Cython > helps to reduce them (or even avoid them? sorry, I don't know well > Cython). > > The C *API* is tidely linked to the *ABI*. I tried to explain it with > an example in my article: > > https://vstinner.github.io/new-python-c-api.html > > A known ABI issue is that it's not possible to load C extensions > compiled in "release mode" on a Python compiled in "debug mode". The > debug mode changes the API in a subtle way which changes the ABI and > so makes C extensions incompatible. This issue prevents many people to > use a Python debug build, whereas debug builds are very helpful to > detect bugs earlier. > > Another issue is that C extensions must be recompiled for each Python > release (3.5, 3.6, 3.7, etc.). Linux vendors like Red Hat cannot > provide a single binary for multiple Python versions which prevents to > upgade the "system" Python in the lifecycle of a distribution major > version (or at least, it makes things much more complicated in term of > packaging, it multiply packages for each Python version...). > > . > . > . > > Don't worry (be happy!), I'm not writing this email to complain, but > to propose a solution. Aha! > > I propose to modify the API step by step to add more functions to the > "stable ABI" (PEP 384) to be able to compile more and more C > extensions using the "stable API" (please try to follow, first it was > B, now it's P... seriously, the difference between the ABI and the API > is subtle, to simplify let's say that it's the same thing, ok? :-)). > > I wrote a draft PEP, but I never found time to update it after the two > rounds of feedbacks (python-ideas and the sprint) to write a proper > PEP. So I will only give a link to my draft, sorry! > > https://github.com/vstinner/misc/blob/master/python/pep_c_api.rst > > In short, the plan is to add back the PyTuple_GET_ITEM() *API* to the > "stable API" but change its implementation to a *function call* rather > than the existing macro, so the compiled C extension will use a > function call and so don't rely on the ABI anymore. > > > My plan is to have two main milestones: > > (1) Python 3.7: Extend the *existing* opt-in "stable API" which > requires to compile C extensions in a special mode. Add maybe an > option in distutils to ease the compilation of a C extension with the > "stable API"? > > (2) In Python 3.8, --if the project is successful and the performance > overhead is acceptable compared the advantages of having C extensions > working on multiple Python verisons--, make the "stable API (without > implementation details)" the default, but add a new opt-in option to > give access to the "full API (with implementation details)" for > debuggers and other people who understand what they do (like Cython?). > > Note: currently, the "stable API" is accessible using Py_LIMITED_API > define, and the "full API" is accessible using Py_BUILD_CORE define. > No define gives the current C API. > > > My problem is more on the concrete implementation: > > * Need to provide two different API using the same filenames (like: > #include "Python.h") > > * Need to extend distutils to have a flag to compile a C extension > with one specific API (define Py_LIMITED_API or Py_BUILD_CORE?) > > * Need to test many C extensions and check how many extensions are broken > > > My plan for Python 3.7 is to not touch the current API at all. There > is no risk of backward incompatibility. You should only get issues if > you opt-in for the new API without implementation details. > > > Final note: Nothing new under the sun: PyPy already implemented my > "idea"! Where the idea is a C API without macros; PyTuple_GET_ITEM() > is already a function call in PyPy ;-) > > > Final question: Is it acceptable to iterate on many small changes on > the C API implement this idea in Python 3.7? Maybe only write partial > implementation, and finish it in Python 3.8? > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > k7hoven%40gmail.com > -- + Koos Zevenhoven + http://twitter.com/k7hoven +
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com