Re: [Python-Dev] General concerns about C API changes

2018-11-14 Thread Victor Stinner
Le jeu. 15 nov. 2018 à 01:06, Gregory P. Smith a écrit : > I expect the largest visible impact may be that a profiler may now attribute > CPU cycles takes by these code snippets to the function from the .h file > rather than directly to the functions the macro expanded in in the past due > to

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Gregory P. Smith
> > It seems like the discussion so far is: > > Victor: "I know people when people hear 'new API' they get scared and > think we're going to do a Python-3-like breaking transition, but don't > worry, we're never going to do that." > Nathaniel: "But then what does the new API add?" > Greg: "It lets

Re: [Python-Dev] General concerns about C API changes

2018-11-14 Thread Gregory P. Smith
On Tue, Nov 13, 2018 at 7:06 PM Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > Overall, I support the efforts to improve the C API, but over the last few > weeks have become worried. I don't want to hold up progress with fear, > uncertainty, and doubt. Yet, I would like to be more

Re: [Python-Dev] Clarify required Visual C++ compiler for binary extensions in 3.7

2018-11-14 Thread Victor Stinner
I maintain my own compatibility matrix: https://pythondev.readthedocs.io/windows.html#python-and-visual-studio-version-matrix Ned Deily asked to update the devguide from my table :-) https://github.com/python/devguide/issues/433 Victor Le mer. 14 nov. 2018 à 18:56, Maik Riechert a écrit : > >

[Python-Dev] Clarify required Visual C++ compiler for binary extensions in 3.7

2018-11-14 Thread Maik Riechert
Hi, I just wanted to point out that the page https://wiki.python.org/moin/WindowsCompilers doesn't mention Python 3.7 in the table. Is the compiler still Visual C++ 14.0? Thanks Maik ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Victor Stinner
Le mer. 14 nov. 2018 à 17:28, Paul Moore a écrit : > OK, got it. Thanks for taking the time to clarify and respond to my > concerns. Much appreciated. I'm my fault. I am failing to explain my plan proplerly. It seems like I had to update my website to better explain :-) Victor

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Paul Moore
On Wed, 14 Nov 2018 at 16:00, Victor Stinner wrote: > > In short, you don't have to modify your C extensions and they will > continue to work as before on Python 3.8. [...] > I hope that "later" we will get a faster CPython using new optimizations there>, only compatible with C extensions

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Victor Stinner
In short, you don't have to modify your C extensions and they will continue to work as before on Python 3.8. I only propose to add a new C API, don't touch the existing one in any way. Introducing backward incompatible changes to the existing C API is out of my plan. /usr/bin/python3.8 will

Re: [Python-Dev] General concerns about C API changes

2018-11-14 Thread Jeroen Demeyer
On 2018-11-14 04:06, Raymond Hettinger wrote: With cross module function calls, I'm less confident about what is happening If the functions are "static inline" (as opposed to plain "inline"), those aren't really cross-module function calls. Because the functions are "static" and defined in a

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Paul Moore
On Wed, 14 Nov 2018 at 14:39, Paul Moore wrote: > If it is the case that there's no need for any 3rd party code to > change in order to continue working with 3.8+, then I apologise for > the interruption. This is where being able to edit posts, a la Discourse would be useful :-) It occurs to

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Paul Moore
On Wed, 14 Nov 2018 at 14:28, Victor Stinner wrote: > > assuming the experiment is successful, forced (as opposed to opt-in) > > migration to the new API would be handled in a gradual, > > No, the current C API will remain available. No one is forced to do > anything. That's not part of my plan.

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Victor Stinner
Le mer. 14 nov. 2018 à 14:36, Paul Moore a écrit : > PS What percentage does "top 5" translate to? In terms of both > downloads and actual numbers of extensions? With only 5, it would be > very easy (I suspect) to get only scientific packages, and (for > example) miss out totally on database

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Paul Moore
On Tue, 13 Nov 2018 at 21:02, Victor Stinner wrote: > My plan is to select something like the top five most popular C > extensions based on PyPI download statistics. I cannot test > everything, I have to put practical limits. You should probably also consider embedding applications - these have

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread André Malo
On Dienstag, 13. November 2018 21:59:14 CET Victor Stinner wrote: > Le mar. 13 nov. 2018 à 20:32, André Malo a écrit : > > As long as they are recompiled. However, they will lose a lot of > > performance. Both these points have been mentioned somewhere, I'm > > certain, but it cannot be stressed

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Antoine Pitrou
On Wed, 14 Nov 2018 11:48:15 +0100 Victor Stinner wrote: > Le mer. 14 nov. 2018 à 11:24, Antoine Pitrou a écrit : > > For example in PyArrow we use PySequence_Fast_GET_ITEM() (*) > > Maybe PyArrow is a kind of C extension which should have one > implementation for the new C API (PyPy) and one

Re: [Python-Dev] General concerns about C API changes

2018-11-14 Thread Victor Stinner
> First, I attempted for "force inlining" (ex: > __attribute__((always_inline)) for GCC/Clang), but it has been decided > to not do that. Please read the discussion on the issue for the > rationale. > > https://bugs.python.org/issue35059 It has been decided to use "static inline" syntax since

Re: [Python-Dev] General concerns about C API changes

2018-11-14 Thread Victor Stinner
Hi, I made many different kinds of changes to the C API last weeks. None of them should impact the backward compatibility. If it's the case, the changes causing the backward compatibility should be reverted. The most controversial changes are the conversion of macros to static inline functions,

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Victor Stinner
Le mer. 14 nov. 2018 à 11:24, Antoine Pitrou a écrit : > For example in PyArrow we use PySequence_Fast_GET_ITEM() (*) Maybe PyArrow is a kind of C extension which should have one implementation for the new C API (PyPy) and one implementation for the current C API (CPython)? Cython can be used

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Antoine Pitrou
On Wed, 14 Nov 2018 11:03:49 +0100 Victor Stinner wrote: > > Oh, I should stop to promote my "CPython fork" idea. > > There is already an existing VM which is way faster than CPython but > its performances are limited by the current C API. The VM is called... > PyPy! > > The bet is that

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Victor Stinner
Le mer. 14 nov. 2018 à 03:24, Nathaniel Smith a écrit : > So I think what you're saying is that your goal is to get a > new/better/shinier VM, and the plan to accomplish that is: > > 1. Define a new C API. > 2. Migrate projects to the new C API. > 3. Build a new VM that gets benefits from only