Re: [Python-Dev] Let's change to C API!

2018-08-23 Thread Stefan Behnel
Greg Ewing schrieb am 23.08.2018 um 03:34: > Neil Schemenauer wrote: >> Perhaps a "argument clinic on steroids" would be the proper >> approach.  So, extensions would mostly be written in C.  However, we >> would have a pre-processor that does some "magic" to make using the >> Python API cleaner. >

Re: [Python-Dev] Let's change to C API!

2018-08-23 Thread Stefan Behnel
Antoine Pitrou schrieb am 23.08.2018 um 09:04: > On Thu, 23 Aug 2018 08:07:08 +0200 > Jeroen Demeyer wrote: >>> - the maintenance problem (how do ensure we can change small things in >>>the C API, especially semi-private ones, without having to submit PRs >>>to Cython as well) >> >> Why d

Re: [Python-Dev] Let's change to C API!

2018-08-23 Thread Antoine Pitrou
On Thu, 23 Aug 2018 08:07:08 +0200 Jeroen Demeyer wrote: > > > - the dependency / versioning problem (Cython is a large quick-evolving > >third-party package that we can't decently vendor) > > Is that a real problem? You're sort of doing the same thing with pip > already. Building and de

Re: [Python-Dev] Let's change to C API!

2018-08-22 Thread Jeroen Demeyer
On 2018-08-23 07:36, Antoine Pitrou wrote: - the bootstrap problem (Cython self-compiles with CPython) This is not a big problem: we can make sure that all stdlib dependencies of Cython either have PEP 399 pure Python implementations or we keep them in pure C. - the dependency / versioning

Re: [Python-Dev] Let's change to C API!

2018-08-22 Thread Antoine Pitrou
On Thu, 23 Aug 2018 13:34:02 +1200 Greg Ewing wrote: > Neil Schemenauer wrote: > > Perhaps a "argument clinic on steroids" would be the proper > > approach. So, extensions would mostly be written in C. However, we > > would have a pre-processor that does some "magic" to make using the > > Python

Re: [Python-Dev] Let's change to C API!

2018-08-22 Thread Greg Ewing
Neil Schemenauer wrote: Perhaps a "argument clinic on steroids" would be the proper approach. So, extensions would mostly be written in C. However, we would have a pre-processor that does some "magic" to make using the Python API cleaner. You seem to have started on the train of thought that

Re: [Python-Dev] Let's change to C API!

2018-08-22 Thread Neil Schemenauer
On 2018-07-31, Victor Stinner wrote: > I would be nice to be able to use something to "generate" C > extensions, maybe even from pure Python code. But someone has to > work on a full solution to implement that. Perhaps a "argument clinic on steroids" would be the proper approach. So, extensions w

Re: [Python-Dev] Let's change to C API!

2018-08-12 Thread Armin Rigo
Hi Antoine, On 11 August 2018 at 15:19, Antoine Pitrou wrote: > Jython and IronPython never got significant manpower AFAIK, so even > without being hindered by the C API, chances are they would never have > gotten very far. Both do not even seem to have stable releases > implementing the Python

Re: [Python-Dev] Let's change to C API!

2018-08-11 Thread Stefan Behnel
Antoine Pitrou schrieb am 11.08.2018 um 15:19: > On Fri, 10 Aug 2018 19:15:11 +0200 Armin Rigo wrote: >> Currently, the C API only allows Psyco-style JITting (much slower than >> PyPy). All three other points might not be possible at all without a >> seriously modified C API. Why? I have no proo

Re: [Python-Dev] Let's change to C API!

2018-08-11 Thread Antoine Pitrou
Hi Armin, On Fri, 10 Aug 2018 19:15:11 +0200 Armin Rigo wrote: > Currently, the C API only allows Psyco-style JITting (much slower than > PyPy). All three other points might not be possible at all without a > seriously modified C API. Why? I have no proof, but only > circumstantial evidence.

Re: [Python-Dev] Let's change to C API!

2018-08-10 Thread Armin Rigo
Hi, On 31 July 2018 at 13:55, Antoine Pitrou wrote: > It's just that I disagree that removing the C API will make CPython 2x > faster. > > Actually, important modern optimizations for dynamic languages (such as > inlining, type specialization, inline caches, object unboxing) don't > seem to depen

Re: [Python-Dev] Let's change to C API!

2018-08-04 Thread Nick Coghlan
On 31 July 2018 at 17:45, Antoine Pitrou wrote: > On Tue, 31 Jul 2018 09:27:03 +0200 > Jeroen Demeyer wrote: >> On 2018-07-31 08:58, Antoine Pitrou wrote: >> > I think Stefan is right that we >> > should push people towards Cython and alternatives, rather than direct >> > use of the C API (which

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Stefan Behnel
Jeroen Demeyer schrieb am 31.07.2018 um 14:01: > On 2018-07-31 12:56, Victor Stinner wrote: >> I would be nice to be able to use something to "generate" C >> extensions, maybe even from pure Python code. > > Cython has a "pure Python mode" which does exactly that. There are several > ways to inclu

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Stefan Behnel
Antoine Pitrou schrieb am 31.07.2018 um 09:45: > On Tue, 31 Jul 2018 09:27:03 +0200 > Jeroen Demeyer wrote: >> On 2018-07-31 08:58, Antoine Pitrou wrote: >>> I think Stefan is right that we >>> should push people towards Cython and alternatives, rather than direct >>> use of the C API (which peopl

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Brett Cannon
On Tue, 31 Jul 2018 at 10:32 Antoine Pitrou wrote: > > Well, I tried to subscribe to capi-sig, but I didn't get a > confirmation e-mail. > I subscribed yesterday without issue. I would email postmaster to try and find out what happened. -Brett > > Regards > > Antoine. > > > On Tue, 31 Jul 201

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Antoine Pitrou
Well, I tried to subscribe to capi-sig, but I didn't get a confirmation e-mail. Regards Antoine. On Tue, 31 Jul 2018 18:25:25 +0200 Victor Stinner wrote: > I replied on capi-sig. > > 2018-07-31 18:03 GMT+02:00 Antoine Pitrou : > > On Tue, 31 Jul 2018 15:34:05 +0200 > > Victor Stinner wrot

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
I replied on capi-sig. 2018-07-31 18:03 GMT+02:00 Antoine Pitrou : > On Tue, 31 Jul 2018 15:34:05 +0200 > Victor Stinner wrote: >> Antoine: would you mind to subscribe to the capi-sig mailing list? As >> expected, they are many interesting points discussed here, but I would >> like to move all C

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Eric Snow
On Tue, Jul 31, 2018 at 7:35 AM Victor Stinner wrote: > 2018-07-31 14:01 GMT+02:00 Jeroen Demeyer : > > Anyway, I know that this is probably not going to happen, but I just wanted > > to bring it up in case people would find it a great idea. But maybe not many > > CPython core developers actually

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Antoine Pitrou
On Tue, 31 Jul 2018 15:34:05 +0200 Victor Stinner wrote: > Antoine: would you mind to subscribe to the capi-sig mailing list? As > expected, they are many interesting points discussed here, but I would > like to move all C API discussions to capi-sig. I only continue on > python-dev since you star

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Jeroen Demeyer
On 2018-07-31 15:34, Victor Stinner wrote: But I never used Cython nor cffi, so I'm not sure which one is the most appropriate depending on the use case. Cython is a build-time tool, while cffi is a run-time tool. But Cython does a lot more than just FFI. It is a Python->C compiler which can

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
2018-07-31 14:01 GMT+02:00 Jeroen Demeyer : > Anyway, I know that this is probably not going to happen, but I just wanted > to bring it up in case people would find it a great idea. But maybe not many > CPython core developers actually know and use Cython? I know that Yury wants to use Cython for

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
Antoine: would you mind to subscribe to the capi-sig mailing list? As expected, they are many interesting points discussed here, but I would like to move all C API discussions to capi-sig. I only continue on python-dev since you started here (and ignored my request to start discussing my idea on ca

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Jeroen Demeyer
On 2018-07-31 12:56, Victor Stinner wrote: We try to make CPython build as simple as possible. I'm quite sure that Cython rely on the stdlib. It does rely on modules like "re" and "functools". Would depending on Cython open a chicken-and-egg issue? Yes, that's a problem but it's not unsolva

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Antoine Pitrou
On Tue, 31 Jul 2018 12:51:23 +0200 Victor Stinner wrote: > 2018-07-31 8:58 GMT+02:00 Antoine Pitrou : > > What exactly in the C API made it slow or non-promising? > > > >> The C API requires that your implementations make almost all the same > >> design choices that CPython made 25 years ago (C

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
2018-07-31 9:27 GMT+02:00 Jeroen Demeyer : > On 2018-07-31 08:58, Antoine Pitrou wrote: >> >> I think Stefan is right that we >> should push people towards Cython and alternatives, rather than direct >> use of the C API (which people often fail to use correctly, in my >> experience). > > > I know t

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
2018-07-31 8:58 GMT+02:00 Antoine Pitrou : > What exactly in the C API made it slow or non-promising? > >> The C API requires that your implementations make almost all the same >> design choices that CPython made 25 years ago (C structures, memory >> allocators, reference couting, specific GC imple

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Antoine Pitrou
On Tue, 31 Jul 2018 09:27:03 +0200 Jeroen Demeyer wrote: > On 2018-07-31 08:58, Antoine Pitrou wrote: > > I think Stefan is right that we > > should push people towards Cython and alternatives, rather than direct > > use of the C API (which people often fail to use correctly, in my > > experience)

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Jeroen Demeyer
On 2018-07-31 08:58, Antoine Pitrou wrote: I think Stefan is right that we should push people towards Cython and alternatives, rather than direct use of the C API (which people often fail to use correctly, in my experience). I know this probably isn't the correct place to bring it up, but I'm

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Antoine Pitrou
On Tue, 31 Jul 2018 02:29:42 +0200 Victor Stinner wrote: > >> Last year, I gave a talk at the Language Summit (during Pycon) to > >> explain that CPython should become 2x faster to remain competitive. > >> IMHO all attempts to optimize Python (CPython forks) have failed > >> because they have been

Re: [Python-Dev] Let's change to C API!

2018-07-30 Thread Victor Stinner
>> Last year, I gave a talk at the Language Summit (during Pycon) to >> explain that CPython should become 2x faster to remain competitive. >> IMHO all attempts to optimize Python (CPython forks) have failed >> because they have been blocked by the C API which implies strict >> constraints. > > Wel

Re: [Python-Dev] Let's change to C API!

2018-07-30 Thread Ronald Oussoren via Python-Dev
> On 30 Jul 2018, at 10:20, Victor Stinner wrote: > > The API leaking all implementation details will remain available as an opt-in > option for Cython, cffi and debug tools. But this API will only be usable on > the "slow" Python runtime, the one which keeps maximum backward > compatibility

Re: [Python-Dev] Let's change to C API!

2018-07-30 Thread Antoine Pitrou
Hi Victor, On Sun, 29 Jul 2018 21:47:51 +0200 Victor Stinner wrote: > > I just sent an email to the capi-sig mailing list. Since this mailing > list was idle for months, I copy my email here to get a wider > audience. But if possible, I would prefer that you join me on capi-sig > to reply ;-)

Re: [Python-Dev] Let's change to C API!

2018-07-30 Thread Victor Stinner
The API leaking all implementation details will remain available as an opt-in option for Cython, cffi and debug tools. But this API will only be usable on the "slow" Python runtime, the one which keeps maximum backward compatibility. To get new optimizations, you have to use Py_INCREF() and avoid a

Re: [Python-Dev] Let's change to C API!

2018-07-29 Thread Chris Angelico
On Mon, Jul 30, 2018 at 10:46 AM, Victor Stinner wrote: > 2018-07-29 23:41 GMT+02:00 Jeroen Demeyer : >> For example, you mention that you want to make Py_INCREF() a function call >> instead of a macro. But since Py_INCREF is very common, I would guess that >> this would make performance worse (no

Re: [Python-Dev] Let's change to C API!

2018-07-29 Thread Victor Stinner
2018-07-29 23:41 GMT+02:00 Jeroen Demeyer : > For example, you mention that you want to make Py_INCREF() a function call > instead of a macro. But since Py_INCREF is very common, I would guess that > this would make performance worse (not by much maybe but surely measurable). For the very specific

Re: [Python-Dev] Let's change to C API!

2018-07-29 Thread Jeroen Demeyer
My first impression is that making things faster and hiding implementation details in the ABI are contrary goals. I agree with hiding implementation details in the API but not in the ABI. For example, you mention that you want to make Py_INCREF() a function call instead of a macro. But since P

[Python-Dev] Let's change to C API!

2018-07-29 Thread Victor Stinner
Hi, I just sent an email to the capi-sig mailing list. Since this mailing list was idle for months, I copy my email here to get a wider audience. But if possible, I would prefer that you join me on capi-sig to reply ;-) -- Hi, Last year, I gave a talk at the Language Summit (during Pycon) to ex