[Python-Dev] Re: Making code object APIs unstable

2021-09-07 Thread Stefan Behnel
Guido van Rossum schrieb am 07.09.21 um 00:44: In addition, I just heard from the SC that they've approved the exception. So we will remove these two APIs from 3.11 without deprecation. Erm, hang on – when I wrote that I'm fine with *changing* them, I wasn't thinking of actually *removing*

[Python-Dev] Re: Making code object APIs unstable

2021-08-31 Thread Stefan Behnel
Guido van Rossum schrieb am 13.08.21 um 19:24: In 3.11 we're changing a lot of details about code objects. Part of this is the "Faster CPython" work, part of it is other things (e.g. PEP 657 -- Fine Grained Error Locations in Tracebacks). As a result, the set of fields of the code object is

[Python-Dev] Re: PR checks hang because travis does not report back to github

2020-10-06 Thread Stefan Behnel
Victor Stinner schrieb am 05.10.20 um 12:25: > Would you mind reporting the issue to > https://github.com/python/core-workflow/issues so we can aggregate > information about this issue? Done. Stefan ___ Python-Dev mailing list -- python-dev@python.org

[Python-Dev] Re: PR checks hang because travis does not report back to github

2020-10-05 Thread Stefan Behnel
Ned Deily schrieb am 05.10.20 um 01:19: > On Oct 4, 2020, at 15:55, Terry Reedy wrote: >> >> On 10/4/2020 2:32 PM, Mariatta wrote: >>> This is a known issue and I have brought it up in GitHub OS Maintainers >>> Feedback Group. It happens to other projects as well. >>> Currently we have branch

[Python-Dev] PR checks hang because travis does not report back to github

2020-10-04 Thread Stefan Behnel
Hi devs, I have a trivial documentation PR https://github.com/python/cpython/pull/22464 for which travis (unsurprisingly) had a successful run, https://travis-ci.com/github/python/cpython/builds/187435578 but github lists the travis build as "created" instead of "passed".

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-07-02 Thread Stefan Behnel
Victor Stinner schrieb am 02.07.20 um 00:07: > Le mer. 1 juil. 2020 à 23:43, Eric V. Smith a écrit : >> >> On 7/1/2020 3:43 PM, Stefan Behnel wrote: >>> Petr Viktorin schrieb am 30.06.20 um 14:51: >>>> For example, could we only deprecate the bad p

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-07-01 Thread Stefan Behnel
Petr Viktorin schrieb am 30.06.20 um 14:51: > For example, could we only deprecate the bad parts, but not remove them > until the experiments actually show that they are preventing a beneficial > change? Big nod on this one. Stefan ___ Python-Dev

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-06-26 Thread Stefan Behnel
Victor Stinner schrieb am 26.06.20 um 14:39: > Well, the general problem is to track when the caller ends using a resource. Although that is less of a problem if you only allow exposing the internal data representation and nothing else. In that case, you can tie the lifetime of the data access to

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-06-24 Thread Stefan Behnel
Gustavo Carneiro schrieb am 24.06.20 um 19:19: > On Wed, 24 Jun 2020 at 17:22, Victor Stinner wrote: >> The question becomes: how to promote the limited C API? Should it >> become the default, rather than an opt-in option? > > It would be interesting to find out what is the performance impact of

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-06-24 Thread Stefan Behnel
Victor Stinner schrieb am 24.06.20 um 17:40: > My practical problem is how to prevent C extensions accessing the > PyFloatObject.ob_fval member directly. Do extensions really do that in their code? I mean, there *is* a macro for doing exactly this thing, which suggests that users should exactly

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-06-24 Thread Stefan Behnel
Victor Stinner schrieb am 24.06.20 um 02:14: > Le mar. 23 juin 2020 à 16:56, Stefan Behnel a écrit : >>> Members of ``PyObject`` and ``PyTupleObject`` structures have not >>> changed since the "Initial revision" commit (1990) >> >> While I see an

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

2020-06-23 Thread Stefan Behnel
Hi Victor, thanks for your continued work on improving the C-API. I'll comment on the PEP inline. Victor Stinner schrieb am 22.06.20 um 14:10: > PEP available at: https://www.python.org/dev/peps/pep-0620/ > [...] > Motivation > == > > The C API blocks CPython evolutions >

[Python-Dev] Re: (PEP 620) C API for efficient loop iterating on a sequence of PyObject** or other C types

2020-06-23 Thread Stefan Behnel
Victor Stinner schrieb am 23.06.20 um 11:18: > Maybe an object can > generate a temporary PyObject** view which requires to allocate > resources (like memory) and the release function would release these > resources. I agree that this is more explicit when it comes to resource management, but

[Python-Dev] Re: Cython and incompatible C API changes

2020-06-22 Thread Stefan Behnel
Victor Stinner schrieb am 17.06.20 um 13:25: > Le mer. 17 juin 2020 à 12:38, Petr Viktorin a écrit : >>> There is an ongoing discussion about always requiring to run Cython >>> when installing a C extension which uses Cython. >> >> Do you have a link to that discussion? Yeah, I was wondering,

[Python-Dev] Re: PEP: Modify the C API to hide implementation details

2020-04-14 Thread Stefan Behnel
Paul Moore schrieb am 13.04.20 um 14:25: > On a related but different note, what is the recommended policy > (assuming it's not to use the C API) for embedding Python, and for > exposing the embedding app to Python as a C extension? My standard > example of this is the Vim interface to Python -

[Python-Dev] Re: PEP: Modify the C API to hide implementation details

2020-04-14 Thread Stefan Behnel
André Malo schrieb am 14.04.20 um 13:39: > I think, it does not serve well as a policy for CPython. Since we're talking > hypotheticals right now, if Cython vanishes tomorrow, we're kind of left > empty > handed. Such kind of a runtime, if considered part of the compatibility > "promise",

[Python-Dev] Re: PEP: Modify the C API to hide implementation details

2020-04-14 Thread Stefan Behnel
Steve Dower schrieb am 14.04.20 um 00:27: > On 13Apr2020 2308, André Malo wrote: >> For one thing, if you open up APIs for Cython, they're open for everybody >> (Cython being "just" another C extension). >> More to the point: The ABIs have the same problem as they have now, >> regardless >> how

[Python-Dev] Accepting PEP 573 (Module State Access from C Extension Methods)

2020-03-23 Thread Stefan Behnel
As (first-time) BDFL delegate, I accept PEP 573 for Python 3.9, "Module State Access from C Extension Methods" https://www.python.org/dev/peps/pep-0573/ Petr, Nick, Eric and Marcel, thank you for your work and intensive discussions on this PEP, and also to everyone else who got involved on

[Python-Dev] Re: Last call for comments on PEP 573 (Module State Access from C Extension Methods)

2020-03-10 Thread Stefan Behnel
Hi Petr! Petr Viktorin schrieb am 14.01.20 um 14:37: > It also includes a more drastic change: it removes the MRO walker from the > proposal. > Reflecting on the feedback, it became clear to me that a MRO walker, as it > was described, won't give correct results in all cases: specifically, is a >

[Python-Dev] Last call for comments on PEP 573 (Module State Access from C Extension Methods)

2019-11-25 Thread Stefan Behnel
Hi all, I think that PEP 573 is ready to be accepted, to greatly improve the state of extension modules in CPython 3.9. https://www.python.org/dev/peps/pep-0573/ It has come a long way since the original proposal and went through several iterations and discussions by various interested people,

[Python-Dev] Re: Pass the Python thread state to internal C functions

2019-11-15 Thread Stefan Behnel
Victor Stinner schrieb am 12.11.19 um 23:03: > Are you ok to modify internal C functions to pass explicitly tstate? FWIW, I started doing the same internally in Cython a while back, because like others, I also considered it wasteful to look it up all over the place, often multiple times inside of

[Python-Dev] Re: The Python 2 death march

2019-09-10 Thread Stefan Behnel
Ned Batchelder schrieb am 10.09.19 um 16:54: > this seems confusing to me > What does the "official EOL date" mean if there's a release in April? Also, what day in April? For example, planning the release for the 1st could possibly add further to the confusion. Stefan

[Python-Dev] Re: Comparing dict.values()

2019-07-29 Thread Stefan Behnel
Kristian Klette schrieb am 23.07.19 um 22:59: > During the sprints after EuroPython, I made an attempt at adding support for > comparing the results from `.values()` of two dicts. > > Currently the following works as expected: > > ``` > d = {'a': 1234} > > d.keys() == d.keys() > d.items() ==

[Python-Dev] Re: Expected stability of PyCode_New() and types.CodeType() signatures

2019-06-11 Thread Stefan Behnel
Victor Stinner schrieb am 12.06.19 um 00:09: > So yeah, the PyCode_New() change is very annoying in practical, since > every single project using Cython requires a new release in practice. I think Cython's deal with regard to this is: """ If you use Cython, we will try hard to cover up the

[Python-Dev] Re: Expected stability of PyCode_New() and types.CodeType() signatures

2019-06-11 Thread Stefan Behnel
Neil Schemenauer schrieb am 08.06.19 um 22:46: > It would be great if we had a system that did CI testing with the > top PyPI modules. E.g. pull the latest versions of the top 100 PyPI > modules and test them with the latest CPython branch. FWIW, travis-ci provides the latest CPython master

Re: [Python-Dev] Expected stability of PyCode_New() and types.CodeType() signatures

2019-06-01 Thread Stefan Behnel
Serhiy Storchaka schrieb am 01.06.19 um 09:02: > I have a related proposition. Yesterday I have reported two bugs (and Pablo > quickly fixed them) related to handling positional-only arguments. These > bugs were occurred due to subtle changing the meaning of co_argcount. When > we make some

Re: [Python-Dev] PEP 580/590 discussion

2019-05-10 Thread Stefan Behnel
Petr Viktorin schrieb am 10.05.19 um 00:07: > On 5/9/19 5:33 PM, Jeroen Demeyer wrote: >> Maybe you misunderstood my proposal. I want to allow both for extra >> flexibility: >> >> - METH_FASTCALL (possibly combined with METH_KEYWORDS) continues to work >> as before. If you don't want to care about

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-27 Thread Stefan Behnel
Matthias Klose schrieb am 25.04.19 um 13:48: > Are there use cases where you only want to load *some* > debug extensions, even if more are installed? Not sure if there are _important_ use cases (that could justify certain design decisions), but I can certainly imagine using a non-debug (and

Re: [Python-Dev] Any core dev event plans for EP19?

2019-04-26 Thread Stefan Behnel
Berker Peksağ schrieb am 26.04.19 um 01:15: > Note that this year's core dev sprint will be held in London. See > https://discuss.python.org/t/2019-core-dev-sprint-location-date/489 > for the previous discussion. There are only two months between both > events, so perhaps we can leave things like

[Python-Dev] Any core dev event plans for EP19?

2019-04-25 Thread Stefan Behnel
Hi core devs, there are several core dev events happening at the US PyCon this year, so I was wondering if we could organise something similar at EuroPython. Does anyone have any plans or ideas already? And, how many of us are planning to attend EP19 in Basel this year? Unless there's something

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

2019-04-24 Thread Stefan Behnel
Jeroen Demeyer schrieb am 24.04.19 um 09:24: > On 2019-04-24 01:44, Victor Stinner wrote: >> I would like to >> be able to run C extensions compiled in release mode on a Python >> compiled in debug mode > > That seems like a very good idea. I would certainly use the debug mode > while developing

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Stefan Behnel
Serhiy Storchaka schrieb am 11.04.19 um 17:30: > If reducing the Python memory footprint is an argument for disabling > Py_TRACE_REFS, it is a weak argument because there is larger overhead in > the debug build. I think what Victor is argueing is rather that we have better ways to debug memory

Re: [Python-Dev] Replacement for array.array('u')?

2019-03-22 Thread Stefan Behnel
Antoine Pitrou schrieb am 22.03.19 um 11:39: > On Fri, 22 Mar 2019 20:31:33 +1300 Greg Ewing wrote: >> A poster on comp.lang.python is asking about array.array('u'). >> He wants an efficient mutable collection of unicode characters >> that can be initialised from a string. > > TBH, I think anyone

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-21 Thread Stefan Behnel
Victor Stinner schrieb am 21.03.19 um 01:22: > Alternatives have been proposed like a recipe to sort node attributes > before serialization, but honestly, it's way too complex. Hm, really? Five lines of simple and obvious Python code, that provide a fast and completely Python-version agnostic

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Stefan Behnel
Ned Batchelder schrieb am 19.03.19 um 12:53: > I need to re-engineer my tests. … or sort the attributes before serialisation, or use C14N always, or change your code to create the attributes in sorted-by-name order. The new behaviour allows for a couple of ways to deal with the issue of backwards

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Stefan Behnel
Nathaniel Smith schrieb am 19.03.19 um 00:15: > That seems potentially simpler to implement than canonical XML > serialization C14N is already implemented for ElementTree, just needs to be ported to Py3.8 and merged. https://bugs.python.org/issue13611 Stefan

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-19 Thread Stefan Behnel
Steve Dower schrieb am 19.02.19 um 21:40: > On 19Feb2019 1212, Stefan Behnel wrote: >> Then it's up to the users to decide >> how much work they want to invest into keeping up with C-API changes vs. >> potentially sub-optimal but stable C-API usage. > [...] > And it's no

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-19 Thread Stefan Behnel
Nick Coghlan schrieb am 19.02.19 um 15:00: > On Tue, 19 Feb 2019 at 20:41, Antoine Pitrou wrote: >> On Mon, 18 Feb 2019 19:04:31 -0800 Steve Dower wrote: >>> If you always rebuild your extension for every micro version (3.x.y) of >>> CPython, then sure, go ahead and use this. >> >> Usually we

Re: [Python-Dev] Add more SyntaxWarnings?

2019-01-30 Thread Stefan Behnel
MRAB schrieb am 29.01.19 um 19:55: > On 2019-01-29 13:44, Nick Coghlan wrote: >> FWIW, we have pretty decent evidence that error messages don't have to >> provide a wonderful explanation on their own in order to be helpful: >> they just need to be distinctive enough that a web search will >>

Re: [Python-Dev] C API changes

2018-11-26 Thread Stefan Behnel
Armin Rigo schrieb am 26.11.18 um 06:37: > On Sun, 25 Nov 2018 at 10:15, Stefan Behnel wrote: >> Overall, this seems like something that PyPy could try out as an >> experiment, by just taking a simple extension module and replacing all >> increfs with newref assignments. And o

Re: [Python-Dev] C API changes

2018-11-25 Thread Stefan Behnel
Hi Armin, Armin Rigo schrieb am 25.11.18 um 06:15: > On Sat, 24 Nov 2018 at 22:17, Stefan Behnel wrote: >> Couldn't this also be achieved via reference counting? Count only in C >> space, and delete the "open object" when the refcount goes to 0? > > The point

Re: [Python-Dev] C API changes

2018-11-24 Thread Stefan Behnel
Armin Rigo schrieb am 23.11.18 um 14:15: > In PyPy we'd have a global table of > "open objects", and a handle would be an index in that table; closing > a handle means writing NULL into that table entry. No emulated > reference counting needed: we simply use the existing GC to keep alive >

Re: [Python-Dev] General concerns about C API changes

2018-11-18 Thread Stefan Behnel
Gregory P. Smith schrieb am 15.11.18 um 01:03: > From my point of view: A static inline function is a much nicer modern code > style than a C preprocessor macro. It's also slower to compile, given that function inlining happens at a much later point in the compiler pipeline than macro expansion.

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-18 Thread Stefan Behnel
Neil Schemenauer schrieb am 17.11.18 um 00:10: > I think making PyObject an opaque pointer would help. ... well, as long as type checks are still as fast as with "ob_type", and visible to the C compiler so that it can eliminate redundant ones, I wouldn't mind. :) > - Borrowed references are a

Re: [Python-Dev] The future of the wchar_t cache

2018-10-20 Thread Stefan Behnel
Serhiy Storchaka schrieb am 20.10.2018 um 13:06: > Currently the PyUnicode object contains two caches: for UTF-8 > representation and for wchar_t representation. They are needed not for > optimization but for supporting C API which returns borrowed references for > such representations. > > The

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

2018-09-21 Thread Stefan Behnel
Guido van Rossum schrieb am 21.09.2018 um 19:35: > Though now I start worrying about interned strings. That's a concept that's > a little closer to being a feature. True. While there's the general '"ab"+"cd" is (not) "abcd"' caveat, I'm sure quite a bit of code out there assumes that parsed

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

2018-09-21 Thread Stefan Behnel
Larry Hastings schrieb am 14.09.2018 um 23:27: > What the patch does: it takes all the Python modules that are loaded as > part of interpreter startup and deserializes the marshalled .pyc file into > precreated objects stored as static C data. What about the small integers cache? The C

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

2018-09-20 Thread Stefan Behnel
Carl Shapiro schrieb am 20.09.2018 um 20:21: > On Wed, Sep 19, 2018 at 12:32 AM, Stefan Behnel wrote: > >> Also, one thing that would be interesting to find out is whether constant >> Python data structures can actually be pre-allocated in the data segment >> (and I m

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

2018-09-19 Thread Stefan Behnel
Carl Shapiro schrieb am 18.09.2018 um 22:44: > How might people feel about using the linker to bundle a list of pre-loaded > modules into a single-file executable? One way to do that would be to compile Python modules with Cython and link them in statically, instead of compiling them to .pyc

Re: [Python-Dev] Use of Cython

2018-09-04 Thread Stefan Behnel
Yury Selivanov schrieb am 04.09.2018 um 18:19: > On Sat, Sep 1, 2018 at 6:12 PM Stefan Behnel wrote: >> Yury Selivanov schrieb am 07.08.2018 um 19:34: >>> The first goal is to compile mypy with it to make it faster, so I hope >>> that the project will be completed. >&

Re: [Python-Dev] Use of Cython

2018-09-01 Thread Stefan Behnel
Yury, given that people are starting to quote enthusiastically the comments you made below, let me set a couple of things straight. Yury Selivanov schrieb am 07.08.2018 um 19:34: > On Mon, Aug 6, 2018 at 11:49 AM Ronald Oussoren via Python-Dev wrote: > >> I have no strong opinion on using

Re: [Python-Dev] Let's change to C API!

2018-08-23 Thread Stefan Behnel
Greg Ewing schrieb am 23.08.2018 um 03:34: > Neil Schemenauer wrote: >> Perhaps a "argument clinic on steroids" would be the proper >> approach.  So, extensions would mostly be written in C.  However, we >> would have a pre-processor that does some "magic" to make using the >> Python API cleaner.

Re: [Python-Dev] Let's change to C API!

2018-08-23 Thread Stefan Behnel
Antoine Pitrou schrieb am 23.08.2018 um 09:04: > On Thu, 23 Aug 2018 08:07:08 +0200 > Jeroen Demeyer wrote: >>> - the maintenance problem (how do ensure we can change small things in >>>the C API, especially semi-private ones, without having to submit PRs >>>to Cython as well) >> >> Why

Re: [Python-Dev] Can we split PEP 489 (extension module init) ?

2018-08-11 Thread Stefan Behnel
Petr Viktorin schrieb am 10.08.2018 um 13:48: > Would this be better than a flag + raising an error on init? Ok, I've implemented this in Cython for now, to finally move the PEP-489 support forward. The somewhat annoying drawback is that module reloading previously *seemed* to work, simply

Re: [Python-Dev] Let's change to C API!

2018-08-11 Thread Stefan Behnel
Antoine Pitrou schrieb am 11.08.2018 um 15:19: > On Fri, 10 Aug 2018 19:15:11 +0200 Armin Rigo wrote: >> Currently, the C API only allows Psyco-style JITting (much slower than >> PyPy). All three other points might not be possible at all without a >> seriously modified C API. Why? I have no

Re: [Python-Dev] Can we split PEP 489 (extension module init) ?

2018-08-10 Thread Stefan Behnel
Petr Viktorin schrieb am 10.08.2018 um 11:51: > On 08/10/18 11:21, Stefan Behnel wrote: >> coming back to PEP 489 [1], the multi-phase extension module >> initialization. We originally designed it as an "all or nothing" feature, >> but as it turns out, the "

[Python-Dev] Can we split PEP 489 (extension module init) ?

2018-08-10 Thread Stefan Behnel
Hi, coming back to PEP 489 [1], the multi-phase extension module initialization. We originally designed it as an "all or nothing" feature, but as it turns out, the "all" part is so difficult to achieve that most potential users end up with "nothing". So, my question is: could we split it up so

Re: [Python-Dev] Use of Cython

2018-08-09 Thread Stefan Behnel
Hi, this is getting a bit out of scope for this list. I propose to move further questions about general Cython usage to he cython-users mailing list. Matěj Cepl schrieb am 08.08.2018 um 12:44: > On 2018-08-06, 15:13 GMT, Stefan Behnel wrote: >> Not sure I understand this correctly, bu

Re: [Python-Dev] Use of Cython

2018-08-06 Thread Stefan Behnel
Ronald Oussoren via Python-Dev schrieb am 06.08.2018 um 15:25: >> On 5 Aug 2018, at 18:14, Nick Coghlan wrote: >> On 5 August 2018 at 18:06, Ronald Oussoren wrote: >>> I’m not sure if I understand this, ctypes and cffi are used to access C APIs >>> without writing C code including the CPython API

Re: [Python-Dev] Use of Cython

2018-08-04 Thread Stefan Behnel
Antoine Pitrou schrieb am 04.08.2018 um 15:57: > Le 04/08/2018 à 15:13, Nick Coghlan a écrit : >> >> It'd be *really* nice to at least be able to write some of the C API >> tests directly in Cython rather than having to fiddle about with >> splitting the test between the regrtest parts that

Re: [Python-Dev] Using Cython for the stdlib (was: Let's change to C API!)

2018-08-01 Thread Stefan Behnel
Brett Cannon schrieb am 01.08.2018 um 18:17: > On Tue, 31 Jul 2018 at 13:42 Stefan Behnel wrote: >> Antoine Pitrou schrieb am 31.07.2018 um 09:45: >>> Also, a C extension can be built-in (linked statically into the >>> interpreter), which I think would be hard to do

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Stefan Behnel
Jeroen Demeyer schrieb am 31.07.2018 um 14:01: > On 2018-07-31 12:56, Victor Stinner wrote: >> I would be nice to be able to use something to "generate" C >> extensions, maybe even from pure Python code. > > Cython has a "pure Python mode" which does exactly that. There are several > ways to

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Stefan Behnel
Antoine Pitrou schrieb am 31.07.2018 um 09:45: > On Tue, 31 Jul 2018 09:27:03 +0200 > Jeroen Demeyer wrote: >> On 2018-07-31 08:58, Antoine Pitrou wrote: >>> I think Stefan is right that we >>> should push people towards Cython and alternatives, rather than direct >>> use of the C API (which

Re: [Python-Dev] [PEP 576/580] Reserve one type slot for Cython

2018-07-30 Thread Stefan Behnel
Jeroen Demeyer schrieb am 30.07.2018 um 16:40: > On 2018-07-30 15:35, INADA Naoki wrote: >> As repeatedly said, PEP 580 is very complicated protocol >> when just implementing callable object. > > Can you be more concrete what you find complicated? Maybe I can improve the > PEP to explain it more.

Re: [Python-Dev] Benchmarks why we need PEP 576/579/580

2018-07-22 Thread Stefan Behnel
Jeroen Demeyer schrieb am 22.07.2018 um 22:54: > On 2018-07-22 22:32, Antoine Pitrou wrote: >> - more importantly, issue26110 is entirely internal changes, while you >>    are proposing to add a new protocol (which is like a new API) > > Just to make sure I understand you: your point is that it's

Re: [Python-Dev] Benchmarks why we need PEP 576/579/580

2018-07-22 Thread Stefan Behnel
Guido van Rossum schrieb am 22.07.2018 um 01:14: > The cost would be if we were to end up maintaining all that code and it > wouldn’t make much difference. Well, this is exactly my point. Someone has to maintain the *existing* code base and help newcomers to get into it and understand it. This is

Re: [Python-Dev] Benchmarks why we need PEP 576/579/580

2018-07-21 Thread Stefan Behnel
Guido van Rossum schrieb am 21.07.2018 um 22:46: > Given the cost of a mistake here I recommend a higher standard. May I ask what you think the "cost of a mistake" is here? Jeroen has already implemented most of this, and is willing to provide essentially a shrink-wrapped implementation. He has

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Stefan Behnel
INADA Naoki schrieb am 11.07.2018 um 02:01: > On Wed, Jul 11, 2018 at 7:47 AM Victor Stinner wrote: >> I proposed something simpler, but nobody tried to implement it. >> Instead of calling the long and complex PyArg_Parse...() functions, >> why not generating C code to parse arguments instead? The

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Stefan Behnel
INADA Naoki schrieb am 11.07.2018 um 02:12: > On Tue, Jul 10, 2018 at 10:20 PM Antoine Pitrou wrote: >> On Tue, 10 Jul 2018 21:59:28 +0900 >> INADA Naoki wrote: >>> >>> Then, the function is called from another C extension like this: >>> >>> PyObject_CallFunction(func, "n", 42); >>> >>> Currently,

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-08 Thread Stefan Behnel
Jeroen Demeyer schrieb am 08.07.2018 um 09:07: > On 2018-07-07 10:55, Serhiy Storchaka wrote: >> The first part of >> handling arguments can be made outside of the C function, by the calling >> API. > > Sure, it could be done but I don't see the advantage. I don't think you > will gain

Re: [Python-Dev] PEP 575, 576, 579 and 580

2018-07-07 Thread Stefan Behnel
INADA Naoki schrieb am 07.07.2018 um 17:16: >> 2. The new API should be used internally so that 3rd party extensions >> are not second class citizens in term of call performance. > > These PEPs proposes new public protocol which can be implemented > by 3rd party extensions, especially Cython. >

Re: [Python-Dev] PEP 575, 576, 579 and 580

2018-07-07 Thread Stefan Behnel
Nick Coghlan schrieb am 07.07.2018 um 16:14: > when the new calling > convention is tied to a protocol that any type can implement (as PEP > 580 proposes), the concern doesn't even arise. Nick, +1 to all of what you said in your reply, and I also really like the fact that this proposal is

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-07 Thread Stefan Behnel
Hi Serhiy! Serhiy Storchaka schrieb am 07.07.2018 um 10:55: > There is my idea. Split every of keyword argument parsing functions on two > parts. The first part linearize keyword arguments, it converts positional > and keyword arguments (in whatever form they were presented) into a linear > array

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods

2018-07-07 Thread Stefan Behnel
INADA Naoki schrieb am 07.07.2018 um 10:08: > Thank you. Do you plan to make it default when PEP 580 is accepted > and implemented? It will become the default at some point, I'm sure. Note that we will still have to support older Python versions, though, currently 2.6+, which would not have the

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods

2018-07-07 Thread Stefan Behnel
INADA Naoki schrieb am 07.07.2018 um 06:10: > How often "custom method type" are used? > > I thought Cython use it by default. > But when I read code generated by Cython, I can't find it. > It uses normal PyMethodDef and tp_methods. > > I found CyFunction in Cython repository, but I can't find >

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-07 Thread Stefan Behnel
Jeroen Demeyer schrieb am 05.07.2018 um 16:53: > The only case when this handling of keywords is suboptimal is when using > **kwargs. In that case, a dict must be converted to a tuple. It looks hard > to me to support efficiently both the case of fixed keyword arguments > (f(foo=x)) and a keyword

Re: [Python-Dev] Can we make METH_FASTCALL public, from Python 3.7? (ref: PEP 579

2018-06-20 Thread Stefan Behnel
Serhiy Storchaka schrieb am 20.06.2018 um 18:56: > 20.06.18 18:42, INADA Naoki пише: >> I don't have any idea about changing METH_FASTCALL more. >> If Victor and Serhiy think so, and PyPy maintainers like it too, I want >> to make it public as soon as possible. > > I don't have objections against

[Python-Dev] C-level calling (was: PEP 575 (Unifying function/method classes) update)

2018-06-20 Thread Stefan Behnel
Victor Stinner schrieb am 19.06.2018 um 16:59: > 2018-06-19 13:58 GMT+02:00 Jeroen Demeyer : >> Personally, I think that you are exaggerating these issues. > > I'm not trying to convince you to abandon the idea. I would be happy > to be able to use FASTCALL in more cases! I just tried to explain

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update

2018-06-18 Thread Stefan Behnel
Victor Stinner schrieb am 18.06.2018 um 15:09: > I tried two options to add support for FASTCALL on calling an object: > add a flag in tp_flags and reuse tp_call, or add a new tp_fastcall > slot. I failed to implement correctly any of these two options. > > There are multiple issues with

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update

2018-06-17 Thread Stefan Behnel
Ronald Oussoren schrieb am 17.06.2018 um 14:50: > Why did you add a tp_ccalloffset slot to the type with the actual information > in instances instead of storing the information in a slot? If the configuration of the callable was in the type, you would need a separate type for each kind of

Re: [Python-Dev] PEP 574 (pickle 5) implementation and backport available

2018-05-26 Thread Stefan Behnel
Antoine Pitrou schrieb am 25.05.2018 um 23:11: > On Fri, 25 May 2018 14:50:57 -0600 > Neil Schemenauer wrote: >> On 2018-05-25, Antoine Pitrou wrote: >>> Do you have something specific in mind? >> >> I think compressed by default is a good idea. My quick proposal: >> >> - Use fast compression

Re: [Python-Dev] PEP: 576 Title: Rationalize Built-in function classes

2018-05-19 Thread Stefan Behnel
mark schrieb am 19.05.2018 um 11:15: > At the language summit this year, there was some discussion of PEP 575. > I wanted to simplify the PEP, but rather than modify that PEP, Nick Coghlan > encouraged me to write an alternative PEP instead. > > PEP 576 aims to fulfill the same goals as PEP 575,

Re: [Python-Dev] PEP 575 (Unifying function/method classes) update

2018-05-16 Thread Stefan Behnel
Petr Viktorin schrieb am 15.05.2018 um 18:36: > On 05/15/18 05:15, Jeroen Demeyer wrote: >> An important note is that it was never my goal to create a minimal PEP. I >> did not aim for changing as little as possible. I was thinking: we are >> changing functions, what would be the best way to

Re: [Python-Dev] PEP 575: Unifying function/method classes

2018-05-03 Thread Stefan Behnel
Hi, let me start by saying that I'm much in favour of this change. It cleans up a lot of the function implementation and makes it much easier to integrate efficiently with external wrapper tools. Guido van Rossum schrieb am 14.04.2018 um 23:14: > On Sat, Apr 14, 2018 at 2:17 AM, Jeroen Demeyer

Re: [Python-Dev] Subtle difference between f-strings and str.format()

2018-03-31 Thread Stefan Behnel
Serhiy Storchaka schrieb am 28.03.2018 um 17:27: > There is a subtle semantic difference between str.format() and "equivalent" > f-string. > >     '{}{}'.format(a, b) >     f'{a}{b}' > > In the former case b is evaluated before formatting a. This is equivalent to > >     t1 = a >     t2 = b >   

Re: [Python-Dev] Subtle difference between f-strings and str.format()

2018-03-28 Thread Stefan Behnel
Serhiy Storchaka schrieb am 28.03.2018 um 17:27: > There is a subtle semantic difference between str.format() and "equivalent" > f-string. > >     '{}{}'.format(a, b) >     f'{a}{b}' > > In the former case b is evaluated before formatting a. This is equivalent to > >     t1 = a >     t2 = b >   

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

2018-02-23 Thread Stefan Behnel
Chris Barker schrieb am 23.02.2018 um 20:23: > BTW, would it be even a tiny bit more efficient to use a tuple in the inner > loop? > > [g(y) for x in range(5) for y in (f(x),)] Serhiy's optimisation does not use a loop at all anymore and folds it into a direct assignment "y=f(x)" instead. But

Re: [Python-Dev] The `for y in [x]` idiom in comprehensions

2018-02-23 Thread Stefan Behnel
Stephen J. Turnbull schrieb am 23.02.2018 um 03:31: > Barry Warsaw writes: > > rather than having to pause to reason about what that 1-element > > list-like syntax actually means, and 2) will this encourage even > > more complicated comprehensions that are less readable than just > > expanding

Re: [Python-Dev] Make re.compile faster

2017-10-03 Thread Stefan Behnel
INADA Naoki schrieb am 03.10.2017 um 05:29: > Before deferring re.compile, can we make it faster? I tried cythonizing both sre_compile.py and sre_parse.py, which gave me a speedup of a bit more than 2x. There is definitely space left for further improvements since I didn't know much about the

Re: [Python-Dev] PEP 550 v4

2017-08-26 Thread Stefan Behnel
Hi, I'm aware that the current implementation is not final, but I already adapted the coroutine changes for Cython to allow for some initial integration testing with real external (i.e. non-Python coroutine) targets. I haven't adapted the tests yet, so the changes are currently unused and mostly

[Python-Dev] Cython compiled stdlib modules - Re: Python startup time

2017-07-21 Thread Stefan Behnel
Nick Coghlan schrieb am 21.07.2017 um 08:23: > I'll also note that in these cases where the import overhead is > proportionally significant for always-imported modules, we may want to look > at the benefits of freezing them (if they otherwise remain as pure Python > modules), or compiling them as

Re: [Python-Dev] Python startup time

2017-07-20 Thread Stefan Behnel
Ivan Levkivskyi schrieb am 20.07.2017 um 13:24: > I agree the start-up time is important. There is something that is related. > ABCMeta is currently implemented in Python. > This makes it slow, creation of an ABC is 2x slower than creation of a > normal class. > However, ABCs are used by many

Re: [Python-Dev] PEP 7 and braces { .... } on if

2017-06-05 Thread Stefan Behnel
Serhiy Storchaka schrieb am 03.06.2017 um 18:25: > Yet about braces. PEP 7 implicitly forbids breaking the line before an > opening brace. An opening brace should stay at the end the line of the > outer compound statement. > > if (mro != NULL) { > ... > } > else { >

Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects

2016-08-31 Thread Stefan Behnel
Nick Coghlan schrieb am 31.08.2016 um 06:30: > On 31 August 2016 at 04:55, Serhiy Storchaka wrote: >> On 30.08.16 21:20, Antoine Pitrou wrote: >>> But the performance overhead of iterating over a 1-element list >>> is small enough (it's just an array access after a pointer dereference) >>> that it

Re: [Python-Dev] RFC: PEP 509: Add a private version to dict

2016-04-15 Thread Stefan Behnel
Victor Stinner schrieb am 15.04.2016 um 10:20: > Le vendredi 15 avril 2016, Stefan Behnel a écrit : > >> How can that be achieved? If the tag is just a sequentially growing number, >> creating two dicts and applying one operation to the first one should give >> both the

Re: [Python-Dev] RFC: PEP 509: Add a private version to dict

2016-04-14 Thread Stefan Behnel
Victor Stinner schrieb am 15.04.2016 um 00:33: > 2016-04-15 0:22 GMT+02:00 Brett Cannon: >> And even if it was GIL-free you do run the risk of two dicts ending up at >> the same version # by simply mutating the same number of times if the >> counters were per-dict instead of process-wide. > > For

Re: [Python-Dev] RFC: PEP 509: Add a private version to dict

2016-04-14 Thread Stefan Behnel
Victor Stinner schrieb am 14.04.2016 um 21:56: > Which kind of usage do you see in Cython? Mainly caching, I guess. We could avoid global/module name lookups in some cases, especially inside of loops. > Off-topic (PEP 510): > > I really want to experiment automatic generation of Cython code

Re: [Python-Dev] RFC: PEP 509: Add a private version to dict

2016-04-14 Thread Stefan Behnel
+1 from me, too. I'm sure we can make some use of this in Cython. Stefan Victor Stinner schrieb am 14.04.2016 um 17:19: > PEP: 509 > Title: Add a private version to dict ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

2016-02-07 Thread Stefan Behnel
M.-A. Lemburg schrieb am 04.02.2016 um 13:54: > On 04.02.2016 13:29, Victor Stinner wrote: >> But, why not PyObject_Malloc() & PObject_Free() were not used in the >> first place? > > Good question. I guess developers simply thought of PyObject_Malloc() > being for PyObjects, not arbitrary memory

Re: [Python-Dev] Speeding up CPython 5-10%

2016-01-29 Thread Stefan Behnel
Yury Selivanov schrieb am 27.01.2016 um 19:25: > tl;dr The summary is that I have a patch that improves CPython performance > up to 5-10% on macro benchmarks. Benchmarks results on Macbook Pro/Mac OS > X, desktop CPU/Linux, server CPU/Linux are available at [1]. There are no > slowdowns that I

  1   2   3   4   5   6   7   >