On Mon, 30 Oct 2023 at 00:12, Stefan Behnel <stefan...@behnel.de> wrote:
> Hi all, > > given the latest blow against exposing implementation details of CPython > in > their C-API (see https://github.com/cython/cython/pull/5767 for the > endless > story), In this new world order of political correctness, they will not say it directly, but the fact that Cython is probably the biggest consumer of these private APIs, this is a way to hard-press Cython to stop doing that once and for all. I remember some exchanges in GitHub where Python core devs manifested quite a bit of dislike/disdain for what Cython is doing. > I seriously start wondering if we shouldn't just define > "Py_BUILD_CORE" This sounds just too much, and it's like a war declaration on what they are trying to do. > (or have our own "CYTHON_USE_CPYTHON_CORE_DETAILS" macro > guard that triggers its #define) and include the internal "pycore_*.h" > CPython header files from here: > > https://github.com/python/cpython/tree/main/Include/internal > > and this is totally reasonable, as it is under Cython's control. However, how much would this be different from CYTHON_LIMITED_API, or how the new define would interact with CYTHON_LIMITED_API? > > There's a risk, clearly, that these internals change even during point > releases. Maybe not a big risk, but not impossible either. We'd have to > deal with that and so would our users. > And that would be the biggest blow, actually. If this is truly a possibility, then I will have to turn off the use of CPython internals in my project. I don't want my already released packages to stop building just after a CPython patch release. There is no way I'm willing to deal with this mess, I already have too much maintenance work going on. > OTOH, having a single macro switch would make it easy for users to adapt > if > something breaks on their side, and also easy to benchmark if it makes a > difference for their code. > Definitely > > We could also leave it off by default and simply allow users with high > performance needs to enable it manually. Or start by leaving it off until > a > new CPython X.Y release has stabilised and its (used-by-us) internals have > proven not to change, and then switch it on for that release series. In > any > case, having a single switch for this feels like it could be easy to > handle. > Given that there is no guarantee of stability across point releases, I would argue that the macro should be off by default. Power users and packagers (that is, people that usually know how things work) can easily turn it on with "CFLAGS=-DFOO=1". What you do not want is novices and occasional users with little or no knowledge of Cython or even Python to flood your issue trackers with issues titled "I cannot install your thing" after a pip install failure. What about adding a Cython compiler directive to set the default value of the new macro? That way, it is up to the project's authors/maintainers to decide on how to handle this, they are in a better position to weigh the performance vs. portability trade-off. You still have to decide on the default value if the compiler directive is not set, but in this case I call for Stefan to pick it according to his preference/convenience, he certainly deserves the right to do so. -- Lisandro Dalcin ============ Senior Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
