On Fri, Nov 11, 2022 at 1:52 PM Sebastian Berg <sebast...@sipsolutions.net> wrote:
> On Fri, 2022-11-11 at 12:27 +0100, Ralf Gommers wrote: > > Hi all, > > > > With distutils now removed from the stdlib in the Python 3.12 release > > cycle, the clock is ticking a bit for dealing with our build system > > situation. With SciPy's move to Meson now basically complete - there > > are > > always loose ends & improvements, but the 1.9 releases have gone well > > - > > it's time to look at NumPy doing the same thing. Scikit-image has > > also > > merged support for Meson, and Pandas is about to. So all the generic > > & > > Python packaging related things we need should be in place. NumPy > > does have > > some hairy stuff of course that those other projects don't have (lots > > of > > config checks and platform-specific behavior, extensive SIMD > > support). It > > shouldn't be too difficult to get a baseline build - Linux/macOS with > > baseline SIMD flags - working, but there'll be a long tail of things > > that > > are hard to test or will need to be upstreamed to Meson. > > > > Here is a tracking issue where I wrote up a plan for how to approach > > the > > transition: https://github.com/numpy/numpy/issues/22546. Linked from > > there > > is a GitHub project board that we plan to use to divide up the work. > > And > > there is a `meson` branch in the repo with a start to the > > implementation. > > > > NumPy should default to building with Meson in the 1.25.0 release, > > which > > should also still build with `numpy.distutils`. And `numpy.distutils` > > will > > continue to be shipped for Python <3.12 for a while after that, until > > we > > determine that it's no longer needed. > > > > If anyone has ideas or concerns regarding the current plan, it'd be > > great > > to hear them - here or on the issue. > > > Thanks for working on this! I fully trusting you, Stéfan, and everyone > else involved to push this forward. > > It seems like most of the decisions were really made for us and the > open points might be mainly about the how and maybe details about the > what. > I think there will be quite a few decisions on the way that different people may want to weigh in on. So we need kind of a "decision log" somewhere and a way to bubble up those decisions - which fits with the NEP idea (as long as it remains draft and can be extended). To give you a couple examples just from spending a few hours on build system changes today: (1) I noticed we still have a `oldnumeric.h` header with a comment that it can probably be removed. So I used the new GitHub code search, found the two actively developed projects that still use it, and asked them to get rid of their usages so that we can indeed safely remove it. And then added code + comment in the meson branch to not install it. (2) a more important one, the `.c.src` format. In SciPy we got rid of it, and we're not going to make Meson understand an ad-hoc templating method that only NumPy uses. So we have two choices: also get rid of it, or write a new custom preprocessing utility for NumPy's Meson build. I think we have too much code using it to remove it, so probably should go with the "new utility" option. But in case we did want to get rid of it, now would be a good time. > To me it would also seem fine if you work on the main branch, but I > guess that would just make things harder to iterate at this point. > Yes, that'd be too disruptive at this point. > > Since "NEP" was mentioned somewhere. There would be two goals of > having a short one: > 1. Settling on a specific approach for our build system > 2. Informing users (about breakages/workarounds and how things work) > > I doubt you/we need it for 1. at this point or details for why meson, > but if anyone disagrees then maybe we do ;). But 2. may very much be > worthwhile. > We should have regular html docs for everything, so the NEP on that point would be more an FYI with a pointer to those docs. > > - Sebastian > > > PS: I do like the idea of having short NEPs. My feeling is that the > "painful" ones are the ones that are technically tricky. For those it > is clear that they are necessary. > The other issue is about document intent (sometimes a NEP may be more > of a roadmap proposal then a concrete implementation one). > That does make sense to me. NEPs are not good as documentation - we should have more design & architecture docs that are maintained over time for that - but to document intent at the time a change was made they are quite nice. Cheers, Ralf
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
