On Fri, 2026-04-24 at 10:54 +0200, Ralf Gommers via NumPy-Discussion wrote: > On Thu, Apr 23, 2026 at 3:40 PM Sebastian Berg > <[email protected]> > wrote: > > > Hi all, > > > > I would like a project like `ml_dtypes` to be able to transition to > > the > > new DType API, I believe that is vital to make easy. > > If anyone has concerns around this, please chime in. IMO, we need > > to > > just do this to move forward. > > > > This seems like a good idea. The details are a bit complex it looks > like, > but if it works for the known users of the legacy API, then that > should > give enough confidence. > > It probably does need some integration test like building `ml_dtypes` > against main, and then testing it against whatever the minimum > version is > that `ml_dtypes` supports. >
Yeah, I am honestly not yet sure what is useful here. `ml_dtypes` already tests with NumPy nightlies and compiles with the newest released NumPy available. Adding a test there that compiles with nightlies but runs with the oldest version is straight-forward. So `ml_dtypes` has better test coverage than what we can really hope to add into NumPy itself. I don't mind adding it too much, but it would be a first one for us and unless to run downstream tests (and maybe even build the project). (We definitely need to have some general tests for the new API of course.) - Sebastian > Cheers, > Ralf > > > > > > I.e. provide an API that allows using the new DType API all the way > > back with NumPy 2.0 (by compiling with NumPy 2.5 or vendoring some > > code). [1] > > The trick being that we can use the new DType API, so long the > > DType > > still is "legacy" and thus uses all the old code paths (unless the > > new > > API clearly replaced them). > > > > Without such "backport" there is a problem that any regression/bug > > means that we can't use the new API until the buggy NumPy version > > is > > dropped and since nothing is bug-free that can be years of > > waiting... > > > > The path is the following PR > > https://github.com/numpy/numpy/pull/31302 > > * Create a new DType registration slot, where the old defining > > struct > > is passed in. (We'll use less and less of it over time.) > > * We have a `static inline` function in NumPy 2.5 (which downstream > > can > > vendor to compile with older NumPy versions!). > > This static inline function does a bit of a crazy dance, but the > > end > > result is a working "old but new" DType (`ml_dtypes` tests pass). > > > > This gives a path, and a path mean we can actually start > > deprecating > > things soon. Even if some things are baked in slightly more after > > this > > (I think this is rather little in practice), having a path now is > > still > > better than not. > > Now is also a great time, because it is a good time to drop NumPy > > <2. > > > > Testing is a bit hard. The truly exhaustive tests is probably just > > indirectly `ml_dtypes` (which google tests exhaustively). (The > > `ml_dtypes` pass completely with this approach.) > > > > Cheers, > > > > Sebastian > > > > > > > > [1] There is exactly one corner-case I saw, and that is that we > > print: > > `np.array(..., dtype=repr(dtype))` for these DTypes, while > > legacy > > ones printed `dtype=dtype.name`, which is a bit shorter. > > (We can change that in NumPy 2.5+, but not on old versions.) > > _______________________________________________ > > NumPy-Discussion mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > https://mail.python.org/mailman3//lists/numpy-discussion.python.org > > Member address: [email protected] > > > _______________________________________________ > NumPy-Discussion mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3//lists/numpy-discussion.python.org > Member address: [email protected] _______________________________________________ NumPy-Discussion mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/numpy-discussion.python.org Member address: [email protected]
