Hi all, I would like a project like `ml_dtypes` to be able to transition to the new DType API, I believe that is vital to make easy. If anyone has concerns around this, please chime in. IMO, we need to just do this to move forward.
I.e. provide an API that allows using the new DType API all the way back with NumPy 2.0 (by compiling with NumPy 2.5 or vendoring some code). [1] The trick being that we can use the new DType API, so long the DType still is "legacy" and thus uses all the old code paths (unless the new API clearly replaced them). Without such "backport" there is a problem that any regression/bug means that we can't use the new API until the buggy NumPy version is dropped and since nothing is bug-free that can be years of waiting... The path is the following PR https://github.com/numpy/numpy/pull/31302 * Create a new DType registration slot, where the old defining struct is passed in. (We'll use less and less of it over time.) * We have a `static inline` function in NumPy 2.5 (which downstream can vendor to compile with older NumPy versions!). This static inline function does a bit of a crazy dance, but the end result is a working "old but new" DType (`ml_dtypes` tests pass). This gives a path, and a path mean we can actually start deprecating things soon. Even if some things are baked in slightly more after this (I think this is rather little in practice), having a path now is still better than not. Now is also a great time, because it is a good time to drop NumPy <2. Testing is a bit hard. The truly exhaustive tests is probably just indirectly `ml_dtypes` (which google tests exhaustively). (The `ml_dtypes` pass completely with this approach.) Cheers, Sebastian [1] There is exactly one corner-case I saw, and that is that we print: `np.array(..., dtype=repr(dtype))` for these DTypes, while legacy ones printed `dtype=dtype.name`, which is a bit shorter. (We can change that in NumPy 2.5+, but not on old versions.) _______________________________________________ NumPy-Discussion mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/numpy-discussion.python.org Member address: [email protected]
