jorisvandenbossche opened a new pull request, #51680: URL: https://github.com/apache/arrow/pull/51680
I recently stumbled upon https://github.com/jon-edward/stubgen-pyx, and this might be an interesting option to explore for the pyarrow typing situation. It is essentially an improved stubgen specifically for cython code (in the past, when we explored options here, the existing stubgens didn't work sufficiently well for cython to be considered). And also https://github.com/NVIDIA/cuda-python is (experimenting with) using this. This PR, just for illustration purposes, includes the full generated output for the entire pyarrow package (with `.pyi` files located next to the cython file, but this could of course be put elsewhere as well): `stubgen-pyx python/pyarrow/*.pyx`. The interesting part that I learned from looking at the cuda-python code, is that you can actually use python type annotations inside the cython code, and that way we could have inline annotations. Whatever information the tool cannot extract from the cython information, we can augment with adding python type annotations. So essentially that would give an option to have all the custom type annotations inline (which the current PRs for https://github.com/apache/arrow/issues/32609, such as https://github.com/apache/arrow/pull/48622, are putting in separate stub files), and have the stub files be fully generated (i.e. allow us to have only one source of truth / one place to edit). To illustrate that part, I am going to open a separate PR with just the generated stub file for `types.pxi` (so it can be compared to https://github.com/apache/arrow/pull/48622), and with a few customizations to improve the generated stubs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
