jorisvandenbossche commented on PR #51680: URL: https://github.com/apache/arrow/pull/51680#issuecomment-6014795660
> Alternative process would be: > > 1. Cover all stubs with manual stubs > 2. Enable conversion form cython/python into stubs with stubgen-pyx > 3. Gradually enrich cython/python code with types from manual stubs (this will have the benefit of using internal types from the start) while removing manual stubs IMO the problem with this approach is that this first item is the bottleneck. The current manual stubs are super complex, and hard to all get in (the stalled PRs). (unless you would go for starting with lighter manual stubs, and then also gradually enrich those, but lighter manual stubs are essentially the auto-generated ones, potentially) My (not validated) proposal would be: 1. Start with adding the auto-generated stubs for cython and compute.py, and inline annotations for other python files. 2. Make minimal changes in the cython and python sources (eg add basic return types) to make the annotations minimally useful 3. Ensure we have good test coverage for the annoations (run type checkers on our tests, and add specific type-annotation tests) 4. Re-enable pyarrow as being py.typed, and try to have at least this in the next release 5. Gradually move richer annotations from the current manual stubs to the inline sources (so that the auto-generated annotations get updated as well) As I mentioned in the pyarrow dev meeting last week, one aspect that might actually not work out with the outline above is the last step: I don't know if stubgen-pyx (well, actually cython) will support the more complex parts of the type annotations syntax in python functions in cython files. That is something that would have to be tested, and if that does not work, something to be discussed how important those additional features are (vs the maintenance benefit of being able to auto-generate stubs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
