rok opened a new pull request, #55597: URL: https://github.com/apache/arrow/pull/55597
### Rationale for this change Closes #55596. The runtime docstrings are inserted into the type stubs by a custom libcst codemod (`python/scripts/update_stub_docstrings.py`), which needs a hand-written mapping from stub files to runtime modules and makes libcst a build requirement of every source build. [docstring-adder](https://github.com/astral-sh/docstring-adder) does the same job generically. Stacked on #55264; only the last two commits belong to this PR. ### What changes are included in this PR? - Squash the stub-only `pyarrow.__lib` package into `lib.pyi`, one section per `.pxi` file, matching the single `pyarrow.lib` extension module. This drops the per-file `__all__` lists and the module aliasing in `run_stubtest.py`, and lets docstring-adder map `lib.pyi` to `pyarrow.lib` by name. - `update_stub_docstrings.py` now runs docstring-adder on the installed stubs (it still assembles an importable pyarrow from the source and install trees) and fails if `lib.pyi` got no docstrings. - Build Cython with `embedsignature.format=clinic`, so the embedded signature goes to `__text_signature__` instead of the first lines of `__doc__` and out of the stub docstrings. - Install the stubs into wheels again and add the docstrings in the wheel builds (`PYARROW_REQUIRE_STUB_DOCSTRINGS=ON`). `py.typed` stays excluded from wheels until the stubs are complete (GH-49831). - libcst is no longer a build requirement. docstring-adder isn't on PyPI, so `requirements-wheel-build.txt` pins it to a commit. The macOS Python 3.15 workaround for pyyaml (pulled in by libcst) is removed. ### Are these changes tested? Locally with a minimal Arrow C++ build and `python -m build --wheel --no-isolation` with `PYARROW_REQUIRE_STUB_DOCSTRINGS=ON`: - The wheel contains the stubs; 631 of the 709 classes and functions in `lib.pyi` have docstrings (the rest have none at runtime, e.g. exception classes and Cython properties), and none starts with a signature line. - stubtest, the stub tests, mypy, pyright and ty pass. - `inspect.signature` of the `pyarrow.lib` functions and methods is unchanged by the clinic format. ### Are there any user-facing changes? Wheels ship the type stubs with docstrings again (without `py.typed`, so mypy keeps ignoring them). The `__doc__` of Cython functions no longer starts with the signature; `help()` and `inspect.signature()` still show it. ### Was AI used for this PR? In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR. **PR code and description written by:** - [ ] Human - [x] AI **Reviewed before submission by:** - [ ] Human - [ ] AI - [ ] Not reviewed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
