rok commented on PR #55264:
URL: https://github.com/apache/arrow/pull/55264#issuecomment-6946304456

   <details><summary>Detailed AI generated open questions</summary>
   
   ## Open questions
   
   Summary of what is still open on this PR after going through all comments 
and review threads (head `374093b`).
   
   ### Needs an answer or a decision
   
   1. **Stubs and non-editable installs** (Joris, in the `development.rst` 
thread). The docs say "With a non-editable install ... the stubs aren't 
installed yet". This is because of #49831: the CMake block that installs the 
stubs has been commented out since June, so no build installs them yet. Still 
unexplained from the same thread: `mypy pyarrow/tests/test_table.py` finds the 
same issues with or without `pyarrow-stubs` on the command line.
   2. **IDE support with an editable install.** VS Code did not pick up the 
stubs for Joris on 6 Nov. The layout has changed since 
(`pyarrow-stubs/pyarrow/` became `pyarrow-stubs/`), which fixed PyCharm, but VS 
Code has not been confirmed.
   3. **Changes to `pyarrow/tests`.** The suggestion was to either type check 
those tests in CI or leave the edits out of this PR. Most were removed, but 11 
`# type: ignore` lines across six test files remain, and CI doesn't check them.
   4. **`--follow-untyped-imports` for mypy.** Raised in review and tested 
locally, but no decision is recorded and it isn't in `pyproject.toml`.
   5. **`dictionary()` and `run_end_encoded()` reject a plain `DataType`** 
(Copilot). `pa.dictionary(arr.type, pa.string())` works at runtime and fails in 
mypy and pyright, because the parameter is typed with the stub-only integer 
types. Should the "parameters take the base `Array`" rule also apply to 
`DataType`?
   6. **`Array` operators and `Array.from_dlpack` are missing from the stubs** 
(Copilot). `arr + arr`, `-arr`, `arr & arr` and `pa.Array.from_dlpack(...)` 
work on the nightly and are rejected by both checkers. Add them here or in a 
follow-up?
   
   ### Deferred, tracked in an issue
   
   - #55499: `pa.array()` can return a `ChunkedArray`; annotated as `Array` for 
now.
   - #55418: `pa.record_batch` raises for tuple data.
   - #55450: remove the deprecated `ChunkedArray.data`.
   - #55486: `test_asarray` never runs.
   - #48970: remove the `__getattr__ -> Any` fallbacks before the stubs ship.
   
   ### Deferred, no issue yet
   
   - **Squash `__lib/*.pyi` into `lib.pyi`**, and decide whether 
`_stubs_typing.pyi` stays separate. Proposed for after the whole series is 
merged.
   - **`Literal` versus `str` for string options** (`unit` etc.). Left strict, 
as in NumPy and pandas-stubs; to revisit once we see downstream impact.
   - **`to_pandas` return type per class.** Left as a TODO.
   - **`KeyValueMetadata.keys/values/items`** return generators at runtime 
rather than mapping views. Worth a separate issue.
   - **`Type_*` constants and private helpers** (`string_to_tzinfo`, 
`tzinfo_to_string`, `ensure_metadata`, `table_to_blocks`) are not in the stubs. 
Add them or allowlist them once stubtest stops skipping missing names.
   - **Re-check the `Self` rule before merge** ("use `Self` when it is correct, 
a concrete type otherwise").
   - **`Array1D`** for parameters that only accept 1-d NumPy arrays.
   - **`Iterable` / `Sequence` / `list` for array-like inputs.** Handled case 
by case; no general rule written down.
   - **`Buffer` placeholder is an empty class**, so `buffer.to_pybytes()` or 
`len(buffer)` are rejected. Waiting for the I/O stubs PR.
   
   ### Additional findings from a local pass
   
   Checked against the current head with mypy and pyright; not raised in review 
so far.
   
   - **Dict schemas:** `pa.schema({"a": pa.int8()})` and `pa.struct({...})` are 
rejected. `Mapping[str | bytes, ...]` doesn't accept a `dict[str, ...]`, 
because `Mapping` is invariant in its key type.
   - **Metadata dicts:** for the same reason, a `dict[str, str]` variable is 
rejected as `metadata=` in `pa.table`, `pa.record_batch`, `from_pydict` and 
`from_pylist`.
   - **`take` / `filter` with a `ChunkedArray`:** `Array.take` and 
`Array.filter` are typed as returning `Array`, but the runtime returns a 
`ChunkedArray`. `RecordBatch.take(ChunkedArray)` type checks and raises at 
runtime.
   - **Tuples:** rejected for `names=` in `pa.table` / `pa.record_batch`, and 
in `rename_columns`, `drop_columns` and `Table.drop`, although they work at 
runtime. `RecordBatch.select` rejects mixed names and indices, unlike 
`Table.select`.
   - **Housekeeping:** a per-file `# mypy: warn-unused-ignores` in 
`test_tensor.py` and `test_device.py` (already set globally), 
`pytest.importorskip("numpy")` in two stub tests, the `_compute.pyi` header 
comment naming `pyarrow.lib`, and `= ...` defaults in `Bool8Array.to_numpy` and 
`binary()`.
   <details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to