tadeja commented on PR #55264: URL: https://github.com/apache/arrow/pull/55264#issuecomment-6474638776
Ehhh... I've fallen into this interesting rabbit hole - There are over 1.5k mypy errors `Module not callable` in `pyarrow/tests` because of the change from [upstream's zen-xu/pyarrow-stubs having subfolder `__lib_pxi`](https://github.com/zen-xu/pyarrow-stubs/tree/148368c532f96f953cf39f34120da9d1043bb4dc/pyarrow-stubs/__lib_pxi) to current PR's flat layout without a subfolder (Past discussion https://github.com/rok/arrow/pull/45#issuecomment-3291316851 was about the naming, but not really why removing subfolder `__lib_pxi`) ```console % mypy --check-untyped-defs --warn-unused-ignores pyarrow/tests/test_convert_builtin.py pyarrow/tests/test_convert_builtin.py:78: error: Module not callable [operator] pyarrow/tests/test_convert_builtin.py:79: error: Module not callable [operator] ... pyarrow/tests/test_convert_builtin.py:168: error: Module not callable [operator] ... Found 321 errors in 1 file (checked 1 source file) % mypy --check-untyped-defs --warn-unused-ignores pyarrow/tests/test_*.py Found 1866 errors in 35 files (checked 44 source files) % ... | grep -c "Module not callable" 1562 ``` [test_convert_builtin.py:168 : pa.array(...](https://github.com/apache/arrow/pull/55264/changes#diff-2447096102a9002ccdf1e69f271d85c16f1c1af2fdf66aab46c51485e038510bR168) A stub `pyarrow/array.pyi` declares pyarrow.array submodule that doesn't exist at runtime because `array.pxi` gets compiled into `lib`. Then `pa.array`, `pa.table` or `pa.scalar` can resolve to stub modules instead of functions. The count of mypy errors may be even higher, depending on module processing order and mypy cache. (optionally run `rm -rf .mypy_cache`) **So I propose we reintroduce subfolder** `__lib_pxi/` (or renamed subfolder `_lib_pxi/` or just `_lib/`) in this PR (for files: array.pyi, builder.pyi, config.pyi, device.pyi, error.pyi, io.pyi, memory.pyi, scalar.pyi, table.pyi, tensor.pyi). Locally that seems to resolve all 1.5k `Module not callable` errors and still passes type checkers' tests for me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
