Mohit-25-tech commented on issue #52997: URL: https://github.com/apache/arrow/issues/52997#issuecomment-6099180586
Hi maintainers, I've prepared a focused PR addressing this issue. **Root cause:** `ParquetFile._build_nested_paths()` creates identical lookup keys for a literal top-level field `a.b` and a nested path `a → b`, causing column projection to select both fields unexpectedly. **Proposed fix:** - Give exact top-level column names precedence over ambiguous dotted nested paths. - Preserve normal nested-field selection when no exact top-level match exists. - Add regression tests covering `read()`, `read_row_group()`, `iter_batches()`, deeper nested paths, duplicate selections, and pandas metadata. **Validation:** - Independently reproduced the original issue with PyArrow 26.0.0 in Kaggle. - All 8 independent Kaggle checks passed using an adapted patch. - 4 focused pytest tests passed using the modified Python methods. - Flake8, syntax checks, and `git diff --check` passed. - Full repository tests remain unverified locally. **PR:** #53010 The precedence change is intentional, and I'd appreciate feedback on whether this is the preferred API behavior. Thanks for your time and review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
