Yicong-Huang opened a new pull request, #53994: URL: https://github.com/apache/spark/pull/53994
### What changes were proposed in this pull request? Fix PyArrow version compatibility for struct cast tests in `test_pyarrow_array_cast.py`. Some struct cast behaviors differ across PyArrow versions: | Feature | PyArrow < 19 | PyArrow 19-20 | PyArrow >= 21 | |---------|--------------|---------------|---------------| | struct cast: field name mismatch (missing fields → null) | ArrowTypeError | supported | supported | | struct cast: target has more fields (extra fields → null) | ArrowTypeError | supported | supported | | struct cast: field reorder (same fields, different order) | ArrowTypeError | ArrowTypeError | supported | This PR: 1. Adds version compatibility documentation to the test file header 2. Introduces `pyarrow_19_or_greater` and `pyarrow_21_or_greater` version flags 3. Updates `test_struct_casts` to expect `ArrowTypeError` for unsupported operations on older PyArrow versions ### Why are the changes needed? The existing tests assumed all struct cast operations were supported in all PyArrow versions, causing test failures on PyArrow 18-20. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested with PyArrow 18, 19, 20, 21, and 22 - all versions pass. ### Was this patch authored or co-authored using generative AI tooling? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
