[
https://issues.apache.org/jira/browse/BEAM-11211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Hulette updated BEAM-11211:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Open)
> Support multiple major pyarrow versions
> ---------------------------------------
>
> Key: BEAM-11211
> URL: https://issues.apache.org/jira/browse/BEAM-11211
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Brian Hulette
> Assignee: Brian Hulette
> Priority: P2
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> We should support using ParquetIO with multiple pyarrow versions, up to 2.x.
> Specific actions:
> [X] Change pyarrow requirement from >=0.15.1,<0.18.0 to >=0.15.1,<3.0.0
> [X] There's a limitation in 1.x where it can't write LZ4 compression, we
> should catch attempts to do this at construction time and raise a useful
> error (See ARROW-9424).
> [X] Add the ability to test with different pyarrow versions and run in
> PostCommit to verify.
> [ ] Update pyarrow in base_image_requirements.txt to 2.x
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
