yihua opened a new pull request, #711: URL: https://github.com/apache/hudi-rs/pull/711
## Description closes #710 Moves the workspace to arrow 58.4, parquet 58.4, object_store 0.13.2, DataFusion 54.1, Lance 11, and pyo3 0.28. Lance is what sets the ceiling: every published lance release, and its unreleased `main`, requires `arrow ^58`, so 58 is the highest arrow this repo can take while the Lance base-file reader still compiles, and DataFusion 54 follows from arrow 58. Most of the churn is mechanical (object_store 0.13 moved `head`/`get`/`copy` onto `ObjectStoreExt` and made `delete_stream`/`copy_opts` required; DataFusion 54 dropped `as_any` and returns `Arc` from `properties`/`partition_statistics`; Lance 11 moved `ReaderProjection::from_column_names` to a free function and made `read_stream` async; pyo3 0.28 replaced the capsule name-then-read pair with `pointer_checked`). Two things are worth a reviewer's attention: DataFusion 54 now pushes the projection and the struct-field predicate into the Parquet source rather than planning a `ProjectionExec` and a `FilterExec`, so the plan assertions in `plan_tests` were rewritten against the new shape (they assert the same projection, filter, and file-group count, just where the plan puts them now); and `benchmark/tpch` cannot follow the workspace, because tpchgen-arrow ships an arrow-57 line and an arrow-59 line and nothing on 58, so its self-contained generator builds against its own renamed arrow and parquet. DataFusion 55 stays out of reach until Lance publishes an arrow-59 release and datafusion-python catches up (54.0.0 is the newest, on datafusion-ffi 54), which is what #698 is blocked on. ## How are the changes test-covered - [x] Automated tests (unit and/or integration tests) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
