PingLiuPing commented on issue #1027: URL: https://github.com/apache/iceberg-cpp/issues/1027#issuecomment-6312347727
@wgtmac Thank you for looking at this issue. > May I ask why Velox pins Arrow 18.0.0? Is it possible and a smaller effort to bump the Arrow version on the Velox side? Dig little deeper on this. Velox has tried to move past Arrow 18 a few times since 2025. The upgrade from 15 to 18 landed in February 2026 ([#16366](https://github.com/facebookincubator/velox/pull/16366)). The move to Arrow 21 ([#14102](https://github.com/facebookincubator/velox/pull/14102)) was merged and then reverted, and a later attempt to revive it also ran into build problems. Moving to 21 or later is more than a version bump, there are four reasons: 1. **Velox's Parquet code uses Arrow's private helpers.** The Velox Parquet reader and writer call internal Arrow functions, such as the bit-unpacking code in the reader. Newer Arrow releases renamed or removed these helpers, so the reader and writer need code changes. 2. **Meta's internal build.** Meta compiles Velox against its own internal copy of Arrow, which was on 18 as of February 2026. Velox has to keep compiling against that version. 3. **Downstream projects.** Gluten builds Velox against Gluten's own Arrow. Gluten is on 18 and has asked Velox to keep supporting older Arrow for a few releases. Presto builds Arrow through Velox's setup scripts for its Arrow Flight connector, so its Flight code has to move to the new version too. 4. **Docker images and CI.** The Velox dev images install Arrow through the same setup scripts, so they need to be rebuilt and retested. The Arrow 21 attempt was reverted because of build breaks of this kind; those specific breaks have since been fixed though. So just changing the pinned version won't work and it requires work across communities. We are pushing this forward with the Velox community. Even so, it will take some time before Velox moves to Arrow 25. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
