baibaichen opened a new pull request, #12625:
URL: https://github.com/apache/gluten/pull/12625
## What changes are proposed in this pull request?
Fixes #7139.
This PR makes vcpkg the owner of native Arrow and the compile-time
dependency graph used by Velox builds.
- Add an Arrow 18.0.0 vcpkg overlay with a `testing` feature that installs
`libarrow_testing.a` without building Arrow's full test suite.
- Align modular Boost ports to 1.84 and patch Arrow's static testing
dependency resolution.
- Force vcpkg-enabled Velox builds to use SYSTEM dependencies and prevent
fallback to bundled or host packages.
- Keep Velox's Arrow module discovery scoped separately from Gluten's
Config-package discovery so both consume the expected CMake targets.
- Link Gluten through Arrow's exported static target and restore usage
requirements lost when Velox static archives are imported by path.
- Isolate CMake and pkg-config discovery from `/usr/local`, inherited
prefixes, and package registries while preserving OS/runtime dependencies.
- Deprecate the standalone native Arrow build path in vcpkg mode and update
Docker, workflow, and notebook callers.
The result is that a clean `package-vcpkg.sh` build no longer requires a
preinstalled `/usr/local` Arrow or a separate `dev/build-arrow.sh` invocation.
Java Arrow dependencies remain managed by Maven.
## How was this patch tested?
- Built the Arrow vcpkg overlay and confirmed:
- `libarrow.a`
- `libarrow_testing.a`
- `arrow/api.h`
- all active modular Boost packages resolve to 1.84.0
- Ran a poisoned-prefix Debug matrix with tests, benchmarks, S3, GCS, and
HDFS enabled:
- Velox: 976/976 build steps
- Gluten: 234/234 build steps
- Ran the end-to-end build:
```bash
./dev/buildbundle-veloxbe.sh \
--enable_vcpkg=ON \
--build_tests=ON \
--build_arrow=OFF \
--build_benchmarks=ON \
--enable_s3=ON \
--enable_gcs=ON \
--enable_hdfs=ON \
--spark_version=4.1 \
--build_type=Debug
```
All ten Maven reactor modules completed successfully.
- Regenerated Debug and Release Velox build graphs and confirmed there are
no bare `-larrow` or `-larrow_testing` links.
- Confirmed the compiler dependency graph contains no headers from
`/usr/local/include`.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI 1.0.75
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
