github-actions[bot] commented on code in PR #65304:
URL: https://github.com/apache/doris/pull/65304#discussion_r3578103385
##########
thirdparty/build-thirdparty.sh:
##########
@@ -2126,6 +2126,79 @@ build_paimon_cpp() {
echo "Paimon-cpp internal dependencies installed successfully"
}
+# lance-c
+build_lance_c() {
+ check_if_source_exist "${LANCE_C_SOURCE}"
+ cd "${TP_SOURCE_DIR}/${LANCE_C_SOURCE}"
+
+ rm -rf "${BUILD_DIR}"
+ mkdir -p "${BUILD_DIR}"
+
+ local cargo_bin="${LANCE_C_CARGO:-${CARGO:-cargo}}"
+ if ! command -v "${cargo_bin}" >/dev/null 2>&1; then
+ echo "cargo is required to build lance-c. Install Rust 1.91.0 or set
LANCE_C_CARGO."
+ exit 1
+ fi
+ if [[ ! -x "${TP_INSTALL_DIR}/bin/protoc" ]]; then
+ echo "protoc is required to build lance-c. Build protobuf first."
+ exit 1
+ fi
+
+ local cargo_env=(
+ "CARGO_BUILD_JOBS=${PARALLEL}"
+ "CARGO_TARGET_DIR=${PWD}/${BUILD_DIR}"
+ "PROTOC=${TP_INSTALL_DIR}/bin/protoc"
+ )
+ if command -v rustup >/dev/null 2>&1 && [[ -z "${RUSTUP_TOOLCHAIN}" ]];
then
+ if rustup toolchain list | grep -q '^1.91.0'; then
+ cargo_env+=("RUSTUP_TOOLCHAIN=1.91.0")
+ fi
+ fi
+
+ local required_rust_version="1.91.0"
+ local cargo_version
+ if ! cargo_version="$(env "${cargo_env[@]}" "${cargo_bin}" --version | awk
'{print $2}')"; then
+ echo "failed to get cargo version for lance-c. Install Rust
${required_rust_version} or set LANCE_C_CARGO/RUSTUP_TOOLCHAIN."
+ exit 1
+ fi
+ if ! awk -v required="${required_rust_version}" -v
actual="${cargo_version}" 'BEGIN {
+ split(required, r, ".");
+ split(actual, a, ".");
+ for (i = 1; i <= 3; i++) {
+ if ((a[i] + 0) > (r[i] + 0)) {
+ exit 0;
+ }
+ if ((a[i] + 0) < (r[i] + 0)) {
+ exit 1;
+ }
+ }
+ exit 0;
+ }'; then
+ echo "lance-c requires Rust/Cargo ${required_rust_version} or newer,
but found ${cargo_version}."
+ echo "Install Rust ${required_rust_version} or set
LANCE_C_CARGO/RUSTUP_TOOLCHAIN."
+ exit 1
+ fi
+
+ if [[ "${KERNEL}" != 'Darwin' ]]; then
+ cargo_env+=("CFLAGS=${CFLAGS:-} -std=gnu17")
+ fi
+
+ local cargo_args=(build --release --locked)
+ if [[ "$(echo "${LANCE_C_CARGO_OFFLINE}" | tr '[:lower:]' '[:upper:]')" ==
"ON" ]]; then
+ cargo_args+=(--offline)
+ fi
+ env "${cargo_env[@]}" "${cargo_bin}" "${cargo_args[@]}"
Review Comment:
This makes the default third-party source build depend on Cargo downloading
crates outside Doris' third-party archive/mirror flow. `vars.sh` adds only the
`lance-c-v0.1.2.tar.gz` source archive to `TP_ARCHIVES`, and the upstream
archive has `Cargo.toml`/`Cargo.lock` but no `.cargo` mirror config, `vendor/`
directory, or crate tarballs; the lockfile entries still point at crates.io. As
a result, an offline or Doris-mirror-only builder can successfully run
`download-thirdparty.sh` and still fail here when Cargo tries to reach the
registry. Please make the Rust dependencies part of the Doris third-party
source flow, for example by vendoring/mirroring the Cargo registry artifacts
and building with `--offline`/`--frozen`.
##########
thirdparty/build-thirdparty.sh:
##########
@@ -2166,6 +2239,7 @@ if [[ "${#packages[@]}" -eq 0 ]]; then
cares
grpc # after cares, protobuf
arrow
Review Comment:
Once `lance_c` is in the default package list, every Dockerfile that runs
the default `thirdparty/build-thirdparty.sh` needs Cargo available, because
`build_lance_c` exits immediately when `cargo` is missing. The main compilation
image was updated with an `install Rust for lance-c` block, but the ARM and
legacy GCC image builders still invoke the default third-party build without
installing Rust/Cargo. Those image builds will now fail before producing
`thirdparty/installed`. Please provision Rust/Cargo 1.91+ in the ARM/gcc7/gcc10
compilation Dockerfiles too, or keep `lance_c` out of those default builds
until the toolchain is guaranteed.
##########
dist/licenses/LICENSE-lance-c-rust-crates.txt:
##########
@@ -0,0 +1,625 @@
+Lance-C Rust crate license list
+
+Generated from lance-c v0.1.2 Cargo.lock using:
+cargo +1.91.0 tree --locked --edges normal,build
Review Comment:
This license inventory is incomplete for the new static Rust dependency. The
command shown here is host-target only; it does not use `--target all` or
per-target sections, and the resulting list already shows the gap:
`rustls-native-certs` is included, but its upstream lockfile dependencies
`schannel`, `security-framework`, `security-framework-sys`, `core-foundation`,
and `core-foundation-sys` are absent, even though this third-party build has a
Darwin path. Also, the file only records SPDX expressions and does not add or
point to the actual license texts for several mandatory non-Apache/MIT terms in
the graph, such as 0BSD, ISC, CDLA-Permissive-2.0, BSL-1.0, MPL-2.0, Unlicense,
and zlib-acknowledgement. Please regenerate the inventory for all supported
targets and ship or explicitly reference the corresponding license texts for
every selected/mandatory term.
##########
be/cmake/thirdparty.cmake:
##########
@@ -110,6 +110,7 @@ add_thirdparty(arrow_flight_sql LIB64)
add_thirdparty(arrow_dataset LIB64)
add_thirdparty(arrow_acero LIB64)
add_thirdparty(parquet LIB64)
+add_thirdparty(lance_c LIB64)
Review Comment:
This new imported target is unconditional, so every normal BE/tool link now
expects `${DORIS_THIRDPARTY}/installed/lib64/liblance_c.a`. The source-build
install path for `lance_c` is correct, but the prebuilt/local freshness check
in `build.sh` still only looks for the old sentinel libraries
(`hadoop_hdfs/native/libhdfs.a` on Linux or `libbrotlienc.a` on Darwin). A
stale prebuilt `thirdparty/installed` tree can therefore skip rebuilding
third-party libraries while missing `liblance_c.a`, and then fail the normal
BE/UT/tool build before any Lance code is used. Please update the prebuilt
bundle and the freshness check to require `lib64/liblance_c.a`, or gate this
dependency until the archive is guaranteed to exist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
