github-actions[bot] commented on code in PR #66546:
URL: https://github.com/apache/doris/pull/66546#discussion_r3842728754
##########
be/CMakeLists.txt:
##########
@@ -200,7 +200,8 @@ option(BUILD_FILE_CACHE_LRU_TOOL "ON for building file
cache lru tool or OFF for
message(STATUS "build file cache lru tool: ${BUILD_FILE_CACHE_LRU_TOOL}")
option(ENABLE_PAIMON_CPP "Enable Paimon C++ integration" ON)
-set(PAIMON_HOME "" CACHE PATH "Paimon install prefix")
+set(ARROW_HOME "${THIRDPARTY_DIR}/arrow-24.0.0" CACHE PATH "Selected Arrow
install prefix")
Review Comment:
Make preflight and CMake share one tracked Arrow/Paimon selection. Today a
reused BE cache can keep this automatic `ARROW_HOME` on the old tree after
`DORIS_THIRDPARTY` changes, or keep auto-derived `PAIMON_HOME` stale after an
`ARROW_HOME` override. The fresh explicit-prefix path is split too: `build.sh`
always validates/rebuilds `${DORIS_THIRDPARTY}/installed`, so it can abort
before CMake uses a valid separate `ARROW_HOME`/`PAIMON_HOME`, while a passing
default tree does not certify the pair CMake actually imports. These states
pass the local existence checks but can cross incompatible static-library/Arrow
ABI boundaries. Track explicit versus automatic roots, refresh automatic values
from their current input, and preflight the exact matched pair that will be
compiled and linked (or reject unsupported selections clearly).
##########
build.sh:
##########
@@ -535,10 +535,12 @@ rebuild_thirdparty_libraries() {
if [[ ! -f "${DORIS_THIRDPARTY}/installed/lib/${LAST_THIRDPARTY_LIB}" ]]; then
echo "Thirdparty libraries need to be build ..."
rebuild_thirdparty_libraries true
-elif [[ "${NEED_ARROW_PAIMON_THIRDPARTY}" == "true" ]] &&
- ! arrow_paimon_prebuilt_valid "${DORIS_THIRDPARTY}/installed"; then
- echo "Arrow/Paimon thirdparty libraries need to be rebuilt ..."
- rebuild_thirdparty_libraries false "${ARROW_PAIMON_BUILD_PACKAGES[@]}"
+elif [[ "${NEED_ARROW_PAIMON_THIRDPARTY}" == "true" ]]; then
Review Comment:
Compute this recovery gate from the normalized targets that actually consume
the stacks. It currently has both inverse errors: `BUILD_CLOUD` makes an
external Cloud-only prefix require root Arrow/Paimon 17 plus versioned
Arrow/Paimon 24 even though `cloud/cmake/thirdparty.cmake` imports neither;
meanwhile standalone `--compile-bench` sets only `COMPILE_BENCH` until lines
732-742, so this branch is skipped even though that documented mode later
enables BE. A complete pre-PR prebuilt then reaches the new BE CMake import
without `${THIRDPARTY_DIR}/arrow-24.0.0` and fails instead of migrating.
Normalize compile-bench before dependency selection and gate shared-stack
validation on real consumers; add lifecycle cases for Cloud-only external
prefixes and standalone compile-bench.
##########
thirdparty/build-thirdparty.sh:
##########
@@ -1185,15 +1199,31 @@ build_arrow() {
"${BUILD_SYSTEM}" install
#copy dep libs
- cp -rf ./brotli_ep/src/brotli_ep-install/lib/libbrotlienc-static.a
"${TP_INSTALL_DIR}/lib64/libbrotlienc.a"
- cp -rf ./brotli_ep/src/brotli_ep-install/lib/libbrotlidec-static.a
"${TP_INSTALL_DIR}/lib64/libbrotlidec.a"
- cp -rf ./brotli_ep/src/brotli_ep-install/lib/libbrotlicommon-static.a
"${TP_INSTALL_DIR}/lib64/libbrotlicommon.a"
- strip_lib libarrow.a
- strip_lib libarrow_compute.a
- strip_lib libparquet.a
- strip_lib libarrow_dataset.a
- strip_lib libarrow_acero.a
+ cp -rf ./brotli_ep/src/brotli_ep-install/lib/libbrotlienc-static.a
"${install_dir}/lib64/libbrotlienc.a"
+ cp -rf ./brotli_ep/src/brotli_ep-install/lib/libbrotlidec-static.a
"${install_dir}/lib64/libbrotlidec.a"
+ cp -rf ./brotli_ep/src/brotli_ep-install/lib/libbrotlicommon-static.a
"${install_dir}/lib64/libbrotlicommon.a"
+ strip_lib_at "${install_dir}" libarrow.a
+ if [[ "${has_separate_compute_archive}" == "true" ]]; then
+ strip_lib_at "${install_dir}" libarrow_compute.a
+ fi
+ strip_lib_at "${install_dir}" libparquet.a
+ strip_lib_at "${install_dir}" libarrow_dataset.a
+ strip_lib_at "${install_dir}" libarrow_acero.a
+}
+build_arrow_17() {
+ invalidate_arrow_17_prebuilt_marker "${TP_INSTALL_DIR}"
+ clean_arrow_artifacts_in "${TP_INSTALL_DIR}"
Review Comment:
Clear the co-located Paimon 24 artifacts before downgrading this legacy
prefix to Arrow 17. The migration starts from the old shared root containing
Arrow 24 plus Paimon built against it, but this function replaces and
republishes only Arrow; Paimon is not invalidated or cleaned until
`build_paimon_cpp_17` much later. If that next build fails or the process
stops, an unchanged pre-upgrade branch-4.1 sees all expected root libraries,
passes its unrelated final-library sentinel, and links Arrow 17 with Paimon 24
despite their public Arrow ABI boundary. Invalidate/remove root Paimon before
cleaning Arrow so failure leaves an incomplete prefix that legacy consumers
reject, and add the interrupted-migration case to the lifecycle test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
