morningman opened a new pull request, #66615: URL: https://github.com/apache/doris/pull/66615
> Split out of **https://github.com/apache/doris/pull/66510**. That PR carries the > whole BE build-time batch and its end-to-end measurements — **please refer to > #66510 for the complete benefit numbers**. This PR is the standalone, > pre-existing engineering-debt part of it: it has no textual or semantic > dependency on the rest and can be reviewed and merged on its own. ### What problem does this PR solve? Related PR: #66510 Problem Summary: `sh run-be-ut.sh` and `BUILD_BENCHMARK=ON` do not build at all on macOS arm64, and digging into it surfaced a bug that affects **every** platform. #### 1. `-fno-access-control` has never reached `doris_be_test` (all platforms) `add_definitions(-D OS_LINUX)` / `(-D OS_MACOSX)` passes `-D` and the macro name as two separate argv entries. On the `doris_be_test` target the flag-injection order leaves the dangling `-D` immediately before the target's `COMPILE_FLAGS`, so clang parses `-D -fno-access-control` as a (rejected) macro definition and only warns `macro name must be an identifier`. **The test target has been compiled without `-fno-access-control` all along.** Ten tests compensated with `#define private public`, which breaks any TU whose include graph reaches libc++'s `<ranges>`: a macro cannot rewrite a class's *default*-private region, so `lazy_split_view`'s member declarations come out `redeclared with public access`. simdjson pulls `<ranges>` in via `segment_iterator.h`, which is why this kept resurfacing. `e27d10362ac` attributed that failure to `<ranges>`; the actual cause is the swallowed flag, and this PR corrects that attribution. Fix: write `-DOS_LINUX` / `-DOS_MACOSX` so the flag survives, and delete the `private`/`protected` `=public` defines from the ten tests — white-box access now comes from the (finally effective) `-fno-access-control`, which coexists with `<ranges>` fine. #### 2. Three macOS arm64 link failures in `doris_be_test` - **tcmalloc branch out of range.** The Debug test binary's `.text` exceeds arm64's ±128MB direct-branch reach, and Apple's linker emits no branch islands for the prebuilt `libtcmalloc.a` (`fixup error (kind=arm64_b26) ... B/BL out of range`). `-Og` no longer keeps it under the limit. Stop linking `${MALLOCLIB}` into macOS-arm `MAKE_TEST` builds — unit tests do not need a custom allocator, and ASAN builds never linked it anyway (which is why they never hit this). - **Unconditionally referenced gperftools symbols.** `HeapAction`'s HTTP handler and brpc's `MallocExtension_ReleaseFreeMemory` hint get no-op stubs, compiled only under `__APPLE__ && __aarch64__`. - **Parallel link race.** The APPLE branch links `vector_search_test` through `-Wl,-force_load,$<TARGET_FILE:...>`, which creates no target-level dependency, so a parallel ninja could reach `doris_be_test`'s link step before the archive exists (`library libvector_search_test.a not found`). Added the missing `add_dependencies`. `BUILD_BENCHMARK` inflates `.text` past the same ±128MB BL reach, so the system-malloc fallback and the stub TU are extended to `benchmark_test` too. #### 3. Six BE headers drop `<ranges>` Src-side follow-up to `e27d10362ac`'s test-side cleanup. These headers only used range algorithms that `<algorithm>` already provides (`std::ranges::sort` / `is_sorted` / `find_if` / `any_of` / `nth_element` / `max_element`) or trivially rewritable views (`reverse_view` → `rbegin`/`rend` loop, `views::values` → structured-binding loop). `object_pool.h` alone is included by hundreds of TUs and was paying for the whole `<ranges>` header for one reverse loop. **This is the one hunk here with a real compile-time payoff.** #### 4. Two test portability fixes, one shell fix - `file_scanner_v2_test` kept a local anonymous-namespace copy of `kIcebergPositionDeleteContent` / `kIcebergDeletionVectorContent` after `iceberg_scan_semantics.h` began exporting the same names into `namespace doris`, making unqualified references ambiguous. - `variant_jsonb_parse_test` constructed `Decimal64` from a bare `long long` literal, ambiguous on macOS where `int64_t` is `long long` vs Linux's `long` (same family as `e3724df7cb8`). - `sh run-fe-ut.sh` — the invocation the script's own usage text documents — died at parse time with ``syntax error near unexpected token `<' `` before building anything: `/bin/sh` is bash, and bash in POSIX mode rejects process substitution at parse time, so `done < <(find ...)` took the whole script down. Regressed in `82646c38c00`. Replaced with a here-string (not a pipe, which would put the loop body in a subshell and discard `broken`). ### Release note None ### Check List (For Author) - Test - [x] Unit Test — this PR is what makes `doris_be_test` build and link on macOS arm64 in the first place; the existing suites are the test. The ten tests that lost `#define private public` still compile and pass, now relying on the (finally effective) `-fno-access-control`. - [x] Manual test - `BUILD_TYPE_UT=Debug sh run-be-ut.sh -j8 --run` builds and links on macOS 26.5 arm64 / llvm 20 (it did not before this PR). - `sh run-fe-ut.sh` reaches and passes fe-core's `test-compile` (1333 test sources, 62 modules, zero errors). The parse fix was verified under `sh` against the real function: a nested-module report with failures returns 1 and lists only that module, a clean report returns 0 with a quoted `failures="7"` inside a stack trace correctly ignored, and no reports at all returns 0 — the empty input being the case the here-string could have broken. - Behavior changed: - [x] No. Build/test-tooling only; no runtime code path is touched. The `<ranges>` → `<algorithm>` rewrites are behavior-preserving (`std::ranges::*` algorithms live in `<algorithm>`; the two view rewrites iterate the same elements in the same order). - Does this need documentation? - [x] No. ### Proactive disclosure - The tcmalloc opt-out is deliberately scoped to `OS_MACOSX AND ARCH_ARM AND (MAKE_TEST OR BUILD_BENCHMARK)`. Linux CI, and macOS `doris_be` itself, keep linking `${MALLOCLIB}` exactly as before — this cannot change allocator behavior for any shipped binary. - Verified on macOS arm64 only for the link fixes (they are macOS-arm-specific by construction). Items 1, 3 and 4 are cross-platform and want a look from the Linux CI lines: item 1 changes how `OS_LINUX` is defined for **every** target, and while `-DOS_LINUX` is strictly more correct than `-D OS_LINUX`, it also means `-fno-access-control` starts being honoured on Linux `doris_be_test` too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
