Jeff-Shen-00 opened a new pull request, #70779:
URL: https://github.com/apache/doris/pull/70779

   
   ### Background
   
   The Backend is compiled with `-msse4.2` and, by default, `-mavx2`
   (`USE_AVX2` in `be/CMakeLists.txt`). Code guarded by `__AVX512F__` /
   `__AVX512BW__` / `__AVX512VL__` / `__AVX512DQ__` is therefore never
   compiled, even on CPUs that support AVX-512. `build.sh` exposes build
   options as environment variables and forwards them to CMake, with
   `build.sh --help` as the user-facing documentation; `run-be-ut.sh` does
   the same for the unit-test binary.
   
   ### What problem does this PR solve?
   
   Issue Number: close #70773
   Related PR: None
   PR Type: Build/CI/Chore
   
   Problem Summary:
   
   Recent Intel and AMD server CPUs support AVX-512, but there is no way to
   build the Backend with AVX-512 enabled, so every AVX-512 code path stays
   dead. This adds a `USE_AVX512` option (default OFF) that adds
   `-mavx512f -mavx512bw -mavx512vl -mavx512dq` and `-DUSE_AVX512`, and
   plumbs it through `build.sh` and `run-be-ut.sh`. Default builds are
   unchanged and keep the AVX2 CPU floor.
   
   ### Details
   
   - **Affected:** `be/CMakeLists.txt` (option, flags, AVX2 guard), `build.sh`
     (help text, default, argument summary, `-DUSE_AVX512`), `run-be-ut.sh`
     (default and `-DUSE_AVX512`).
   - **Developer and release impact:** `USE_AVX512=ON ./build.sh --be` builds
     an AVX-512 binary; the default (`OFF`) produces the same binary as
     before. An AVX-512 build requires an AVX-512 capable CPU at run time and
     faults on older hosts, which the help text and the option description
     state. The UT binary follows the same variable so tests can be built with
     the same instruction set as the BE.
   - **License and third-party:** no new dependency. The flag set includes DQ
     because the flags are inherited by `contrib/faiss`, whose `__AVX512F__`
     code paths use DQ intrinsics such as `_mm512_inserti32x8`; without DQ
     those files fail with `'__builtin_ia32_inserti32x8' needs target feature
     avx512dq,evex512`.
   - **Choice of flags:** F for 512-bit integer operations, BW for byte/word
     operations, VL for EVEX-encoded 256-bit operations, DQ for the faiss
     paths above. `-mavx512f` also defines `__AVX2__`, so `USE_AVX512=ON`
     together with `USE_AVX2=OFF` is rejected at configure time instead of
     silently producing a binary that assumes AVX2.
   - **Rollback:** revert the commit. Default builds are unaffected, and an
     AVX-512 build can still be produced by putting the flags in
     `EXTRA_CXX_FLAGS`.
   
   ### Testing
   
   - Full ASAN BE build with this flag set: 14006/14006 targets, exit 0.
     Without DQ it failed in `contrib/faiss`, which is why DQ is in the set.
   - `cmake` configure, default: `-- USE_AVX512 is OFF`; 0 of 1258 Doris
     source TUs in `compile_commands.json` carry `-mavx512*`.
   - `cmake` configure, `-DUSE_AVX512=ON`: 1258 of 1258 Doris source TUs carry
     `-mavx512f -mavx512bw -mavx512vl -mavx512dq` and `-DUSE_AVX512`.
   - `cmake` configure, `-DUSE_AVX512=ON -DUSE_AVX2=OFF`: fails with
     `USE_AVX512 requires USE_AVX2=ON: -mavx512f implies AVX2.`
     (`be/CMakeLists.txt:402`), as intended.
   - `bash -n build.sh run-be-ut.sh` passes.
   - The branch was rebased onto master `635fab23d235` before submission; the
     change itself is identical after the rebase, and the configure checks
     above were run against it on the branch's original base. This checkout
     cannot configure current master end to end because its
     `thirdparty/installed` predates mongo-c-driver (no `bson-2.0` /
     `mongoc-2.0`), which is unrelated to this change.
   
   ### Notes for Reviewers
   
   - The flags are added in the existing `if (... x86 OR x86_64)` SIMD block,
     next to `USE_AVX2` and before the `add_subdirectory` calls, so every
     Doris target and every contrib library built from `be/CMakeLists.txt`
     inherits them.
   - `USE_AVX512` is a `BOOL` cache variable, so a manual
     `cmake -DUSE_AVX512=ON` works without going through `build.sh`.
   
   ### Release note
   
   New build option `USE_AVX512` (default OFF) to build the Backend with
   AVX-512 instructions (F/BW/VL/DQ). The resulting binary requires an
   AVX-512 capable CPU.
   
   ### Check List (For Author)
   
   - Behavior changed: No by default; Yes only with `USE_AVX512=ON` (see
     Details)
   - Does this need documentation: No, `build.sh --help` documents the option.
   - Pre-submission status: Partially verified: configure-level checks and
     `bash -n` were re-run; the full BE build was done on the branch base, not
     after the rebase.
   
   ### AI Disclosure
   
   - Tool: Claude Code
   - Generated: the original branch commit (which carries a Claude Code
     co-author), the rebase onto master, and this description.
   - Human review: pending; the author has not yet reviewed the generated
     parts.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to