github-actions[bot] commented on code in PR #65055: URL: https://github.com/apache/doris/pull/65055#discussion_r3502844600
########## tools/fast-fe-ut.sh: ########## @@ -0,0 +1,227 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# fast-fe-ut.sh — Fast FE unit test runner +# +# Usage: +# ./fast-fe-ut.sh # Run all UT +# ./fast-fe-ut.sh org.apache.doris.utframe.Demo # Run a specific test class +# ./fast-fe-ut.sh DemoTest#testMethod1+testMethod2 # Run specific test methods +# ./fast-fe-ut.sh org.apache.doris.Demo,org.apache.doris.Demo2 # Run multiple test classes +# +# How it works: +# 1. Fast incremental compile (main + test) via tools/fast-compile-fe.sh +# 2. If compilation succeeds → run tests via mvn surefire:test (skips Maven compile) +# 3. If compilation fails → fall back to run-fe-ut.sh --run <test_spec> + +set -eo pipefail + +DORIS_HOME="$(cd "$(dirname "$0")/.." && pwd)" +export DORIS_HOME + +# Save original arguments for potential fallback to run-fe-ut.sh +ORIGINAL_ARGS=("$@") Review Comment: This assignment is never read, and the repository ShellCheck workflow checks newly added shell scripts. Locally, `shellcheck tools/fast-fe-ut.sh` reports `SC2034` here, so this PR will fail the `ShellCheck` job unless the variable is removed, used, or explicitly suppressed with a real justification. ########## tools/fast-fe-ut.sh: ########## @@ -0,0 +1,227 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# fast-fe-ut.sh — Fast FE unit test runner +# +# Usage: +# ./fast-fe-ut.sh # Run all UT +# ./fast-fe-ut.sh org.apache.doris.utframe.Demo # Run a specific test class +# ./fast-fe-ut.sh DemoTest#testMethod1+testMethod2 # Run specific test methods +# ./fast-fe-ut.sh org.apache.doris.Demo,org.apache.doris.Demo2 # Run multiple test classes +# +# How it works: +# 1. Fast incremental compile (main + test) via tools/fast-compile-fe.sh +# 2. If compilation succeeds → run tests via mvn surefire:test (skips Maven compile) +# 3. If compilation fails → fall back to run-fe-ut.sh --run <test_spec> + +set -eo pipefail + +DORIS_HOME="$(cd "$(dirname "$0")/.." && pwd)" +export DORIS_HOME + +# Save original arguments for potential fallback to run-fe-ut.sh +ORIGINAL_ARGS=("$@") + +. "${DORIS_HOME}/env.sh" + +# ─── Color Output ──────────────────────────────────────────────────────────────── +RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m' +info() { echo -e "${GREEN}[INFO]${NC} $*"; } +warn() { echo -e "${YELLOW}[WARN]${NC} $*"; } +error() { echo -e "${RED}[ERROR]${NC} $*" >&2; } + +# ─── Helper functions (from run-fe-ut.sh) ───────────────────────────────────── + +trim_whitespace() { + local value="$1" + value="${value#"${value%%[![:space:]]*}"}" + value="${value%"${value##*[![:space:]]}"}" + printf '%s' "${value}" +} + +is_valid_extra_module_feature() { + local feature="$1" + [[ "${feature}" =~ ^[A-Za-z][A-Za-z0-9_-]*$ ]] +} + +parse_extra_fe_modules() { + local spec_value="$1" + local entry feature module_path existing + local -a feature_keys=() + + FE_EXTRA_MODULE_PATHS=() + if [[ -z "${spec_value}" ]]; then + return + fi + + IFS=',' read -r -a entries <<<"${spec_value}" + for entry in "${entries[@]}"; do + entry="$(trim_whitespace "${entry}")" + if [[ -z "${entry}" || "${entry}" != *=* ]]; then + echo "Invalid EXTRA_FE_MODULES entry '${entry}': expected feature=module_path" + exit 1 + fi + + feature="$(trim_whitespace "${entry%%=*}")" + module_path="$(trim_whitespace "${entry#*=}")" + if [[ -z "${feature}" || -z "${module_path}" ]]; then + echo "Invalid EXTRA_FE_MODULES entry '${entry}': feature and module_path must be non-empty" + exit 1 + fi + if ! is_valid_extra_module_feature "${feature}"; then + echo "Invalid EXTRA_FE_MODULES feature '${feature}'" + exit 1 + fi + for existing in "${feature_keys[@]}"; do + if [[ "${existing}" == "${feature}" ]]; then + echo "Duplicate EXTRA_FE_MODULES feature '${feature}'" + exit 1 + fi + done + if [[ ! -f "${DORIS_HOME}/fe/${module_path}/pom.xml" ]]; then + echo "Missing EXTRA_FE_MODULES module: ${DORIS_HOME}/fe/${module_path}/pom.xml" + exit 1 + fi + feature_keys+=("${feature}") + FE_EXTRA_MODULE_PATHS+=("${module_path}") + done +} + +# ─── Usage ────────────────────────────────────────────────────────────────────── + +usage() { + echo " +Usage: $0 [<test_spec>] + test_spec Optional. Class name, method, or comma-separated list. + If omitted, runs all FE unit tests. + + Environment variables: + EXTRA_FE_MODULES Optional FE feature modules in feature=module_path format, separated by commas. + + Eg. + $0 run all UT + $0 org.apache.doris.utframe.Demo run a specific test class + $0 org.apache.doris.utframe.Demo#testCreateDbAndTable run specific test methods + $0 org.apache.doris.Demo,org.apache.doris.Demo2 run multiple test classes + " + exit 1 +} + +# ─── Argument parsing ───────────────────────────────────────────────────────── + +TEST_SPEC="" +if [[ $# -ge 1 ]]; then + TEST_SPEC="$1" +fi + +# ─── Build module list ───────────────────────────────────────────────────────── + +EXTRA_FE_MODULES="${EXTRA_FE_MODULES:-}" +parse_extra_fe_modules "${EXTRA_FE_MODULES}" + +FE_MODULES=("fe-common" "fe-core") +for extra_module_path in "${FE_EXTRA_MODULE_PATHS[@]}"; do + FE_MODULES+=("${extra_module_path}") +done +MVN_MODULES="$(IFS=','; echo "${FE_MODULES[*]}")" + +echo "Fast FE UT Runner + TEST_SPEC -- ${TEST_SPEC:-<all>} + EXTRA_FE_MODULES -- ${EXTRA_FE_MODULES} + MVN_MODULES -- ${MVN_MODULES} +" + +echo "**************************************" +echo " Fast DorisFe Unittest (incremental) " +echo "**************************************" + +# ─── Pre-build: generated sources ────────────────────────────────────────────── + +bash "${DORIS_HOME}/generated-source.sh" + +cd "${DORIS_HOME}/fe" +mkdir -p build/compile + +if [[ -z "${FE_UT_PARALLEL}" ]]; then + export FE_UT_PARALLEL=1 +fi +echo "Unit test parallel is: ${FE_UT_PARALLEL}" + +# ─── Fallback helper ─────────────────────────────────────────────────────────── + +fallback() { + local fallback_args=() + if [[ -n "${TEST_SPEC}" ]]; then + fallback_args=(--run "${TEST_SPEC}") + fi + warn "Falling back to run-fe-ut.sh ${fallback_args[*]} ..." + exec bash "${DORIS_HOME}/run-fe-ut.sh" "${fallback_args[@]}" +} + +# ─── Step 1: Fast compile main sources ──────────────────────────────────────── + +echo "" +echo "=== Step 1/3: Fast compile main sources ===" +FAST_COMPILE_SCRIPT="${DORIS_HOME}/tools/fast-compile-fe.sh" + +if [[ ! -f "${FAST_COMPILE_SCRIPT}" ]]; then + warn "fast-compile-fe.sh not found" + fallback +fi + +if ! bash "${FAST_COMPILE_SCRIPT}" 2>&1; then + warn "Fast compile (main) failed" + fallback +fi + +# ─── Step 2: Fast compile test sources ──────────────────────────────────────── + +echo "" +echo "=== Step 2/3: Fast compile test sources ===" + +if ! bash "${FAST_COMPILE_SCRIPT}" --test 2>&1; then + warn "Fast compile (test) failed" + fallback +fi + +# ─── Step 3: Run tests via surefire ─────────────────────────────────────────── + +echo "" +echo "=== Step 3/3: Run unit tests (surefire) ===" + +# Build the surefire command. +# -pl selects the modules to run tests in, -am includes their dependencies in the +# reactor so that ${revision} placeholders are resolved correctly for sibling +# modules (fe-foundation, fe-catalog, etc.) without triggering compilation. +SUREFIRE_ARGS=( + surefire:test Review Comment: This shortcut still lets `surefire:test` run even though the only compiler it invokes is `tools/fast-compile-fe.sh`, which is hard-coded to refresh `fe-core` source and test output. The Maven reactor selected here includes `fe-common` and any `EXTRA_FE_MODULES`, and normal `mvn test` also runs lifecycle work that this call skips, including test-compile/resource processing and the `generate-sources` executions that rebuild Nereids `GeneratedPlanPatterns`/`GeneratedMemoPatterns`. After changing `fe-common`, an extra FE module, a test resource, or a Nereids pattern-generation input, this helper can therefore execute tests against previously compiled/generated artifacts and still report success. Please either run the Maven lifecycle far enough for every selected module/generator before Surefire, or restrict/fallback when the fast compiler cannot refresh the affected module/generation step. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
