AlexStocks commented on code in PR #23: URL: https://github.com/apache/dubbo-net/pull/23#discussion_r4463446272
########## Tests/EndToEndTestEnvironment/wait-for-environment.sh: ########## @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -euo pipefail + +script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +log_dir=${1:-"$script_dir/logs"} +provider_host=${DUBBONET_PROVIDER_HOST:-} +if [[ -z "$provider_host" && -f "$log_dir/provider-host" ]]; then + provider_host=$(<"$log_dir/provider-host") +fi +if [[ -z "$provider_host" ]]; then + echo "Cannot determine the provider host." >&2 + exit 1 +fi + +ready_timeout=${DUBBONET_E2E_READY_TIMEOUT:-180} +if [[ ! "$ready_timeout" =~ ^[1-9][0-9]*$ ]]; then + echo "DUBBONET_E2E_READY_TIMEOUT must be a positive number of seconds." >&2 + exit 1 +fi +ready_deadline=$((SECONDS + ready_timeout)) + +check_provider_processes() { + if [[ ! -s "$log_dir/provider-pids" ]]; then + echo "No provider PID file found; start the providers first." >&2 + return 1 + fi + while IFS= read -r provider_pid; do + if [[ ! "$provider_pid" =~ ^[0-9]+$ ]] || ! kill -0 "$provider_pid" 2>/dev/null; then + echo "Provider process $provider_pid exited during startup. Inspect $log_dir." >&2 + return 1 + fi + done < "$log_dir/provider-pids" +} + +wait_for_tcp() { + local host=$1 + local port=$2 + local label=$3 + + while (( SECONDS < ready_deadline )); do + check_provider_processes + if timeout 2 bash -c 'echo > "/dev/tcp/$1/$2"' _ "$host" "$port" >/dev/null 2>&1; then + echo "$label is ready on $host:$port." + return 0 + fi + sleep 2 + done + + echo "Timed out waiting for $label on $host:$port." >&2 + return 1 +} + +wait_for_tcp 127.0.0.1 2181 ZooKeeper +wait_for_tcp 127.0.0.1 8848 Nacos +wait_for_tcp 127.0.0.1 9848 "Nacos gRPC" +wait_for_tcp "$provider_host" 20883 "Dubbo 2.7.23 provider" +wait_for_tcp "$provider_host" 20882 "Dubbo 3.1.11 provider" +wait_for_tcp "$provider_host" 20881 "Dubbo 3.2.20 provider" +wait_for_tcp "$provider_host" 20880 "Dubbo 3.3.6 provider" +wait_for_tcp "$provider_host" 50051 "Dubbo 3.3.6 Triple provider" + +check_provider_processes + +# Ports open before registry metadata is always visible. Give both registries a +# short, deterministic settling window before the first discovery request. +sleep 10 Review Comment: 🟡 **`sleep 10` 是这套「可复现」环境里唯一的非确定性等待** ```bash # Ports open before registry metadata is always visible. Give both registries a # short, deterministic settling window before the first discovery request. sleep 10 ``` 注释说「deterministic」,但固定睡眠恰恰是**不确定**的反面:它把「需要多久」写死成一个与机器负载无关的常数。端口先于元数据可见是真的(Dubbo provider 先 bind 再注册,ZK/Nacos 写入还有一跳),但这段延迟在快机器上是 0.5 s、在慢机器或 Nacos 元数据加载慢时可能超过 10 s,两种情况下这个脚本都做了错误的事——一个白等 9.5 s,一个直接导致首批用例发现不到 provider。 而这个脚本**已经具备了轮询的全部零件**:`ready_deadline`、`DUBBONET_E2E_READY_TIMEOUT` 参数校验、`timeout 2 bash -c 'echo > /dev/tcp/...'` 探活、失败时打印 `$log_dir`。把固定睡眠换成「读注册中心直到看到预期 provider 数」只是一次改造,不是新机制。 **Suggestion(示意,ZooKeeper 侧):** ```bash wait_for_providers() { local root=$1 expected=$2 while (( SECONDS < ready_deadline )); do check_provider_processes # 每个 provider 版本注册 5 个服务,共享同一 root local seen seen=$(docker compose -f "$script_dir/compose.yaml" exec -T zookeeper zkCli.sh -server 127.0.0.1:2181 ls "/$root" 2>/dev/null | grep -c . || true) if (( seen >= expected )); then echo "Registry /$root has $seen services." return 0 fi sleep 2 done echo "Timed out waiting for /$root to expose $expected services." >&2 return 1 } wait_for_providers dubbo-2.7.23 5 # 与 EndToEndTestCatalog.Methods 的服务数一致 ``` Nacos 侧可以走 `curl -s "127.0.0.1:8848/nacos/v1/ns/catalog/services?hasIpCount=true&withInstances=false&pageNo=1&pageSize=100&namespaceId=&groupName=dubbo-3.3.6"`。两者都比 `sleep 10` 更慢不了多少,但**失败时给出的是「哪个 root 少了几个服务」,而不是后面某条用例莫名其妙的 `QuerySuccess=False`**。 如果不想在这里引入 `zkCli.sh` / `curl` 依赖,退一步也行:把 `sleep 10` 改成「轮询一个最轻量的元数据读」——哪怕是让 `QueryDemo --print-config` 之外再加一个只读发现的小步骤。核心是**不要用一个与目标状态无关的时长来代表目标状态**。 ########## .github/workflows/end-to-end.yml: ########## @@ -0,0 +1,180 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +name: End-to-end tests + +on: + push: + branches: [main] + pull_request: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: end-to-end-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true Review Comment: 🟡 **`cancel-in-progress: true` 对 `push: main` 也用上了** ```yaml concurrency: group: end-to-end-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: true ``` `group` 的写法是对的(PR 按号、push 按 ref)。但 `cancel-in-progress: true` 对两种事件的效果完全不同: - **PR**:新 push 取消旧运行,正是想要的——省资源、且结论只关心最新 commit。 - **`push: main`**:连续两个提交落到 main(squash merge 连发时很常见)会**取消**上一轮 main 上的端到端运行。结果是 main 上留下一个 `cancelled` 的 run,而那个 commit 其实从来没被端到端验证过。目前这不是 required check 所以只是噪音,但一旦有人把它加进 ruleset 的 required checks,`cancelled` 会变成「main 红」,而且用 rerun 也很难稳定复现。 `Build and test` 那边没有配 concurrency(我确认过 `build.yml` 没有 `concurrency` 段),所以两个 workflow 在这点上不一致。 **Suggestion:** 只对 PR 取消,main 上排队: ```yaml concurrency: group: end-to-end-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: ${{ github.event_name == 'pull_request' }} ``` 顺带说明一下为什么值得较真:这个 workflow 的价值在于「每个进入 main 的 commit 都过过真实 provider」。main 上的运行被静默取消,恰好破坏的就是这个保证。 ########## Tests/EndToEndTestEnvironment/compose.yaml: ########## @@ -0,0 +1,38 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +name: dubbonet-e2e Review Comment: 💭 **这个 `name:` 在 CI 里其实不生效,本地跑又会和 CI 不是同一个 project** ```yaml name: dubbonet-e2e ``` Docker Compose 的 project 名优先级是:`-p` 参数 > `COMPOSE_PROJECT_NAME` 环境变量 > 顶层 `name:` > 目录名。workflow 里设了 ```yaml env: COMPOSE_PROJECT_NAME: dubbonet-e2e-${{ github.run_id }}-${{ github.run_attempt }} ``` 所以 CI 里 `name: dubbonet-e2e` 被覆盖(这正是你要的效果:按 run 隔离),`run-local.sh` 也自己设了 `dubbonet-e2e-local-$$`。**结果是这个字段在任何真实路径下都不生效**,只有人手工 `docker compose -f .../compose.yaml up -d` 调试时才生效——而那时它产生的是一个和 CI 不同名的 project,`docker compose down` 忘了带同一个 `-f`/env 就会留下 `dubbonet-e2e_zookeeper-data` 这类卷。 不影响功能,但它是一个「看起来在配置、实际不生效」的字段,容易误导排查。 **Suggestion:** 删掉这一行让 project 名完全由调用方决定;或者保留但在上方注释写清「仅在未设置 `COMPOSE_PROJECT_NAME` 时生效,CI 与 run-local.sh 都会覆盖它」。我更倾向删掉——少一个需要解释的例外。 ########## Tests/EndToEndTestEnvironment/run-local.sh: ########## @@ -0,0 +1,83 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -euo pipefail + +script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +repo_root=$(CDPATH= cd -- "$script_dir/../.." && pwd) +cd "$repo_root" + +for required in docker java mvn dotnet timeout; do + if ! command -v "$required" >/dev/null 2>&1; then + echo "Required tool is missing: $required. See $script_dir/README.md." >&2 + exit 1 + fi +done +docker compose version >/dev/null +docker info >/dev/null + +log_dir=${1:-$(mktemp -d "${TMPDIR:-/tmp}/dubbonet-e2e.XXXXXX")} +mkdir -p "$log_dir" +log_dir=$(CDPATH= cd -- "$log_dir" && pwd) +export COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME:-dubbonet-e2e-local-$$} + +cleanup() { + local status=$? + trap - EXIT + docker compose -f "$script_dir/compose.yaml" logs --no-color > "$log_dir/registries.log" 2>&1 || true + "$script_dir/stop-providers.sh" "$log_dir" || status=1 + docker compose -f "$script_dir/compose.yaml" down --volumes --remove-orphans || status=1 + echo "End-to-end logs: $log_dir" + exit "$status" +} +trap cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM + +run_logged() { + local name=$1 + shift + "$@" 2>&1 | tee "$log_dir/$name.log" +} + +docker compose -f "$script_dir/compose.yaml" up -d +run_logged java-build mvn -B -ntp -f "$script_dir/JavaProviders/pom.xml" package -DskipTests +"$script_dir/start-providers.sh" "$log_dir" +export DUBBONET_PROVIDER_HOST Review Comment: 💭 **`export` 写在赋值之前,读起来像 bug(虽然实际有效)** ```bash export DUBBONET_PROVIDER_HOST DUBBONET_PROVIDER_HOST=$(<"$log_dir/provider-host") ``` 在 bash 里这是有效的:`export NAME` 只是给变量打上 export 属性,之后赋值会继承该属性,子进程能看到新值。所以 `start-providers.sh` / `wait-for-environment.sh` / `dotnet run` 都能拿到。 但它和上面第 46 行的写法形成了对照——那里是 `start-providers.sh` 自己算 host 并写文件,这里又从文件读回来。两个顺序放反的意图(「先声明导出、后取值」)几乎肯定不是有人刻意选的,而是写的时候顺手;**下一个人做 code review 时会花时间确认「这会不会导致 host 为空」**,而 `set -u` 下 `$(<file)` 读空文件会得到空字符串(不报错),后面 `--provider-host ""` 又会被 `TestEndpointOptions.Value()` 的 `IsNullOrWhiteSpace` 拦下——路径很长,读者得自己走一遍才能放心。 **Suggestion:** ```bash DUBBONET_PROVIDER_HOST=$(<"$log_dir/provider-host") export DUBBONET_PROVIDER_HOST ``` 一行之差,语义就从「需要推理才确定正确」变成「扫一眼就对」。 ########## .github/workflows/end-to-end.yml: ########## @@ -0,0 +1,180 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +name: End-to-end tests + +on: + push: + branches: [main] + pull_request: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: end-to-end-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + end-to-end: + name: ZooKeeper, Nacos and direct protocols + runs-on: ubuntu-22.04 + timeout-minutes: 30 + defaults: + run: + shell: bash + env: + COMPOSE_PROJECT_NAME: dubbonet-e2e-${{ github.run_id }}-${{ github.run_attempt }} + + steps: + - name: Check out source + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Configure the end-to-end log directory + run: echo "E2E_LOG_DIR=$RUNNER_TEMP/dubbonet-e2e-logs" >> "$GITHUB_ENV" + + - name: Set up Java + uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5 + with: + distribution: temurin + java-version: "8" + cache: maven + cache-dependency-path: | + Tests/EndToEndTestEnvironment/JavaProviders/pom.xml + Tests/EndToEndTestEnvironment/JavaProviders/*/pom.xml + + - name: Set up .NET + uses: actions/setup-dotnet@a98b56852c35b8e3190ac28c8c2271da59106c68 # v6.0.0 + with: + dotnet-version: 10.0.x + + - name: Start temporary registries + run: docker compose -f Tests/EndToEndTestEnvironment/compose.yaml up -d + + - name: Resolve the Runner provider address + run: | + provider_host=$(ip -4 route get 1.1.1.1 | awk '{ for (field = 1; field <= NF; field++) if ($field == "src") { print $(field + 1); exit } }') + if [[ -z "$provider_host" || "$provider_host" == 127.* ]]; then + echo "Cannot resolve a non-loopback Runner IPv4 address." >&2 + exit 1 + fi + echo "DUBBONET_PROVIDER_HOST=$provider_host" >> "$GITHUB_ENV" + echo "Providers will register $provider_host." + + - name: Build Java providers + run: >- + mvn -B -ntp + -f Tests/EndToEndTestEnvironment/JavaProviders/pom.xml + package + -DskipTests + + - name: Start Java providers + run: Tests/EndToEndTestEnvironment/start-providers.sh "$E2E_LOG_DIR" + + - name: Wait for the temporary environment + run: Tests/EndToEndTestEnvironment/wait-for-environment.sh "$E2E_LOG_DIR" + + - name: Restore .NET projects + run: dotnet restore DubboNet.sln + + - name: Test every provider method end to end + env: + DUBBONET_E2E_ENABLED: "true" + DUBBONET_E2E_ZOOKEEPER_ADDRESS: 127.0.0.1:2181 + DUBBONET_E2E_NACOS_ADDRESS: nacos://127.0.0.1:8848 + run: >- + dotnet test + Tests/DubboNetEndToEndTests/DubboNetEndToEndTests.csproj + --configuration Release + --no-restore + -p:GeneratePackageOnBuild=false + --logger "console;verbosity=detailed" + --logger "trx;LogFileName=dubbonet-end-to-end.trx" + --results-directory "$E2E_LOG_DIR/test-results" + 2>&1 | tee "$E2E_LOG_DIR/end-to-end-tests.log" + + - name: Test ZooKeeper discovery and all Dubbo versions + run: >- + dotnet run + --project Tests/DubboClientIntegrationTests/DubboClientIntegrationTests.csproj + --configuration Release + --no-restore + -p:GeneratePackageOnBuild=false + -- --registry 127.0.0.1:2181 --provider-host "$DUBBONET_PROVIDER_HOST" + 2>&1 | tee "$E2E_LOG_DIR/integration-zookeeper.log" + + - name: Test Nacos discovery + run: >- + dotnet run + --project Tests/DubboClientIntegrationTests/DubboClientIntegrationTests.csproj + --configuration Release + --no-restore + -p:GeneratePackageOnBuild=false + -- --registry nacos://127.0.0.1:8848 --provider-host "$DUBBONET_PROVIDER_HOST" + 2>&1 | tee "$E2E_LOG_DIR/integration-nacos.log" + + - name: Run QueryDemo with ZooKeeper + run: >- + dotnet run + --project Tests/QueryDemo/QueryDemo.csproj + --configuration Release + --no-restore + -p:GeneratePackageOnBuild=false + -- --registry 127.0.0.1:2181 --provider-host "$DUBBONET_PROVIDER_HOST" + 2>&1 | tee "$E2E_LOG_DIR/query-demo-zookeeper.log" + + - name: Run QueryDemo with Nacos + run: >- + dotnet run + --project Tests/QueryDemo/QueryDemo.csproj + --configuration Release + --no-restore + -p:GeneratePackageOnBuild=false + -- --registry nacos://127.0.0.1:8848 --provider-host "$DUBBONET_PROVIDER_HOST" + 2>&1 | tee "$E2E_LOG_DIR/query-demo-nacos.log" + + - name: Collect registry logs + if: always() + run: | + mkdir -p "$E2E_LOG_DIR" + docker compose -f Tests/EndToEndTestEnvironment/compose.yaml logs --no-color > "$E2E_LOG_DIR/registries.log" 2>&1 || true + + - name: Upload end-to-end logs + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7 + with: + name: end-to-end-logs-${{ github.run_attempt }} + path: ${{ env.E2E_LOG_DIR }} + retention-days: 7 + if-no-files-found: warn + + - name: Upload end-to-end test results + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7 + with: + name: end-to-end-test-results-${{ github.run_attempt }} + path: ${{ env.E2E_LOG_DIR }}/test-results + retention-days: 7 + if-no-files-found: warn + + - name: Stop temporary environment + if: always() + run: | + Tests/EndToEndTestEnvironment/stop-providers.sh "$E2E_LOG_DIR" Review Comment: 🟡 **清理步骤失败会把全绿的测试变成红 job,而且 `docker compose down` 会被跳过** ```yaml - name: Stop temporary environment if: always() run: | Tests/EndToEndTestEnvironment/stop-providers.sh "$E2E_LOG_DIR" docker compose -f Tests/EndToEndTestEnvironment/compose.yaml down --volumes --remove-orphans ``` GitHub 对 `run: |` 的默认 shell 是 `/usr/bin/bash --noprofile --norc -e -o pipefail`,所以: 1. `stop-providers.sh` 一旦非零退出,**第 2 行的 `docker compose down` 不会执行**——容器和 volume 留在 runner 上。runner 是一次性的,泄漏本身影响有限,但这刚好让「清理」在真正需要它的时候失效。 2. 更实际的问题:这一步带 `if: always()`,所以它在测试全绿之后也会跑;**它失败会把整个 job 标红**。于是出现「156/156 通过但 PR 是红的」,而且失败原因藏在日志最底部,排查成本很高。 `run-local.sh` 里对同一件事的处理反而更严谨(`stop-providers.sh ... || status=1` 后仍然执行 `docker compose down`),两处不一致。 **Suggestion:** 清理做成「尽力而为 + 不改变测试结论」,顺序也调成先拆容器(更可能卡住的是 provider 进程,让它的失败不影响拆容器): ```yaml - name: Stop temporary environment if: always() run: | docker compose -f Tests/EndToEndTestEnvironment/compose.yaml down --volumes --remove-orphans || true Tests/EndToEndTestEnvironment/stop-providers.sh "$E2E_LOG_DIR" || true ``` 如果确实希望「清理失败可见」,那就拆成两步并给清理步加 `continue-on-error: true`,让它以黄色警告出现而不是把 job 判红——前提是这个 runner 已经上传完日志(现在顺序是对的,两个上传都在第 18/19 步,早于清理,这点没问题)。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
