nic-6443 opened a new pull request, #13923: URL: https://github.com/apache/apisix/pull/13923
`rerun_flaky_tests()` in `ci/common.sh` decides the job's exit status entirely from rerunning the files its awk could pick out of the prove summary — and that pattern only matches names ending in `.t`. Anything it cannot parse is not just skipped, it is forgiven. Two ways it goes wrong: - **A failure it cannot rerun is discarded.** If a `.t` and a non-`.t` both fail in the first round and the `.t` passes on rerun, the rerun exits 0, that becomes the function's status, and the job is green having never rerun the non-`.t` at all. - **With nothing matching, it reruns the wrong thing.** `tests` ends up empty and the last line degrades to a bare `prove --timer -I./test-nginx/lib -I./`, which runs the default `t/` directory instead of the failures in hand. (`n_test` does not catch this: `echo "" | wc -l` is 1, not 0.) The fix compares the files prove reported as failed against the subset that can be rerun, and refuses when they differ or when the rerunnable set is empty. A failure this function cannot rerun is a failure, not a flake. Behaviour, driven against synthetic prove summaries with a stub `prove`: | first-round failures | before | after | | --- | --- | --- | | `Result: PASS` | exit 0 | exit 0 | | one `.t` | reruns it | reruns it | | one `.t` + one non-`.t` | reruns only the `.t`, **exit 0** | **exit 1** | | only a non-`.t` | `prove` with no file arguments, **exit 0** | **exit 1** | The first two rows are unchanged, which is the point — this only removes the cases where a real failure was being converted into a pass. **This is latent on master, not a currently red job.** No non-`.t` file is executed by prove in any `build` shard: `t/cli`'s shell scripts sit under a directory named in a test target, and `prove -r` on a directory collects only `.t`, so they run in the `CLI Test` workflow instead. I found it in a downstream fork where a `.sh` is named directly in a test target and had been failing for a long time without ever turning a job red. Sending it here because the mechanism is identical and the same shape can arise upstream the moment a non-`.t` is added to a shard. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
