Claus Ibsen created CAMEL-26271:
-----------------------------------
Summary: AGENTS.md - add guidelines for writing stable tests that
run in parallel
Key: CAMEL-26271
URL: https://issues.apache.org/jira/browse/CAMEL-26271
Project: Camel
Issue Type: Task
Components: tests
Reporter: Claus Ibsen
{{AGENTS.md}} has no guidance on how to write tests that are stable in CI. Many
PRs are now written or reviewed by AI agents, and the build runs with {{-T1C}}
and parallel JUnit tests, so a test that is not safe to run in parallel becomes
a flaky test on Jenkins and GitHub Actions.
The causes and fixes found while stabilizing the Camel Core CI (see the blog
post "Scaling the Apache Camel Core CI",
https://github.com/apache/camel-website/pull/1831) could become a short
"Writing stable tests" section in {{AGENTS.md}}.
h3. Common causes of flaky tests
* ports that clash for services used in tests (use the test-infra /
{{AvailablePortFinder}} helpers, never fixed ports)
* shared systems: tear-downs that clear all contents, or names and paths reused
across tests
* route ids or endpoint URIs that collide between tests
* {{Thread.sleep}} instead of waiting for a condition
* the next step of a test not waiting for the system to reach the expected state
h3. Preferred fixes
* wait with Awaitility for the system to be ready, or for the expected state
* {{@Isolated}} when a test cannot share the JVM with others (it costs build
time, so use it as a last resort)
* do not just increase a timeout: first check whether the passing test is close
to the timeout; if not, the problem is elsewhere and a longer timeout only
delays the failure
* check that a new or changed test passes when run in parallel, not only on its
own
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
