[
https://issues.apache.org/jira/browse/GEODE-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004351#comment-17004351
]
ASF subversion and git services commented on GEODE-7554:
--------------------------------------------------------
Commit 6e7681ddeeb9f669b9c74941424676f6293c2b7e in geode's branch
refs/heads/release/1.11.0 from Helena Bales
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=6e7681d ]
GEODE-7554: Add retry mechanism for failed tests (#4461)
* GEODE-7554: Add retry mechanism for failed tests
There are still some flaky benchmarks. Retry tests that have failed, up
to 5 times, to determine if the failure is legitimate or just a flaky
test. If the test fails 5 times in a row, we know that it is a
legitimate failure.
With each run of run_against_baseline (in the apache/geode-benchmarks
repo), the class names of all failed tests will be written to a file.
The run_benchmarks script called by CI will read the failed tests from
the file and run only those tests. Once the return code is 0, or once
we've tried 5 times, exit.
* build failedFile path dynamically
Signed-off-by: Helena A. Bales <[email protected]>
* add print of iteration number for CI debugging of failures
Signed-off-by: Robert Houghton <[email protected]>
* triggering CI again. Previous status never updated
Signed-off-by: Helena A. Bales <[email protected]>
* refactor if statement for readability
* restarting CI
* remove hardcoded cluster tag
(cherry picked from commit 278c2470a5cac2c332d13914b935f0618b820a91)
> Benchmark CI: rerun failed tests
> --------------------------------
>
> Key: GEODE-7554
> URL: https://issues.apache.org/jira/browse/GEODE-7554
> Project: Geode
> Issue Type: Improvement
> Components: benchmarks
> Reporter: Helena Bales
> Assignee: Helena Bales
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When tests run in CI fail, they will be written to a file. Read that file and
> rerun the failed tests until the list is empty, or until the run has been
> retried 5 times. Any tests that fail 5 times in a row are not flaky and need
> to be addressed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
