[
https://issues.apache.org/jira/browse/FLINK-29198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602202#comment-17602202
]
Ryan Skraba commented on FLINK-29198:
-------------------------------------
{{@RetryOnFailure}} works as as expected, but I can reproduce this for
{{@RetryOnException}} :
{code:java}
@TestTemplate
@RetryOnException(times = 3 exception = IllegalArgumentException.class)
void testFailForever() {
throw new IllegalArgumentException();
} {code}
This test never passes, but it never fails either. The test case is aborted
and is counted as "skipped". According to the
[Javadoc|https://github.com/apache/flink/blob/8a4538ae4551713591bbe92fd3ec46b1f212e36e/flink-test-utils-parent/flink-test-utils-junit/src/main/java/org/apache/flink/testutils/junit/RetryOnException.java#L47],
it should fail after 4 tries.
It looks like the new RetryExtension explicitly does not fail on the given
exception class ever. I think it might be surprising to some developers!
Can you assign this to me?
> RetryExtension doesn't make the test fail if the retries are exhausted
> ----------------------------------------------------------------------
>
> Key: FLINK-29198
> URL: https://issues.apache.org/jira/browse/FLINK-29198
> Project: Flink
> Issue Type: Improvement
> Components: Tests
> Affects Versions: 1.15.0, 1.16.0, 1.17.0
> Reporter: Matthias Pohl
> Assignee: Ryan Skraba
> Priority: Major
> Labels: starter
>
> FLINK-24627 introduced retry functionality for JUnit5-based tests. It appears
> that the retry mechanism doesn't have the desired behavior: If the retries
> are exhausted without the test ever succeeding will result in the test being
> ignored. I would expect the test to fail in that case. Otherwise, a CI run
> would succeed without anyone noticing the malfunctioning of the test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
