[
https://issues.apache.org/jira/browse/RATIS-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17797942#comment-17797942
]
Attila Doroszlai commented on RATIS-1980:
-----------------------------------------
{code:title=https://github.com/apache/ratis/blob/2728ba8cee7a462854d495ee95051e1ebad3761f/ratis-test/src/test/java/org/apache/ratis/grpc/TestRetryCacheWithGrpc.java#L125-L132}
stateMachine.blockApplyTransaction();
final List<CompletableFuture<Long>> asyncCalls = new ArrayList<>();
for (int i = 0; i < 5; i++) {
// Since applyTransaction is blocked, the replied call id remains the
same.
asyncCalls.add(sendAsync(client));
}
// async call will invalidate blocking calls even if applyTransaction
is blocked.
assertRetryCacheEntry(client, lastBlockingCallId, false);
{code}
With some additional debug log:
{code}
2023-12-17 16:27:29,103 [Time-limited test] INFO
impl.SimpleStateMachine4Testing (SimpleStateMachine4Testing.java:block(135)) -
block APPLY_TRANSACTION
...
2023-12-17 16:27:29,110 [Time-limited test] TRACE server.RetryCache
(RetryCacheImpl.java:getIfPresent(258)) - getIfPresent 5@client-706FB9B42965
returns 5@client-706FB9B42965:done
2023-12-17 16:27:29,110 [Time-limited test] INFO grpc.TestRetryCacheWithGrpc
(TestRetryCacheWithGrpc.java:assertRetryCacheEntry(86)) - assertRetryCacheEntry
5@client-706FB9B42965 absent: false
...
2023-12-17 16:27:29,110 [s1-client-thread1] DEBUG server.RaftServer$Division
(RaftServerImpl.java:submitClientRequestAsync(905)) - s1@group-832C038C25B0:
receive client
request(RaftClientRequest:client-706FB9B42965->s1@group-832C038C25B0, cid=6,
seq=1*, Watch(0), Message:<EMPTY>)
2023-12-17 16:27:29,110 [s1-client-thread1] DEBUG server.RetryCache
(RetryCacheImpl.java:invalidateRepliedRequests(246)) -
invalidateRepliedRequests callIds [5] for client-706FB9B42965
2023-12-17 16:27:29,110 [s1-client-thread1] TRACE server.RetryCache
(RetryCacheImpl.java:lambda$new$0(188)) - removed 5@client-706FB9B42965 =>
5@client-706FB9B42965:done
{code}
RetryCache entry is removed asynchronously (in {{s1-client-thread1}} here) when
sending async request. The problem is that the test proceeds to assert without
waiting for the async request to be sent.
> Intermittent failure in TestRetryCacheWithGrpc#testInvalidateRepliedCalls
> -------------------------------------------------------------------------
>
> Key: RATIS-1980
> URL: https://issues.apache.org/jira/browse/RATIS-1980
> Project: Ratis
> Issue Type: Bug
> Components: gRPC
> Affects Versions: 3.0.0
> Reporter: Attila Doroszlai
> Priority: Major
>
> Intermittent failure:
> {code:title=https://github.com/adoroszlai/ratis-build-results/blob/master/2023/12/13/2862/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt}
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 10.213 s <<<
> FAILURE! - in org.apache.ratis.grpc.TestRetryCacheWithGrpc
> org.apache.ratis.grpc.TestRetryCacheWithGrpc.testInvalidateRepliedCalls Time
> elapsed: 1.61 s <<< FAILURE!
> java.lang.AssertionError: expected null, but was:<5@client-18AC80E03421:done>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotNull(Assert.java:756)
> at org.junit.Assert.assertNull(Assert.java:738)
> at org.junit.Assert.assertNull(Assert.java:748)
> at
> org.apache.ratis.grpc.TestRetryCacheWithGrpc$InvalidateRepliedCallsTest.assertRetryCacheEntry(TestRetryCacheWithGrpc.java:88)
> at
> org.apache.ratis.grpc.TestRetryCacheWithGrpc$InvalidateRepliedCallsTest.run(TestRetryCacheWithGrpc.java:132)
> at
> org.apache.ratis.grpc.TestRetryCacheWithGrpc.lambda$testInvalidateRepliedCalls$0(TestRetryCacheWithGrpc.java:61)
> at
> org.apache.ratis.server.impl.MiniRaftCluster$Factory$Get.runWithNewCluster(MiniRaftCluster.java:141)
> at
> org.apache.ratis.server.impl.MiniRaftCluster$Factory$Get.runWithNewCluster(MiniRaftCluster.java:121)
> at
> org.apache.ratis.grpc.TestRetryCacheWithGrpc.testInvalidateRepliedCalls(TestRetryCacheWithGrpc.java:61)
> {code}
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/10/31/2737/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/11/07/2761/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/11/20/2793/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/11/28/2799/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/11/29/2800/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/12/01/2809/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/12/10/2833/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
> *
> https://github.com/adoroszlai/ratis-build-results/blob/master/2023/12/13/2862/unit-grpc/ratis-test/org.apache.ratis.grpc.TestRetryCacheWithGrpc.txt
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
