Thanks for replying. Just confirming for now the workaround is to disable the retry as suggested in the linked GitHub issue.
On Monday 26 February 2024 at 23:43:24 UTC+5:30 Larry Safran wrote: > There was an issue with a test in Google's internal repository, so this > fix was temporarily backed out and is now targeted for 1.63. > > On Mon, Feb 26, 2024 at 9:45 AM 'Yuri Golobokov' via grpc.io < > grp...@googlegroups.com> wrote: > >> Maybe you are hitting this bug: >> https://github.com/grpc/grpc-java/issues/10838 >> It is fixed and backported to 1.59 and later versions. >> >> On Monday, February 26, 2024 at 7:00:57 AM UTC-8 tuk wrote: >> >>> I am using gRPC 1.55.1 and observing an issue similar to the one >>> discussed in the below GitHub issue >>> >>> - https://github.com/grpc/grpc-java/issues/9069 >>> >>> I have set the KeepAlive on both the client and server-side as suggested >>> in the above issue >>> >>> I am creating the stub like below >>> >>> var channel = ManagedChannelBuilder.forAddress(network.getIp(), >>> network.getPort()) >>> .keepAliveTime(130, TimeUnit.SECONDS) >>> .maxInboundMessageSize(maxInboundMessageSize) >>> .maxInboundMetadataSize(maxInboundMetadataSize) >>> .enableRetry() >>> .build(); >>> >>> var stub = >>> HelloServiceGrpc.newBlockingStub(channel).withDeadline(Deadline.after(115, >>> TimeUnit.SECONDS)); >>> stub.sayHello(); >>> stub.sayHello(); >>> >>> In server side also keepAliveTime is set as suggested in the above >>> github issue. >>> >>> Grpc.newServerBuilderForPort(port, InsecureServerCredentials.create()) >>> .addService(new GreeterImpl()) >>> .keepAliveTime(130, TimeUnit.SECONDS) >>> .build() >>> .start(); >>> >>> In my case, client calls server 1 then server 1 acts as a client to >>> server 2. >>> >>> I am observing that when the deadline is exceeded in server2 the same >>> deadline error is coming to server 1 like below which is also coming to >>> client. It is working as expected. >>> >>> io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: context timed out >>> at >>> io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271) >>> at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252) >>> at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165) >>> >>> >>> But on some rare occasions I am observing the client thread hanging like >>> the below and not getting the deadline exceeded error >>> >>> at jdk.internal.misc.Unsafe.park(java...@17.0.9/Native Method) >>> - parking to wait for <0x0000000767a53a00> (a >>> io.grpc.stub.ClientCalls$ThreadlessExecutor) >>> at >>> java.util.concurrent.locks.LockSupport.park(java...@17.0.9/LockSupport.java:211) >>> at >>> io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:748) >>> at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:157) >>> >>> I waited for about 2 hours & it did not recover. The only way to recover >>> from this is to restart the client application. It happened 3-4 times in >>> the last couple of months for us. >>> >>> Can someone let me know >>> >>> 1. What I am doing wrong or is there any known issue in the grpc-java >>> 1.55.1 that I am using? >>> 2. Is there any timeout config I can set on the gRPC client side so >>> that the client threads do not hang indefinitely? >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to grpc-io+u...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/grpc-io/d2700afd-762e-49ee-a15b-7f7901a6d10cn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/grpc-io/d2700afd-762e-49ee-a15b-7f7901a6d10cn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/80d5231c-d509-4c20-824b-34d71ee97f18n%40googlegroups.com.
