The client automatically recovered after a few hours. This is the only thread that is relevant. I did not explicitly set waitForReady, and I think by default it is disabled. I can provide more thread dumps when it happen again next time. Let me know if any information other than thread dump is needed.
On Thu, Oct 19, 2017 at 10:31 AM 'Carl Mastrangelo' via grpc.io < [email protected]> wrote: > Is that the only thread? Also, are you using waitForReady? > > > On Wednesday, October 18, 2017 at 3:40:19 PM UTC-7, [email protected] > wrote: >> >> The problem happened to me again even with keep-alive. >> netstats suggests that underlying tcp connection is established. >> Client thread dump follows: >> java.lang.Thread.State: WAITING (parking) >> at sun.misc.Unsafe.park(Native Method) >> - parking to wait for <0x00007f1624000038> (a >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) >> at >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) >> at >> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) >> at >> io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:572) >> at >> io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:120) >> >> All server threads are idle. It is likely server receives no request. >> >> >> On Wednesday, October 11, 2017 at 7:57:28 AM UTC-7, Taehyun Park wrote: >>> >>> This is what I did to avoid this problem in production. I wrapped all >>> grpc calls with RxJava and used Retry to re-initialize channel when >>> DEADLINE_EXCEEDED is thrown. >>> >>> >>> >>> On Tuesday, September 26, 2017 at 7:19:14 AM UTC+9, [email protected] >>> wrote: >>>> >>>> Language: java >>>> Version: 1.5 >>>> >>>> I ran into weird issue multiple times recently: all RPCs from one >>>> client failed with DEADLINE_EXCEEDED. From server log, it looks like these >>>> failed requests didn't arrive at server at all. Other GRPC clients worked >>>> fine during that time. The issue was fixed by restarting client >>>> application. >>>> Keep-alive feature is not used in client. From my understanding, client >>>> channel should manage the underlying connection properly even keep-alive is >>>> off. >>>> This issue happens occasionally and I haven't find a reliable way to >>>> reproduce. >>>> >>> -- > You received this message because you are subscribed to a topic in the > Google Groups "grpc.io" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/grpc-io/0t9n1-2GTms/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/grpc-io. > To view this discussion on the web visit > https://groups.google.com/d/msgid/grpc-io/178d7d15-fc27-4c0a-b53d-39f48fc6f9f9%40googlegroups.com > <https://groups.google.com/d/msgid/grpc-io/178d7d15-fc27-4c0a-b53d-39f48fc6f9f9%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAOFje1u15wCKKAK8yUd0aDWV9mVse%2B6o5zfgWNjubjYvpA6x6w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
