Sure, I understand that part. But what I didn't understand was why I continued to get DEADLINE_EXCEEDED errors after the UNAVAILABLE errors? I would have though that since the old instances were terminated, I wouldn't be able to even maintain a connection to the old instances, so theoretically I would keep trying to reconnect and eventually open up a connection to the new instance.
On Wednesday, January 23, 2019 at 11:29:35 AM UTC-8, Srini Polavarapu wrote: > > That's right. When a subchannel goes down, the channel re-resolves DNS in > round-robin LB. Depending on your DNS TTL, the OS may still be returning > cached DNS entry which might still contain the server IP that went down. > Regardless, depending on DNS updates will not fully meet your LB > requirements. This is because gRPC client does not periodically re-resolve > DNS. This means when new backends are added, gRPC client will not know > about those. See this <https://github.com/grpc/grpc/issues/14492> and this > <https://groups.google.com/forum/#!topic/grpc-io/2a5zsRNBXLI>. > > On Sunday, January 20, 2019 at 9:27:49 PM UTC-8, [email protected] > wrote: >> >> I'm trying to set up a python gRPC simple client and server, where the >> client uses round robin load balancing against a single DNS record, where I >> have multiple servers (instances) in the DNS record. >> >> In the beginning, I'm able to connect and issue queries fine, but when I >> try a re-deploy of my servers, I get some weird behavior that I was hoping >> would be resolved automatically by the client library. In my re-deploy, I >> first bring up new servers, set the DNS record to the IPs of the new >> servers, and then destroy the old servers. Everything seems to work until I >> destroy the old servers, at which point I get a couple of >> UNAVAILABLE_ERRORs followed by DEADLINE_EXCEEDED until I kill the client. >> From what I understand, when the sub-channels go down (i.e. the server >> instances are killed), the channel should re-resolve the DNS record and >> attempt to re-connect to the new instances. Am I interpreting this >> incorrectly? Is there some channel and/or server option I need to set in >> order for this to work? >> >> Sample client below: >> >> channel = grpc.insecure_channel("localhost:10000", >> options=(("grpc.lb_policy_name", "round_robin"),)) >> fut = grpc.channel_ready_future(channel) >> fut.result() >> print("done waiting") >> stub = test_pb2_grpc.TestStub(channel) >> while True: >> try: >> print(stub.Test(request, timeout=5)) >> except grpc.RpcError as e: >> print("{} {}".format(time.time(), e)) >> time.sleep(0.5) >> >> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/62460209-cfbc-48b5-840b-c72351c216b0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
