It looks like you are re-creating channels when the backends change. That is unfortunate; I would encourage you to instead create a NameResolver that will provide updated server addresses when they change. That will prevent needing to shut down perfectly good connections and avoids you having to deal with many races when swapping out the Channel.
Are you sure you are using RoundRobin? The last channel would likely only send RPCs to 9095 if it was using the default PickFirst. On Fri, Jan 11, 2019 at 2:43 PM <[email protected]> wrote: > Hi Kun, > > please see below the logs from the gRPC client, so server1 > (localhost:9095) is running first, then the client start making requests, > afterward, I started up server2 (localhost:9096), then I see the following > logs, and the request is not sent to server2. > > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Created with target localhost:9095 > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Created with target localhost:9095 > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Exiting idle mode > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Exiting idle mode > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > resolved address: [[addrs=[localhost/127.0.0.1:9095], attrs={}], > [addrs=[localhost/0:0:0:0:0:0:0:1:9095], attrs={}]], config={} > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > resolved address: [[addrs=[localhost/127.0.0.1:9095], attrs={}], > [addrs=[localhost/0:0:0:0:0:0:0:1:9095], attrs={}]], config={} > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > io.grpc.internal.InternalSubchannel-14 created for [[addrs=[localhost/ > 127.0.0.1:9095], attrs={}], [addrs=[localhost/0:0:0:0:0:0:0:1:9095], > attrs={}]] > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > io.grpc.internal.InternalSubchannel-14 created for [[addrs=[localhost/ > 127.0.0.1:9095], attrs={}], [addrs=[localhost/0:0:0:0:0:0:0:1:9095], > attrs={}]] > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > shutdownNow() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > shutdownNow() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > shutdown() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > shutdown() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Shutting down > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Shutting down > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Created with target localhost:9096 > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Created with target localhost:9096 > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Terminated > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-12] > Terminated > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Exiting idle mode > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Exiting idle mode > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > resolved address: [[addrs=[localhost/127.0.0.1:9096], attrs={}], > [addrs=[localhost/0:0:0:0:0:0:0:1:9096], attrs={}]], config={} > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > resolved address: [[addrs=[localhost/127.0.0.1:9096], attrs={}], > [addrs=[localhost/0:0:0:0:0:0:0:1:9096], attrs={}]], config={} > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > io.grpc.internal.InternalSubchannel-18 created for [[addrs=[localhost/ > 127.0.0.1:9096], attrs={}], [addrs=[localhost/0:0:0:0:0:0:0:1:9096], > attrs={}]] > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > io.grpc.internal.InternalSubchannel-18 created for [[addrs=[localhost/ > 127.0.0.1:9096], attrs={}], [addrs=[localhost/0:0:0:0:0:0:0:1:9096], > attrs={}]] > > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > shutdownNow() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > shutdownNow() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > shutdown() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > shutdown() called > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Shutting down > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Shutting down > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-4] > resolved address: [[addrs=[localhost/127.0.0.1:9095], attrs={}], > [addrs=[localhost/127.0.0.1:9096], attrs={}]], config={} > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-4] > resolved address: [[addrs=[localhost/127.0.0.1:9095], attrs={}], > [addrs=[localhost/127.0.0.1:9096], attrs={}]], config={} > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-4] > io.grpc.internal.InternalSubchannel-20 created for [[addrs=[localhost/ > 127.0.0.1:9096], attrs={}]] > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-4] > io.grpc.internal.InternalSubchannel-20 created for [[addrs=[localhost/ > 127.0.0.1:9096], attrs={}]] > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Terminated > [io.grpc.internal.ManagedChannelImpl][io.grpc.internal.ManagedChannelImpl-16] > Terminated > > > On Thursday, January 10, 2019 at 5:37:55 PM UTC-8, Kun Zhang wrote: >> >> SubChannel getting created for the new server means round-robin is aware >> of this new server and tries to connect. >> The creation log starts with the logId of the Subchannel. Do you see any >> other logs related to that logId? >> My suspicion is that the Subchannel couldn't get connected. >> >> On Thursday, January 10, 2019 at 4:34:58 PM UTC-8, [email protected] >> wrote: >>> >>> Hi Kun, >>> >>> Thanks for your reply, I did see that new SubChannel gets created for >>> the new server, do you mean that so long as the new server's subchannel >>> gets created, it should take effect immediately, meaning the new server >>> should also get the traffic? >>> >>> Thanks a lot! >>> >>> On Thursday, January 10, 2019 at 4:00:28 PM UTC-8, Kun Zhang wrote: >>>> >>>> Can you find logs from InternalSubchannel that mention the new server? >>>> If the new server can not be connected, round-robin won't use it. >>>> >>>> On Wednesday, January 9, 2019 at 10:18:47 AM UTC-8, [email protected] >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> in my java gRPC client, when I create the ManagedChannel, I am passing >>>>> my custom NameResolver, and using RoundRobinLoadBalancer. When my >>>>> NameResolver is notified with a change to the server list (new server >>>>> added), it will call Listener.onAddress and pass the updated the list. >>>>> >>>>> I see from the Log: the onAddress is called from >>>>> NameResolverListenerImpl, (9097 is the new server address added) >>>>> >>>>> resolved address: [[addrs=[localhost/127.0.0.1:9096], attrs={}], >>>>> [addrs=[localhost/127.0.0.1:9097], attrs={}]], config={} >>>>> >>>>> >>>>> however, the traffic is not coming to the new server, did I miss >>>>> anything? >>>>> >>>>> >>>>> Thanks a lot! >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- > You received this message because you are subscribed to the Google Groups " > grpc.io" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/grpc-io. > To view this discussion on the web visit > https://groups.google.com/d/msgid/grpc-io/b869c723-3d66-4305-8dd7-80208fc18066%40googlegroups.com > <https://groups.google.com/d/msgid/grpc-io/b869c723-3d66-4305-8dd7-80208fc18066%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CA%2B4M1oMO0ZjsyFH5qz_9-smM%3DN742J4KM_ker_8cgg%3DWAHKekg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
smime.p7s
Description: S/MIME Cryptographic Signature
