The plans you have all seem related. 1. It true that "round_robin" requires your dns to return a list of backend address. 2. WRR inherits "round_robin". It also requires your DNS server to return a list of backend address to round robin with. So not feasible in your case. (FWIW WRR would require backend servers to return metrics to the clients to determine the weights dynamically, not static weight distribution as you configure in DNS.)
On solution I can think of is to 1. Write your own NameResolverProvider <https://grpc.github.io/grpc-java/javadoc/io/grpc/NameResolverProvider.html>. It will periodically resolve dns and update the address to the load balancer so that new backend will kick in. The load balancer will kill the connection with the previous address. It might not be horrible as your thought because you can delegate to grpc DnsNameResolverProvider with the same , just with the same scheme <https://grpc.github.io/grpc-java/javadoc/io/grpc/NameResolverProvider.html#getScheme()> but higher priority <https://grpc.github.io/grpc-java/javadoc/io/grpc/NameResolverProvider.html#priority()> . 2. Write your own LoadBalancerProvider that periodically do refreshNameResolution <https://grpc.github.io/grpc-java/javadoc/io/grpc/LoadBalancer.Helper.html#refreshNameResolution()> For guide of writing your own custom name resolver or loadbalancer, see guide: https://grpc.io/docs/guides/custom-load-balancing/ https://grpc.io/docs/guides/custom-name-resolution/ On Monday, November 20, 2023 at 8:22:06 PM UTC-8 Ryan Li wrote: > Hello grpc community, > > We are trying to figure out load balancing across weighted endpoints > behind a primary domain from client-side. We use grpc-java 1.3 > > The context: > > - We have a single long running client which establishes a GRPC > channel to a cluster of GRPC Services behind a single L7 Application-level > Load Balancer which load balances round-robin style > - Our client is high traffic streaming calls to the services behind > each load balancer > - A single load balancer and cluster can be considered a single single > unit. We want to spin up more units, and map each load balancer behind a > primary domain name, which the client would then call. > - We have configured a weighted routing policy for this primary DNS, > with a weight assigned to each load balancer > > Example: > > 1. We have Loadbalancer A, Loadbalancer B, and DNS P > 2. We associate A and B behind P, each with a weight of .5 and .5 > respectively > 3. If we query DNS P 10,000 times, then it resolves to LB A ~5000 > times, and LB B ~5000 times > > The issue comes up though when actually sending requests through the > client, which establishes a long running grpc channel with the primary DNS. > > Found that the behavior is not what we expected: > > - The amount of traffic actually routing to each cluster is not > proportional to the weights we assigned to them. > - The client establishes a connection to a single cluster, then it > doesn't change and all requests are sent to that single cluster rather > than > being balanced across multiple clusters accordingly > > > This seems like it already has a solution if we configure MAX_CONNECTION_AGE > on the server side, but for certain reasons we cannot modify this property > from the server side, and so want to investigate if there are possible > solutions solely from a client side. > > - > https://github.com/grpc/proposal/blob/master/A9-server-side-conn-mgt.md > > > Some things we have found or are planning to look into: > > 1. defaultLoadBalancingPolicy("round_robin") > - > > https://www.javadoc.io/doc/io.grpc/grpc-all/1.30.1/io/grpc/ForwardingChannelBuilder.html#defaultLoadBalancingPolicy-java.lang.String- > - This initially seemed promising, but unsure of its feasibility > since our DNS will only resolve to a single LoadBalancer as described > above, rather than provide the entire address list of all units > 2. Weighted round robin > - https://github.com/grpc/proposal/pull/343/files > - Our weighted routing policy is configured externally to the grpc > service, server or client side, so this doesn't seem exactly usable for > our > scenario, although its naming sounds very similar to what we are trying > to > do. > - This seems like a newer change, and it is infeasible for us to > upgrade to a newer version of grpc-java for now > 3. Recreating the channel on some interval > - This can be ruled out since it goes against grpc best practices > and creating channels so often is expensive > > Any ideas or suggestions would be greatly appreciated, thanks > > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/389a4e4e-76a2-4cfa-a35a-b891e73e8785n%40googlegroups.com.
