Hi Vitaly, Please see this post <https://groups.google.com/g/grpc-io/c/0yGihF-EFQo/m/1XsLZpKsAgAJ> if you are planning to use gRPCLB. gRPC has moved away from gRPCLB protocol. Instead, gRPC is adopting xDS protocol. A number of xDS features <https://github.com/grpc/grpc/blob/master/doc/grpc_xds_features.md>, including round robin LB, are already supported in gRPC. This project <https://github.com/asishrs/proxyless-grpc-lb> might be useful to you but I think it is blocked on this issue <https://github.com/envoyproxy/go-control-plane/issues/349>. This <https://github.com/salrashid123/grpc_xds> project might be useful too.
On Friday, February 19, 2021 at 3:47:22 PM UTC-8 vitaly....@gmail.com wrote: > Thanks Srini, > > I haven't tested option 2 yet, I would expect though that since client is > unaware of what is happening we should see some request failures/latency > spikes until new connection is established. That's why I would consider it > mostly for disaster prevention rather than for general connection balancing. > I'm actually now more interested in exploring option 4 as it looks like we > can achieve safe setup if we keep proxy in front of servers and expose a > separate proxy port for each server. > Can someone recommend a good opensource grpclb implementation? I've found > bsm/grpclb <https://github.com/bsm/grpclb> which looks reasonable but > wasn't sure if there is anything else available. > > On Friday, February 19, 2021 at 12:50:17 PM UTC-8 Srini Polavarapu wrote: > >> Hi, >> >> Option 3 is ideal but since you don't have that as well as option 4 >> available, option 2 is worth exploring. Are the concerns with options 2 >> based on some experiments you have done or is it just a hunch? This >> comment >> <https://github.com/grpc/grpc/issues/12295#issuecomment-650364080> has >> some relevant info that you could use. >> >> On Thursday, February 18, 2021 at 7:06:37 PM UTC-8 vitaly....@gmail.com >> wrote: >> >>> >>> Hey folks, >>> >>> I'm trying to solve a problem of even load (or at least connection) >>> distribution between grpc clients and our backend servers. >>> >>> First of all let me describe our setup: >>> We are using network load balancing (L4) in front of our grpc servers. >>> Clients will see one endpoint (LB) and connect to it. This means that >>> standard client-side load balancing features like round robing wouldn't >>> work as there will only be one sub-channel for client-server communication. >>> >>> One issue with this approach can be demonstrated by the following >>> example: >>> Let's say we have 2 servers running and 20 clients connect to them. At >>> the beginning, since we go through the network load balancer, connections >>> will be distributed evenly (or close to that), so we'll roughly have 50% of >>> connections to each server. Now let's assume these servers reboot one after >>> another, like in a deployment. What would happen is that server that comes >>> up first would get all 20 worker connections and server that comes up later >>> would have zero. This situation won't change unless client or server would >>> drop a connection periodically or more clients request connections. >>> >>> I've considered a few options for solving this: >>> 1. Connection management on the client side - do something to reset the >>> channel (like [enterIdle]( >>> https://grpc.github.io/grpc-java/javadoc/io/grpc/ManagedChannel.html#enterIdle) >>> >>> in grpc-java). Downside - it seems that this feature has been developed for >>> android and I can't find similar functionality in grpc-go. >>> 2. Connection management on the server side - drop connections >>> periodically on the server. Downside - this approach looks less graceful >>> than the client side one and may impact request latency and result in >>> request failures on the client side. >>> 3. Use request based grpc-aware L7 LB, this way client would connect to >>> the LB, which would fan out requests to the servers. Downside - I've been >>> told by our infra guys that it is hard to implement in our setup due to the >>> way we use TLS and manage certificates. >>> 4. Expose our servers outside and use grpc-lb or client side load >>> balancing. Downside - it seems less secure and would make it harder to >>> protect against DDoS attacks if we go this route. I think this downside >>> makes this approach unviable. >>> >>> My bias is towards going with option 3 and doing request based load >>> balancing because it allows much more fine grained control based on load, >>> but since our infra can not support it at the moment, I might be forced to >>> use option 1 or 2 in the short to mid term. Option 2 I like the least, as >>> it might result in latency spikes and errors on the client side. >>> >>> My questions are: >>> 1. Which approach is generally preferable? >>> 2. Are there other options to consider? >>> 3. Is it possible to influence grpc channel state in grpc-go, which >>> would trigger resolver and balancer to establish a new connection similar >>> to what enterIdle does in java? From what I see in the [clientconn.go]( >>> https://github.com/grpc/grpc-go/blob/master/clientconn.go) there is no >>> option to change the channel state to idle or trigger a reconnect in some >>> other way. >>> 4. Is there a way to implement server side connection management cleanly >>> without impacting client-side severely? >>> >>> Here are links that I find useful for some context: >>> grpc/load-balancing.md at master · grpc/grpc >>> <https://www.google.com/url?q=https://github.com/grpc/grpc/blob/master/doc/load-balancing.md&sa=D&source=calendar&ust=1614134668829000&usg=AOvVaw21tfy7_lvaEmuQ_VRla1tY> >>> proposal/A9-server-side-conn-mgt.md at master · grpc/proposal >>> <https://www.google.com/url?q=https://github.com/grpc/proposal/blob/master/A9-server-side-conn-mgt.md&sa=D&source=calendar&ust=1614134668829000&usg=AOvVaw3CEasUxdbyoBhDZoX9oYB3> >>> >>> proposal/A8-client-side-keepalive.md at master · grpc/proposal >>> <https://www.google.com/url?q=https://github.com/grpc/proposal/blob/master/A8-client-side-keepalive.md&sa=D&source=calendar&ust=1614134668829000&usg=AOvVaw2EuL2EScC-WhnwJStxikI4> >>> >>> grpc/keepalive.md at master · grpc/grpc >>> <https://www.google.com/url?q=https://github.com/grpc/grpc/blob/master/doc/keepalive.md&sa=D&source=calendar&ust=1614134668829000&usg=AOvVaw1T5WVe-QM5uc6UzblVzhKp> >>> >>> >>> Sorry for the long read, >>> Vitaly >>> >> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/58dc4e02-6d2d-4c5d-a431-947fbf34e711n%40googlegroups.com.
