Re: [grpc-io] Client Loadbalancing with Kubernetes SVC

[julien . senon]

Thanks a lot for your explanation. I will have a look on 2 solution 
provided.



On Tuesday, September 18, 2018 at 5:40:45 PM UTC+2, Josh Humphries wrote:
>
> Unless things have changed recently, the default kubedns result for a 
> standard k8s service will have a single IP: the virtual IP of the service. 
> This in turn causes the gRPC client to configure just a single socket 
> connection and route all requests on the one socket. Since kubeproxy load 
> balancing through that virtual IP is layer 4, this can result in very poor 
> load balancing, especially if you have unbalanced clients (e.g. a small 
> number of clients that constitute the majority of RPC traffic).
>
> If you use "headless" services in k8s on the other hand, the DNS query 
> should return multiple pod IPs. This causes the gRPC client to in turn 
> maintain multiple connections, one to each destination pod. However, I am 
> not sure how responsive this will be to topology changes (as pod instances 
> are auto-scaled, or killed and rescheduled, they will move around and 
> change IP address). It would require disabling DNS caching and making sure 
> the service host name is resolved/polled regularly.
>
> Another solution is to use a custom gRPC resolver that talks to the k8s 
> API to watch the service topology and convey the results to the gRPC 
> client. For Go, this is implemented in an open-source package: 
> github.com/sercand/kuberesolver
>
> (Most of my experience is with Go. So your mileage may vary if using a 
> runtime other than Go. But I think the various implementations largely 
> behave quite similarly.)
> 
> *Josh Humphries*
> jh...@bluegosling.com 
>
>
> On Tue, Sep 18, 2018 at 7:10 AM > wrote:
>
>> Hello,
>>
>> Does it make sense to have client loadbalancer with gRPC when we are 
>> using gRPC server in a Kubernetes cluster? 
>> Because client will dial a service DNS and will always retrieve IP of 
>> service and not IP of Pods behind it.
>>
>> NB: If already seen this blog entry 
>> https://github.com/grpc/grpc/blob/master/doc/load-balancing.md
>>
>> Thanks
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "grpc.io" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to grpc-io+u...@googlegroups.com .
>> To post to this group, send email to grp...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/grpc-io.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/grpc-io/63490220-692a-49fd-84bd-62ef13b19e09%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/c1dd0c1a-fc49-42a1-aec9-3fe11903f2f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [grpc-io] Client Loadbalancing with Kubernetes SVC

[Josh Humphries]

Unless things have changed recently, the default kubedns result for a
standard k8s service will have a single IP: the virtual IP of the service.
This in turn causes the gRPC client to configure just a single socket
connection and route all requests on the one socket. Since kubeproxy load
balancing through that virtual IP is layer 4, this can result in very poor
load balancing, especially if you have unbalanced clients (e.g. a small
number of clients that constitute the majority of RPC traffic).

If you use "headless" services in k8s on the other hand, the DNS query
should return multiple pod IPs. This causes the gRPC client to in turn
maintain multiple connections, one to each destination pod. However, I am
not sure how responsive this will be to topology changes (as pod instances
are auto-scaled, or killed and rescheduled, they will move around and
change IP address). It would require disabling DNS caching and making sure
the service host name is resolved/polled regularly.

Another solution is to use a custom gRPC resolver that talks to the k8s API
to watch the service topology and convey the results to the gRPC client.
For Go, this is implemented in an open-source package:
github.com/sercand/kuberesolver

(Most of my experience is with Go. So your mileage may vary if using a
runtime other than Go. But I think the various implementations largely
behave quite similarly.)

*Josh Humphries*
jh...@bluegosling.com


On Tue, Sep 18, 2018 at 7:10 AM  wrote:

> Hello,
>
> Does it make sense to have client loadbalancer with gRPC when we are using
> gRPC server in a Kubernetes cluster?
> Because client will dial a service DNS and will always retrieve IP of
> service and not IP of Pods behind it.
>
> NB: If already seen this blog entry
> https://github.com/grpc/grpc/blob/master/doc/load-balancing.md
>
> Thanks
>
>
> --
> You received this message because you are subscribed to the Google Groups "
> grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to grpc-io+unsubscr...@googlegroups.com.
> To post to this group, send email to grpc-io@googlegroups.com.
> Visit this group at https://groups.google.com/group/grpc-io.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/grpc-io/63490220-692a-49fd-84bd-62ef13b19e09%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/CAO78j%2BLoC4Pdy50tuhdfZ_qO%2BoEB3LWX%3Dg4DE_-_ODb%2BR_M5fA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[grpc-io] Client Loadbalancing with Kubernetes SVC

[julien . senon]

Hello,

Does it make sense to have client loadbalancer with gRPC when we are using 
gRPC server in a Kubernetes cluster? 
Because client will dial a service DNS and will always retrieve IP of 
service and not IP of Pods behind it.

NB: If already seen this blog 
entry https://github.com/grpc/grpc/blob/master/doc/load-balancing.md

Thanks


-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/63490220-692a-49fd-84bd-62ef13b19e09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.