tustvold commented on issue #2581: URL: https://github.com/apache/arrow-rs/issues/2581#issuecomment-1227621533
> Seems like with 1 service and 2 pods there is still only 1 IP, so it must be executors proxying to each other? That's the ClusterIP, which is a type of [Service](https://kubernetes.io/docs/concepts/services-networking/service/). Its a transparent L4 proxy, when you make a TCP connection to that IP it will transparently DNAT it to one of the pod IPs. If you run `kubectl get pods -o wide` you will get the pod IPs > if there is data locality to the executor that did the processing I have to confess to not being intimately familiar with Ballista, but the following is what I had presumed: * The executors are interchangeable with no data locality. They have some mechanism to distribute data between them, potentially via object storage or some communication protocol * The user makes a single DoGet request and gets a stream of FlightData responses back, as a result even if doing L7 load balancing, there is only HTTP request to route per query and so you don't need to worry about session stickiness or anything -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
