I was able to get drill up and running inside a k8s cluster but I didn't connect to it from outside the cluster, so the DNS names were always resolvable by the client(s).
I had to run it as a statefulset to ensure the DNS names are stable, otherwise the drillbits couldn't talk to each other, either. On 3/24/2020 6:37:44 AM, Jaimes, Rafael - 0993 - MITLL <rafael.jai...@ll.mit.edu> wrote: I’m seeing a problem with scaling the number of pod instances in the replication controller because they aren’t reporting their hostnames properly. This was a common problem that got fixed in scalable architectures like ZooKeeper and Kafka (see reference at bottom I think this was related). In Drill’s case, ZooKeeper is able to see all of the drillbits, however, the hostnames are only locally addressable within the cluster, so as soon as you perform a query it fails since the client can’t find the drillbit that it got assigned, its hostname isn’t externally addressable. Kafka fixes this by allowing an override for advertised names. Has anyone gotten Drill to scale in a K8s cluster? https://issues.apache.org/jira/browse/KAFKA-1070
