I need to be able to overwrite the resolv.conf per pods. If I tweak the kube-dns configmap, I will have additional nameservers per node, not per pods. The scenario is: I have a kubernetes cluster which is shared between different teams, and each team has his own namespace to deploy pods. Anyway these pods must be able to communicate with different consul clusters which are in different environments. Since they are in different envs, we need to point to different DNS. Since pods get dns resolutions from the nodes, but the nodes are shared, we need a way to overwrite the dns settings per pods. I know this is a bad practice - I get what Tim said above in this thread - but this is just a proof of concept and we'd like to know whether there is a way to do this (the cleaner the better :D) Thanks Simone
Il giorno lunedì 25 settembre 2017 15:15:43 UTC+2, Rodrigo Campos ha scritto: > > Can you explain what do you what to achieve? > > Maybe changing the configmap kube-dns uses it's enough (and is prepared to > be changed easily, now). > > On Monday, September 25, 2017, Simone D'Andreta <simone....@gmail.com > <javascript:>> wrote: > >> Hi Tim, >> I know what you mean and that's definitely a big issue on our side. This >> is for us more a proof of concept to understand if we can go this way. >> Since I am not a big expert with Kubernetes, I wanted to know if there are >> solutions I haven't considered that might solve my problem. >> That said, I thank you a lot for your explanations here. I found >> something that can help me, though it sets dns names at node level - not >> per pod: >> >> https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/ >> If I include the different IPs in the stub domains I definitely should >> get an answer for the service discovery part. Not ideal, but I think better >> than mounting and overriding the resolv.conf. >> Cheers >> Simone >> >> Il giorno venerdì 22 settembre 2017 19:10:01 UTC+2, Tim Hockin ha scritto: >>> >>> you're trying to mount a directory (emptyDir) onto a file >>> (/etc/resolv.conf). Without seeing the error that is a wild guess. I >>> can't stop you from doing this, but I strongly encourage you to >>> re-read and internalize what I wrote about multiple nameserver >>> records. >>> >>> On Fri, Sep 22, 2017 at 6:19 AM, Simone D'Andreta >>> <simone....@gmail.com> wrote: >>> > I am trying to mount that volume but my container won't start. I guess >>> I am >>> > doing something wrong. This is the yaml >>> > apiVersion: extensions/v1beta1 >>> > kind: Deployment >>> > metadata: >>> > name: {{ template "fullname" . }} >>> > namespace: code >>> > labels: >>> > chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}" >>> > annotations: >>> > commitSHA: {{ .Chart.AppVersion }} >>> > isNotifiable : "true" >>> > spec: >>> > replicas: {{ .Values.replicaCount }} >>> > template: >>> > metadata: >>> > labels: >>> > app: {{ template "fullname" . }} >>> > >>> > spec: >>> > >>> > containers: >>> > - name: {{ .Chart.Name }} >>> > image: "{{ .Values.image.repository }}:{{ .Values.image.tag >>> }}" >>> > imagePullPolicy: {{ .Values.image.pullPolicy }} >>> > volumeMounts: >>> > - name: new-resolv >>> > mountPath: /etc/resolv.conf >>> > command: ["/bin/sh"] >>> > args: ["-c", "echo nameserver 10.24.26.102 > >>> /etc/resolv.conf"] >>> > >>> > >>> > ports: >>> > - name: frontend >>> > containerPort: {{ .Values.port}} >>> > readinessProbe: >>> > httpGet: >>> > path: {{ .Values.lifecheck }} >>> > port: {{ .Values.port}} >>> > >>> > volumes: >>> > - name: new-resolv >>> > emptyDir: {} >>> > >>> > I am using helm to deploy so the variables get expanded via >>> Values.yaml or >>> > other template files. >>> > I think I am just not able to mount that volume properly.. >>> > Thanks >>> > >>> > >>> > Il giorno giovedì 21 settembre 2017 19:21:39 UTC+2, Tim Hockin ha >>> scritto: >>> >> >>> >> You'd have to craft a new file and mount it onto your resolv.conf, >>> >> which makes it harder to "just add another line" because you don't >>> >> have the base. >>> >> >>> >> But more than that, what you're asking for is really non-standard >>> >> behavior. You can't safely add a nameserver record to resolv.conf >>> >> that produces different results. The behavior of DNS resolvers >>> varies >>> >> widely, and this will cause you pain eventually (kubernetes used to >>> do >>> >> this, it was bad). >>> >> >>> >> Consider this - some resolvers ask all DNS servers in parallel, and >>> >> take the first response. If one resolver can answer a query and >>> >> another can't (NXDOMAIN), your app will sometimes get an address and >>> >> will sometimes fail. This actually happens. >>> >> >>> >> >>> >> On Thu, Sep 21, 2017 at 6:33 AM, Simone D'Andreta >>> >> <simone....@gmail.com> wrote: >>> >> > Bad news, my idea doesn't work. Could you explain me more about the >>> >> > volumeMount? I know how to mount but I don't know how I can >>> effectively >>> >> > add >>> >> > a nameserver on that file. >>> >> > >>> >> > Thanks >>> >> > Simone >>> >> > >>> >> > >>> >> > Il giorno giovedì 21 settembre 2017 10:26:48 UTC+2, Simone >>> D'Andreta ha >>> >> > scritto: >>> >> >> >>> >> >> Hi Tim, >>> >> >> thanks for your answer. I don't actually need to override all the >>> >> >> settings >>> >> >> in the resolv.conf, I just need to add another nameserver at the >>> top of >>> >> >> the >>> >> >> file. How about if I run a command in the pod such as: >>> >> >> >>> >> >> command: ['/bin/sh', '-c', 'echo 'nameserver x.y.z.w' | cat - >>> >> >> /etc/resolv.conf > temp && mv temp /etc/resolv.conf'] >>> >> >> would it work? >>> >> >> Thanks >>> >> >> >>> >> >> Il giorno mercoledì 20 settembre 2017 17:29:16 UTC+2, Tim Hockin >>> ha >>> >> >> scritto: >>> >> >>> >>> >> >>> There's no supported way to do that, in part because it would >>> give up >>> >> >>> all of the Service names that kubernetes provides. I don't know >>> what >>> >> >>> would happen if you tried to volumeMount a file over >>> /etc/resolv.conf >>> >> >>> - might be worth a shot. >>> >> >>> >>> >> >>> On Wed, Sep 20, 2017 at 3:15 AM, Simone D'Andreta >>> >> >>> <simone....@gmail.com> wrote: >>> >> >>> > Hello, >>> >> >>> > I wanted to know if there is a way to override DNS settings for >>> pods >>> >> >>> > (not >>> >> >>> > per cluster). I tried to use HostAliases but it only creates an >>> A >>> >> >>> > record for >>> >> >>> > that entry. I basically need a NS record cause I need to point >>> to >>> >> >>> > different >>> >> >>> > Consul clusters and then Consul must be able to do service >>> >> >>> > discovery. >>> >> >>> > So I >>> >> >>> > was thinking to change the resolv.conf for the pod to use >>> Consul for >>> >> >>> > specific requests. Any idea? >>> >> >>> > Thank you, >>> >> >>> > Simone >>> >> >>> > >>> >> >>> > -- >>> >> >>> > You received this message because you are subscribed to the >>> Google >>> >> >>> > Groups >>> >> >>> > "Kubernetes user discussion and Q&A" group. >>> >> >>> > To unsubscribe from this group and stop receiving emails from >>> it, >>> >> >>> > send >>> >> >>> > an >>> >> >>> > email to kubernetes-use...@googlegroups.com. >>> >> >>> > To post to this group, send email to >>> kubernet...@googlegroups.com. >>> >> >>> > Visit this group at >>> >> >>> > https://groups.google.com/group/kubernetes-users. >>> >> >>> > For more options, visit https://groups.google.com/d/optout. >>> >> > >>> >> > -- >>> >> > You received this message because you are subscribed to the Google >>> >> > Groups >>> >> > "Kubernetes user discussion and Q&A" group. >>> >> > To unsubscribe from this group and stop receiving emails from it, >>> send >>> >> > an >>> >> > email to kubernetes-use...@googlegroups.com. >>> >> > To post to this group, send email to kubernet...@googlegroups.com. >>> >> > Visit this group at >>> https://groups.google.com/group/kubernetes-users. >>> >> > For more options, visit https://groups.google.com/d/optout. >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "Kubernetes user discussion and Q&A" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> an >>> > email to kubernetes-use...@googlegroups.com. >>> > To post to this group, send email to kubernet...@googlegroups.com. >>> > Visit this group at https://groups.google.com/group/kubernetes-users. >>> > For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Kubernetes user discussion and Q&A" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to kubernetes-users+unsubscr...@googlegroups.com. >> To post to this group, send email to kubernetes-users@googlegroups.com. >> Visit this group at https://groups.google.com/group/kubernetes-users. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
