I can't try today unfortunately but that's definitely the issue, I will have to test it to an environment where I can reach the consul subnets. As per what I am trying to achieve (and that's perfectly fine if we cannot find a viable solution, it is a proof of concept after all) I need a way to have different nameservers per namespace or pod. It was suggested before that I could do it via volumeMounts, by creating a new resolv.conf and copying it over the resolv.conf of the pod. I tried but I couldn't get it working, but I believe I am doing something wrong when mounting.. do you have any example I can use and adapt to my needs? Thanks Simone
Il giorno mercoledì 27 settembre 2017 15:14:59 UTC+2, Rodrigo Campos ha scritto: > > Can you ping the IP? There might not be any route to that subnet? Have you > tried/checked that? > > And as far as I know, there is no way. Im not 100% sure what problem you > are trying to solve, exactly, though. You can have a service type external, > configure kube dns stub domains, but having **the same stub domains** > resolve to different NS servers according to the kubernetes namespace the > query is executed doesn't seem supported. Not sure it's something useful > besides your setup, either :-/ > > You can probably patch kubedns or something, but not sure it's worth the > effort. > > On Wednesday, September 27, 2017, Simone D'Andreta <simone....@gmail.com > <javascript:>> wrote: > >> Ah yeah that's a typo.. it's setup as consul.service.domain.io but I >> don't know why I cannot ping it - are the service and the endpoint properly >> declared? >> As per the link you provided - I used the stubDomain and gave the >> domain.io name a list of Consul IPs - all good, but this will overwrite >> the kube-dns configmap in the kube-system namespace so the new dns >> resolution will be per node. I need that per namespace or pod. >> Is there anyway I can do that? >> Hope it's clear and thanks a lot for your help. >> Simone >> >> Il giorno martedì 26 settembre 2017 22:30:12 UTC+2, Rodrigo Campos ha >> scritto: >>> >>> What I tried to say is using this: >>> http://blog.kubernetes.io/2017/04/configuring-private-dns-zones-upstream-nameservers-kubernetes.html?m=1 >>> >>> in kube-dns configuration. Not sure how your consul name is and, with >>> all you said in the previous mail, a service type external will help. >>> >>> As not even able to ping, not sure what you mean. The service has >>> nothing to do with the domain you want to ping, right? Or is that a typo? >>> >>> On Tuesday, September 26, 2017, Simone D'Andreta <simone....@gmail.com> >>> wrote: >>> >>>> It creates records such as myservice.service.domain.io, so your >>>> application must be able to contact a dns forwarder which has the zone for >>>> that domain.io. >>>> What I am using at the moment are stubdomains to include the domain.io >>>> but it's at cluster level. What I need to do is to solve different consul >>>> names per namespace or pod. >>>> I tried to use a service definition as follows: >>>> >>>> kind: Service >>>> apiVersion: v1 >>>> metadata: >>>> name: consul-resolution >>>> namespace: default >>>> spec: >>>> type: ExternalName >>>> externalName: consul.service.domain.io >>>> >>>> And an endpoint: >>>> kind: Endpoints >>>> apiVersion: v1 >>>> metadata: >>>> name: consulresolution >>>> subsets: >>>> - addresses: >>>> - ip: 10.24.7.26 >>>> ports: >>>> - port: 8300 >>>> >>>> But I am not even able to ping consul.service.cnqr.io.. any idea? >>>> Thanks >>>> >>>> Il giorno martedì 26 settembre 2017 16:03:16 UTC+2, Rodrigo Campos ha >>>> scritto: >>>>> >>>>> Sorry, never used consul and I don't follow what you said. >>>>> >>>>> Does it create records like <service name>.k8s-service that won't work >>>>> with just a k8s external type service? >>>>> >>>>> Then it might be possible to say to kube dns to use some upstream to >>>>> some domains, probably? I've not played with it, as I don't need it. But >>>>> I'd guess something like that might be possible to do. And use different >>>>> domains on apps that need to query different consul clusters? >>>>> >>>>> On Tuesday, September 26, 2017, Simone D'Andreta <simone....@gmail.com> >>>>> wrote: >>>>> >>>>>> I am afraid it won't work. I will be able to solve >>>>>> consul.service.domain but I won't be able to solve external services >>>>>> registered within consul, such as mydatabase.service.domain because I >>>>>> need >>>>>> a NS record.. unless I can use that CNAME as a NS record, but I don't >>>>>> think >>>>>> it's the right approach.. >>>>>> >>>>>> >>>>>> Il giorno martedì 26 settembre 2017 10:21:13 UTC+2, Simone D'Andreta >>>>>> ha scritto: >>>>>>> >>>>>>> Hi Rodrigo, >>>>>>> ideally we would need this per pod, but I can give it a try with >>>>>>> creating a service per namespace. >>>>>>> Thanks for the hint, I will let you know how it goes. >>>>>>> Simone >>>>>>> >>>>>>> Il giorno lunedì 25 settembre 2017 18:34:07 UTC+2, Rodrigo Campos ha >>>>>>> scritto: >>>>>>>> >>>>>>>> Sorry, I must be missing something. But if you want to resolve to >>>>>>>> different consul clusters in different kubernetes namespaces, can't >>>>>>>> you >>>>>>>> just use a service type external on each? >>>>>>>> >>>>>>>> You create a service, named as you want them to contact, and is per >>>>>>>> ns and returns a CNAME. >>>>>>>> >>>>>>>> Wouldn't that do the trick? >>>>>>>> >>>>>>>> On Monday, September 25, 2017, Simone D'Andreta < >>>>>>>> simone....@gmail.com> wrote: >>>>>>>> >>>>>>>>> I need to be able to overwrite the resolv.conf per pods. If I >>>>>>>>> tweak the kube-dns configmap, I will have additional nameservers per >>>>>>>>> node, >>>>>>>>> not per pods. >>>>>>>>> The scenario is: I have a kubernetes cluster which is shared >>>>>>>>> between different teams, and each team has his own namespace to >>>>>>>>> deploy >>>>>>>>> pods. Anyway these pods must be able to communicate with different >>>>>>>>> consul >>>>>>>>> clusters which are in different environments. Since they are in >>>>>>>>> different >>>>>>>>> envs, we need to point to different DNS. Since pods get dns >>>>>>>>> resolutions >>>>>>>>> from the nodes, but the nodes are shared, we need a way to overwrite >>>>>>>>> the >>>>>>>>> dns settings per pods. >>>>>>>>> I know this is a bad practice - I get what Tim said above in this >>>>>>>>> thread - but this is just a proof of concept and we'd like to know >>>>>>>>> whether >>>>>>>>> there is a way to do this (the cleaner the better :D) >>>>>>>>> Thanks >>>>>>>>> Simone >>>>>>>>> >>>>>>>>> Il giorno lunedì 25 settembre 2017 15:15:43 UTC+2, Rodrigo Campos >>>>>>>>> ha scritto: >>>>>>>>>> >>>>>>>>>> Can you explain what do you what to achieve? >>>>>>>>>> >>>>>>>>>> Maybe changing the configmap kube-dns uses it's enough (and is >>>>>>>>>> prepared to be changed easily, now). >>>>>>>>>> >>>>>>>>>> On Monday, September 25, 2017, Simone D'Andreta < >>>>>>>>>> simone....@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Tim, >>>>>>>>>>> I know what you mean and that's definitely a big issue on our >>>>>>>>>>> side. This is for us more a proof of concept to understand if we >>>>>>>>>>> can go >>>>>>>>>>> this way. Since I am not a big expert with Kubernetes, I wanted to >>>>>>>>>>> know if >>>>>>>>>>> there are solutions I haven't considered that might solve my >>>>>>>>>>> problem. >>>>>>>>>>> That said, I thank you a lot for your explanations here. I >>>>>>>>>>> found something that can help me, though it sets dns names at node >>>>>>>>>>> level - >>>>>>>>>>> not per pod: >>>>>>>>>>> >>>>>>>>>>> https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/ >>>>>>>>>>> If I include the different IPs in the stub domains I definitely >>>>>>>>>>> should get an answer for the service discovery part. Not ideal, but >>>>>>>>>>> I think >>>>>>>>>>> better than mounting and overriding the resolv.conf. >>>>>>>>>>> Cheers >>>>>>>>>>> Simone >>>>>>>>>>> >>>>>>>>>>> Il giorno venerdì 22 settembre 2017 19:10:01 UTC+2, Tim Hockin >>>>>>>>>>> ha scritto: >>>>>>>>>>>> >>>>>>>>>>>> you're trying to mount a directory (emptyDir) onto a file >>>>>>>>>>>> (/etc/resolv.conf). Without seeing the error that is a wild >>>>>>>>>>>> guess. I >>>>>>>>>>>> can't stop you from doing this, but I strongly encourage you to >>>>>>>>>>>> re-read and internalize what I wrote about multiple nameserver >>>>>>>>>>>> records. >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Sep 22, 2017 at 6:19 AM, Simone D'Andreta >>>>>>>>>>>> <simone....@gmail.com> wrote: >>>>>>>>>>>> > I am trying to mount that volume but my container won't >>>>>>>>>>>> start. I guess I am >>>>>>>>>>>> > doing something wrong. This is the yaml >>>>>>>>>>>> > apiVersion: extensions/v1beta1 >>>>>>>>>>>> > kind: Deployment >>>>>>>>>>>> > metadata: >>>>>>>>>>>> > name: {{ template "fullname" . }} >>>>>>>>>>>> > namespace: code >>>>>>>>>>>> > labels: >>>>>>>>>>>> > chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" >>>>>>>>>>>> "_" }}" >>>>>>>>>>>> > annotations: >>>>>>>>>>>> > commitSHA: {{ .Chart.AppVersion }} >>>>>>>>>>>> > isNotifiable : "true" >>>>>>>>>>>> > spec: >>>>>>>>>>>> > replicas: {{ .Values.replicaCount }} >>>>>>>>>>>> > template: >>>>>>>>>>>> > metadata: >>>>>>>>>>>> > labels: >>>>>>>>>>>> > app: {{ template "fullname" . }} >>>>>>>>>>>> > >>>>>>>>>>>> > spec: >>>>>>>>>>>> > >>>>>>>>>>>> > containers: >>>>>>>>>>>> > - name: {{ .Chart.Name }} >>>>>>>>>>>> > image: "{{ .Values.image.repository }}:{{ >>>>>>>>>>>> .Values.image.tag }}" >>>>>>>>>>>> > imagePullPolicy: {{ .Values.image.pullPolicy }} >>>>>>>>>>>> > volumeMounts: >>>>>>>>>>>> > - name: new-resolv >>>>>>>>>>>> > mountPath: /etc/resolv.conf >>>>>>>>>>>> > command: ["/bin/sh"] >>>>>>>>>>>> > args: ["-c", "echo nameserver 10.24.26.102 > >>>>>>>>>>>> /etc/resolv.conf"] >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > ports: >>>>>>>>>>>> > - name: frontend >>>>>>>>>>>> > containerPort: {{ .Values.port}} >>>>>>>>>>>> > readinessProbe: >>>>>>>>>>>> > httpGet: >>>>>>>>>>>> > path: {{ .Values.lifecheck }} >>>>>>>>>>>> > port: {{ .Values.port}} >>>>>>>>>>>> > >>>>>>>>>>>> > volumes: >>>>>>>>>>>> > - name: new-resolv >>>>>>>>>>>> > emptyDir: {} >>>>>>>>>>>> > >>>>>>>>>>>> > I am using helm to deploy so the variables get expanded via >>>>>>>>>>>> Values.yaml or >>>>>>>>>>>> > other template files. >>>>>>>>>>>> > I think I am just not able to mount that volume properly.. >>>>>>>>>>>> > Thanks >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > Il giorno giovedì 21 settembre 2017 19:21:39 UTC+2, Tim >>>>>>>>>>>> Hockin ha scritto: >>>>>>>>>>>> >> >>>>>>>>>>>> >> You'd have to craft a new file and mount it onto your >>>>>>>>>>>> resolv.conf, >>>>>>>>>>>> >> which makes it harder to "just add another line" because you >>>>>>>>>>>> don't >>>>>>>>>>>> >> have the base. >>>>>>>>>>>> >> >>>>>>>>>>>> >> But more than that, what you're asking for is really >>>>>>>>>>>> non-standard >>>>>>>>>>>> >> behavior. You can't safely add a nameserver record to >>>>>>>>>>>> resolv.conf >>>>>>>>>>>> >> that produces different results. The behavior of DNS >>>>>>>>>>>> resolvers varies >>>>>>>>>>>> >> widely, and this will cause you pain eventually (kubernetes >>>>>>>>>>>> used to do >>>>>>>>>>>> >> this, it was bad). >>>>>>>>>>>> >> >>>>>>>>>>>> >> Consider this - some resolvers ask all DNS servers in >>>>>>>>>>>> parallel, and >>>>>>>>>>>> >> take the first response. If one resolver can answer a query >>>>>>>>>>>> and >>>>>>>>>>>> >> another can't (NXDOMAIN), your app will sometimes get an >>>>>>>>>>>> address and >>>>>>>>>>>> >> will sometimes fail. This actually happens. >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> On Thu, Sep 21, 2017 at 6:33 AM, Simone D'Andreta >>>>>>>>>>>> >> <simone....@gmail.com> wrote: >>>>>>>>>>>> >> > Bad news, my idea doesn't work. Could you explain me more >>>>>>>>>>>> about the >>>>>>>>>>>> >> > volumeMount? I know how to mount but I don't know how I >>>>>>>>>>>> can effectively >>>>>>>>>>>> >> > add >>>>>>>>>>>> >> > a nameserver on that file. >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > Thanks >>>>>>>>>>>> >> > Simone >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > Il giorno giovedì 21 settembre 2017 10:26:48 UTC+2, Simone >>>>>>>>>>>> D'Andreta ha >>>>>>>>>>>> >> > scritto: >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> Hi Tim, >>>>>>>>>>>> >> >> thanks for your answer. I don't actually need to override >>>>>>>>>>>> all the >>>>>>>>>>>> >> >> settings >>>>>>>>>>>> >> >> in the resolv.conf, I just need to add another nameserver >>>>>>>>>>>> at the top of >>>>>>>>>>>> >> >> the >>>>>>>>>>>> >> >> file. How about if I run a command in the pod such as: >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> command: ['/bin/sh', '-c', 'echo 'nameserver x.y.z.w' | >>>>>>>>>>>> cat - >>>>>>>>>>>> >> >> /etc/resolv.conf > temp && mv temp /etc/resolv.conf'] >>>>>>>>>>>> >> >> would it work? >>>>>>>>>>>> >> >> Thanks >>>>>>>>>>>> >> >> >>>>>>>>>>>> >> >> Il giorno mercoledì 20 settembre 2017 17:29:16 UTC+2, Tim >>>>>>>>>>>> Hockin ha >>>>>>>>>>>> >> >> scritto: >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> There's no supported way to do that, in part because it >>>>>>>>>>>> would give up >>>>>>>>>>>> >> >>> all of the Service names that kubernetes provides. I >>>>>>>>>>>> don't know what >>>>>>>>>>>> >> >>> would happen if you tried to volumeMount a file over >>>>>>>>>>>> /etc/resolv.conf >>>>>>>>>>>> >> >>> - might be worth a shot. >>>>>>>>>>>> >> >>> >>>>>>>>>>>> >> >>> On Wed, Sep 20, 2017 at 3:15 AM, Simone D'Andreta >>>>>>>>>>>> >> >>> <simone....@gmail.com> wrote: >>>>>>>>>>>> >> >>> > Hello, >>>>>>>>>>>> >> >>> > I wanted to know if there is a way to override DNS >>>>>>>>>>>> settings for pods >>>>>>>>>>>> >> >>> > (not >>>>>>>>>>>> >> >>> > per cluster). I tried to use HostAliases but it only >>>>>>>>>>>> creates an A >>>>>>>>>>>> >> >>> > record for >>>>>>>>>>>> >> >>> > that entry. I basically need a NS record cause I need >>>>>>>>>>>> to point to >>>>>>>>>>>> >> >>> > different >>>>>>>>>>>> >> >>> > Consul clusters and then Consul must be able to do >>>>>>>>>>>> service >>>>>>>>>>>> >> >>> > discovery. >>>>>>>>>>>> >> >>> > So I >>>>>>>>>>>> >> >>> > was thinking to change the resolv.conf for the pod to >>>>>>>>>>>> use Consul for >>>>>>>>>>>> >> >>> > specific requests. Any idea? >>>>>>>>>>>> >> >>> > Thank you, >>>>>>>>>>>> >> >>> > Simone >>>>>>>>>>>> >> >>> > >>>>>>>>>>>> >> >>> > -- >>>>>>>>>>>> >> >>> > You received this message because you are subscribed >>>>>>>>>>>> to the Google >>>>>>>>>>>> >> >>> > Groups >>>>>>>>>>>> >> >>> > "Kubernetes user discussion and Q&A" group. >>>>>>>>>>>> >> >>> > To unsubscribe from this group and stop receiving >>>>>>>>>>>> emails from it, >>>>>>>>>>>> >> >>> > send >>>>>>>>>>>> >> >>> > an >>>>>>>>>>>> >> >>> > email to kubernetes-use...@googlegroups.com. >>>>>>>>>>>> >> >>> > To post to this group, send email to >>>>>>>>>>>> kubernet...@googlegroups.com. >>>>>>>>>>>> >> >>> > Visit this group at >>>>>>>>>>>> >> >>> > https://groups.google.com/group/kubernetes-users. >>>>>>>>>>>> >> >>> > For more options, visit >>>>>>>>>>>> https://groups.google.com/d/optout. >>>>>>>>>>>> >> > >>>>>>>>>>>> >> > -- >>>>>>>>>>>> >> > You received this message because you are subscribed to >>>>>>>>>>>> the Google >>>>>>>>>>>> >> > Groups >>>>>>>>>>>> >> > "Kubernetes user discussion and Q&A" group. >>>>>>>>>>>> >> > To unsubscribe from this group and stop receiving emails >>>>>>>>>>>> from it, send >>>>>>>>>>>> >> > an >>>>>>>>>>>> >> > email to kubernetes-use...@googlegroups.com. >>>>>>>>>>>> >> > To post to this group, send email to >>>>>>>>>>>> kubernet...@googlegroups.com. >>>>>>>>>>>> >> > Visit this group at >>>>>>>>>>>> https://groups.google.com/group/kubernetes-users. >>>>>>>>>>>> >> > For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>>>>> > -- >>>>>>>>>>>> > You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups >>>>>>>>>>>> > "Kubernetes user discussion and Q&A" group. >>>>>>>>>>>> > To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an >>>>>>>>>>>> > email to kubernetes-use...@googlegroups.com. >>>>>>>>>>>> > To post to this group, send email to >>>>>>>>>>>> kubernet...@googlegroups.com. >>>>>>>>>>>> > Visit this group at >>>>>>>>>>>> https://groups.google.com/group/kubernetes-users. >>>>>>>>>>>> > For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "Kubernetes user discussion and Q&A" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to >>>>>>>>>>> kubernetes-users+unsubscr...@googlegroups.com. >>>>>>>>>>> To post to this group, send email to >>>>>>>>>>> kubernetes-users@googlegroups.com. >>>>>>>>>>> Visit this group at >>>>>>>>>>> https://groups.google.com/group/kubernetes-users. >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Kubernetes user discussion and Q&A" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to kubernetes-users+unsubscr...@googlegroups.com. >>>>>>>>> To post to this group, send email to >>>>>>>>> kubernetes-users@googlegroups.com. >>>>>>>>> Visit this group at >>>>>>>>> https://groups.google.com/group/kubernetes-users. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Kubernetes user discussion and Q&A" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to kubernetes-users+unsubscr...@googlegroups.com. >>>>>> To post to this group, send email to >>>>>> kubernetes-users@googlegroups.com. >>>>>> Visit this group at https://groups.google.com/group/kubernetes-users. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Kubernetes user discussion and Q&A" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to kubernetes-users+unsubscr...@googlegroups.com. >>>> To post to this group, send email to kubernetes-users@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/kubernetes-users. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Kubernetes user discussion and Q&A" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to kubernetes-users+unsubscr...@googlegroups.com. >> To post to this group, send email to kubernetes-users@googlegroups.com. >> Visit this group at https://groups.google.com/group/kubernetes-users. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
