On Friday, February 24, 2017 at 6:02:00 PM UTC-6, ayodele abejide wrote: > Found this in kube-proxy logs, not sure how useful this is: > > > > > > > > > > Feb 24 23:55:15 kube-worker01.mydomain.com kube-proxy[3039]: W0224 > 23:55:15.159640 3039 server.go:468] Failed to retrieve node info: nodes > "kube-worker01.mydomain.com" not foundFeb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: W0224 23:55:15.160308 3039 > proxier.go:249] invalid nodeIP, initialize kube-proxy with 127.0.0.1 as > nodeIPFeb 24 23:55:15 kube-worker01.mydomain.com kube-proxy[3039]: W0224 > 23:55:15.160596 3039 proxier.go:254] clusterCIDR not specified, unable to > distinguish between internal and external trafficFeb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.160832 3039 > server.go:227] Tearing down userspace rules.Feb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.161206 3039 > healthcheck.go:119] Initializing kube-proxy healthcheckerFeb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.170297 3039 > conntrack.go:81] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072Feb 24 > 23:55:15 kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.170731 > 3039 conntrack.go:66] Setting conntrack hashsize to 32768Feb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.170920 3039 > conntrack.go:81] Set sysctl > 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400Feb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.170950 3039 > conntrack.go:81] Set sysctl > 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600Feb 24 23:55:15 > kube-worker01.mydomain.com kube-proxy[3039]: I0224 23:55:15.172295 3039 > proxier.go:802] Not syncing iptables until Services and Endpoints have been > received from masterFeb 24 23:55:15 kube-worker01.mydomain.com > kube-proxy[3039]: I0224 23:55:15.173529 3039 proxier.go:472] Adding new > service "default/kubernetes:https" at 172.32.0.1:443/TCP > > > > > Abejide Ayodele > It always seems impossible until it's done. --Nelson Mandela > > > > On Fri, Feb 24, 2017 at 5:51 PM, 'Vishnu Kannan' via Kubernetes user > discussion and Q&A <kubernet...@googlegroups.com> wrote: > > +sig-cluster-lifecycle > > > On Fri, Feb 24, 2017 at 3:45 PM, Àbéjídé Àyodélé <abejide...@gmail.com> wrote: > > Like I said I have a similar cluster that I setup by hand that works, trying > to automate the cluster I setup by hand is where I am running into problems, > also I am interested in knowing why I am running into this problem, so I can > learn for the future > > > > > Abejide Ayodele > It always seems impossible until it's done. --Nelson Mandela > > > > > > On Fri, Feb 24, 2017 at 5:42 PM, 'Vishnu Kannan' via Kubernetes user > discussion and Q&A <kubernet...@googlegroups.com> wrote: > > > > It depends on how you are trying to setup the cluster. I'd recommend starting > with a known working solution like "kubeadm" to bootstrap your cluster before > customizing custer bringup. > > > > > On Fri, Feb 24, 2017 at 3:39 PM, ayodele abejide <abejide...@gmail.com> wrote: > > > > > Your node bootstrapping is failing. > > > How do I debug this? > > > > On Friday, 24 February 2017 17:28:53 UTC-6, Vishnu Kannan wrote: > Your node bootstrapping is failing. I'd recommend focussing on getting your > node registered and ignore the eviction manager error. The latter is a red > herring. > > > > > On Fri, Feb 24, 2017 at 2:45 PM, ayodele abejide <abejide...@gmail.com> wrote: > > > kubeclt get nodes returns No resources found. I do not know how to verify: > > > > Has the node successfully registered itself with the apiserver? > > > > Relevant code that is generating the failure is here. > > > I sort of have browsed the code, but don't have enough context to get enough > information out of it > > On Friday, 24 February 2017 16:25:49 UTC-6, Vishnu Kannan wrote: > Kubelet is unable to retrieve the "Node" object that represents the node its > running on from the api server. Has the node successfully registered itself > with the apiserver? > > > Relevant code that is generating the failure is here. > > > On Fri, Feb 24, 2017 at 2:19 PM, ayodele abejide <abejide...@gmail.com> wrote: > > Hi, > > > I have searched the internet and found no answers that solve my problem. > > > Background: > > I have a working cluster that I intended to automate it's creation via puppet > and terraform, all seemed to have worked well except I find: > > > > > > Feb 24 22:05:41 kube-worker02.mydomain.com kubelet[18956]: E0224 > 22:05:41.775438 18956 eviction_manager.go:204] eviction manager: unexpected > err: failed GetNode: node 'kube-worker02.mydomain.com' not found > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 > 22:05:51.725950 18956 kubelet.go:1155] Image garbage collection succeeded > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 > 22:05:51.733365 18956 container_manager_linux.go:728] CPUAccounting not > enabled for pid: 6095 > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 > 22:05:51.733381 18956 container_manager_linux.go:731] MemoryAccounting not > enabled for pid: 6095 > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: I0224 > 22:05:51.733391 18956 container_manager_linux.go:434] Discovered runtime > cgroups name: /system.slice/docker.service > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 > 22:05:51.733478 18956 container_manager_linux.go:728] CPUAccounting not > enabled for pid: 18956 > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: W0224 > 22:05:51.733489 18956 container_manager_linux.go:731] MemoryAccounting not > enabled for pid: 18956 > Feb 24 22:05:51 kube-worker02.mydomain.com kubelet[18956]: E0224 > 22:05:51.776184 18956 eviction_manager.go:204] eviction manager: unexpected > err: failed GetNode: node 'kube-worker02.mydomain.com' not found > > > > in the kubelet logs. > > > What have I tried? > > > I have disabled ssl, set authorization-mode=AlwaysAllow, confirmed that > kube-proxy running on the same worker can reach the api-server, also > confirmed that cadvisor is running: > > > > curl localhost:4194/api/v2.0/ > > Supported request types: > "appmetrics,attributes,events,machine,ps,spec,stats,storage,summary,version" > > > > I have also rebooted the worker a couple of times and restarted kubelet and > kube-proxy so many times. > > > I am at loss here and don't know what to try, any help will be appreciated. > > > Thanks! > > > > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-use...@googlegroups.com. > > To post to this group, send email to kubernet...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kubernetes-users. > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-use...@googlegroups.com. > > To post to this group, send email to kubernet...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kubernetes-users. > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-use...@googlegroups.com. > > To post to this group, send email to kubernet...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kubernetes-users. > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > -- > > You received this message because you are subscribed to a topic in the Google > Groups "Kubernetes user discussion and Q&A" group. > > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/kubernetes-users/bpER7WJX-Jc/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > kubernetes-use...@googlegroups.com. > > To post to this group, send email to kubernet...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kubernetes-users. > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-use...@googlegroups.com. > > To post to this group, send email to kubernet...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kubernetes-users. > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > -- > > You received this message because you are subscribed to a topic in the Google > Groups "Kubernetes user discussion and Q&A" group. > > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/kubernetes-users/bpER7WJX-Jc/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > kubernetes-use...@googlegroups.com. > > To post to this group, send email to kubernet...@googlegroups.com. > > Visit this group at https://groups.google.com/group/kubernetes-users. > > For more options, visit https://groups.google.com/d/optout.
Bump... I still don't have a solve to this problem -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.