I'm setting up a kubernetes clusters via "the hard way" but I"m stuck.
Right now I'm using flannel (tried canal too) and the apiserver runs with a 'kubernetes' cert. I get this when I run kubectl: root@host-9c16fd7a ~ # kubectl logs busybox-855686df5d-ln6ww Error from server (Forbidden): Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) ( pods/log busybox-855686df5d-ln6ww) Then on the kubelet node I get the following error (see below) I think what's happening is that the 'kubernetes' user doesn't have the proper permissions but I can't figure out actually how to configure it as the documentation seems sparse/complicated on this issue. I've definitely RTFMd but can't figure this out. Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: I0208 19:24:34.381382 10257 server.go:248] Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: I0208 19:24:34.381576 10257 server.go:796] GET /containerLogs/default/busybox-855686df5d-ln6ww/busybox: (5.610932ms) 403 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: goroutine 963 [running]: Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc42025ea10, 0x193) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xdd Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc42025ea10, 0x193) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.(*Response).WriteHeader(0xc421c3f0e0, 0x193) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/emicklei/go-restful/response.go:201 +0x41 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.(*Response).WriteErrorString(0xc421c3f0e0, 0x193, 0xc420f879a0, 0x48, 0x4, 0xc420f879a0) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/emicklei/go-restful/response.go:181 +0x46 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/pkg/kubelet/server.(*Server).InstallAuthFilter.func1(0xc420febbc0, 0xc421c3f0e0, 0xc420febcb0) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/server.go:249 +0x4b5 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.(*FilterChain).ProcessFilter(0xc420febcb0, 0xc420febbc0, 0xc421c3f0e0) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/emicklei/go-restful/filter.go:19 +0x68 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.(*Container).dispatch(0xc4208c23f0, 0x5769940, 0xc42025ea10, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/emicklei/go-restful/container.go:274 +0x8ff Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.(*Container).(k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.dispatch)-fm(0x5769940, 0xc42025ea10, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/emicklei/go-restful/container.go:120 +0x48 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: net/http.HandlerFunc.ServeHTTP(0xc4201a2240, 0x5769940, 0xc42025ea10, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /usr/local/go/src/net/http/server.go:1918 +0x44 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: net/http.(*ServeMux).ServeHTTP(0xc42075c9c0, 0x5769940, 0xc42025ea10, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /usr/local/go/src/net/http/server.go:2254 +0x130 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/vendor/github.com/emicklei/go-restful.(*Container).ServeHTTP(0xc4208c23f0, 0x5769940, 0xc42025ea10, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/emicklei/go-restful/container.go:292 +0x4d Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: k8s.io/kubernetes/pkg/kubelet/server.(*Server).ServeHTTP(0xc420880690, 0x5769940, 0xc42025ea10, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubelet/server/server.go:795 +0x106 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: net/http.serverHandler.ServeHTTP(0xc420a092b0, 0x576a580, 0xc421797c00, 0xc421036500) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /usr/local/go/src/net/http/server.go:2619 +0xb4 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: net/http.(*conn).serve(0xc4203e03c0, 0x576ca00, 0xc42183ff00) Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /usr/local/go/src/net/http/server.go:1801 +0x71d Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: created by net/http.(*Server).Serve Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: /usr/local/go/src/net/http/server.go:2720 +0x288 Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: logging error output: "Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy)" Feb 08 19:24:34 host-d9c9d5e1.instances.us-west-1.scalefastr.cloud kubelet[10257]: [[Go-http-client/1.1] 195.201.30.240:58019] -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
