I thought source IP was already in the kube-apiserver logs? You may also be able to take a clue from what objects are being watched.
re: the apiserver manifest: You should upgrade to etcd3 & proto. On Wed, Mar 21, 2018 at 4:05 AM <carlos....@gmail.com> wrote: > We have a Kubernetes 1.7.8 clusters deployed with Kops 1.7 in HA with > three masters. The cluster has 10 nodes and around 400 pods. > > The cluster has heapster, prometheus, and ELK (collecting logs for some > pods). > > We are seeing a very high activity in the masters, over 90% of CPU used by > the api-server. > > Checking prometheus numbers we can see that near 5000 requests to the > kube-apiserver are WATCH verbs, the rest are less than 50 request (GET, > LIST, PATCH, PUT). > > Almost all requests are reported with client "Go-Http-client/2.0" (the > default User Agent for the Go HTTP library). > > We increased the masters size (doubling their CPU) and the CPU stilled to > the same high CPU (now the double). > > Is this a normal situation? > > How can we debug which are the pods sending these requests? (How can we > add the source IP to the kube-apiserver logs?) > > [kube-apiserver.manifest][1] > > Thanks, Charles > > The apiserver.manifest can be seen in https://pastebin.com/nGxSXuZb > > -- > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-users+unsubscr...@googlegroups.com. > To post to this group, send email to kubernetes-users@googlegroups.com. > Visit this group at https://groups.google.com/group/kubernetes-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
