Hi I have resolved the problem: add namespace to the job... -Steve
On Wed, Mar 25, 2020 at 3:25 PM Steve <[email protected]> wrote: > Hi > > I have been struggling with a RBAC issue and I cannot figure it out. > > Help please! > > > I have node exporter running in my cluster. > > As you know, it is a deamonSet and there is a node_exporter pod running on > each node. > > I also have a Prometheus server also running in the same namespace as the > node_exporter deamonSet i.e. the default namespace. > > > The scrape job for node _exporter is using a SD configuration for pods as > follows: > > - job_name: prometheus_node_exporter > > honor_timestamps: true > > scrape_interval: 15s > > scrape_timeout: 10s > > metrics_path: /metrics > > scheme: http > > kubernetes_sd_configs: > > - role: pod > > ... > > > > If I setup my Prometheus Server to use a cluster role, the node_exporter > targets are properly discovered. So far so good! > > > Now if I try to reduce the Prometheus Server to use a role instead, then > it does not work. > > > As far as I know if the role includes listing any pods within the same > namespace of the Prometheus Server service account, then the API server > should grant access. > > However, this is not the case. This is the log message I get from > Prometheus Server: > > level=error ts=2020-03-25T13:57:53.652Z caller=klog.go:94 > component=k8s_client_runtime func=ErrorDepth > msg="/app/discovery/kubernetes/kubernetes.go:385: Failed to list *v1.Pod: > pods is forbidden: User \"system:serviceaccount:default:prometheus-server\" > cannot list resource \"pods\" in API group \"\" at the cluster scope" > > > Below is role I used for the Prometheus Server service account: > > apiVersion: rbac.authorization.k8s.io/v1 > > kind: Role > > metadata: > > creationTimestamp: "2020-03-25T13:40:13Z" > > labels: > > app: prometheus > > component: server > > heritage: Helm > > release: my-server > > name: prometheus-server > > namespace: default > > resourceVersion: "1943" > > selfLink: /apis/ > rbac.authorization.k8s.io/v1/namespaces/default/roles/prometheus-server > > uid: 28d3c869-894d-4797-9146-6137f60c7232 > > rules: > > - apiGroups: > > - "" > > resources: > > - pods > > - configmaps > > verbs: > > - get > > - list > > - watch > > > > > > Below is the role binding I used for Prometheus Server service account: > > > > apiVersion: rbac.authorization.k8s.io/v1 > > kind: RoleBinding > > metadata: > > creationTimestamp: "2020-03-25T13:40:13Z" > > labels: > > app: prometheus > > chart: prometheus-10.5.1-steve-server-12 > > component: server > > heritage: Helm > > release: my-server > > name: prometheus-server > > namespace: default > > resourceVersion: "1946" > > selfLink: /apis/ > rbac.authorization.k8s.io/v1/namespaces/default/rolebindings/prometheus-server > > uid: d581c497-52d6-4080-8ade-e33008c019fd > > roleRef: > > apiGroup: rbac.authorization.k8s.io > > kind: Role > > name: prometheus-server > > subjects: > > - kind: ServiceAccount > > name: prometheus-server > > namespace: default > > > > > > Thank you! > > > > Regards > > Steve B > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/2b7bd5cf-4fb8-4b5c-991a-f755aaf86106%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/2b7bd5cf-4fb8-4b5c-991a-f755aaf86106%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAL7AJRsB7S7pwJdcwX9X%2B9b24wOXLe8iYLCQ4p9g%3Dviqzsn_mA%40mail.gmail.com.
