You need a similar -k argument by curl to prometheus. -k, --insecure Allow insecure server connections when using SSL
regards. On Tue, Dec 22, 2020 at 5:12 PM alex he <[email protected]> wrote: > yes . my k8s cluster is based on rancher. all certs are self-signed. > > 在2020年12月22日星期二 UTC+8 下午5:10:24<[email protected]> 写道： > >> *x509: certificate signed by unknown authority"* >> >> It seems you are using a self-signed certificate for authentication. >> This is maybe the issue. >> >> On Tue, Dec 22, 2020 at 4:59 PM alex he <[email protected]> wrote: >> >>> I can use curl to visit k8s apiserver api: >>> >>> *curl https://10.10.10.68:6443/api/v1/nodes >>> <https://10.10.10.68:6443/api/v1/nodes> --cacert kube-ca.pem --cert >>> kube-node.pem --key kube-node-key.pem|head -n 20* >>> >>> "kind": "NodeList", >>> "apiVersion": "v1", >>> "metadata": { >>> "selfLink": "/api/v1/nodes", >>> "resourceVersion": "67299229" >>> }, >>> "items": [ >>> { >>> "metadata": { >>> "name": "k8smaster12", >>> "selfLink": "/api/v1/nodes/k8smaster12", >>> "uid": "060be972-6346-11ea-a193-00155d0a3a00", >>> "resourceVersion": "67299092", >>> "creationTimestamp": "2020-03-11T03:11:38Z", >>> "labels": { >>> "beta.kubernetes.io/arch": "amd64", >>> "beta.kubernetes.io/os": "linux", >>> "kubernetes.io/arch": "amd64", >>> "kubernetes.io/hostname": "k8smaster12", >>> >>> >>> *but I can't use prometheus to visit k8s.this is my prometheus.yml:* >>> root@alextest-55c44cddc8-gqcdt:~/prometheus-2.23.0.linux-amd64# cat >>> prometheus.yml >>> global: >>> scrape_interval: 15s >>> evaluation_interval: 15s >>> >>> alerting: >>> alertmanagers: >>> >>> - static_configs: >>> - targets: >>> >>> rule_files: >>> >>> scrape_configs: >>> >>> - job_name: "alexk8s-apiserver" >>> kubernetes_sd_configs: >>> - role: endpoints >>> api_server: 'https://10.10.10.68:6443' >>> scheme: https >>> tls_config: >>> insecure_skip_verify: true >>> ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt >>> cert_file: /root/ssl/kube-node.pem >>> key_file: /root/ssl/kube-node-key.pem >>> bearer_token_file: /var/run/secrets/ >>> kubernetes.io/serviceaccount/token >>> relabel_configs: >>> - action: labelmap >>> regex: _*meta_kubernetes_node_label*(.+) >>> >>> >>> when I start prometheus, it reports: >>> root@alextest-gqcdt:~/prometheus-2.23.0.linux-amd64# ./prometheus >>> >>> level=info ts=2020-12-22T08:39:27.185Z caller=main.go:322 msg="No time >>> or size retention was set so using the default time retention" duration=15d >>> level=info ts=2020-12-22T08:39:27.185Z caller=main.go:360 msg="Starting >>> Prometheus" version="(version=2.23.0, branch=HEAD, >>> revision=26d89b4b0776fe4cd5a3656dfa520f119a375273)" level=info >>> ts=2020-12-22T08:39:27.185Z caller=main.go:365 build_context="(go=go1.15.5, >>> user=root@37609b3a0a21, date=20201126-10:56:17)" level=info >>> ts=2020-12-22T08:39:27.185Z caller=main.go:366 host_details="(Linux >>> 4.15.0-123-generic #126-Ubuntu SMP Wed Oct 21 09:40:11 UTC 2020 x86_64 >>> alextest-55c44cddc8-gqcdt (none))" level=info ts=2020-12-22T08:39:27.186Z >>> caller=main.go:367 fd_limits="(soft=1048576, hard=1048576)" level=info >>> ts=2020-12-22T08:39:27.186Z caller=main.go:368 vm_limits="(soft=unlimited, >>> hard=unlimited)" level=info ts=2020-12-22T08:39:27.188Z caller=main.go:722 >>> msg="Starting TSDB ..." level=info ts=2020-12-22T08:39:27.188Z >>> caller=web.go:528 component=web msg="Start listening for connections" >>> address=0.0.0.0:9090 level=info ts=2020-12-22T08:39:27.193Z >>> caller=head.go:645 component=tsdb msg="Replaying on-disk memory mappable >>> chunks if any" level=info ts=2020-12-22T08:39:27.193Z caller=head.go:659 >>> component=tsdb msg="On-disk memory mappable chunks replay completed" >>> duration=4.9µs level=info ts=2020-12-22T08:39:27.193Z caller=head.go:665 >>> component=tsdb msg="Replaying WAL, this may take a while" level=info >>> ts=2020-12-22T08:39:27.193Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=0 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.194Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=1 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.195Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=2 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.197Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=3 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.198Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=4 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.199Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=5 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.200Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=6 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.200Z caller=head.go:717 component=tsdb msg="WAL >>> segment loaded" segment=7 maxSegment=7 level=info >>> ts=2020-12-22T08:39:27.200Z caller=head.go:722 component=tsdb msg="WAL >>> replay completed" checkpoint_replay_duration=102.209µs >>> wal_replay_duration=7.33696ms total_replay_duration=7.495874ms level=info >>> ts=2020-12-22T08:39:27.203Z caller=main.go:742 fs_type=794c7630 level=info >>> ts=2020-12-22T08:39:27.203Z caller=main.go:745 msg="TSDB started" >>> level=info ts=2020-12-22T08:39:27.203Z caller=main.go:871 msg="Loading >>> configuration file" filename=prometheus.yml level=info >>> ts=2020-12-22T08:39:27.204Z caller=main.go:902 msg="Completed loading of >>> configuration file" filename=prometheus.yml totalDuration=1.170705ms >>> remote_storage=2µs web_handler=500ns query_engine=1.5µs scrape=252.623µs >>> scrape_sd=336.23µs notify=17.502µs notify_sd=18.502µs rules=1.5µs >>> level=info ts=2020-12-22T08:39:27.204Z caller=main.go:694 msg="Server is >>> ready to receive web requests." level=error ts=2020-12-22T08:39:27.253Z >>> caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth >>> *msg="/app/discovery/kubernetes/kubernetes.go:514: >>> Failed to watch *v1.Node: failed to list *v1.Node: Get >>> \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\ >>> <https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0%5C>": >>> x509: certificate signed by unknown authority" level=error >>> ts=2020-12-22T08:39:28.554Z caller=klog.go:96 component=k8s_client_runtime >>> func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to >>> watch *v1.Node: failed to list *v1.Node: Get >>> \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\ >>> <https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0%5C>": >>> x509: certificate signed by unknown authority" level=error >>> ts=2020-12-22T08:39:31.675Z caller=klog.go:96 component=k8s_client_runtime >>> func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to >>> watch *v1.Node: failed to list *v1.Node: Get >>> \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\ >>> <https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0%5C>": >>> x509: certificate signed by unknown authority" level=error >>> ts=2020-12-22T08:39:37.017Z caller=klog.go:96 component=k8s_client_runtime >>> func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to >>> watch *v1.Node: failed to list *v1.Node: Get >>> \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\ >>> <https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0%5C>": >>> x509: certificate signed by unknown authority"* >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/9caa85ba-8aee-48df-9fae-ef4078a3d1c7n%40googlegroups.com >>> <https://groups.google.com/d/msgid/prometheus-users/9caa85ba-8aee-48df-9fae-ef4078a3d1c7n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/d28aa7b0-52c3-4c96-81cf-c2f9b90d13b9n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/d28aa7b0-52c3-4c96-81cf-c2f9b90d13b9n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAOZE12p9Ee%2BQVZ89cQx-sxV2E8LRCYmaa6UGvBbvv3ybkDv%3Dow%40mail.gmail.com.
