AmjadHussainSyed opened a new issue #6957: URL: https://github.com/apache/skywalking/issues/6957
Please answer these questions before submitting your issue. - Why do you submit this issue? - [x] Question or discussion - [ ] Bug - [ ] Requirement - [ ] Feature or performance improvement ___ ### Question - What do you want to know? I have tried to follow the k8s monitoring in skywalking using the following document. Configured what all the document says but still not able to see the k8s monitoring in UI. From the logs I can say the configuration is loaded. Please advice https://skywalking.apache.org/docs/main/latest/en/setup/backend/backend-infrastructure-monitoring/#setup-1 *`Helm:`* ``` helm install "${SKYWALKING_RELEASE_NAME}" skywalking -n "${SKYWALKING_RELEASE_NAMESPACE}" \ --set oap.image.tag=8.5.0-es7 \ --set oap.storageType=elasticsearch7 \ --set ui.image.tag=8.5.0 \ --set elasticsearch.imageTag=7.5.1 --set oap.env.SW_OTEL_RECEIVER=default --set oap.env.SW_OTEL_RECEIVER_ENABLED_OC_RULES="k8s-cluster\,k8s-service\,k8s-node" \ --set elasticsearch.persistence.enabled=true ``` *`OTEL-LOGS:`* ``` StartTimestamp: 1970-01-01 00:00:00 +0000 UTC Timestamp: 2021-05-17 07:54:17.942 +0000 UTC Value: 0.00000^C -> cluster: xxx-xxx -> failure_domain_beta_kubernetes_io_region: xx-xxxx-1 -> failure_domain_beta_kubernetes_io_zone: xx-xxxxx-1a -> id: /kubepods/podab1be6bf-cea6-47d6-9884-4c2ad3c7e557 -> instanceType: spotInstance -> kubernetes_io_arch: amd64 -> kubernetes_io_hostname: xx-xxx-xx-xx-node-k8s-xx-xxx-v1 -> kubernetes_io_os: linux -> namespace: default -> node: ip-xx-xxx-xx-xxx.xx-xxx-1.compute.internal -> node_kubernetes_io_instance_type: m5d.4xlarge -> pod: xxx-web-xxx-xxx-xxxxxxx-deployment-54544ff5db-g7lg9 -> state: stopped -> topology_kubernetes_io_region: xx-xxxxx-1 -> topology_kubernetes_io_zone: xx-xxxxx-1a ``` *`Logs:`* ``` skywalking-oap-684d845ff8-hs77w oap 2021-05-17 07:39:59,767 - org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoader -642 [main] INFO [] - Get a provider define belong to receiver-otel module, provider name: default skywalking-oap-684d845ff8-hs77w oap 2021-05-17 07:39:59,767 - org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoader -642 [main] INFO [] - Provider=default config=enabledHandlers has been set as oc skywalking-oap-684d845ff8-hs77w oap 2021-05-17 07:39:59,767 - org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoader -642 [main] INFO [] - Provider=default config=enabledOcRules has been set as k8s-cluster,k8s-service,k8s-node ```` *`NOTE:`* I was getting the following error in oap so i added the clusterrole permissions to read metadata as per the document, which fix the following error: ``` skywalking-oap-684d845ff8-wmrt6 oap 2021-05-17 07:25:18,940 - io.kubernetes.client.informer.cache.ReflectorRunnable -102569 [controller-reflector-io.kubernetes.client.openapi.models.V1Service-0] ERROR [] - class io.kubernetes.client.openapi.models.V1Service#Reflector loop failed unexpectedly skywalking-oap-684d845ff8-wmrt6 oap io.kubernetes.client.openapi.ApiException: Forbidden skywalking-oap-684d845ff8-wmrt6 oap at io.kubernetes.client.openapi.ApiClient.handleResponse(ApiClient.java:971) ~[client-java-api-10.0.0.jar:?] skywalking-oap-684d845ff8-wmrt6 oap at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:883) ~[client-java-api-10.0.0.jar:?] skywalking-oap-684d845ff8-wmrt6 oap at io.kubernetes.client.informer.SharedInformerFactory$1.list(SharedInformerFactory.java:182) ~[client-java-10.0.0.jar:?] skywalking-oap-684d845ff8-wmrt6 oap at io.kubernetes.client.informer.cache.ReflectorRunnable.run(ReflectorRunnable.java:74) [client-java-10.0.0.jar:?] skywalking-oap-684d845ff8-wmrt6 oap at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] skywalking-oap-684d845ff8-wmrt6 oap at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] skywalking-oap-684d845ff8-wmrt6 oap at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] skywalking-oap-684d845ff8-wmrt6 oap at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] skywalking-oap-684d845ff8-wmrt6 oap at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] skywalking-oap-684d845ff8-wmrt6 oap at java.lang.Thread.run(Thread.java:834) [?:?] skywalking-oap-684d845ff8-s8cbv oap 2021-05-17 07:25:18,988 - io.kubernetes.client.informer.cache.ReflectorRunnable -102213 [controller-reflector-io.kubernetes.client.openapi.models.V1Pod-0] INFO [] - class io.kubernetes.client.openapi.models.V1Pod#Start listing and watching... skywalking-oap-684d845ff8-s8cbv oap 2021-05-17 07:25:18,988 - io.kubernetes.client.informer.cache.ReflectorRunnable -102213 [controller-reflector-io.kubernetes.client.openapi.models.V1Service-0] INFO [] - class io.kubernetes.client.openapi.models.V1Service#Start listing and watching... skywalking-oap-684d845ff8-s8cbv oap 2021-05-17 07:25:18,989 - io.kubernetes.client.informer.cache.ReflectorRunnable -102214 [controller-reflector-io.kubernetes.client.openapi.models.V1Pod-0] ERROR [] - class io.kubernetes.client.openapi.models.V1Pod#Reflector loop failed unexpectedly skywalking-oap-684d845ff8-s8cbv oap io.kubernetes.client.openapi.ApiException: Forbidden skywalking-oap-684d845ff8-s8cbv oap at io.kubernetes.client.openapi.ApiClient.handleResponse(ApiClient.java:971) ~[client-java-api-10.0.0.jar:?] skywalking-oap-684d845ff8-s8cbv oap at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:883) ~[client-java-api-10.0.0.jar:?] skywalking-oap-684d845ff8-s8cbv oap at io.kubernetes.client.informer.SharedInformerFactory$1.list(SharedInformerFactory.java:182) ~[client-java-10.0.0.jar:?] skywalking-oap-684d845ff8-s8cbv oap at io.kubernetes.client.informer.cache.ReflectorRunnable.run(ReflectorRunnable.java:74) [client-java-10.0.0.jar:?] skywalking-oap-684d845ff8-s8cbv oap at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] skywalking-oap-684d845ff8-s8cbv oap at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] skywalking-oap-684d845ff8-s8cbv oap at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] skywalking-oap-684d845ff8-s8cbv oap at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] skywalking-oap-684d845ff8-s8cbv oap at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] skywalking-oap-684d845ff8-s8cbv oap at java.lang.Thread.run(Thread.java:834) [?:?] skywalking-oap-684d845ff8-s8cbv oap 2021-05-17 07:25:18,989 - io.kubernetes.client.informer.cache.ReflectorRunnable -102214 [controller-reflector-io.kubernetes.client.openapi.models.V1Service-0] ERROR [] - class io.kubernetes.client.openapi.models.V1Service#Reflector loop failed unexpectedly skywalking-oap-684d845ff8-s8cbv oap io.kubernetes.client.openapi.ApiException: Forbidden ``` ___ ### Bug - Which version of SkyWalking, OS, and JRE? 8.5.0 - Which company or project? - What happened? If possible, provide a way to reproduce the error. e.g. demo application, component version. Not able to see k8s monitoring in UI ___ ### Requirement or improvement - Please describe your requirements or improvement suggestions. Need to able to see k8s monitoring in UI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
