wankai123 opened a new pull request #6739:
URL: https://github.com/apache/skywalking/pull/6739


   ### Fix K8s monitoring the incorrect metrics calculates
   - [ ] Add a unit test to verify that the fix works.
   - [X] Explain briefly why the bug exists and how to fix it.
   
   - [X] If this pull request closes/resolves/fixes an existing issue, replace 
the issue number. Closes #6738
   - [X] Update the [`CHANGES` 
log](https://github.com/apache/skywalking/blob/master/CHANGES.md).
   
   
   1. cAdvisor has duplicate metrics in CPU and Memory usage, for example:
   ```
   
container_cpu_usage_seconds_total{container="",cpu="total",id="/kubepods/burstable/podfe75645a-8767-4384-8329-212a6939efe1",image="",name="",namespace="monitoring",pod="otel-collector-65dd55b6fc-q6f95"}
 1.702443889 1618272629007
   
container_cpu_usage_seconds_total{container="POD",cpu="total",id="/kubepods/burstable/podfe75645a-8767-4384-8329-212a6939efe1/944412e147887c8949210db6f3b9dab478789d0da302724777ca59b559ba2568",image="k8s.gcr.io/pause:3.2",name="k8s_POD_otel-collector-65dd55b6fc-q6f95_monitoring_fe75645a-8767-4384-8329-212a6939efe1_0",namespace="monitoring",pod="otel-collector-65dd55b6fc-q6f95"}
 0.022044932 1618272634136
   
container_cpu_usage_seconds_total{container="otel-collector",cpu="total",id="/kubepods/burstable/podfe75645a-8767-4384-8329-212a6939efe1/20ec553676f5a3bbce627d2cef61d739e282a7f2bc963a4433c25e25b64cac48",image="otel/opentelemetry-collector-dev@sha256:7ef3617885a5c1d2383f4d2abef0d1cd881d2b7de50a40b6163ef20b88805cc9",name="k8s_otel-collector_otel-collector-65dd55b6fc-q6f95_monitoring_fe75645a-8767-4384-8329-212a6939efe1_0",namespace="monitoring",pod="otel-collector-65dd55b6fc-q6f95"}
 1.698166589 1618272632536
   ```
   When the metric `container=""` has contained the other usage value, we need 
to filter it.
   
   2. The CPU metrics couldn't show up in time.
   We changed the MAL expression `rate('PT1M')` to `.irate()`
   
   Fixed snapshot:
   
   
![image](https://user-images.githubusercontent.com/16773043/114478514-fce28900-9c30-11eb-839e-10518c1782f3.png)
   
![image](https://user-images.githubusercontent.com/16773043/114478519-ffdd7980-9c30-11eb-832e-50c67b20f346.png)
   
   
![image](https://user-images.githubusercontent.com/16773043/114478531-066bf100-9c31-11eb-9ae2-262aeba98b91.png)
   
![image](https://user-images.githubusercontent.com/16773043/114478535-09ff7800-9c31-11eb-88d1-d38885beda86.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to