wklken commented on issue #10000:
URL: https://github.com/apache/apisix/issues/10000#issuecomment-1673042123

   from the log:
   1. apisix.node_listen port is ok before been restarted, access.log
   2. `prometheus.lua:920  no memory` in error.log
   3. the prometheus can't scrape the `/metrics` from apisix for few hours
   
   -------
   
   the service monitor config
   
   ```
       interval: "30s"
       scrapeTimeout: "30s"
   ```
   
   so, if the response too slow(maybe the metrics data too huge?), it will 
timeout and lost the line in chart.
   
   if the prometheus privileged process keep receiving the `/metrics` for 
hours, all timeout => a high cpu usage  => the livenessProbe/readinessProbe to 
the apisix will affected, timeout (`context deadline exceeded (Client Timeout 
exceeded while awaiting headersi`) => the container been restart
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to