[I] help request: My APISIX is experiencing 100% CPU utilization and has become unresponsive [apisix]

[via GitHub]

zhaoqiang1980 opened a new issue, #12275:
URL: https://github.com/apache/apisix/issues/12275

   ### Description
   
   When the free_size of Prometheus in APISIX's share_dict becomes 0, will it 
trigger a 100% CPU exception? This issue occurred in 2 out of 38 instances. 
Even after removing the traffic, the CPU remains fully occupied. This is what 
was observed using perf tools and flamegraph.    
   ngx_shmtx_lock  ngx_shmtx_unlock
   
   
![Image](https://github.com/user-attachments/assets/b048ff1e-aebc-4d57-b351-5ac79c641c9b)
   strace信息：
   
![Image](https://github.com/user-attachments/assets/97101529-d825-4cd1-a92b-715343a8480a)
   perf top信息：
   
   
![Image](https://github.com/user-attachments/assets/4f28039c-3fc1-4b78-8fa3-e190d30adadc)
   
   We have mounted too many routes, so the metrics are very large. The 
currently allocated memory is 100M.
   what should i do ? scale prometheus share dict?
   
   ### Environment
   
   apisix :2.15
   Linux 4 3.1-1160.92.1.el7.x86_64 #1 SMP Tue Jun 20 11:48:01 UTC 2023 x86_64 
x86_64 x86_64 GNU/Linux
   openresty/1.21.4.2
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@apisix.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org