[GitHub] [apisix-ingress-controller] Donghui0 opened a new issue, #1646: bug: sync k8s crd resources to apisix at intervals, causing etcd dbsize to skyrocket

Fri, 03 Feb 2023 02:54:45 -0800 


Donghui0 opened a new issue, #1646:
URL: https://github.com/apache/apisix-ingress-controller/issues/1646

   ### Issue description
   
   The controller parameter `apisix-resource-sync-interval` is used to control 
whether the k8s crd resource is consistent with apisix(etcd). 
   However, when the number of resources (route/upstream/pods)  is too large, 
the sync of the interval will have a high impact on the load of etcd service. 
And it will continue to increase etcd dbsize.
   
   See below, the dbsize increase to 2GB within 7 days, causing etcd to be 
unable to write new data.   (The etcd cluster consists of 3 nodes,  4c16g)
   <img width="1462" alt="image" 
src="https://user-images.githubusercontent.com/7134928/216584450-84371216-e9d4-46d3-8d23-34a72a6d0cfa.png";>
   
   
   By compressing the data regularly and reducing the dbsize, you can see that 
the dbsize of etcd will still increase at intervals. The interval is basically 
consistent with the parameter apisix-resource-sync-interval, which is 600s.
   <img width="1401" alt="image" 
src="https://user-images.githubusercontent.com/7134928/216581895-dad78201-0010-4b1b-bb3e-6749ffc4155e.png";>
   
   
   
   
   
   
   
   ### Environment
   
   apisix-ingress-controller version: 1.5.0
   number resource in apisix:     ApisixRoutes  300+,   Upstream 20+,  
Upstream.node  60+.
   
   
   ### Minimal test code / Steps to reproduce
   
   Under 300+ routing entries, when the number of Upstream Nodes is large, it 
can be reproduced by running at intervals.
   
   params  apisix-resource-sync-interval  set to  300s or 600s.
   
   
   ### Actual result
   
   etcd dbsize increase.
   
   ### Error log
   
   none
   
   ### Expected result
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to