Reviewed: https://review.opendev.org/c/openstack/neutron/+/932041 Committed: https://opendev.org/openstack/neutron/commit/adbc3e23b7d2251cc7de088e2a757674a41c2f6a Submitter: "Zuul (22348)" Branch: master
commit adbc3e23b7d2251cc7de088e2a757674a41c2f6a Author: Rodolfo Alonso Hernandez <[email protected]> Date: Thu Oct 10 08:49:44 2024 +0000 Optimize the SG rule retrieval There are some operations where the SG DB object can be used instead of the SG OVO. That saves conversion time, including the conversion of the SG rule OVOs, that are child resources of the SG OVO. This optimization applies to the following methods: * SecurityGroupDbMixin.get_security_groups * SecurityGroupDbMixin.update_security_group (partially) The Nova query to retrieve the SG list in the "server list" command, has been benchmarked. The testing environment had a single SG with 250 SG rules. Call: "GET /networking/v2.0/security-groups?id=81f64aa4-2cea-46db-8fea-cd944f106aab &fields=id&fields=name HTTP/1.1" * Without this patch: around 1.25 seconds * With this patch: around 0.025 second (50x improvement). Closes-bug: #2083682 Change-Id: Ibd032ea77c5bfbc1fa80b3b3ee9ba7d5c36bb1bc ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2083682 Title: Slowness of security groups list API Status in neutron: Fix Released Bug description: Input: - OpenStack cluster of 2024.1 release - Total number of VMs = 9k - Total number of security groups = 6.4k - Total number of security groups rules = 122k Problem description: Nova servers list API exceeded 60s timeout in processing request of retrieving detailed information of 1k servers(default limit). OpenStack SDK equivalent call `conn.compute.servers(all_projects=True, paginated=False)`. The debugging showed that it takes <5s the retrieve all info from Nova's db and all remaining time is wasted by calling Neutron to retrieve information about security groups. Nova's logic to retrieve security group info - https://github.com/openstack/nova/blob/stable/2024.1/nova/network/security_group_api.py#L532 : - retrieving all ports for servers. Nova does a separate call to neutron for each 150 items to not exceed URL size limit - https://github.com/openstack/nova/blob/stable/2024.1/nova/network/security_group_api.py#L471-L497. Each such call takes less than 0.5s to complete. - retrieving discovered security groups. Same here, separate call for each 150 items - https://github.com/openstack/nova/blob/stable/2024.1/nova/network/security_group_api.py#L500-L529 . Nova passes fields=["id", "name"] filter to neutron API - https://github.com/openstack/nova/blob/stable/2024.1/nova/network/security_group_api.py#L547 to avoid neutron fetching security group rules which can be a heavy operation. Each such call takes ~9s. https://review.opendev.org/c/openstack/neutron/+/929967 is applied to neutron server's. It improved the case, but has not resolved it. Additional info: Nova uses python-neutronclient library, which in my experiments behaves quicker than openstacksdk. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2083682/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
