Reviewed: https://review.opendev.org/708695 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d874c46bff7045ba25f5dd6e790f7ddb209cb224 Submitter: Zuul Branch: master
commit d874c46bff7045ba25f5dd6e790f7ddb209cb224 Author: Rodolfo Alonso Hernandez <[email protected]> Date: Tue Feb 18 17:08:22 2020 +0000 Filter by owner SGs when retrieving the SG rules Retrieving the SG rules now is used the admin context. This allows to get all possible rules, independently of the user calling. The filters passed and the RBAC policies filter those results, returning only: - The SG rules belonging to the user. - The SG rules belonging to a SG owned by the user. However, if the SG list is too long, the query can take a lot of time. Instead of this, the filtering is done in the DB query. If no filters are passed to "get_security_group_rules" and the context is not the admin context, only the rules specified in the first paragraph will be retrieved. Because overwriting the method "get_objects" is too complex, an intermediate query is done to retrieve the SG rule IDs. Those IDs will be used as a filter in the "get_objects" call. Closes-Bug: #1863201 Change-Id: I25d3da929f8d0b6ee15d7b90ec59b9d58a4ae6a5 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863201 Title: stein regression listing security group rules Status in neutron: Fix Released Bug description: Upgrading neutron from rocky -> stein and get a considerable slow down when listing all security groups for a project. Goes from ~2 seconds to almost 2 minutes. Looking into the code it looks like it is very inefficient because it gets all rules from the DB and then filters after the fact. We have around 7000 rules in our QA env. Very keen to get this sorted but don't know the neutron code base that well so can offer testing of patches if there are any out there already. It looks like this happened with listing ports too for stein and found this https://bugzilla.redhat.com/show_bug.cgi?id=1737012 so wonder if this is related? With Rocky: time openstack security group rule list +--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+ | ID | IP Protocol | Ethertype | IP Range | Port Range | Remote Security Group | Security Group | +--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+ | 01b877cc-1621-44cd-8e69-1345ab01a1ef | None | IPv4 | 0.0.0.0/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 | | 0c744788-6319-42e5-931a-5e7b0df166c4 | None | IPv6 | ::/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 | | 0fc6b79d-d211-4201-ac76-60fb8ea40c9c | None | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 | | 17d6c8a3-7894-42a6-92f2-1bd56a30ef1d | tcp | IPv4 | 0.0.0.0/0 | 80:80 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | | 19d3ba79-65f1-4c89-a1c2-b32049ceb25a | None | IPv6 | ::/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 21f1d173-b99f-47a7-9983-6926f7bc58f3 | icmp | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 3321d5ff-11c3-4104-be13-107c789e4bf8 | None | IPv6 | ::/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | | 381c6816-9b5c-42b7-9dd3-dae12a49c08b | None | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 | | 3886ad10-99ea-4f60-a36c-ffbe80d92907 | None | IPv6 | ::/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | | 5be4853a-75d1-435c-87ca-56c54a243f70 | None | IPv4 | 0.0.0.0/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | | 71656249-4454-410e-8e7d-24910df127ba | None | IPv6 | ::/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 | | 783324ac-6844-4d4d-985c-936015bcb66e | icmp | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 | | 7ca7f0cc-b4df-401f-aaa4-662f17afcfb0 | None | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 825a33ff-b693-456d-811e-a0b494e8e308 | None | IPv6 | ::/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 89fd2d18-45d3-4a86-a020-09d240912e5c | tcp | IPv4 | 128.250.116.173/32 | 22:22 | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 8a1f45f1-e4c8-41e4-b6f3-80ab48b7e38d | None | IPv6 | ::/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 | | 9ebc6d15-e3eb-4d20-88d4-6737367ffc08 | None | IPv4 | 0.0.0.0/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | | 9f29f539-a80a-4a8d-89cc-f714224b5f8c | icmp | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 | | a1bc8f05-3a20-48c2-bae5-a60f4ffed514 | None | IPv4 | 0.0.0.0/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba | | bef999d6-669a-47f6-988c-e69bab6df87a | tcp | IPv4 | 0.0.0.0/0 | 22:22 | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 | | c5ce339b-cd92-492c-9af4-6eab875027ce | tcp | IPv4 | 0.0.0.0/0 | 80:80 | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba | | d9ec0eba-d80d-4331-a588-e4f8c1c75533 | None | IPv6 | ::/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 | | de760e03-92a9-4183-8acc-1d82addc3604 | None | IPv4 | 0.0.0.0/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 | | f4bc4616-1d18-4488-84bb-546516c053bc | tcp | IPv4 | 0.0.0.0/0 | 443:443 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | +--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+ real 0m2.499s user 0m0.642s sys 0m0.053s With Stein: time openstack security group rule list +--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+ | ID | IP Protocol | Ethertype | IP Range | Port Range | Remote Security Group | Security Group | +--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+ | 01b877cc-1621-44cd-8e69-1345ab01a1ef | None | IPv4 | 0.0.0.0/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 | | 0c744788-6319-42e5-931a-5e7b0df166c4 | None | IPv6 | ::/0 | | None | 3dcbd4fa-d017-4361-b0b0-b7508e923087 | | 0fc6b79d-d211-4201-ac76-60fb8ea40c9c | None | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 | | 17d6c8a3-7894-42a6-92f2-1bd56a30ef1d | tcp | IPv4 | 0.0.0.0/0 | 80:80 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | | 19d3ba79-65f1-4c89-a1c2-b32049ceb25a | None | IPv6 | ::/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 21f1d173-b99f-47a7-9983-6926f7bc58f3 | icmp | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 3321d5ff-11c3-4104-be13-107c789e4bf8 | None | IPv6 | ::/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | | 381c6816-9b5c-42b7-9dd3-dae12a49c08b | None | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 | | 3886ad10-99ea-4f60-a36c-ffbe80d92907 | None | IPv6 | ::/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | | 5be4853a-75d1-435c-87ca-56c54a243f70 | None | IPv4 | 0.0.0.0/0 | | None | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | | 71656249-4454-410e-8e7d-24910df127ba | None | IPv6 | ::/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 | | 783324ac-6844-4d4d-985c-936015bcb66e | icmp | IPv4 | 0.0.0.0/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 | | 7ca7f0cc-b4df-401f-aaa4-662f17afcfb0 | None | IPv4 | 0.0.0.0/0 | | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 825a33ff-b693-456d-811e-a0b494e8e308 | None | IPv6 | ::/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 89fd2d18-45d3-4a86-a020-09d240912e5c | tcp | IPv4 | 128.250.116.173/32 | 22:22 | None | 008510a7-d176-4ee5-87e2-e74da06c55ba | | 8a1f45f1-e4c8-41e4-b6f3-80ab48b7e38d | None | IPv6 | ::/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 | | 9ebc6d15-e3eb-4d20-88d4-6737367ffc08 | None | IPv4 | 0.0.0.0/0 | | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | | 9f29f539-a80a-4a8d-89cc-f714224b5f8c | icmp | IPv4 | 0.0.0.0/0 | | None | 8f55c18b-cd8c-4d84-afef-f8b83d5eb128 | | a1bc8f05-3a20-48c2-bae5-a60f4ffed514 | None | IPv4 | 0.0.0.0/0 | | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba | | bef999d6-669a-47f6-988c-e69bab6df87a | tcp | IPv4 | 0.0.0.0/0 | 22:22 | 57cb14de-dd5f-4f0c-b0cf-a7effc36fca5 | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 | | c5ce339b-cd92-492c-9af4-6eab875027ce | tcp | IPv4 | 0.0.0.0/0 | 80:80 | 008510a7-d176-4ee5-87e2-e74da06c55ba | 008510a7-d176-4ee5-87e2-e74da06c55ba | | d9ec0eba-d80d-4331-a588-e4f8c1c75533 | None | IPv6 | ::/0 | | None | 3f63cfbb-87ee-4aa2-8193-7e86cb542881 | | de760e03-92a9-4183-8acc-1d82addc3604 | None | IPv4 | 0.0.0.0/0 | | None | bf7abb53-e5ca-428d-9fce-6a2e37a25ee0 | | f4bc4616-1d18-4488-84bb-546516c053bc | tcp | IPv4 | 0.0.0.0/0 | 443:443 | None | ed257fd7-d825-4014-96a8-c16adfea70f0 | +--------------------------------------+-------------+-----------+--------------------+------------+--------------------------------------+--------------------------------------+ real 1m51.921s user 0m0.624s sys 0m0.077s To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1863201/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
