Public bug reported: While allocating or updating ips for port, _get_subnet and _get_subents are called multiple times.
For example, if update_port is called with below fixed_ips fixed_ips = [{'subnet_id': 'subnet1'}, {'subnet_id': 'v6_dhcp_stateless_subnet'} {'subnet_id': 'v6_slaac_subnet'} {'subnet_id': 'v6_pd_subnet'} {'subnet_id': 'subnet4', 'ip_address': '30.0.0.3'} {'ip_address': '40.0.0.4'} {'ip_address': '50.0.0.5'}} then through _test_fixed_ips_for_port(fixed_ips), "_get_subnet"[1] is called 4 times for subnet1, v6_dhcp_stateless_subnet, v6_slaac_subnet, v6_pd_subnet, subnet4. "_get_subnets" [2] is called 2 times for ip_address 40.0.0.4 and 50.0.0.5. If in case of _test_fixed_ips_for_port called for _allocate_ips_for_port, then _get_subnets is already called at[3] (so increase call count). So incase of _allocate_ips_for_port, if we save subnets from [3] saved in local variable and use that in-memory subnets in further calls, we can avoid above calls to DB. Sometimes when subnet is updated, update_subnet may trigger update_port(fixed_ips)[4] for all ports on the subnet. And in each port's fixed_ips, if we have multiple subnets and ip_addresses then _get_subnet and _get_subnets will be called for multiple times for each port like above example. For example in above case if we have 10 ports on the subnet, then update_subnet will result in (10*6=60) 60 times DB access instead of 10 DB access. When port_update is called for PD subnet, it again calls get_subnet for each fixed_ip[5], to check if subnet is PD subnet or not(after get_subnet and get_subnets called many times in _test_fixed_ips_for_port). In all above cases, for _get_subnet and _get_subnets, we are accessing DB many times. So instead of calling get_subnet or get_subnets for each fixed_ip of a port(at multiple places), we can call get_subnets of a network at begining of _allocate_ips_for_port(for create port) and _update_ips_for_port(during update) and use the in-memory subnets in subsequent private functions. [1] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L311 [2] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L331 [3] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_pluggable_backend.py#L192 [4] https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L785 [5] https://review.openstack.org/#/c/241227/11/neutron/db/ipam_non_pluggable_backend.py Lines 284 and 334. ** Affects: neutron Importance: Undecided Assignee: venkata anil (anil-venkata) Status: New ** Tags: ipv6 l3-ipam-dhcp ** Changed in: neutron Assignee: (unassigned) => venkata anil (anil-venkata) ** Tags added: ipv6 l3-ipam-dhcp -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1554414 Title: Avoid calling _get_subnet(s) multiple times in ipam driver Status in neutron: New Bug description: While allocating or updating ips for port, _get_subnet and _get_subents are called multiple times. For example, if update_port is called with below fixed_ips fixed_ips = [{'subnet_id': 'subnet1'}, {'subnet_id': 'v6_dhcp_stateless_subnet'} {'subnet_id': 'v6_slaac_subnet'} {'subnet_id': 'v6_pd_subnet'} {'subnet_id': 'subnet4', 'ip_address': '30.0.0.3'} {'ip_address': '40.0.0.4'} {'ip_address': '50.0.0.5'}} then through _test_fixed_ips_for_port(fixed_ips), "_get_subnet"[1] is called 4 times for subnet1, v6_dhcp_stateless_subnet, v6_slaac_subnet, v6_pd_subnet, subnet4. "_get_subnets" [2] is called 2 times for ip_address 40.0.0.4 and 50.0.0.5. If in case of _test_fixed_ips_for_port called for _allocate_ips_for_port, then _get_subnets is already called at[3] (so increase call count). So incase of _allocate_ips_for_port, if we save subnets from [3] saved in local variable and use that in-memory subnets in further calls, we can avoid above calls to DB. Sometimes when subnet is updated, update_subnet may trigger update_port(fixed_ips)[4] for all ports on the subnet. And in each port's fixed_ips, if we have multiple subnets and ip_addresses then _get_subnet and _get_subnets will be called for multiple times for each port like above example. For example in above case if we have 10 ports on the subnet, then update_subnet will result in (10*6=60) 60 times DB access instead of 10 DB access. When port_update is called for PD subnet, it again calls get_subnet for each fixed_ip[5], to check if subnet is PD subnet or not(after get_subnet and get_subnets called many times in _test_fixed_ips_for_port). In all above cases, for _get_subnet and _get_subnets, we are accessing DB many times. So instead of calling get_subnet or get_subnets for each fixed_ip of a port(at multiple places), we can call get_subnets of a network at begining of _allocate_ips_for_port(for create port) and _update_ips_for_port(during update) and use the in-memory subnets in subsequent private functions. [1] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L311 [2] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L331 [3] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_pluggable_backend.py#L192 [4] https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L785 [5] https://review.openstack.org/#/c/241227/11/neutron/db/ipam_non_pluggable_backend.py Lines 284 and 334. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1554414/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp