Public bug reported: When listing server details, instance_get_all_by_filters() can be invoked with duplicates in the columns_to_join list. This can result in both a join and a separate query against the potential "manual_join" tables.
This appears to have been introduced with the behavior in this commit: https://github.com/openstack/nova/commit/2e68b2298e94a15d1282c0fb46804b9efa6c8b3a Where the list of expected_attr here: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L55 https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L110 is further extended with the expected_attr list in the compute_api here: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2092 Resulting in a columns_to_join list resembling: ['flavor', 'info_cache', 'metadata', 'metadata', 'system_metadata', 'info_cache', 'security_groups'] In nova.db.sqlalchemy.api:_manual_join_columns(), only the first 'metadata' entry gets removed resulting in both a sqlalchemy joinedload() hint (joining against instance_metadata) and separately querying instance_metadata via _instances_fill_metadata(). The 'metadata' join in particular can be rather inefficient. In some cases this results in about 10x the data being pulled from the database compared to just the "manual join" - this problem is particularly amplified for projects with a large number of associated instances. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1599256 Title: instance_get_all_by_filters can perform unnecessary joins Status in OpenStack Compute (nova): New Bug description: When listing server details, instance_get_all_by_filters() can be invoked with duplicates in the columns_to_join list. This can result in both a join and a separate query against the potential "manual_join" tables. This appears to have been introduced with the behavior in this commit: https://github.com/openstack/nova/commit/2e68b2298e94a15d1282c0fb46804b9efa6c8b3a Where the list of expected_attr here: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L55 https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L110 is further extended with the expected_attr list in the compute_api here: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2092 Resulting in a columns_to_join list resembling: ['flavor', 'info_cache', 'metadata', 'metadata', 'system_metadata', 'info_cache', 'security_groups'] In nova.db.sqlalchemy.api:_manual_join_columns(), only the first 'metadata' entry gets removed resulting in both a sqlalchemy joinedload() hint (joining against instance_metadata) and separately querying instance_metadata via _instances_fill_metadata(). The 'metadata' join in particular can be rather inefficient. In some cases this results in about 10x the data being pulled from the database compared to just the "manual join" - this problem is particularly amplified for projects with a large number of associated instances. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1599256/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
