Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter
Vishvananda Ishaya vishvana...@gmail.com wrote on 01/28/2015 11:32:16 AM: From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 01/28/2015 11:50 AM Subject: Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter On Jan 28, 2015, at 7:05 AM, Steven Kaufer kau...@us.ibm.com wrote: Vishvananda Ishaya vishvana...@gmail.com wrote on 01/27/2015 04:29:50 PM: From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 01/27/2015 04:32 PM Subject: Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter The network info for an instance is cached as a blob of data (neutron has the canonical version in most installs), so it isn’t particularly easy to do at the database layer. You would likely need a pretty complex stored procedure to do it accurately. Vish Vish, Thanks for the reply. I agree with your point about the difficultly in accurately querying the blob of data; however, IMHO, the complexity this fix does not preclude the current behavior as being classified as a bug. With that in mind, I was wondering if anyone in the community has any thoughts on if the current behavior is considered a bug? Yes it should be classified as a bug. Bug filed: https://bugs.launchpad.net/nova/+bug/1417649 If so, how should it be resolved? A couple options that I could think of: 1. Disallow the combination of using both a limit and an IP address filter by raising an error. I think this is the simplest solution. Vish 2. Workaround the problem by removing the limit from the DB query and then manually limiting the list of servers (after manually applying the IP address filter). I have proposed a fix that implements this solution: https://review.openstack.org/#/c/152614 Thanks, Steven Kaufer 3. Break up the query so that the server UUIDs that match the IP filter are retrieved first and then used as a UUID DB filter. As far as I can tell, this type of solution was originally implemented but the network query was deemed to expensive [1]. Is there a less expensive method to determine the UUIDs (possibly querying the cached 'network_info' in the 'instance_info_caches' table)? 4. Figure out how to accurately query the blob of network info that is cached in the nova DB and apply the IP filter at the DB layer. [1]: https://review.openstack.org/#/c/131460/ Thanks, Steven Kaufer__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter
On Jan 28, 2015, at 7:05 AM, Steven Kaufer kau...@us.ibm.com wrote: Vishvananda Ishaya vishvana...@gmail.com wrote on 01/27/2015 04:29:50 PM: From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 01/27/2015 04:32 PM Subject: Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter The network info for an instance is cached as a blob of data (neutron has the canonical version in most installs), so it isn’t particularly easy to do at the database layer. You would likely need a pretty complex stored procedure to do it accurately. Vish Vish, Thanks for the reply. I agree with your point about the difficultly in accurately querying the blob of data; however, IMHO, the complexity this fix does not preclude the current behavior as being classified as a bug. With that in mind, I was wondering if anyone in the community has any thoughts on if the current behavior is considered a bug? Yes it should be classified as a bug. If so, how should it be resolved? A couple options that I could think of: 1. Disallow the combination of using both a limit and an IP address filter by raising an error. I think this is the simplest solution. Vish 2. Workaround the problem by removing the limit from the DB query and then manually limiting the list of servers (after manually applying the IP address filter). 3. Break up the query so that the server UUIDs that match the IP filter are retrieved first and then used as a UUID DB filter. As far as I can tell, this type of solution was originally implemented but the network query was deemed to expensive [1]. Is there a less expensive method to determine the UUIDs (possibly querying the cached 'network_info' in the 'instance_info_caches' table)? 4. Figure out how to accurately query the blob of network info that is cached in the nova DB and apply the IP filter at the DB layer. [1]: https://review.openstack.org/#/c/131460/ Thanks, Steven Kaufer On Jan 27, 2015, at 2:00 PM, Steven Kaufer kau...@us.ibm.com wrote: Hello, When applying an IP address filter to a paginated servers query (eg, supplying servers/detail?ip=192.168limit=100), the IP address filtering is only being applied against the non-filtered page of servers that were retrieved from the DB; see [1]. I believe that the IP address filtering should be done before the limit is applied, returning up to limit servers that match the IP address filter. Currently, if the servers in the page of data returned from the DB do not happen to match the IP address filter (applied in the compute API), then no servers will be returned by the REST API (even if there are servers that match the IP address filter). This seems like a bug to me, shouldn't all filtering be done at the DB layer? [1]: https://github.com/openstack/nova/blob/master/nova/compute/ api.py#L2037-L2042 Thanks, Steven Kaufer __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter
Vishvananda Ishaya vishvana...@gmail.com wrote on 01/27/2015 04:29:50 PM: From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 01/27/2015 04:32 PM Subject: Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter The network info for an instance is cached as a blob of data (neutron has the canonical version in most installs), so it isn’t particularly easy to do at the database layer. You would likely need a pretty complex stored procedure to do it accurately. Vish Vish, Thanks for the reply. I agree with your point about the difficultly in accurately querying the blob of data; however, IMHO, the complexity this fix does not preclude the current behavior as being classified as a bug. With that in mind, I was wondering if anyone in the community has any thoughts on if the current behavior is considered a bug? If so, how should it be resolved? A couple options that I could think of: 1. Disallow the combination of using both a limit and an IP address filter by raising an error. 2. Workaround the problem by removing the limit from the DB query and then manually limiting the list of servers (after manually applying the IP address filter). 3. Break up the query so that the server UUIDs that match the IP filter are retrieved first and then used as a UUID DB filter. As far as I can tell, this type of solution was originally implemented but the network query was deemed to expensive [1]. Is there a less expensive method to determine the UUIDs (possibly querying the cached 'network_info' in the ' instance_info_caches' table)? 4. Figure out how to accurately query the blob of network info that is cached in the nova DB and apply the IP filter at the DB layer. [1]: https://review.openstack.org/#/c/131460/ Thanks, Steven Kaufer On Jan 27, 2015, at 2:00 PM, Steven Kaufer kau...@us.ibm.com wrote: Hello, When applying an IP address filter to a paginated servers query (eg, supplying servers/detail?ip=192.168limit=100), the IP address filtering is only being applied against the non-filtered page of servers that were retrieved from the DB; see [1]. I believe that the IP address filtering should be done before the limit is applied, returning up to limit servers that match the IP address filter. Currently, if the servers in the page of data returned from the DB do not happen to match the IP address filter (applied in the compute API), then no servers will be returned by the REST API (even if there are servers that match the IP address filter). This seems like a bug to me, shouldn't all filtering be done at the DB layer? [1]: https://github.com/openstack/nova/blob/master/nova/compute/ api.py#L2037-L2042 Thanks, Steven Kaufer __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [api] Get servers with limit and IP address filter
The network info for an instance is cached as a blob of data (neutron has the canonical version in most installs), so it isn’t particularly easy to do at the database layer. You would likely need a pretty complex stored procedure to do it accurately. Vish On Jan 27, 2015, at 2:00 PM, Steven Kaufer kau...@us.ibm.com wrote: Hello, When applying an IP address filter to a paginated servers query (eg, supplying servers/detail?ip=192.168limit=100), the IP address filtering is only being applied against the non-filtered page of servers that were retrieved from the DB; see [1]. I believe that the IP address filtering should be done before the limit is applied, returning up to limit servers that match the IP address filter. Currently, if the servers in the page of data returned from the DB do not happen to match the IP address filter (applied in the compute API), then no servers will be returned by the REST API (even if there are servers that match the IP address filter). This seems like a bug to me, shouldn't all filtering be done at the DB layer? [1]: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2037-L2042 Thanks, Steven Kaufer __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
