Re: [openstack-dev] [api][nova] Handling lots of GET query string parameters?

[Salvatore Orlando]

Regarding Jay's proposal, this would be tantamount to defining an API
action for retrieving instances, something currently being discussed here
[1].
The only comment I have is that I am not entirely surely whether using the
POST verb for operations which do no alter at all the server representation
of any object is in accordance with RFC 7231.
A search API like the one pointed out by Julien is interesting; at first
glance I'm not able to comment on its RESTfulness - it definitely has
plenty of use cases and enables users to run complex queries; one possible
downside is that it increases the complexity of simple queries.

For the purpose of the Nova spec I think it might be ok to limit the
functionality to a "small number of instance ids" as expressed in the spec.
On the other hand how crazy it would be to limit the number of bytes in the
URL by allowing to specify contract form of instance UUIDs - in a way
similar to git commits?

[1] https://review.openstack.org/#/c/234994/

On 4 November 2015 at 13:17, Sean Dague  wrote:

> On 11/03/2015 05:45 AM, Julien Danjou wrote:
> > On Tue, Nov 03 2015, Jay Pipes wrote:
> >
> >> My suggestion was to add a new POST /servers/search URI resource that
> can take
> >> a request body containing large numbers of filter arguments, encoded in
> a JSON
> >> object.
> >>
> >> API working group, what thoughts do you have about this? Please add your
> >> comments to the Gerrit spec patch if you have time.
> >
> > FWIW, we already have an extensive support for that in both Ceilometer
> > and Gnocchi. It looks like a small JSON query DSL that we're able to
> > "compile" down to SQL Alchemy filters.
> >
> > A few examples are:
> >
> http://docs.openstack.org/developer/gnocchi/rest.html#searching-for-resources
> >
> > I've planed for a long time to move this code to a library, so if Nova's
> > interested, I can try to move that forward eagerly.
>
> I guess I wonder what the expected interaction with things like
> Searchlight is? Searchlight was largely created for providing this kind
> of fast access to subsets of resources based on arbitrary attribute search.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][nova] Handling lots of GET query string parameters?

[Cory Benfield]

> On 4 Nov 2015, at 13:13, Salvatore Orlando  wrote:
> 
> Regarding Jay's proposal, this would be tantamount to defining an API action 
> for retrieving instances, something currently being discussed here [1].
> The only comment I have is that I am not entirely surely whether using the 
> POST verb for operations which do no alter at all the server representation 
> of any object is in accordance with RFC 7231.

It’s totally fine, so long as you define things appropriately. Jay’s suggestion 
does exactly that, and is entirely in line with RFC 7231.

The analogy here is to things like complex search forms. Many search engines 
allow you to construct very complex search queries (consider something like 
Amazon or eBay, where you can filter on all kinds of interesting criteria). 
These forms are often submitted to POST endpoints rather than GET.

This is totally fine. In fact, the first example from RFC 7231 Section 4.3.3 
(POST) applies here: “POST is used for the following functions (among others): 
Providing a block of data […] to a data-handling process”. In this case, the 
data-handling function is the search function on the server.

The *only* downside of Jay’s approach is that the response cannot really be 
cached. It’s not clear to me whether anyone actually deploys a cache in this 
kind of role though, so it may not hurt too much.

Cory




signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][nova] Handling lots of GET query string parameters?

[Sean Dague]

On 11/03/2015 05:45 AM, Julien Danjou wrote:
> On Tue, Nov 03 2015, Jay Pipes wrote:
> 
>> My suggestion was to add a new POST /servers/search URI resource that can 
>> take
>> a request body containing large numbers of filter arguments, encoded in a 
>> JSON
>> object.
>>
>> API working group, what thoughts do you have about this? Please add your
>> comments to the Gerrit spec patch if you have time.
> 
> FWIW, we already have an extensive support for that in both Ceilometer
> and Gnocchi. It looks like a small JSON query DSL that we're able to
> "compile" down to SQL Alchemy filters.
> 
> A few examples are:
>   
> http://docs.openstack.org/developer/gnocchi/rest.html#searching-for-resources
> 
> I've planed for a long time to move this code to a library, so if Nova's
> interested, I can try to move that forward eagerly.

I guess I wonder what the expected interaction with things like
Searchlight is? Searchlight was largely created for providing this kind
of fast access to subsets of resources based on arbitrary attribute search.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][nova] Handling lots of GET query string parameters?

[Salvatore Orlando]

Inline,
Salvatore

On 4 November 2015 at 15:11, Cory Benfield  wrote:

>
> > On 4 Nov 2015, at 13:13, Salvatore Orlando 
> wrote:
> >
> > Regarding Jay's proposal, this would be tantamount to defining an API
> action for retrieving instances, something currently being discussed here
> [1].
> > The only comment I have is that I am not entirely surely whether using
> the POST verb for operations which do no alter at all the server
> representation of any object is in accordance with RFC 7231.
>
> It’s totally fine, so long as you define things appropriately. Jay’s
> suggestion does exactly that, and is entirely in line with RFC 7231.
>
> The analogy here is to things like complex search forms. Many search
> engines allow you to construct very complex search queries (consider
> something like Amazon or eBay, where you can filter on all kinds of
> interesting criteria). These forms are often submitted to POST endpoints
> rather than GET.
>
> This is totally fine. In fact, the first example from RFC 7231 Section
> 4.3.3 (POST) applies here: “POST is used for the following functions (among
> others): Providing a block of data […] to a data-handling process”. In this
> case, the data-handling function is the search function on the server.
>

I looked back at the RFC and indeed it does not state anywhere that a POST
operation is required to change somehow the state of any object, so the
approach is entirely fine from this aspect as well.


>
> The *only* downside of Jay’s approach is that the response cannot really
> be cached. It’s not clear to me whether anyone actually deploys a cache in
> this kind of role though, so it may not hurt too much.
>

I believe there will be not a great advantage from caching this kind of
responses, as cache hits would be very low anyway.


> Cory
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [api][nova] Handling lots of GET query string parameters?

[Jay Pipes]

Hi all,

A spec [1] that proposes adding a new server_ids query string parameter 
to the existing GET /servers/detail URI resource has highlighted an 
interesting issue.


The point of the spec is to add an ability to filter the results for the 
GET /servers/detail API call to a specified set of instance UUIDs. 
However, Tony Breeds points out that there will be a rather small limit 
(~55 or so, maximum) on the number of UUIDs that can be specified in the 
query parameters due to length limitations of the URI.


My suggestion was to add a new POST /servers/search URI resource that 
can take a request body containing large numbers of filter arguments, 
encoded in a JSON object.


API working group, what thoughts do you have about this? Please add your 
comments to the Gerrit spec patch if you have time.


Thank you!
-jay

[1] https://review.openstack.org/#/c/239286/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][nova] Handling lots of GET query string parameters?

[Julien Danjou]

On Tue, Nov 03 2015, Jay Pipes wrote:

> My suggestion was to add a new POST /servers/search URI resource that can take
> a request body containing large numbers of filter arguments, encoded in a JSON
> object.
>
> API working group, what thoughts do you have about this? Please add your
> comments to the Gerrit spec patch if you have time.

FWIW, we already have an extensive support for that in both Ceilometer
and Gnocchi. It looks like a small JSON query DSL that we're able to
"compile" down to SQL Alchemy filters.

A few examples are:
  http://docs.openstack.org/developer/gnocchi/rest.html#searching-for-resources

I've planed for a long time to move this code to a library, so if Nova's
interested, I can try to move that forward eagerly.

-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev