John,

> Our resources are not. We've also had specific requests to prevent
> > header bloat because it impacts the HTTP caching systems. Also, it's
> > pretty clear that headers are really not where you want to put volatile
> > information, which this is.
> Hmm, you do make a good point about caching.



Caching is useful only in such cases when you would like to return same
data many times.
In our case we are interested in latest state of resource, such kinds of
things can't be cached.


> I think we should step back here and figure out what the actual problem
> > is, and what ways we might go about solving it. This has jumped directly
> > to a point in time optimized fast poll loop. It will shave a few cycles
> > off right now on our current implementation, but will still be orders of
> > magnitude more costly that consuming the Nova notifications if the only
> > thing that is cared about is task state transitions. And it's an API
> > change we have to live with largely *forever* so short term optimization
> > is not what we want to go for.
> I do agree with that.


The thing here is that we have to have Async API, because we have long
running operations.
And basically there are 3 approaches to understand that operation is done:
1) pub/sub
2) polling resource status
3) long polling requests

All approaches have pros and cons, however the "actual" problem will stay
the same and you can't fix that..


Best regards,
Boris Pavlovic

On Thu, Nov 5, 2015 at 12:18 AM, John Garbutt <j...@johngarbutt.com> wrote:

> On 4 November 2015 at 15:00, Sean Dague <s...@dague.net> wrote:
> > On 11/04/2015 09:49 AM, Jay Pipes wrote:
> >> On 11/04/2015 09:32 AM, Sean Dague wrote:
> >>> On 11/04/2015 09:00 AM, Jay Pipes wrote:
> >>>> On 11/03/2015 05:20 PM, Boris Pavlovic wrote:
> >>>>> Hi stackers,
> >>>>>
> >>>>> Usually such projects like Heat, Tempest, Rally, Scalar, and other
> tool
> >>>>> that works with OpenStack are working with resources (e.g. VM,
> Volumes,
> >>>>> Images, ..) in the next way:
> >>>>>
> >>>>>   >>> resource = api.resouce_do_some_stuff()
> >>>>>   >>> while api.resource_get(resource["uuid"]) != expected_status
> >>>>>   >>>    sleep(a_bit)
> >>>>>
> >>>>> For each async operation they are polling and call many times
> >>>>> resource_get() which creates significant load on API and DB layers
> due
> >>>>> the nature of this request. (Usually getting full information about
> >>>>> resources produces SQL requests that contains multiple JOINs, e,g for
> >>>>> nova vm it's 6 joins).
> >>>>>
> >>>>> What if we add new API method that will just resturn resource status
> by
> >>>>> UUID? Or even just extend get request with the new argument that
> >>>>> returns
> >>>>> only status?
> >>>>
> >>>> +1
> >>>>
> >>>> All APIs should have an HTTP HEAD call on important resources for
> >>>> retrieving quick status information for the resource.
> >>>>
> >>>> In fact, I proposed exactly this in my Compute "vNext" API proposal:
> >>>>
> >>>> http://docs.oscomputevnext.apiary.io/#reference/server/serversid/head
> >>>>
> >>>> Swift's API supports HEAD for accounts:
> >>>>
> >>>>
> http://developer.openstack.org/api-ref-objectstorage-v1.html#showAccountMeta
> >>>>
> >>>>
> >>>>
> >>>> containers:
> >>>>
> >>>>
> http://developer.openstack.org/api-ref-objectstorage-v1.html#showContainerMeta
> >>>>
> >>>>
> >>>>
> >>>> and objects:
> >>>>
> >>>>
> http://developer.openstack.org/api-ref-objectstorage-v1.html#showObjectMeta
> >>>>
> >>>>
> >>>> So, yeah, I agree.
> >>>> -jay
> >>>
> >>> How would you expect this to work on "servers"? HEAD specifically
> >>> forbids returning a body, and, unlike swift, we don't return very much
> >>> information in our headers.
> >>
> >> I didn't propose doing it on a collection resource like "servers". Only
> >> on an entity resource like a single "server".
> >>
> >> HEAD /v2/{tenant}/servers/{uuid}
> >> HTTP/1.1 200 OK
> >> Content-Length: 1022
> >> Last-Modified: Thu, 16 Jan 2014 21:12:31 GMT
> >> Content-Type: application/json
> >> Date: Thu, 16 Jan 2014 21:13:19 GMT
> >> OpenStack-Compute-API-Server-VM-State: ACTIVE
> >> OpenStack-Compute-API-Server-Power-State: RUNNING
> >> OpenStack-Compute-API-Server-Task-State: NONE
> >
> > Right, but these headers aren't in the normal resource. They are
> > returned in the body only.
> >
> > The point of HEAD is give me the same thing as GET without the body,
> > because I only care about the headers. Swift resources are structured in
> > a way where this information is useful.
>
> I guess we would have to add this to GET requests, for consistency,
> which feels like duplication.
>
> > Our resources are not. We've also had specific requests to prevent
> > header bloat because it impacts the HTTP caching systems. Also, it's
> > pretty clear that headers are really not where you want to put volatile
> > information, which this is.
>
> Hmm, you do make a good point about caching.
>
> > I think we should step back here and figure out what the actual problem
> > is, and what ways we might go about solving it. This has jumped directly
> > to a point in time optimized fast poll loop. It will shave a few cycles
> > off right now on our current implementation, but will still be orders of
> > magnitude more costly that consuming the Nova notifications if the only
> > thing that is cared about is task state transitions. And it's an API
> > change we have to live with largely *forever* so short term optimization
> > is not what we want to go for.
>
> I do agree with that.
>
> > We should focus on the long term game here.
>
> The long term plan being the end user async API? Maybe using
> websockets, or similar?
> https://etherpad.openstack.org/p/liberty-cross-project-user-notifications
>
> Thanks,
> johnthetubaguy
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to