Jay, Flavio, thanks for this interesting discussion. I get your points and they really make sense to me.
I'll go for a specific driver that will inherits from the HTTP Store for the read path and implements the write path. Jordan On Tue, Feb 17, 2015 at 12:52 PM, Flavio Percoco <[email protected]> wrote: > On 13/02/15 17:01 +0100, Jordan Pittier wrote: > >> Humm this doesn't have to be complicated, for a start. >> >> > Sorry for my late reply > > - Figuring out the http method the server expects (POST/PUT) >>> >> Yeah, I agree. There"s no definitive answer to this but I think PUT makes >> sense >> here. I googled 'post vs put' and I found that the "idempotent" and "who >> is in >> charge of the actual resource location choice" (the client vs the server), >> favors PUT. >> > > Right but that's not what the remote server may be expecting. One of > the problems about the HTTP store is that there's not real "API" > besides what the HTTP protocol allows you to do. That is to say that a > remote server may accept POST/PUT and in order to keep the > implementation non-opinionated, you'd need to have a way for these > things to be specified. > > >> - Adding support for at least few HTTP auth methods >>> >> Why should the "write" path be more secured/more flexible than the read >> path ? >> If I take a look at the current HTTP store, only basic auth is supported >> (ie >> http://user:pass@server1/myLinuxImage). I suggest the write path (ie the >> add() >> method) should support the same auth mecanism. The cloud admin could also >> add >> some firewall rules to make sure the HTTP backend server can only be >> accessed >> by the Glance-api servers. >> > > I didn't say the read path was correct :P > > That said, I agree that we should keep both paths consistent. > > - Having a sufixed URL where we're sure glance will have proper >>> permissions to upload data. >>> >> That's up the the cloud admin/operator to make it work. The HTTP >> glance_store >> could have 2 config flags : >> a) "http_server", a string with the scheme (http vs https) and the >> hostname of >> the HTTP server, ie 'http://server1' >> b) "path_prefix". A string that will prefix the "path" part of the image >> URL. >> This config flag could be left empty/is optional. >> > > Yes, it was probably not clear from my previous email that these were > not "ands" but things that would need to be brought up. > > >> Handling HTTP responses from the server >>> >> That's of course to be discussed. But, IMO, this should be as simple as >> "if >> response.code is 200 or 202 then OKAY else raise GlanceStoreException". I >> am >> not sure any other glance store is more "granular" than this. >> > > Again, this assumes "too much" from the server. So, either we impose > some kind of rules as to how Glance expects the HTTP server to behave > or we try to be bullet proof API-wise. > > How can we handle quota? >>> >> I am new to glance_store, is there a notion of quotas in glance stores ? I >> though Glance (API) was handling this. What kind of quotas are we talking >> about >> here ? >> > > Glance handles quotas. The problem is that when the data is sent to > the remote store, glance looses some control on it. A user may upload > some data, the HTTP push could fail and we may try to delete the data > without any proof that it will be correctly deleted. > > Also, without auth, we will have to force the user to send all image > data through glance. The reason is that we don't know whether the HTTP > store has support for HEAD to report the image size when using > `--location`. > > Sorry if all the above sounds confusing. The problem with the HTTP > store is that we have basically no control over it and that is > worrisome from a security and implementation perspective. > > Flavio > > > Frankly, it shouldn't add that much code. I feel we can make it clean if >> we >> leverage the different Python modules (httplib etc.) >> >> Regards, >> Jordan >> >> >> On Fri, Feb 13, 2015 at 4:20 PM, Flavio Percoco <[email protected]> >> wrote: >> >> On 13/02/15 16:01 +0100, Jordan Pittier wrote: >> >> What is the difference between just calling the Glance API to >> upload an image, >> >> versus adding add() functionality to the HTTP image store? >> You mean using "glance image-create --location http://server1/ >> myLinuxImage [..] >> " ? If so, I guess adding the add() functionality will save the >> user >> from >> having to find the right POST curl/wget command to properly upload >> his >> image. >> >> >> I believe it's more complex than this. Having an `add` method for the >> HTTP store implies: >> >> - Figuring out the http method the server expects (POST/PUT) >> - Adding support for at least few HTTP auth methods >> - Having a sufixed URL where we're sure glance will have proper >> permissions to upload data. >> - Handling HTTP responses from the server w.r.t the status of the data >> upload. For example: What happens if the remote http server runs out >> of space? What's the response status going to be like? How can we >> make glance agnostic to these discrepancies across HTTP servers so >> that it's consistent in its responses to glance users? >> - How can we handle quota? >> >> I'm not fully opposed, although it sounds like not worth it code-wise, >> maintenance-wise and performance-wise. The user will have to run just >> 1 command but at the cost of all of the above. >> >> Do the points listed above make sense to you? >> >> Cheers, >> Flavio >> >> >> >> >> On Fri, Feb 13, 2015 at 3:55 PM, Jay Pipes <[email protected]> >> wrote: >> >> On 02/13/2015 09:47 AM, Jordan Pittier wrote: >> Hi list, >> >> I would like to add the 'add' capability to the HTTP glance >> store. >> >> Let's say I (as an operator or cloud admin) provide an HTTP >> server >> where >> (authenticated/trusted) users/clients can make the following >> HTTP >> request : >> >> POST http://server1/myLinuxImage HTTP/1.1 >> Host: server1 >> Content-Length: 256000000 >> Content-Type: application/octet-stream >> >> mybinarydata[..] >> >> Then the HTTP server will store the binary data, somewhere >> (for >> instance >> locally), some how (for instance in a plain file), so that >> the >> data is >> later on accessible by a simple GET >> http://server1/myLinuxImage >> >> In that case, this HTTP server could easily be a full >> fleshed >> Glance >> store. >> >> Questions : >> 1) Has this been already discussed/proposed ? If so, could >> someone give >> me a pointer to this work ? >> 2) Can I start working on this ? (the 2 main work items are >> : >> 'add an >> add method to glance_store._drivers.http.__Store' and 'add >> a >> delete >> method to glance_store._drivers.http.__Store (HTTP DELETE >> method)' >> >> >> What is the difference between just calling the Glance API to >> upload >> an >> image, versus adding add() functionality to the HTTP image >> store? >> >> Best, >> -jay >> >> >> ___________________________________________________________ >> _______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]? >> subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/ >> openstack-dev >> >> >> >> >> ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]? >> subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> @flaper87 >> Flavio Percoco >> ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject: >> unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> > ____________________________________________________________ >> ______________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject: >> unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > -- > @flaper87 > Flavio Percoco > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
