On 12/9/2015 12:01 PM, Boris Schrijver wrote:
I was trying out a few things in qemu-img, a virtualisation utility which
depends on libcurl. And with the signed-urls recently becoming more common, I
stumbled upon the following issue. The signed-urls I will be talking about are
for S3 [1].
A signed-url can have it's HTTP method be included in the signature, so a
signed-url with the GET method included in it's signature, when used by a HTTP
HEAD method will return a 403 forbidden.
Program's like qemu-img will want to first get the Content-Length. Normally this
would mean a HEAD request. But that's not possible, hence the signed-url.
I don't understand why you're signing for the GET method when the HEAD
method is what you're supposed to be using and therefore is what is
supposed to be signed, isn't it? You sign the method the client is using
(if you accept it), at least that's my interpretation of [1]. Do you
have code that shows how you're doing this?
Or do you have a third party somewhere that signs the request for you,
and that's why you can only use GET? In other words, you are saying that
a third party giving you access to one of their Amazon resources gives
you an authentication request signed for GET even when you use HEAD? (I
tried this just now with GitHub => AmazonAWS and they do that, hence my
hunch).
What I basically want is: curl -X GET -I http://random.url/object
If you can't use HEAD for no body I guess you could try limiting the
body to 1 via Range like curl -r 0-0. If it works you'll get 206 and a
reply header like Content-Range: bytes 0-0/1048576 so you can get the
size from that. I'd stay away from -X if you can [2].
I tried to implement it in qemu-img [2] but ended up always getting a
CURLE_WRITE_ERROR, due to the body not being written anywhere.
Is it possible to include a curl_easy_setopt() option to discard the response
body and stop the connection after the HEADER? So that the curl_easy_perform()
will still return 0 on success, and we don't need to check for a
CURLE_WRITE_ERROR?
[1]
http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#ConstructingTheAuthenticationHeader
[2] https://lists.nongnu.org/archive/html/qemu-devel/2015-12/msg01131.html
I'd guess that should be handled in qemu, it's not expecting a body so
if it's going to receive something as the body they'll have to deal with
that. In other words if you are looking to make a modification I think
you should focus on qemu not libcurl.
[1]: https://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html
[2]: http://daniel.haxx.se/blog/2015/09/11/unnecessary-use-of-curl-x/
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
