[
https://issues.apache.org/jira/browse/JCLOUDS-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Filip Panovski updated JCLOUDS-1571:
------------------------------------
Description:
(Disclaimer: I'm not 100% sure whether this is a bug report or a feature
request, as I've found only scant information available)
We have the use case where we wish to provide a signed aws-s3 {{PUT}} URL to a
browser client which will be used to process a file upload (non-chunked, single
URL). We are using version {{2.3.0}} (the latest public release as of this
writing).
The {{jclouds}} API has support for this via {{BlobStoreContext#getSigner}} ,
i.e. {{BlobRequestSigner#signPutBlob(String containerName, Blob blob, long
timeInSeconds)}}.
However, there seems to be no support for signing custom headers as part of the
URL.
The relevant AWS Documentation (link
[here|https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html])
states that:
{code:java}
For added security, you should sign all the request headers that you plan to
include in your request.
{code}
I dug around the {{jclouds-core}} code and saw that, for pre-signed {{PUT}}
requests, a {{PhantomPayload}} is used which appears, for all intents and
purposes, as a real payload with checksum and content-length.
However, although this information is available at the time of signing, the
only headers which are part of the signed headers are those which are either
specifically annotated as such (e.g.: {{org.jclouds.s3.S3Client#putObject}},
which contains {{@Headers(keys = EXPECT, values = "100-continue"}}) or are
otherwise part of normal request flow depending on media type (i.e.
{{content-type}}, {{accept}}).
The way I access the API is as follows:
{code:java}
@Override
public URI getPreSignedPutUrl(final String blobId, final long contentLength,
final String checksum,
final String mediaType) {
BlobStoreContext context = blobStoreContextProvider.getBlobStoreContext();
BlobRequestSigner signer = context.getSigner();
Blob expectedBlob = context.getBlobStore().blobBuilder(blobId)
.forSigning()
.contentType(mediaType)
.contentMD5(HashCode.fromString(checksum))
.contentLength(contentLength)
.build();
HttpRequest request = signer.signPutBlob(this.bucket.getName(),
expectedBlob, 60);
return request.getEndpoint();
}
{code}
What I _expected_ to happen was:
# A pre-signed URL is created with the blob I pass it.
# The blob's content-md5 and content-length values are passed through as part
of the signed headers.
# The pre-signed URL includes these values so that only a specific file (or
something that has exactly that checksum and content length) can be uploaded to
said URL.
What _did_ happen was:
# A pre-signed URL is created with the blob I pass it.
# {{RestAnnotationProcessor}} and the default {{S3Client}} API both seem to
not take any of the blob metadata into account
# The pre-signed URL includes only the {{host}} header as a signed header.
As a result, I can upload any file I wish to the pre-signed {{PUT}} URL, not
just the one that I specifically intended, although metadata like content
checksum and length is available and could also be signed within the request.
For what it's worth, I did try to extend the existing {{BlogRequestSigner}} to
include {{PutOptions}} (which have an effect on signed headers, as far as I can
tell, see:
{{[RestAnnotationProcessor#apply:282|https://github.com/apache/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java#L282]}}),
but got lots of Guice binding errors that I was unable to resolve. I am
relatively new to Guice, so that might have been user error - as the
documentation is lacking though, I'm unsure if that's the preferred way.
Lastly, https://issues.apache.org/jira/browse/JCLOUDS-1161 exists which seems
loosely related to my issue. As I understand it though, the "workaround"
described therein should no longer be necessary (since V4 is automatically used
for {{aws-s3}} since ~{{2.2.0}}) and it does not help in my case either.
Is the current behaviour, as described above, correct? Is the lack of a support
for additional signed headers intentional? Is there another way that I'm not
quite seeing?
Any feedback would be greatly appreciated.
was:
(Disclaimer: I'm not 100% sure whether this is a bug report or a feature
request, as I've found only scant information available)
We have the use case where we wish to provide a signed aws-s3 {{PUT}} URL to a
browser client which will be used to process a file upload (non-chunked, single
URL). We are using version {{2.3.0}} (the latest public release).
The {{jclouds}} API has support for this via {{BlobStoreContext#getSigner}} ,
i.e. {{BlobRequestSigner#signPutBlob(String containerName, Blob blob, long
timeInSeconds)}}.
However, there seems to be no support for signing custom headers as part of
the URL.
The relevant AWS Documentation (link
[here|https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html])
states that:
{code:java}
For added security, you should sign all the request headers that you plan to
include in your request.
{code}
I dug around the {{jclouds-core}} code and saw that, for pre-signed {{PUT}}
requests, a {{PhantomPayload}} is used which appears, for all intents and
purposes, as a real payload with checksum and content-length.
However, although this information is available at the time of signing, the
only headers which are part of the signed headers are those which are either
specifically annotated as such (e.g.: {{org.jclouds.s3.S3Client#putObject}},
which contains {{@Headers(keys = EXPECT, values = "100-continue"}}) or are
otherwise part of normal request flow depending on media type (i.e.
{{content-type}}, {{accept}}).
The way I access the API is as follows:
{code:java}
@Override
public URI getPreSignedPutUrl(final String blobId, final long contentLength,
final String checksum,
final String mediaType) {
BlobStoreContext context = blobStoreContextProvider.getBlobStoreContext();
BlobRequestSigner signer = context.getSigner();
Blob expectedBlob = context.getBlobStore().blobBuilder(blobId)
.forSigning()
.contentType(mediaType)
.contentMD5(HashCode.fromString(checksum))
.contentLength(contentLength)
.build();
HttpRequest request = signer.signPutBlob(this.bucket.getName(),
expectedBlob, 60);
return request.getEndpoint();
}
{code}
What I _expected_ to happen was:
# A pre-signed URL is created with the blob I pass it.
# The blob's content-md5 and content-length values are passed through as part
of the signed headers.
# The pre-signed URL includes these values so that only a specific file (or
something that has exactly that checksum and content length) can be uploaded to
said URL.
What _did_ happen was:
# A pre-signed URL is created with the blob I pass it.
# {{RestAnnotationProcessor}} and the default {{S3Client}} API both seem to
not take any of the blob metadata into account
# The pre-signed URL includes only the {{host}} header as a signed header.
As a result, I can upload any file I wish to the pre-signed {{PUT}} URL, not
just the one that I specifically intended, although metadata like content
checksum and length is available and could also be signed within the request.
For what it's worth, I did try to extend the existing {{BlogRequestSigner}} to
include {{PutOptions}} (which have an effect on signed headers, as far as I can
tell, see:
{{[RestAnnotationProcessor#apply:282|https://github.com/apache/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java#L282]}}),
but got lots of Guice binding errors that I was unable to resolve. I am
relatively new to Guice, so that might have been user error - as the
documentation is lacking though, I'm unsure if that's the preferred way.
Lastly, https://issues.apache.org/jira/browse/JCLOUDS-1161 exists which seems
loosely related to my issue. As I understand it though, the "workaround"
described therein should no longer be necessary (since V4 is automatically used
for {{aws-s3}} since ~{{2.2.0}}) and it does not help in my case either.
Is the current behaviour, as described above, correct? Is the lack of a support
for additional signed headers intentional? Is there another way that I'm not
quite seeing?
Any feedback would be greatly appreciated.
> Unable to provide custom signed headers using BlobRequestSigner#signPutBlob
> ---------------------------------------------------------------------------
>
> Key: JCLOUDS-1571
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1571
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore, jclouds-core
> Affects Versions: 2.3.0
> Environment: Debian
> Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
> Reporter: Filip Panovski
> Priority: Major
>
> (Disclaimer: I'm not 100% sure whether this is a bug report or a feature
> request, as I've found only scant information available)
> We have the use case where we wish to provide a signed aws-s3 {{PUT}} URL to
> a browser client which will be used to process a file upload (non-chunked,
> single URL). We are using version {{2.3.0}} (the latest public release as of
> this writing).
> The {{jclouds}} API has support for this via {{BlobStoreContext#getSigner}} ,
> i.e. {{BlobRequestSigner#signPutBlob(String containerName, Blob blob, long
> timeInSeconds)}}.
> However, there seems to be no support for signing custom headers as part of
> the URL.
> The relevant AWS Documentation (link
> [here|https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html])
> states that:
> {code:java}
> For added security, you should sign all the request headers that you plan to
> include in your request.
> {code}
> I dug around the {{jclouds-core}} code and saw that, for pre-signed {{PUT}}
> requests, a {{PhantomPayload}} is used which appears, for all intents and
> purposes, as a real payload with checksum and content-length.
> However, although this information is available at the time of signing, the
> only headers which are part of the signed headers are those which are either
> specifically annotated as such (e.g.: {{org.jclouds.s3.S3Client#putObject}},
> which contains {{@Headers(keys = EXPECT, values = "100-continue"}}) or are
> otherwise part of normal request flow depending on media type (i.e.
> {{content-type}}, {{accept}}).
> The way I access the API is as follows:
> {code:java}
> @Override
> public URI getPreSignedPutUrl(final String blobId, final long contentLength,
> final String checksum,
> final String mediaType) {
> BlobStoreContext context = blobStoreContextProvider.getBlobStoreContext();
> BlobRequestSigner signer = context.getSigner();
> Blob expectedBlob = context.getBlobStore().blobBuilder(blobId)
> .forSigning()
> .contentType(mediaType)
> .contentMD5(HashCode.fromString(checksum))
> .contentLength(contentLength)
> .build();
> HttpRequest request = signer.signPutBlob(this.bucket.getName(),
> expectedBlob, 60);
> return request.getEndpoint();
> }
> {code}
> What I _expected_ to happen was:
> # A pre-signed URL is created with the blob I pass it.
> # The blob's content-md5 and content-length values are passed through as
> part of the signed headers.
> # The pre-signed URL includes these values so that only a specific file (or
> something that has exactly that checksum and content length) can be uploaded
> to said URL.
> What _did_ happen was:
> # A pre-signed URL is created with the blob I pass it.
> # {{RestAnnotationProcessor}} and the default {{S3Client}} API both seem to
> not take any of the blob metadata into account
> # The pre-signed URL includes only the {{host}} header as a signed header.
> As a result, I can upload any file I wish to the pre-signed {{PUT}} URL, not
> just the one that I specifically intended, although metadata like content
> checksum and length is available and could also be signed within the request.
> For what it's worth, I did try to extend the existing {{BlogRequestSigner}}
> to include {{PutOptions}} (which have an effect on signed headers, as far as
> I can tell, see:
> {{[RestAnnotationProcessor#apply:282|https://github.com/apache/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java#L282]}}),
> but got lots of Guice binding errors that I was unable to resolve. I am
> relatively new to Guice, so that might have been user error - as the
> documentation is lacking though, I'm unsure if that's the preferred way.
> Lastly, https://issues.apache.org/jira/browse/JCLOUDS-1161 exists which seems
> loosely related to my issue. As I understand it though, the "workaround"
> described therein should no longer be necessary (since V4 is automatically
> used for {{aws-s3}} since ~{{2.2.0}}) and it does not help in my case either.
> Is the current behaviour, as described above, correct? Is the lack of a
> support for additional signed headers intentional? Is there another way that
> I'm not quite seeing?
> Any feedback would be greatly appreciated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
