[
https://issues.apache.org/jira/browse/JCLOUDS-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Filip Panovski updated JCLOUDS-1571:
------------------------------------
Description:
(Disclaimer: I'm not 100% sure whether this is a bug report or a feature
request, as I've found only scant information available)
We have the use case where we wish to provide a signed aws-s3 {{PUT}} URL to a
browser client which will be used to process a file upload (non-chunked, single
URL). We are using version {{2.3.0}} (the latest public release as of this
writing).
The {{jclouds}} API has support for this via {{BlobStoreContext#getSigner}} ,
i.e. {{BlobRequestSigner#signPutBlob(String containerName, Blob blob, long
timeInSeconds)}}.
However, there seems to be no support for signing custom headers as part of the
URL.
The relevant AWS Documentation (link
[here|https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html])
states that:
{code:java}
For added security, you should sign all the request headers that you plan to
include in your request.
{code}
I dug around the {{jclouds-core}} code and saw that, for pre-signed {{PUT}}
requests, a {{PhantomPayload}} is used which appears, for all intents and
purposes, as a real payload with checksum and content-length.
However, although this information is available at the time of signing, the
only headers which are part of the signed headers are those which are either
specifically annotated as such (e.g.: {{org.jclouds.s3.S3Client#putObject}},
which contains {{@Headers(keys = EXPECT, values = "100-continue"}}) or are
otherwise part of normal request flow depending on media type (i.e.
{{content-type}}, {{accept}}).
The way I access the API is as follows:
{code:java}
@Override
public URI getPreSignedPutUrl(final String blobId, final long contentLength,
final String checksum,
final String mediaType) {
BlobStoreContext context = blobStoreContextProvider.getBlobStoreContext();
BlobRequestSigner signer = context.getSigner();
Blob expectedBlob = context.getBlobStore().blobBuilder(blobId)
.forSigning()
.contentType(mediaType)
.contentMD5(HashCode.fromString(checksum))
.contentLength(contentLength)
.build();
HttpRequest request = signer.signPutBlob(this.bucket.getName(),
expectedBlob, 60);
return request.getEndpoint();
}
{code}
What I _expected_ to happen was:
# A pre-signed URL is created with the blob I pass it.
# The blob's content-md5 and content-length values are passed through as part
of the signed headers.
# The pre-signed URL includes these values so that only a specific file (or
something that has exactly that checksum and content length) can be uploaded to
said URL.
What _did_ happen was:
# A pre-signed URL is created with the blob I pass it.
# {{RestAnnotationProcessor}} and the default {{S3Client}} API both seem to
not take any of the blob metadata into account
# The pre-signed URL includes only the {{host}} header as a signed header.
As a result, I can upload any file I wish to the pre-signed {{PUT}} URL, not
just the one that I specifically intended, although metadata like content
checksum and length is available and could also be signed within the request.
For what it's worth, I did try to extend the existing {{BlogRequestSigner}} to
include {{PutOptions}} (which have an effect on signed headers, as far as I can
tell, see:
{{[RestAnnotationProcessor#apply:282|https://github.com/apache/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java#L282]}}),
but got lots of Guice binding errors that I was unable to resolve. I am
relatively new to Guice, so that might have been user error - as the
documentation is lacking though, I'm unsure if that's the preferred way.
Lastly, https://issues.apache.org/jira/browse/JCLOUDS-1161 exists which seems
loosely related to my issue. As I understand it though, the "workaround"
described therein should no longer be necessary (since V4 is automatically used
for {{aws-s3}} since ~{{2.2.0}}) and it does not help in my case either as it
has more to do with which signer is used, and not with the tuning of the
resulting URL.
Is the current behaviour, as described above, correct? Is the lack of a support
for additional signed headers intentional? Is there another way that I'm not
quite seeing?
Any feedback would be greatly appreciated.
was:
(Disclaimer: I'm not 100% sure whether this is a bug report or a feature
request, as I've found only scant information available)
We have the use case where we wish to provide a signed aws-s3 {{PUT}} URL to a
browser client which will be used to process a file upload (non-chunked, single
URL). We are using version {{2.3.0}} (the latest public release as of this
writing).
The {{jclouds}} API has support for this via {{BlobStoreContext#getSigner}} ,
i.e. {{BlobRequestSigner#signPutBlob(String containerName, Blob blob, long
timeInSeconds)}}.
However, there seems to be no support for signing custom headers as part of the
URL.
The relevant AWS Documentation (link
[here|https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html])
states that:
{code:java}
For added security, you should sign all the request headers that you plan to
include in your request.
{code}
I dug around the {{jclouds-core}} code and saw that, for pre-signed {{PUT}}
requests, a {{PhantomPayload}} is used which appears, for all intents and
purposes, as a real payload with checksum and content-length.
However, although this information is available at the time of signing, the
only headers which are part of the signed headers are those which are either
specifically annotated as such (e.g.: {{org.jclouds.s3.S3Client#putObject}},
which contains {{@Headers(keys = EXPECT, values = "100-continue"}}) or are
otherwise part of normal request flow depending on media type (i.e.
{{content-type}}, {{accept}}).
The way I access the API is as follows:
{code:java}
@Override
public URI getPreSignedPutUrl(final String blobId, final long contentLength,
final String checksum,
final String mediaType) {
BlobStoreContext context = blobStoreContextProvider.getBlobStoreContext();
BlobRequestSigner signer = context.getSigner();
Blob expectedBlob = context.getBlobStore().blobBuilder(blobId)
.forSigning()
.contentType(mediaType)
.contentMD5(HashCode.fromString(checksum))
.contentLength(contentLength)
.build();
HttpRequest request = signer.signPutBlob(this.bucket.getName(),
expectedBlob, 60);
return request.getEndpoint();
}
{code}
What I _expected_ to happen was:
# A pre-signed URL is created with the blob I pass it.
# The blob's content-md5 and content-length values are passed through as part
of the signed headers.
# The pre-signed URL includes these values so that only a specific file (or
something that has exactly that checksum and content length) can be uploaded to
said URL.
What _did_ happen was:
# A pre-signed URL is created with the blob I pass it.
# {{RestAnnotationProcessor}} and the default {{S3Client}} API both seem to
not take any of the blob metadata into account
# The pre-signed URL includes only the {{host}} header as a signed header.
As a result, I can upload any file I wish to the pre-signed {{PUT}} URL, not
just the one that I specifically intended, although metadata like content
checksum and length is available and could also be signed within the request.
For what it's worth, I did try to extend the existing {{BlogRequestSigner}} to
include {{PutOptions}} (which have an effect on signed headers, as far as I can
tell, see:
{{[RestAnnotationProcessor#apply:282|https://github.com/apache/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java#L282]}}),
but got lots of Guice binding errors that I was unable to resolve. I am
relatively new to Guice, so that might have been user error - as the
documentation is lacking though, I'm unsure if that's the preferred way.
Lastly, https://issues.apache.org/jira/browse/JCLOUDS-1161 exists which seems
loosely related to my issue. As I understand it though, the "workaround"
described therein should no longer be necessary (since V4 is automatically used
for {{aws-s3}} since ~{{2.2.0}}) and it does not help in my case either.
Is the current behaviour, as described above, correct? Is the lack of a support
for additional signed headers intentional? Is there another way that I'm not
quite seeing?
Any feedback would be greatly appreciated.
> Unable to provide custom signed headers using BlobRequestSigner#signPutBlob
> ---------------------------------------------------------------------------
>
> Key: JCLOUDS-1571
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1571
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore, jclouds-core
> Affects Versions: 2.3.0
> Environment: Debian
> Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
> Reporter: Filip Panovski
> Priority: Major
>
> (Disclaimer: I'm not 100% sure whether this is a bug report or a feature
> request, as I've found only scant information available)
> We have the use case where we wish to provide a signed aws-s3 {{PUT}} URL to
> a browser client which will be used to process a file upload (non-chunked,
> single URL). We are using version {{2.3.0}} (the latest public release as of
> this writing).
> The {{jclouds}} API has support for this via {{BlobStoreContext#getSigner}} ,
> i.e. {{BlobRequestSigner#signPutBlob(String containerName, Blob blob, long
> timeInSeconds)}}.
> However, there seems to be no support for signing custom headers as part of
> the URL.
> The relevant AWS Documentation (link
> [here|https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html])
> states that:
> {code:java}
> For added security, you should sign all the request headers that you plan to
> include in your request.
> {code}
> I dug around the {{jclouds-core}} code and saw that, for pre-signed {{PUT}}
> requests, a {{PhantomPayload}} is used which appears, for all intents and
> purposes, as a real payload with checksum and content-length.
> However, although this information is available at the time of signing, the
> only headers which are part of the signed headers are those which are either
> specifically annotated as such (e.g.: {{org.jclouds.s3.S3Client#putObject}},
> which contains {{@Headers(keys = EXPECT, values = "100-continue"}}) or are
> otherwise part of normal request flow depending on media type (i.e.
> {{content-type}}, {{accept}}).
> The way I access the API is as follows:
> {code:java}
> @Override
> public URI getPreSignedPutUrl(final String blobId, final long contentLength,
> final String checksum,
> final String mediaType) {
> BlobStoreContext context = blobStoreContextProvider.getBlobStoreContext();
> BlobRequestSigner signer = context.getSigner();
> Blob expectedBlob = context.getBlobStore().blobBuilder(blobId)
> .forSigning()
> .contentType(mediaType)
> .contentMD5(HashCode.fromString(checksum))
> .contentLength(contentLength)
> .build();
> HttpRequest request = signer.signPutBlob(this.bucket.getName(),
> expectedBlob, 60);
> return request.getEndpoint();
> }
> {code}
> What I _expected_ to happen was:
> # A pre-signed URL is created with the blob I pass it.
> # The blob's content-md5 and content-length values are passed through as
> part of the signed headers.
> # The pre-signed URL includes these values so that only a specific file (or
> something that has exactly that checksum and content length) can be uploaded
> to said URL.
> What _did_ happen was:
> # A pre-signed URL is created with the blob I pass it.
> # {{RestAnnotationProcessor}} and the default {{S3Client}} API both seem to
> not take any of the blob metadata into account
> # The pre-signed URL includes only the {{host}} header as a signed header.
> As a result, I can upload any file I wish to the pre-signed {{PUT}} URL, not
> just the one that I specifically intended, although metadata like content
> checksum and length is available and could also be signed within the request.
> For what it's worth, I did try to extend the existing {{BlogRequestSigner}}
> to include {{PutOptions}} (which have an effect on signed headers, as far as
> I can tell, see:
> {{[RestAnnotationProcessor#apply:282|https://github.com/apache/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java#L282]}}),
> but got lots of Guice binding errors that I was unable to resolve. I am
> relatively new to Guice, so that might have been user error - as the
> documentation is lacking though, I'm unsure if that's the preferred way.
> Lastly, https://issues.apache.org/jira/browse/JCLOUDS-1161 exists which seems
> loosely related to my issue. As I understand it though, the "workaround"
> described therein should no longer be necessary (since V4 is automatically
> used for {{aws-s3}} since ~{{2.2.0}}) and it does not help in my case either
> as it has more to do with which signer is used, and not with the tuning of
> the resulting URL.
> Is the current behaviour, as described above, correct? Is the lack of a
> support for additional signed headers intentional? Is there another way that
> I'm not quite seeing?
> Any feedback would be greatly appreciated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
