Some background, there is a generic s3 api with most of the
functionality and an aws-s3 provider which inherits from the former and
overrides the endpoints. The aws-s3 provider also overrides the signer
that you can see in AWSS3HttpApiModule.bindRequestSigner and
AWSS3BlobStoreContextModule.bindRequestSigner. Instead
S3HttpApiModule.bindRequestSigner and
S3BlobStoreContextModule.bindRequestSigner should set this conditionally
based on a property. The Guice declarative style is confusing but you
can see some examples like @Named(Constants.PROPERTY_SESSION_INTERVAL)
which allow setting values based on a default or a user-provided value.
Fixing this will require moving AWSS3BlobRequestSignerV4 and its tests
from the aws-s3 provider to the s3 api.
jclouds has two styles of tests. You can run the unit tests via:
mvn test -pl :s3,:aws-s3
and the integration tests via:
mvn integration-test -pl :s3 -Plive \
-Dtest.s3.endpoint="${JCLOUDS_ENDPOINT}" \
-Dtest.s3.identity="${JCLOUDS_IDENTITY}" \
-Dtest.s3.credential="${JCLOUDS_CREDENTIAL}"
mvn integration-test -pl :aws-s3 -Plive \
-Dtest.aws-s3.identity="${JCLOUDS_IDENTITY}" \
-Dtest.aws-s3.credential="${JCLOUDS_CREDENTIAL}"
The ideal initial state would be to allow the s3 api to default to V2
and opt into V4 while the aws-s3 provider continues to default to V4.
There is a reasonable question about what the s3 api default should be
which we could address in a subsequent commit.
If you get stuck it might make sense to share a work-in-progress PR on
GitHub. I hope this helps!
On Sat, Dec 12, 2020 at 06:43:45AM -0000, John Calcote wrote:
> Andrew,
>
> I would love to submit this PR. But I'm going to need some guidance. I'm a
> pretty good coder, but I must tell you that there are big gaps in my
> understanding of how the jclouds code works.
>
> Can you please point me in the right direction? Tell me which files to look
> at and I'll try to decipher what needs to be done. And any additional
> information you think might be relevant. My hope is that I can do the work,
> saving you that effort, if you will spend just a few minutes jotting down how
> you would approach the patch since you obviously understand the code better
> than most.
>
> Thanks very much for the opportunity to help.
>
>
> On 2020/12/12 03:04:12, Andrew Gaul <[email protected]> wrote:
> > 1) and 2) are hacky but will work. However we should really properly
> > support this by giving the generic S3 provider a property. This would
> > require reparenting some code from aws-s3 to s3. Would you be able to
> > submit a PR for this?
> >
> > The unsigned payload is also a desired feature and easy to add after
> > fixing the above.
> >
> > We plan to release 2.3.0 later this month so it could include these
> > features if you can help out!
> >
> > On Thu, Dec 10, 2020 at 07:42:22PM -0000, John Calcote wrote:
> > > I see someone asked this very question a while back (2018):
> > > https://lists.apache.org/thread.html/3553946dcbe7c72fc8f82c6353c9a5c56d7b587da18a4ebe12df1e26%40%3Cuser.jclouds.apache.org%3E
> > >
> > > Did anyone ever solve the double read issue regarding disabling
> > > data-hashing?
> > >
> > > On 2020/12/10 18:03:13, John Calcote <[email protected]> wrote:
> > > > Hi all -
> > > >
> > > > We have a client who would like to use jclouds with ONTAP 9.8, which
> > > > requires V4 signatures AND path-based URL naming. I believe the true
> > > > AWS S3 provider requires virtual bucket URL naming. Anyone have any
> > > > ideas on how to either:
> > > >
> > > > 1) coerce jclouds into using the AWS S3 provider with path-based naming
> > > > or
> > > > 2) coerce jclouds into using the generic S3 provider with V4 signing
> > > >
> > > > Thanks in advance.
> > > > John
> > > >
> >
> > --
> > Andrew Gaul
> > http://gaul.org/
> >
--
Andrew Gaul
http://gaul.org/
