Eric, I believe you are correct that Amazon is pushing for v4, and compliance is only optional in certain regions, for a while. I can see NiFi getting caught between AWS users wanting latest-greatest support and compatible service users preferring legacy support. I believe signature v4 is mature enough at this point that most compatible services like Riak and Ceph have had a chance to update, but I'm not very familiar with the space.
Joe, I like your thinking, I opened a JIRA and a PR for an AWS SDK update in 1.0. In recent SDKs, Amazon is in fact making signature v4 the default in more (all?) regions, so upgrading may have a side effect for compatible service users. https://issues.apache.org/jira/browse/NIFI-2062 https://github.com/apache/nifi/pull/562 We also had a brief discussion in PR 362 about how to support SSE-KMS, but keep the signature version configurable ( https://github.com/apache/nifi/pull/362). Thanks, James On Thu, Jun 23, 2016 at 10:47 AM, Eric Olson <eol...@softnas.com> wrote: > I will tell you that there is some AWS regions that will only work with V4 > when it comes to S3. I know for sure this is true of Frankfurt region for > example. So it is something worth considering > > Eric > > On 6/23/16, 1:44 PM, "Joe Skora" <jsk...@gmail.com> wrote: > > >Jim, > > > >I don't have time to research this more fully at the moment, but the S3 > >processors use the AWS libraries that should know about the API versions. > > > >We may be bundling older library versions that pre-date the new API, but > >that should be easy to verify. It looks like we bundle > >"com.amazonaws:aws-java-sdk:1.10.32" currently. If that is too old and > >updating to a current library requires a lot of code changes, that seems > >like it would probably be a 1.0.0 change. > > > >Regards, > >Joe > > > >On Mon, Jun 20, 2016 at 2:40 PM, Tony Kurc <trk...@gmail.com> wrote: > > > >> James, > >> I briefly used s3 ninja, and experienced no issues, but I was chiefly > >>using > >> it for integration testing. > >> > >> Tony > >> > >> On Mon, Jun 20, 2016 at 2:05 PM, James Wing <jvw...@gmail.com> wrote: > >> > >> > Is anyone using NiFi's S3 processors to work with non-Amazon, > >> S3-compatible > >> > services? Would you be willing to share which service you are using > >>and > >> if > >> > you have experienced compatibility issues? > >> > > >> > Are you aware of the difference between S3's API signature versions 2 > >>and > >> > 4? NiFi's current default S3 signature version is version 2. Would > >>you > >> be > >> > concerned about upgrading to a more recent AWS SDK where more > >>operations > >> > default to version 4? > >> > > >> > Changes in Signature Version 4 > >> > http://docs.aws.amazon.com/general/latest/gr/sigv4_changes.html > >> > > >> > > >> > Thanks, > >> > > >> > James > >> > > >> > >
