[
https://issues.apache.org/jira/browse/HADOOP-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562525#comment-16562525
]
Steve Loughran commented on HADOOP-15642:
-----------------------------------------
Proposed process for SDK update, something I'd add to the testing.md file along
with an intro "updating the AWS SDK is non-trivial and can take a while to bed
in"
# plan this a few weeks before are release, to allow for (a) time to see what
the issues are, (b) time to roll back and (c) possibly time to get any big
issues fixed in the AWS Release cycle.
# increase {{aws-java-sdk.version}} to the new version
# rebuild, retest
# distinguish test failures from our code being wrong (repair) vs incompatible
AWS SDK changes (file)
# look at logs of generated test runs. Are new warning messages appearing?
Stack traces?
# build hadoop CLI, perform manual operations. Again, does everything "look"
right
# build and run any downstream tests you have to hand
# do a mvn dependency dump of hadoop-aws: has the new shaded SDK forgotten to
shade everything? If so: file issue with AWS; exclude this dependency
(HADOOP-15264)
Policy w.r.t deprecated items
* it depends. Treat as a hint more than a demand, but do consider why the old
version is being deprecated.
Policy w.r.t backporting SDK updates
* Updates allowed for branch-2 and trunk
* No to <=2.8.x (unshaded, separate JAR era)
* Maybe to 2.9.x, 3.0.x, 3.1.x, if there are significant security implications
or other major bug fixes.
> Update to latest/recent version of aws-sdk
> ------------------------------------------
>
> Key: HADOOP-15642
> URL: https://issues.apache.org/jira/browse/HADOOP-15642
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, fs/s3
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Priority: Blocker
> Attachments: Screen Shot 2018-07-30 at 14.11.22.png
>
>
> Move to a later version of the AWS SDK library for a different set of
> features and issues.
> One thing which doesn't work on the one we ship with is the ability to create
> assumed role sessions >1h, as there's a check in the client lib for
> role-duration <= 3600 seconds. I'll assume more recent SDKs delegate duration
> checks to the far end.
> see:
> [https://aws.amazon.com/about-aws/whats-new/2018/03/longer-role-sessions/]
> * assuming later versions will extend assumed role life, docs will need
> changing,
> * Adding a test in HADOOP-15583 which expects an error message if you ask for
> a duration of 3h; this should act as the test to see what happens.
> * think this time would be good to explicitly write down the SDK update
> process
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
