[
https://issues.apache.org/jira/browse/HADOOP-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-15553:
------------------------------------
Priority: Minor (was: Major)
Component/s: (was: tools)
fs/s3
documentation
> DOCO - not clear how to integrate hadoop-aws and aws-sdk jars with S3A
> ----------------------------------------------------------------------
>
> Key: HADOOP-15553
> URL: https://issues.apache.org/jira/browse/HADOOP-15553
> Project: Hadoop Common
> Issue Type: New Feature
> Components: documentation, fs/s3
> Affects Versions: 2.8.3
> Reporter: t oo
> Priority: Minor
>
> Documentation update, a compatibility matrix would be nice, especially since
> upgrading to new aws java sdk does that work :
>
> On a hadoop common 2.7.5 cluster, I take AWS Java SDK
> [1.7.5|https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk/1.7.5]
> and Hadoop-aws
> [2.7.5|https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws/2.7.5]
> and use them in my Spark app writing to S3a, this is fine.
> On another cluster (this time hadoop common 2.8.3 cluster) I take AWS Java
> SDK 1.10.6 and Hadoop-aws 2.8.3 and try to use them in my Spark app writing
> to S3a, but get errors about missing classes. Turns out this version of aws
> java sdk is tiny with basically nothing inside.
> Couldn't find any documentation helping users on this incompatibility.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
