[
https://issues.apache.org/jira/browse/HIVE-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386228#comment-16386228
]
Steve Loughran commented on HIVE-18861:
---------------------------------------
Dependencies before the patch when built against hadoop branch-3.1
{code}
[INFO] | +- io.druid.extensions:druid-hdfs-storage:jar:0.11.0:compile
[INFO] | | +- (org.apache.hadoop:hadoop-client:jar:3.1.0-SNAPSHOT:compile -
version managed from 2.7.3; omitted for duplicate)
[INFO] | | +- org.apache.hadoop:hadoop-aws:jar:2.7.3:compile
[INFO] | | | \- (org.apache.hadoop:hadoop-common:jar:3.1.0-SNAPSHOT:compile
- version managed from 2.7.3; omitted for duplicate)
[INFO] | | \- com.amazonaws:aws-java-sdk-s3:jar:1.10.77:compile
[INFO] | | +- com.amazonaws:aws-java-sdk-kms:jar:1.10.77:compile
[INFO] | | | \- (com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile -
omitted for duplicate)
[INFO] | | \- com.amazonaws:aws-java-sdk-core:jar:1.10.77:compile
[INFO] | | +- (commons-logging:commons-logging:jar:1.1.3:compile -
omitted for conflict with 1.0.4)
[INFO] | | +- (org.apache.httpcomponents:httpclient:jar:4.5.2:compile -
version managed from 4.3.6; omitted for duplicate)
[INFO] | | +-
com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.5.3:compile
{code}
> druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating
> classpath problems on hadoop 3.x
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-18861
> URL: https://issues.apache.org/jira/browse/HIVE-18861
> Project: Hive
> Issue Type: Sub-task
> Components: Druid integration
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HIVE-18861-001.patch
>
>
> druid-hdfs-storage JAR is transitively pulling in hadoop-aws JAR 2.7.3, which
> creates classpath problems as a set of aws-sdk 1.10.77 JARs get on the CP,
> even with Hadoop 3 & its move to a full aws-sdk-bundle JAR.
> Two options
> # exclude the dependency
> # force it up to whatever ${hadoop.version} is, so make it consistent
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
