[jira] [Updated] (HADOOP-13692) hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.
[ https://issues.apache.org/jira/browse/HADOOP-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-13692: - Fix Version/s: (was: 3.3.6) > hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent > classpath conflicts. > --- > > Key: HADOOP-13692 > URL: https://issues.apache.org/jira/browse/HADOOP-13692 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HADOOP-13692-branch-2.001.patch > > > If an end user's application has a dependency on hadoop-aws and no other > Hadoop artifacts, then it picks up a transitive dependency on Jackson 2.5.3 > jars through the AWS SDK. This can cause conflicts at deployment time, > because Hadoop has a dependency on version 2.2.3, and the 2 versions are not > compatible with one another. We can prevent this problem by changing > hadoop-aws to declare explicit dependencies on the Jackson artifacts, at the > version Hadoop wants. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13692) hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.
[ https://issues.apache.org/jira/browse/HADOOP-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13692: - Fix Version/s: 3.0.0-alpha2 > hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent > classpath conflicts. > --- > > Key: HADOOP-13692 > URL: https://issues.apache.org/jira/browse/HADOOP-13692 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HADOOP-13692-branch-2.001.patch > > > If an end user's application has a dependency on hadoop-aws and no other > Hadoop artifacts, then it picks up a transitive dependency on Jackson 2.5.3 > jars through the AWS SDK. This can cause conflicts at deployment time, > because Hadoop has a dependency on version 2.2.3, and the 2 versions are not > compatible with one another. We can prevent this problem by changing > hadoop-aws to declare explicit dependencies on the Jackson artifacts, at the > version Hadoop wants. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13692) hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.
[ https://issues.apache.org/jira/browse/HADOOP-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13692: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Thank you, Steve. I committed this to trunk, branch-2 and branch-2.8. > hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent > classpath conflicts. > --- > > Key: HADOOP-13692 > URL: https://issues.apache.org/jira/browse/HADOOP-13692 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 2.8.0 > > Attachments: HADOOP-13692-branch-2.001.patch > > > If an end user's application has a dependency on hadoop-aws and no other > Hadoop artifacts, then it picks up a transitive dependency on Jackson 2.5.3 > jars through the AWS SDK. This can cause conflicts at deployment time, > because Hadoop has a dependency on version 2.2.3, and the 2 versions are not > compatible with one another. We can prevent this problem by changing > hadoop-aws to declare explicit dependencies on the Jackson artifacts, at the > version Hadoop wants. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13692) hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.
[ https://issues.apache.org/jira/browse/HADOOP-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-13692: --- Status: Patch Available (was: Open) > hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent > classpath conflicts. > --- > > Key: HADOOP-13692 > URL: https://issues.apache.org/jira/browse/HADOOP-13692 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Attachments: HADOOP-13692-branch-2.001.patch > > > If an end user's application has a dependency on hadoop-aws and no other > Hadoop artifacts, then it picks up a transitive dependency on Jackson 2.5.3 > jars through the AWS SDK. This can cause conflicts at deployment time, > because Hadoop has a dependency on version 2.2.3, and the 2 versions are not > compatible with one another. We can prevent this problem by changing > hadoop-aws to declare explicit dependencies on the Jackson artifacts, at the > version Hadoop wants. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13692) hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.
[
https://issues.apache.org/jira/browse/HADOOP-13692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-13692:
---
Attachment: HADOOP-13692-branch-2.001.patch
I'm attaching patch 001. We can solve the problem simply by listing the
Jackson 2 dependencies in hadoop-aws/pom.xml. See below for a before and after
comparison of the dependency tree for an application with a dependency on
hadoop-aws. Before, you'll see jackson-databind 2.5.3, coming in transitively
through aws-java-sdk-core. After, you'll see jackson-databind 2.2.3, coming in
transitively as a direct dependency of hadoop-aws.
For testing, I completed a full test run against an S3 bucket in US-west-2. I
also tested some S3A operations through a throw-away client application with a
dependency on hadoop-aws.
We have a similar dependency management challenge for joda-time, but that one
is already listed in hadoop-aws/pom.xml, so we don't need to do anything about
it in this patch.
h3. Before
{code}
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @
test-hadoop-cloud-storage ---
[INFO] cnauroth:test-hadoop-cloud-storage:jar:0.0.1-SNAPSHOT
[INFO] \- org.apache.hadoop:hadoop-aws:jar:2.9.0-SNAPSHOT:compile
[INFO]+- org.apache.hadoop:hadoop-common:jar:2.9.0-SNAPSHOT:compile
[INFO]| +- org.apache.hadoop:hadoop-annotations:jar:2.9.0-SNAPSHOT:compile
[INFO]| | \- jdk.tools:jdk.tools:jar:1.8:system
[INFO]| +- com.google.guava:guava:jar:11.0.2:compile
[INFO]| +- commons-cli:commons-cli:jar:1.2:compile
[INFO]| +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO]| +- xmlenc:xmlenc:jar:0.52:compile
[INFO]| +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile
[INFO]| | \- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO]| +- commons-codec:commons-codec:jar:1.4:compile
[INFO]| +- commons-io:commons-io:jar:2.4:compile
[INFO]| +- commons-net:commons-net:jar:3.1:compile
[INFO]| +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO]| +- javax.servlet:servlet-api:jar:2.5:compile
[INFO]| +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO]| +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO]| +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26:compile
[INFO]| +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO]| +- com.sun.jersey:jersey-core:jar:1.9:compile
[INFO]| +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO]| | +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO]| | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO]| | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO]| | | +- javax.xml.stream:stax-api:jar:1.0-2:compile
[INFO]| | | \- javax.activation:activation:jar:1.1:compile
[INFO]| | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile
[INFO]| | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO]| +- com.sun.jersey:jersey-server:jar:1.9:compile
[INFO]| | \- asm:asm:jar:3.1:compile
[INFO]| +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO]| +- log4j:log4j:jar:1.2.17:compile
[INFO]| +- net.java.dev.jets3t:jets3t:jar:0.9.0:compile
[INFO]| | \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile
[INFO]| +- commons-lang:commons-lang:jar:2.6:compile
[INFO]| +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO]| | +- commons-digester:commons-digester:jar:1.8:compile
[INFO]| | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO]| | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO]| +- org.slf4j:slf4j-api:jar:1.7.10:compile
[INFO]| +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile
[INFO]| +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO]| +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO]| +- org.apache.avro:avro:jar:1.7.4:compile
[INFO]| | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO]| | \- org.xerial.snappy:snappy-java:jar:1.0.4.1:compile
[INFO]| +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO]| +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO]| +- org.apache.hadoop:hadoop-auth:jar:2.9.0-SNAPSHOT:compile
[INFO]| | +- com.nimbusds:nimbus-jose-jwt:jar:3.9:compile
[INFO]| | | +- net.jcip:jcip-annotations:jar:1.0:compile
[INFO]| | | \- net.minidev:json-smart:jar:1.1.1:compile
[INFO]| | +-
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO]| | | +-
org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO]| | | +-
org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO]| | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO]| | \- org.apache.curator:curator-framework:jar:2.7.1:compile
[INFO]
