[ https://issues.apache.org/jira/browse/SPARK-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14545636#comment-14545636 ]
Steve Loughran commented on SPARK-7669: --------------------------------------- snippet of the maven dependencies with {{-Phadoop-2.4 -Dhadoop.version=2.6.0 }} {code} [INFO] | | | +- org.apache.curator:curator-client:jar:2.6.0:compile [INFO] | | | | +- (org.slf4j:slf4j-api:jar:1.7.10:compile - version managed from 1.7.6; omitted for duplicate) [INFO] | | | | +- (org.apache.zookeeper:zookeeper:jar:3.4.5:compile - version managed from 3.4.6; omitted for duplicate) [INFO] | | | | \- (com.google.guava:guava:jar:14.0.1:provided - version managed from 11.0.2; scope managed from compile; omitted for duplicate) [INFO] | | | +- (org.apache.curator:curator-recipes:jar:2.4.0:compile - version managed from 2.6.0; omitted for duplicate) {code} What's happened is that the curator-recipes version is being set by spark, but no version of curator-client is set. Nor can you fix the versions on the CLI, as the curator version isn't yet property driven. I have a patch to make things consistent, with a hadoop 2.6 profile which sets curator.version=2.6,0, zookeeper.version=3.4.6, and drives the curator-recipes version off that curator.version property. There's one possible enhancement to this: declare the version of curator-client &c directly, putting the spark build in control of which version gets picked up. > Builds against Hadoop 2.6+ get inconsistent curator dependencies > ---------------------------------------------------------------- > > Key: SPARK-7669 > URL: https://issues.apache.org/jira/browse/SPARK-7669 > Project: Spark > Issue Type: Bug > Components: Build > Affects Versions: 1.3.1, 1.4.0 > Environment: Hadoop 2.6 > Reporter: Steve Loughran > Priority: Minor > > If you build spark against Hadoop 2.6 you end up with an inconsistent set of > curator dependencies -curator-recipe 2.4.0 with curator 2.6.0. > A dedicated hadoop-2.6 profile along with extraction of curator version into > a property can keep the curator versions in sync, along with ZK. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org