[
https://issues.apache.org/jira/browse/SPARK-13933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197206#comment-15197206
]
Steve Loughran commented on SPARK-13933:
----------------------------------------
No custom guava built. Hadoop dealt with the problem by
-experimentally building curator with guava.version set to 11 to see what broke
(me: HADOOP-11102 )
-fixing the one needed operation by duplicating the code in hadoop code with it
fixed to work with guava 11.
There's no problem AFAIK; the reason for the hadoop update was to get some ZK
fixes related to NN HA in.
Even so, having a library that is known to be incompatible with a shared
dependency is dangerous; I wasn't happy with the change going in without much
discussion, and if HADOOP-11102 hadn't reassured me about compatibility, I'd
have reverted things.
The Hadoop change notes flag the API changes as:
----
* Binary incompatible change:
{{org.apache.curator.utils.PathUtils.validatePath(String)}} changed return
types. Downstream users of this method will need to recompile. *Not directly
used in spark codebase*
* Source incompatible change:
{{org.apache.curator.framework.recipes.shared.SharedCountReader}} added a
method to its interface definition. Downstream users with custom
implementations of this interface can continue without binary compatibility
problems but will need to modify their source code to recompile. *Not directly
used in spark codebase*
* Source incompatible change:
{{org.apache.curator.framework.recipes.shared.SharedValueReader}} added a
method to its interface definition. Downstream users with custom
implementations of this interface can continue without binary compatibility
problems but will need to modify their source code to recompile. *Not directly
used in spark codebase*
----
As none of them are directly used, there's no direct risk of things breaking if
spark depended on 2.6 and someone ran it with 2.7.1. Tachyon-client does use
curator; someone would need to see its versioning/compatibility there; it may
already be on a later version.
How about I just add a one-liner to the profile saying "deliberately held back
for guava reasons"
> hadoop-2.7 profile's curator version should be 2.7.1
> ----------------------------------------------------
>
> Key: SPARK-13933
> URL: https://issues.apache.org/jira/browse/SPARK-13933
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.0.0
> Reporter: Steve Loughran
> Priority: Minor
>
> This is pretty minor, more due diligence than any binary compatibility.
> # the {{hadoop-2.7}} profile declares the curator version to be 2.6.0
> # the actual hadoop-2.7.1 dependency is of curator 2.7.1; this came from
> HADOOP-11492
> For consistency, the profile can/should be changed. However, note that as
> well as some incompatibilities defined in HADOOP-11492; the version of Guava
> that curator asserts a need for is 15.x. HADOOP-11612 showed what needed to
> be done to address compatibility problems there; one of the Curator classes
> had to be forked to make compatible with guava 11+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
