Steve Loughran commented on YARN-2092:

# code using Apache Curator was one -in YARN and on the client if you picked up 
the locally installed hadoop CP. And we couldn't upload a newer version of 
Jackson, because again, what's on the CP is what you get. 
# If, client-side you abandon that CP and use/redist your entire hadoop binary 
set then you can reduce the risk here, at the expense of taking away from ops 
any control of the versions of things you run on the cluster, but then you now 
have to deal with older files in the cluster.
# or you ignore yarn.lib.classpath entirely, *somehow* work out the values of 
yarn-site.xml &c, and re-upload every single hadoop-*.jar and its chosen 
binaries into every single container. having a YARN artifact repo will reduce 
the cost of that, but add a new one: bug fixes in hadoop will only propagate 
when the apps are rebuilt.
# ..if you look at the HADOOP-9991 issue you can see links to some places where 
the outdated JARs in Hadoop cause problems for other ASF projects.
# Tez appears to have broken because it was explicity putting the 1.8.x JARs on 
its list of binaries to upload. It only worked because it was using exactly the 
same version. 
# if you adopt a policy of change no dependencies that break apps that upload 
duplicate JARs to the CP -then this goes beyond Jackson, it says "hadoop cannot 
update any of its dependencies". That would go for 2.x and no doubt even if we 
did update things for 3.x, then we'll still get "you broke my code that 
uploaded jackson 
1.8" issues.
# ...and we haven't gone near Guava yet, which is frozen because it really is 
so brittle, but that means we can't pick up the guava 16.x-only fixes needed to 
work with the latest JVMs. 

If you do want to revoke jackson, I'm not going to veto it -but it goes beyond 
YARN, and we may as well revert every single HADOOP-9991-related upgrade. 

> Incompatible org.codehaus.jackson* dependencies when moving from 2.4.0 to 
> 2.5.0-SNAPSHOT
> ----------------------------------------------------------------------------------------
>                 Key: YARN-2092
>                 URL: https://issues.apache.org/jira/browse/YARN-2092
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Hitesh Shah
> Came across this when trying to integrate with the timeline server. Using a 
> 1.8.8 dependency of jackson works fine against 2.4.0 but fails against 
> 2.5.0-SNAPSHOT which needs 1.9.13. This is in the scenario where the user 
> jars are first in the classpath.  

This message was sent by Atlassian JIRA

Reply via email to