The use of an unnecessarily old dependency encourages problems like HDFS-7040. The current Guava dependency is a big problem for downstream apps and I'd really like to see it addressed.
On Tue, Sep 23, 2014 at 2:09 PM, Steve Loughran <ste...@hortonworks.com> wrote: > I'm using curator elsewhere, it does log a lot (as does the ZK client), but > it solves a lot of problem. It's being adopted more downstream too. > > I'm wondering if we can move the code to the extent we know it works with > Guava 16, with the hadoop core being 16-compatible, but not actually > migrated to 16.x only. Then hadoop ships with 16 for curator & downstream > apps, but we say "you can probably roll back to 11 provided you don't use > features x-y-z". > > On 23 September 2014 21:55, Robert Kanter <rkan...@cloudera.com> wrote: > > > At the same time, not being able to use Curator will require a lot of > extra > > code, a lot of which we probably already have from the ZKRMStateStore, > but > > it's not available to use in hadoop-auth. We'd need to create our own ZK > > libraries that Hadoop components can use, but (a) that's going to take a > > while, and (b) it seems silly to reinvent the wheel when Curator already > > does all this. > > > > I agree that upgrading Guava will be a compatibility problem though... > > > > On Tue, Sep 23, 2014 at 9:30 AM, Sandy Ryza <sandy.r...@cloudera.com> > > wrote: > > > > > If we've broken compatibility in branch-2, that's a bug that we need to > > > fix. HADOOP-10868 has not yet made it into a release; I don't see it > as a > > > justification for solidifying the breakage. > > > > > > -1 to upgrading Guava in branch-2. > > > > > > On Tue, Sep 23, 2014 at 3:06 AM, Steve Loughran < > ste...@hortonworks.com> > > > wrote: > > > > > > > +1 to upgrading guava. Irrespective of downstream apps, the hadoop > > source > > > > tree is now internally inconsistent > > > > > > > > On 22 September 2014 17:56, Sangjin Lee <sj...@apache.org> wrote: > > > > > > > > > I agree that a more robust solution is to have better classloading > > > > > isolation. > > > > > > > > > > Still, IMHO guava (and possibly protobuf as well) sticks out like a > > > sore > > > > > thumb. There are just too many issues in trying to support both > guava > > > 11 > > > > > and guava 16. Independent of what we may do with the classloading > > > > > isolation, we should still consider upgrading guava. > > > > > > > > > > My 2 cents. > > > > > > > > > > On Sun, Sep 21, 2014 at 3:11 PM, Karthik Kambatla < > > ka...@cloudera.com> > > > > > wrote: > > > > > > > > > > > Upgrading Guava version is tricky. While it helps in many cases, > it > > > can > > > > > > break existing applications/deployments. I understand we do not > > have > > > a > > > > > > policy for updating dependencies, but still we should be careful > > with > > > > > > Guava. > > > > > > > > > > > > I would be more inclined towards a more permanent solution to > this > > > > > problem > > > > > > - how about prioritizing classpath isolation so applications > aren't > > > > > > affected by Hadoop dependency updates at all? I understand that > > will > > > > also > > > > > > break user applications, but it might be the driving feature for > > > Hadoop > > > > > > 3.0? > > > > > > > > > > > > On Fri, Sep 19, 2014 at 5:13 PM, Sangjin Lee <sj...@apache.org> > > > wrote: > > > > > > > > > > > > > I would also agree on upgrading guava. Yes I am aware of the > > > > potential > > > > > > > impact on customers who might rely on hadoop bringing in guava > > 11. > > > > > > However, > > > > > > > IMHO the balance tipped over to the other side a while ago; > i.e. > > I > > > > > think > > > > > > > there are far more people using guava 16 in their code and > > > scrambling > > > > > to > > > > > > > make things work than the other way around. > > > > > > > > > > > > > > On Thu, Sep 18, 2014 at 2:40 PM, Steve Loughran < > > > > > ste...@hortonworks.com> > > > > > > > wrote: > > > > > > > > > > > > > > > I know we've been ignoring the Guava version problem, but > > > > > HADOOP-10868 > > > > > > > > added a transitive dependency on Guava 16 by way of Curator > > 2.6. > > > > > > > > > > > > > > > > Maven currently forces the build to use Guava 11.0.2, but > this > > is > > > > > > hiding > > > > > > > at > > > > > > > > compile timeall code paths from curator which may use > classes & > > > > > methods > > > > > > > > that aren't there. > > > > > > > > > > > > > > > > I need curator for my own work (2.4.1 & Guava 14.0 was what > I'd > > > > been > > > > > > > > using), so don't think we can go back. > > > > > > > > > > > > > > > > HADOOP-11102 covers the problem -but doesn't propose a > specific > > > > > > solution. > > > > > > > > But to me the one that seems most likely to work is: update > > Guava > > > > > > > > > > > > > > > > -steve > > > > > > > > > > > > > > > > -- > > > > > > > > CONFIDENTIALITY NOTICE > > > > > > > > NOTICE: This message is intended for the use of the > individual > > or > > > > > > entity > > > > > > > to > > > > > > > > which it is addressed and may contain information that is > > > > > confidential, > > > > > > > > privileged and exempt from disclosure under applicable law. > If > > > the > > > > > > reader > > > > > > > > of this message is not the intended recipient, you are hereby > > > > > notified > > > > > > > that > > > > > > > > any printing, copying, dissemination, distribution, > disclosure > > or > > > > > > > > forwarding of this communication is strictly prohibited. If > you > > > > have > > > > > > > > received this communication in error, please contact the > sender > > > > > > > immediately > > > > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > entity > > > to > > > > which it is addressed and may contain information that is > confidential, > > > > privileged and exempt from disclosure under applicable law. If the > > reader > > > > of this message is not the intended recipient, you are hereby > notified > > > that > > > > any printing, copying, dissemination, distribution, disclosure or > > > > forwarding of this communication is strictly prohibited. If you have > > > > received this communication in error, please contact the sender > > > immediately > > > > and delete it from your system. Thank You. > > > > > > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
