If Calcite ends up dropping the minimum to 16, that would certainly be helpful, and appreciated, since it would mean Druid can upgrade Calcite without needing to also tackle the Hadoop 2 problem. Please let us know if it seems like that would be in the cards!
In case Calcite sticks on Guava 19+, I asked on the Druid issue (https://github.com/apache/druid/issues/13532) to see if anyone's tried Hadoop 2 with Guava 19. I'm not sure if we've tried that specific combination; if not, maybe that would work for us. If that doesn't work out, I think our best remaining idea would be to build our own version of Calcite that uses a relocated Guava. On 2022/12/08 20:49:17 Julian Hyde wrote: > If it helps, it wouldn’t be impossible for Calcite to move its minimum Guava > version down to 16. (If we rely on APIs introduced after 16 we can create a > facade for those APIs.) > > I am frustrated that the Hadoop folks get locked into a particular version of > Guava (and JDK… don’t get me started on that one) and in Calcite I would like > to do the opposite. > > Julian > > > > On Dec 8, 2022, at 9:24 AM, Gian Merlino <[email protected]> wrote: > > > > It pains me to admit that the reason Druid still uses Guava 16 is that we > > still support Hadoop 2, which relies on APIs that have been removed in > > newer versions of Guava. From time to time it is proposed that the project > > drops support for Hadoop 2 in order to be free to move up Guava versions, > > and also achieve various other benefits. There are generally concerns from > > people that are stuck with Hadoop 2 for various reasons. The most recent > > thread is here: > > https://lists.apache.org/thread/1j5w9dmt1gp8hx31tvrmyomcko4mlp03. I hope at > > some point we decide to bite the bullet and do it, but the community has > > not yet come to that decision. > > > > In the meantime, we're stuck on an older Calcite version, and really want > > to update it, so we're looking into options. There's a GitHub issue here > > with our thoughts: https://github.com/apache/druid/issues/13532 > > > > On 2022/03/30 17:55:44 Julian Hyde wrote: > >> Wow, I had no idea that anyone was on a version of Guava that old. > >> > >> I checked our history of Guava versions: > >> * In https://issues.apache.org/jira/browse/CALCITE-1590 > >> <https://issues.apache.org/jira/browse/CALCITE-1590> (Calcite 1.12) we > >> changed the default from 19 to 20, but still supported Guava 14.0.1 and > >> higher. > >> * In https://issues.apache.org/jira/browse/CALCITE-1715 > >> <https://issues.apache.org/jira/browse/CALCITE-1715> (also Calcite 1.12) > >> we changed the default version from 20 to 19 > >> * In https://issues.apache.org/jira/browse/CALCITE-2191 > >> <https://issues.apache.org/jira/browse/CALCITE-2191> (Calcite 1.16) we > >> moved the minimum from 14.0.1 to 19 and cited a Druid email thread [1] > >> about upgrading Druid’s version of Guava > >> > >> Foolish of me to assume that 4 years later Druid would have done something. > >> > >> Julian > >> > >> [1] https://groups.google.com/g/druid-development/c/Dw2Qu1CWbuQ > >> > >>> On Mar 30, 2022, at 10:22 AM, Abhishek Agarwal > >>> <[email protected]> wrote: > >>> > >>> Thank you, Alessandro and Stamatis, for the info. Currently, the guava > >>> version being used in druid is 16. So simple exclusion is unlikely to > >>> work. > >>> I will still give it a try. If it doesn't, we will shade the jars on our > >>> end. > >>> > >>> On Tue, Mar 29, 2022 at 3:38 PM Alessandro Solimando < > >>> [email protected]> wrote: > >>> > >>>> Hello Abhishek, > >>>> maybe I am re-stating what Stamatis suggested without realising, but > >>>> since > >>>> Calcite works well with a broad range of guava versions, can't you simply > >>>> exclude guava when you include it in druid? > >>>> > >>>> At that point if the guava version of druid is in the list of those > >>>> supported by Calcite (>= 19 IIRC) all should work fine. > >>>> > >>>> I feel like Stamatis concerning shading. > >>>> > >>>> Best regards, > >>>> Alessandro > >>>> > >>>> On Tue, 29 Mar 2022 at 11:18, Stamatis Zampetakis <[email protected]> > >>>> wrote: > >>>> > >>>>> Hi Abhishek, > >>>>> > >>>>> Calcite supports multiple versions of Guava so in principle it should > >>>> work > >>>>> without problems with older versions. > >>>>> Instead of shading you could possibly select explicitly the version > >>>>> that you want to use via maven/gradle. > >>>>> > >>>>> I share your pain but shading is quite complex to get right so I am not > >>>> in > >>>>> favor of putting this burden on Calcite. > >>>>> > >>>>> I've seen it used in many projects and it rarely works as expected. Most > >>>> of > >>>>> the time it creates additional problems that need to be resolved. > >>>>> > >>>>> Best, > >>>>> Stamatis > >>>>> > >>>> > >> > >> > >
