Thank you! I've built a copy of main and have started working on integrating it into Druid. I'll let the group know if anything interesting comes up. There is also a good chunk of work to do here, as it's been a while since we updated our version of Calcite.
On 2022/12/11 18:43:08 Julian Hyde wrote: > The bug is now fixed. (I started from your PR, but I had to do quite a lot of > extra work. Despite what I said earlier, there were several places where > Calcite was using APIs added between Guava 16 and 19.) > > > On Dec 9, 2022, at 1:30 PM, Gian Merlino <g...@apache.org> wrote: > > > > Hmm. In that case I'm not sure why we came to the belief that newer > > Calcites could not be used with Guava 16.0.1. Maybe it was from reading the > > Calcite release notes rather than actually testing it. It would be great if > > this just works. > > > > I raised this PR: https://github.com/apache/calcite/pull/2999 and this > > JIRA: https://issues.apache.org/jira/browse/CALCITE-5428. > > > >> On 2022/12/09 20:54:46 Julian Hyde wrote: > >> I ran with '-Dguava.version=16.0.1' and it seems to just work. > >> > >> If you want this to be the new minimum, please create a Jira case and > >> PR. Changes will be needed to main.yml (GitHub CI), comments in > >> gradle.properties, and history.md (release notes for the upcoming > >> release). > >> > >> Julian > >> > >>> On Fri, Dec 9, 2022 at 12:30 PM Gian Merlino <g...@apache.org> wrote: > >>> > >>> If Calcite ends up dropping the minimum to 16, that would certainly be > >>> helpful, and appreciated, since it would mean Druid can upgrade Calcite > >>> without needing to also tackle the Hadoop 2 problem. Please let us know > >>> if it seems like that would be in the cards! > >>> > >>> In case Calcite sticks on Guava 19+, I asked on the Druid issue > >>> (https://github.com/apache/druid/issues/13532) to see if anyone's tried > >>> Hadoop 2 with Guava 19. I'm not sure if we've tried that specific > >>> combination; if not, maybe that would work for us. If that doesn't work > >>> out, I think our best remaining idea would be to build our own version of > >>> Calcite that uses a relocated Guava. > >>> > >>> On 2022/12/08 20:49:17 Julian Hyde wrote: > >>>> If it helps, it wouldn’t be impossible for Calcite to move its minimum > >>>> Guava version down to 16. (If we rely on APIs introduced after 16 we can > >>>> create a facade for those APIs.) > >>>> > >>>> I am frustrated that the Hadoop folks get locked into a particular > >>>> version of Guava (and JDK… don’t get me started on that one) and in > >>>> Calcite I would like to do the opposite. > >>>> > >>>> Julian > >>>> > >>>> > >>>>> On Dec 8, 2022, at 9:24 AM, Gian Merlino <g...@apache.org> wrote: > >>>>> > >>>>> It pains me to admit that the reason Druid still uses Guava 16 is that > >>>>> we still support Hadoop 2, which relies on APIs that have been removed > >>>>> in newer versions of Guava. From time to time it is proposed that the > >>>>> project drops support for Hadoop 2 in order to be free to move up Guava > >>>>> versions, and also achieve various other benefits. There are generally > >>>>> concerns from people that are stuck with Hadoop 2 for various reasons. > >>>>> The most recent thread is here: > >>>>> https://lists.apache.org/thread/1j5w9dmt1gp8hx31tvrmyomcko4mlp03. I > >>>>> hope at some point we decide to bite the bullet and do it, but the > >>>>> community has not yet come to that decision. > >>>>> > >>>>> In the meantime, we're stuck on an older Calcite version, and really > >>>>> want to update it, so we're looking into options. There's a GitHub > >>>>> issue here with our thoughts: > >>>>> https://github.com/apache/druid/issues/13532 > >>>>> > >>>>> On 2022/03/30 17:55:44 Julian Hyde wrote: > >>>>>> Wow, I had no idea that anyone was on a version of Guava that old. > >>>>>> > >>>>>> I checked our history of Guava versions: > >>>>>> * In https://issues.apache.org/jira/browse/CALCITE-1590 > >>>>>> <https://issues.apache.org/jira/browse/CALCITE-1590> (Calcite 1.12) we > >>>>>> changed the default from 19 to 20, but still supported Guava 14.0.1 > >>>>>> and higher. > >>>>>> * In https://issues.apache.org/jira/browse/CALCITE-1715 > >>>>>> <https://issues.apache.org/jira/browse/CALCITE-1715> (also Calcite > >>>>>> 1.12) we changed the default version from 20 to 19 > >>>>>> * In https://issues.apache.org/jira/browse/CALCITE-2191 > >>>>>> <https://issues.apache.org/jira/browse/CALCITE-2191> (Calcite 1.16) we > >>>>>> moved the minimum from 14.0.1 to 19 and cited a Druid email thread [1] > >>>>>> about upgrading Druid’s version of Guava > >>>>>> > >>>>>> Foolish of me to assume that 4 years later Druid would have done > >>>>>> something. > >>>>>> > >>>>>> Julian > >>>>>> > >>>>>> [1] https://groups.google.com/g/druid-development/c/Dw2Qu1CWbuQ > >>>>>> > >>>>>>> On Mar 30, 2022, at 10:22 AM, Abhishek Agarwal > >>>>>>> <abhishekagarwal.i...@gmail.com> wrote: > >>>>>>> > >>>>>>> Thank you, Alessandro and Stamatis, for the info. Currently, the guava > >>>>>>> version being used in druid is 16. So simple exclusion is unlikely to > >>>>>>> work. > >>>>>>> I will still give it a try. If it doesn't, we will shade the jars on > >>>>>>> our > >>>>>>> end. > >>>>>>> > >>>>>>> On Tue, Mar 29, 2022 at 3:38 PM Alessandro Solimando < > >>>>>>> alessandro.solima...@gmail.com> wrote: > >>>>>>> > >>>>>>>> Hello Abhishek, > >>>>>>>> maybe I am re-stating what Stamatis suggested without realising, but > >>>>>>>> since > >>>>>>>> Calcite works well with a broad range of guava versions, can't you > >>>>>>>> simply > >>>>>>>> exclude guava when you include it in druid? > >>>>>>>> > >>>>>>>> At that point if the guava version of druid is in the list of those > >>>>>>>> supported by Calcite (>= 19 IIRC) all should work fine. > >>>>>>>> > >>>>>>>> I feel like Stamatis concerning shading. > >>>>>>>> > >>>>>>>> Best regards, > >>>>>>>> Alessandro > >>>>>>>> > >>>>>>>> On Tue, 29 Mar 2022 at 11:18, Stamatis Zampetakis <zabe...@gmail.com> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi Abhishek, > >>>>>>>>> > >>>>>>>>> Calcite supports multiple versions of Guava so in principle it > >>>>>>>>> should > >>>>>>>> work > >>>>>>>>> without problems with older versions. > >>>>>>>>> Instead of shading you could possibly select explicitly the version > >>>>>>>>> that you want to use via maven/gradle. > >>>>>>>>> > >>>>>>>>> I share your pain but shading is quite complex to get right so I am > >>>>>>>>> not > >>>>>>>> in > >>>>>>>>> favor of putting this burden on Calcite. > >>>>>>>>> > >>>>>>>>> I've seen it used in many projects and it rarely works as expected. > >>>>>>>>> Most > >>>>>>>> of > >>>>>>>>> the time it creates additional problems that need to be resolved. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Stamatis > >>>>>>>>> > >>>>>>>> > >>>>>> > >>>>>> > >>>> > >>>> > >> >
