The bug is now fixed. (I started from your PR, but I had to do quite a lot of extra work. Despite what I said earlier, there were several places where Calcite was using APIs added between Guava 16 and 19.)
> On Dec 9, 2022, at 1:30 PM, Gian Merlino <[email protected]> wrote: > > Hmm. In that case I'm not sure why we came to the belief that newer Calcites > could not be used with Guava 16.0.1. Maybe it was from reading the Calcite > release notes rather than actually testing it. It would be great if this just > works. > > I raised this PR: https://github.com/apache/calcite/pull/2999 and this JIRA: > https://issues.apache.org/jira/browse/CALCITE-5428. > >> On 2022/12/09 20:54:46 Julian Hyde wrote: >> I ran with '-Dguava.version=16.0.1' and it seems to just work. >> >> If you want this to be the new minimum, please create a Jira case and >> PR. Changes will be needed to main.yml (GitHub CI), comments in >> gradle.properties, and history.md (release notes for the upcoming >> release). >> >> Julian >> >>> On Fri, Dec 9, 2022 at 12:30 PM Gian Merlino <[email protected]> wrote: >>> >>> If Calcite ends up dropping the minimum to 16, that would certainly be >>> helpful, and appreciated, since it would mean Druid can upgrade Calcite >>> without needing to also tackle the Hadoop 2 problem. Please let us know if >>> it seems like that would be in the cards! >>> >>> In case Calcite sticks on Guava 19+, I asked on the Druid issue >>> (https://github.com/apache/druid/issues/13532) to see if anyone's tried >>> Hadoop 2 with Guava 19. I'm not sure if we've tried that specific >>> combination; if not, maybe that would work for us. If that doesn't work >>> out, I think our best remaining idea would be to build our own version of >>> Calcite that uses a relocated Guava. >>> >>> On 2022/12/08 20:49:17 Julian Hyde wrote: >>>> If it helps, it wouldn’t be impossible for Calcite to move its minimum >>>> Guava version down to 16. (If we rely on APIs introduced after 16 we can >>>> create a facade for those APIs.) >>>> >>>> I am frustrated that the Hadoop folks get locked into a particular version >>>> of Guava (and JDK… don’t get me started on that one) and in Calcite I >>>> would like to do the opposite. >>>> >>>> Julian >>>> >>>> >>>>> On Dec 8, 2022, at 9:24 AM, Gian Merlino <[email protected]> wrote: >>>>> >>>>> It pains me to admit that the reason Druid still uses Guava 16 is that we >>>>> still support Hadoop 2, which relies on APIs that have been removed in >>>>> newer versions of Guava. From time to time it is proposed that the >>>>> project drops support for Hadoop 2 in order to be free to move up Guava >>>>> versions, and also achieve various other benefits. There are generally >>>>> concerns from people that are stuck with Hadoop 2 for various reasons. >>>>> The most recent thread is here: >>>>> https://lists.apache.org/thread/1j5w9dmt1gp8hx31tvrmyomcko4mlp03. I hope >>>>> at some point we decide to bite the bullet and do it, but the community >>>>> has not yet come to that decision. >>>>> >>>>> In the meantime, we're stuck on an older Calcite version, and really want >>>>> to update it, so we're looking into options. There's a GitHub issue here >>>>> with our thoughts: https://github.com/apache/druid/issues/13532 >>>>> >>>>> On 2022/03/30 17:55:44 Julian Hyde wrote: >>>>>> Wow, I had no idea that anyone was on a version of Guava that old. >>>>>> >>>>>> I checked our history of Guava versions: >>>>>> * In https://issues.apache.org/jira/browse/CALCITE-1590 >>>>>> <https://issues.apache.org/jira/browse/CALCITE-1590> (Calcite 1.12) we >>>>>> changed the default from 19 to 20, but still supported Guava 14.0.1 and >>>>>> higher. >>>>>> * In https://issues.apache.org/jira/browse/CALCITE-1715 >>>>>> <https://issues.apache.org/jira/browse/CALCITE-1715> (also Calcite 1.12) >>>>>> we changed the default version from 20 to 19 >>>>>> * In https://issues.apache.org/jira/browse/CALCITE-2191 >>>>>> <https://issues.apache.org/jira/browse/CALCITE-2191> (Calcite 1.16) we >>>>>> moved the minimum from 14.0.1 to 19 and cited a Druid email thread [1] >>>>>> about upgrading Druid’s version of Guava >>>>>> >>>>>> Foolish of me to assume that 4 years later Druid would have done >>>>>> something. >>>>>> >>>>>> Julian >>>>>> >>>>>> [1] https://groups.google.com/g/druid-development/c/Dw2Qu1CWbuQ >>>>>> >>>>>>> On Mar 30, 2022, at 10:22 AM, Abhishek Agarwal >>>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> Thank you, Alessandro and Stamatis, for the info. Currently, the guava >>>>>>> version being used in druid is 16. So simple exclusion is unlikely to >>>>>>> work. >>>>>>> I will still give it a try. If it doesn't, we will shade the jars on our >>>>>>> end. >>>>>>> >>>>>>> On Tue, Mar 29, 2022 at 3:38 PM Alessandro Solimando < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hello Abhishek, >>>>>>>> maybe I am re-stating what Stamatis suggested without realising, but >>>>>>>> since >>>>>>>> Calcite works well with a broad range of guava versions, can't you >>>>>>>> simply >>>>>>>> exclude guava when you include it in druid? >>>>>>>> >>>>>>>> At that point if the guava version of druid is in the list of those >>>>>>>> supported by Calcite (>= 19 IIRC) all should work fine. >>>>>>>> >>>>>>>> I feel like Stamatis concerning shading. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Alessandro >>>>>>>> >>>>>>>> On Tue, 29 Mar 2022 at 11:18, Stamatis Zampetakis <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Abhishek, >>>>>>>>> >>>>>>>>> Calcite supports multiple versions of Guava so in principle it should >>>>>>>> work >>>>>>>>> without problems with older versions. >>>>>>>>> Instead of shading you could possibly select explicitly the version >>>>>>>>> that you want to use via maven/gradle. >>>>>>>>> >>>>>>>>> I share your pain but shading is quite complex to get right so I am >>>>>>>>> not >>>>>>>> in >>>>>>>>> favor of putting this burden on Calcite. >>>>>>>>> >>>>>>>>> I've seen it used in many projects and it rarely works as expected. >>>>>>>>> Most >>>>>>>> of >>>>>>>>> the time it creates additional problems that need to be resolved. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Stamatis >>>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>> >>>> >>
