Fantastic news! Thanks to all who made this happen. -- Michael Mior [email protected]
2018-01-19 5:43 GMT-05:00 Arina Ielchiieva <[email protected]>: > I am happy to announce that Apache Drill has rebased on Calcite 1.15. All > information can be found in DRILL-3993 and in Google Doc [1]. > Currently changes are in master and are intended to be part of Drill 1.13.0 > release unless something critical pops up. Feel free to test them and give > your feedback. > > Thanks Roman Kulyk and Volodymyr Vysotskyi for making this happen. > Aman Sinha and Chunhui Shi for the code review. > And of course Julian Hyde for helping to address / review Calcite specific > issues. > > [1] > https://docs.google.com/document/d/1Lqk9NoKQviz0YimBmov4z1pu > i7QjJGjDVwMa1p0emPk/edit#heading=h.i3rowg20vxv4 > > > Kind regards > Arina > > On Fri, Sep 1, 2017 at 8:01 PM, Roman Kulyk <[email protected]> wrote: > > > Team, thanks for your answers! > > > > Julian, сontinuing the topic of Calcite tests I just want to note that I > > have ignored 7 test in 5ba68240 commit in Calcite. Here is a list of > > ignored tests: > > > > TraitConversionTest.testTraitConversion > > JdbcAdapterTest.testCrossJoinWithJoinKeyPlan > > JdbcAdapterTest.testCrossJoinWithJoinKeyAndFilterPlan > > JdbcTest.testJoinManyWay (it was also ignored by 8a87ad1e5) > > LatticeTest.testTileAlgorithm (it was also ignored by 2e076e1a) > > LatticeTest.testTileAlgorithm2 (it was also ignored by 8a87ad1e5) > > LatticeTest.testTwoLattices > > > > You can see detailed information in my comments. Also, I want to notice > > that there are 2 required commits in my Calcite branch: 5ba68240 and > > 1f5145b1. Without them, we lose a big part of Drill functionality. > > > > Best regards, > > Roman Kulyk > > > > 2017-08-31 18:43 GMT+00:00 Julian Hyde <[email protected]>: > > > > > +dev@calcite. > > > > > > Excellent work, Roman. > > > > > > Your changes to Calcite don’t seem to be very major, and I ran the test > > > suite on your https://github.com/KulykRoman/incubator-calcite/commits/ > > > DrillCalcite1.13.0_rc1 branch and all the tests pass. So when you are > > > ready, I think we can expedite getting these changes back into Calcite. > > > > > > Calcite will release 1.14 probably in the next two weeks. So, when > you’re > > > on to 1.13 let’s continue the momentum and get up to the latest > release. > > I > > > would love to get to the situation that we can ask Drill to verify a > > > Calcite release before we make it. There are possibly breaking changes > in > > > the area of schema and grouping-sets aggregate but it should be mostly > > > straightforward. > > > > > > Julian > > > > > > > > > > On Aug 31, 2017, at 8:12 AM, Jinfeng Ni <[email protected]> wrote: > > > > > > > > It's great to hear that someone is working on moving Drill on new > > Calcite > > > > release! As someone who did similar job in last two rebase work, I > > could > > > > see that there would be many regressions to fix. Please let me know > if > > I > > > > can provide any help. > > > > > > > > I thought we have pushed almost Drill specific commits to Calcite. > I'll > > > > take a look at your Calcite branch. Ideally, we want to get rid of > all > > > > Drill specific commits, after the rebase work this time. > > > > > > > > Thanks, > > > > > > > > Jinfeng > > > > > > > > > > > > > > > > > > > > On Thu, Aug 31, 2017 at 8:04 AM, Aman Sinha <[email protected]> > > > wrote: > > > > > > > >> Yes, this is long overdue ! Thanks for working on it Roman. If > > > needed, > > > >> we can do a separate hangout with a few selected folks who have > worked > > > on > > > >> Drill+Calcite to provide feedback on how to resolve the test > failures. > > > >> Although, at this stage you probably know a lot more about the > > > integration > > > >> than many of us. > > > >> > > > >> -Aman > > > >> > > > >> On Thu, Aug 31, 2017 at 7:33 AM, Roman Kulyk <[email protected]> > > > wrote: > > > >> > > > >>> Hi, team! > > > >>> > > > >>> I want to inform that I am working on updating Drill Calcite > version > > to > > > >>> 1.13. I forked from Calcite "branch-1.13" and added some commits: > > > >>> - DRILL-1455: Add return type-inference strategy for arithmetic > > > operators > > > >>> when one of the arguments is ANY type. (was not in Calcite) > > > >>> - Add new method to ViewExpander interface to allow passing > > SchemaRoot. > > > >>> (was not in Calcite) > > > >>> - Allow a MAP literal type. (was not in Calcite) > > > >>> - DRILL-4047: Modify table functions to allow querying a table with > > > >> options > > > >>> in Drill (was not in Calcite) > > > >>> - Drill-specific change: Add back AbstractConverter in RelSet.java > > ... > > > >>> (small changes after a3bc0d8 commit) > > > >>> - CALCITE-628 related but not fix the problem: Ensure target traits > > are > > > >>> simple when use Frameworks or RelOptRule.convert() method. (small > > > changes > > > >>> after a3bc0d8 commit) > > > >>> - Support select * from schema-less table in execution engine like > > > Drill > > > >>> (small changes after 9bd7d75 commit) > > > >>> > > > >>> Only after adding these commits to Calcite, a part of Drill > > > >> functionality > > > >>> started to work as before. Currently, I am at Drill integration > > stage. > > > I > > > >> am > > > >>> fixing Drill unit tests. Progress can be tracked in my branches. > > > >>> > > > >>> Drill: https://github.com/KulykRoman/drill/commits/ > > > CalciteForkRebase_rc1 > > > >>> Calcite: > > > >>> https://github.com/KulykRoman/incubator-calcite/commits/ > > > >>> DrillCalcite1.13.0_rc1 > > > >>> > > > >>> Currently, I face ~160 errors in java-exec module unit tests. I > have > > > >>> divided them into seven groups: > > > >>> 1) SYSTEM ERROR: ClassCastException > > > >>> 2) Unexpected column errors > > > >>> 3) Runtime Exceptions > > > >>> 4) validation errors > > > >>> 5) RpcExceptions (generalized group: some of them should be fixed > by > > > the > > > >>> previous groups) > > > >>> 6) IllegalState errors (generalized group: some of them should be > > fixed > > > >> by > > > >>> the previous group) > > > >>> 7) Other errors > > > >>> > > > >>> Best regards, > > > >>> Roman Kulyk > > > >>> > > > >> > > > > > > > > >
