I'd like to propose that we do a Hackathon to try to close up the gap. Hopefully, we could get Julian to join and work through these issues with him. I'm sure that there is a way that we can either merge our patches (with different options) or change them to use new extension points. It is bad for everyone to maintain a fork so I think everyone will be equally invested in getting things merged.
Thoughts? -- Jacques Nadeau CTO and Co-Founder, Dremio On Sun, Aug 30, 2015 at 9:49 PM, Jinfeng Ni <[email protected]> wrote: > You are exactly right. We have 26 patches. Here they are > (--pretty=format:"%an %h %s") > > Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in order > to avoid rule naming conflict error. > Jinfeng Ni ba2166f Make CalciteSchema extendable for different > implemetations by refactoring CalciteSchema and making it an interface > Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of > LogicalWindow to permute the columns in the same order as specified in the > SELECT list > Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter in > RelSet.java and make sure AbstractConverter is not created when subset has > "NONE" conersion. > Mehant Baid 7ae335c Remove redundant invocation to ensureType in > RexBuilder.makeOver(). Don't invoke ensureType() for rewriting aggregate > expressions part of window functions. > Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure > target traits are simple when use Frameworks or RelOptRule.convert() > method. > vkorukanti e1876f0 Add new method to ViewExpander interface to allow > passing SchemaRoot. > Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule. > Jason Altekruse a29f007 Enable inheriting from previously singleton filter > and calc constant reduction rules. > Jason Altekruse 15d645f Make the consent expression Executor configurable > in FrameworkConfig. > Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter. > Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate names > will be renamed using the same policy as 0.9. > Jinfeng Ni 84bae71 Return validated row type and validated SqlNode when > call Planner.validate(). > Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive > comparison when creating output row type of a JoinRel. > Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate until > CALCITE-777 is fixed. > Jinfeng Ni fcacd35 Make public the constructor of > SqlSumEmptyIsZeroAggFunction, > Aman Sinha 13d6449 Make certain push project constructors protected such > that derived classes can access them. > Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for > arithmetic operators when one of the arguments is ANY type. > Mehant Baid 32accb4 Some changes related to Sql ANY type. > Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name > comparison, in order to fix Sql Validator Exception when group by a > function expression. > Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select list is > expanded at most once. > Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from left, > when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery. This is required to > support * column in schema-less system. > Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier for > star column. > Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting on a > star which cannot be expanded at planning time, use ITEM operator to wrap > this column > Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356, which > regresses AggChecker logic, after * query in schema-less table is added. > Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less table in > execution engine like Drill. Include support for view, subquery, CTE. > > I have pushed at least 2 patches to Calcite in the timeframe of > 1.4.0-release. While working on the rebasing work, I also try to wrap some > patches (including the "famous" calcite Schema change!) and make they ready > for pushing back to Calcite. > > In general, I agree that we should set a target to remove the forked > version. For the time frame of > 1.2 release, I will try to push at least 2 patches to Calcite master. I > also would like to have others > listed above spend sometime to push their corresponding patches to > Calcite. For example, I talked > to Jason before, and he kindly agreed to push his two patches to Calcite ( > they are pretty ready to > merge to calcite master). There is also [CALCITE-827], which has been > already in Calcite next > release, but not in 1.4.0-release. > > However, I'm not sure if we could reduce the # of patches to 13 in 1.2, and > especially reduce to 2 > by 1.3 release. > - Many patches are to solve Drill's specific issues, for instance, Star > column in schema-less > system, AbstractConverter in VolcanoPlanner). That's why we even do not > create Calcite > JIRA for them yet. In other words, Calcite would work perfectly > without those patches, while > Drill would fail lots of queries. > - For some patches, we have tried to push to Calcite, but Calcite > community does not accept > the patches. What should we do in such scenarios? > > I would welcome any help, if someone else would like to jump in and offer > his/her help. > > Regards, > > Jinfeng > > > > > > On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <[email protected]> > wrote: > > > Its great that we have rebased on top of Calcite 1.4. However, it looks > > like we're still on a forked version of Calcite. When I look here: > > https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0, I > > count ~26 patches that still need to be merged (many of them without > even a > > Calcite bug id). > > > > I think we need to have a target number of non-merged patches or we'll > > never get off the fork. What do people think about bounding the release > on > > merging at least half of the items listed on that branch before we > release > > Drill 1.2 (~13) and we get down to no more than 2 for 1.3? > > >
