Hackathon seems to be a brilliant idea. Definitely, it will help a lot if we could get Julian to join and help to review / work through these issues.
On Mon, Aug 31, 2015 at 9:33 AM, Jacques Nadeau <[email protected]> wrote: > I'd like to propose that we do a Hackathon to try to close up the gap. > Hopefully, we could get Julian to join and work through these issues with > him. I'm sure that there is a way that we can either merge our patches > (with different options) or change them to use new extension points. It is > bad for everyone to maintain a fork so I think everyone will be equally > invested in getting things merged. > > Thoughts? > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Sun, Aug 30, 2015 at 9:49 PM, Jinfeng Ni <[email protected]> wrote: > > > You are exactly right. We have 26 patches. Here they are > > (--pretty=format:"%an %h %s") > > > > Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in > order > > to avoid rule naming conflict error. > > Jinfeng Ni ba2166f Make CalciteSchema extendable for different > > implemetations by refactoring CalciteSchema and making it an interface > > Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of > > LogicalWindow to permute the columns in the same order as specified in > the > > SELECT list > > Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter in > > RelSet.java and make sure AbstractConverter is not created when subset > has > > "NONE" conersion. > > Mehant Baid 7ae335c Remove redundant invocation to ensureType in > > RexBuilder.makeOver(). Don't invoke ensureType() for rewriting aggregate > > expressions part of window functions. > > Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure > > target traits are simple when use Frameworks or RelOptRule.convert() > > method. > > vkorukanti e1876f0 Add new method to ViewExpander interface to allow > > passing SchemaRoot. > > Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule. > > Jason Altekruse a29f007 Enable inheriting from previously singleton > filter > > and calc constant reduction rules. > > Jason Altekruse 15d645f Make the consent expression Executor configurable > > in FrameworkConfig. > > Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter. > > Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate > names > > will be renamed using the same policy as 0.9. > > Jinfeng Ni 84bae71 Return validated row type and validated SqlNode when > > call Planner.validate(). > > Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive > > comparison when creating output row type of a JoinRel. > > Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate > until > > CALCITE-777 is fixed. > > Jinfeng Ni fcacd35 Make public the constructor of > > SqlSumEmptyIsZeroAggFunction, > > Aman Sinha 13d6449 Make certain push project constructors protected such > > that derived classes can access them. > > Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for > > arithmetic operators when one of the arguments is ANY type. > > Mehant Baid 32accb4 Some changes related to Sql ANY type. > > Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name > > comparison, in order to fix Sql Validator Exception when group by a > > function expression. > > Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select list > is > > expanded at most once. > > Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from left, > > when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery. This is required > to > > support * column in schema-less system. > > Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier for > > star column. > > Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting on a > > star which cannot be expanded at planning time, use ITEM operator to wrap > > this column > > Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356, which > > regresses AggChecker logic, after * query in schema-less table is added. > > Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less table > in > > execution engine like Drill. Include support for view, subquery, CTE. > > > > I have pushed at least 2 patches to Calcite in the timeframe of > > 1.4.0-release. While working on the rebasing work, I also try to wrap > some > > patches (including the "famous" calcite Schema change!) and make they > ready > > for pushing back to Calcite. > > > > In general, I agree that we should set a target to remove the forked > > version. For the time frame of > > 1.2 release, I will try to push at least 2 patches to Calcite master. I > > also would like to have others > > listed above spend sometime to push their corresponding patches to > > Calcite. For example, I talked > > to Jason before, and he kindly agreed to push his two patches to Calcite > ( > > they are pretty ready to > > merge to calcite master). There is also [CALCITE-827], which has been > > already in Calcite next > > release, but not in 1.4.0-release. > > > > However, I'm not sure if we could reduce the # of patches to 13 in 1.2, > and > > especially reduce to 2 > > by 1.3 release. > > - Many patches are to solve Drill's specific issues, for instance, Star > > column in schema-less > > system, AbstractConverter in VolcanoPlanner). That's why we even do > not > > create Calcite > > JIRA for them yet. In other words, Calcite would work perfectly > > without those patches, while > > Drill would fail lots of queries. > > - For some patches, we have tried to push to Calcite, but Calcite > > community does not accept > > the patches. What should we do in such scenarios? > > > > I would welcome any help, if someone else would like to jump in and offer > > his/her help. > > > > Regards, > > > > Jinfeng > > > > > > > > > > > > On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <[email protected]> > > wrote: > > > > > Its great that we have rebased on top of Calcite 1.4. However, it > looks > > > like we're still on a forked version of Calcite. When I look here: > > > https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0, I > > > count ~26 patches that still need to be merged (many of them without > > even a > > > Calcite bug id). > > > > > > I think we need to have a target number of non-merged patches or we'll > > > never get off the fork. What do people think about bounding the > release > > on > > > merging at least half of the items listed on that branch before we > > release > > > Drill 1.2 (~13) and we get down to no more than 2 for 1.3? > > > > > >
