Just to clarify, what time frame are we looking for the Hackathon? Given the fact that 1.2 release is quite close, it probably makes more sense to do Hackathon during the 1.3 release time frame.
Thoughts? On Mon, Aug 31, 2015 at 9:58 AM, Jinfeng Ni <[email protected]> wrote: > Hackathon seems to be a brilliant idea. Definitely, it will help a lot if > we > could get Julian to join and help to review / work through these issues. > > > > On Mon, Aug 31, 2015 at 9:33 AM, Jacques Nadeau <[email protected]> > wrote: > >> I'd like to propose that we do a Hackathon to try to close up the gap. >> Hopefully, we could get Julian to join and work through these issues with >> him. I'm sure that there is a way that we can either merge our patches >> (with different options) or change them to use new extension points. It is >> bad for everyone to maintain a fork so I think everyone will be equally >> invested in getting things merged. >> >> Thoughts? >> >> -- >> Jacques Nadeau >> CTO and Co-Founder, Dremio >> >> On Sun, Aug 30, 2015 at 9:49 PM, Jinfeng Ni <[email protected]> >> wrote: >> >> > You are exactly right. We have 26 patches. Here they are >> > (--pretty=format:"%an %h %s") >> > >> > Jinfeng Ni 857ee95 VolcanoPlanner.clear() should clear ruleNames, in >> order >> > to avoid rule naming conflict error. >> > Jinfeng Ni ba2166f Make CalciteSchema extendable for different >> > implemetations by refactoring CalciteSchema and making it an interface >> > Hsuan-Yi Chu 046302d [CALCITE-827] Add a LogicalProject on top of >> > LogicalWindow to permute the columns in the same order as specified in >> the >> > SELECT list >> > Jinfeng Ni 2e076e1 Drill-specific change: Add back AbstractConverter in >> > RelSet.java and make sure AbstractConverter is not created when subset >> has >> > "NONE" conersion. >> > Mehant Baid 7ae335c Remove redundant invocation to ensureType in >> > RexBuilder.makeOver(). Don't invoke ensureType() for rewriting aggregate >> > expressions part of window functions. >> > Jinfeng Ni 3bcce80 CALCITE-628 related but not fix theproblem: Ensure >> > target traits are simple when use Frameworks or RelOptRule.convert() >> > method. >> > vkorukanti e1876f0 Add new method to ViewExpander interface to allow >> > passing SchemaRoot. >> > Jinfeng Ni 8c8acee Match Logical Rel in ReduceExpressionRule. >> > Jason Altekruse a29f007 Enable inheriting from previously singleton >> filter >> > and calc constant reduction rules. >> > Jason Altekruse 15d645f Make the consent expression Executor >> configurable >> > in FrameworkConfig. >> > Jinfeng Ni f176aa5 Disable the nullability cast checking in Filter. >> > Jinfeng Ni d0e2690 Add a new field name Suggester, so that duplicate >> names >> > will be renamed using the same policy as 0.9. >> > Jinfeng Ni 84bae71 Return validated row type and validated SqlNode when >> > call Planner.validate(). >> > Aman Sinha 034e6e9 Workaround for CALCITE-528: Use case-insensitive >> > comparison when creating output row type of a JoinRel. >> > Aman Sinha 1f2fb1c Don't use 'SumEmptyIsZero' (SUM0) window aggregate >> until >> > CALCITE-777 is fixed. >> > Jinfeng Ni fcacd35 Make public the constructor of >> > SqlSumEmptyIsZeroAggFunction, >> > Aman Sinha 13d6449 Make certain push project constructors protected such >> > that derived classes can access them. >> > Aman Sinha aebbb34 DRILL-1455: Add return type-inference strategy for >> > arithmetic operators when one of the arguments is ANY type. >> > Mehant Baid 32accb4 Some changes related to Sql ANY type. >> > Jinfeng Ni e182e2c [StarColumn] Use case insensitive in function name >> > comparison, in order to fix Sql Validator Exception when group by a >> > function expression. >> > Jinfeng Ni b75b374 [StarColumn] Make sure expression in the select list >> is >> > expanded at most once. >> > Jinfeng Ni bca12a6 [StarColumn] Keep the original field names from left, >> > when rewrite a IN/EXISTS/NOT IN /NOT EXISTS subquery. This is required >> to >> > support * column in schema-less system. >> > Jinfeng Ni eba9342 [StarColumn] Fix issues related to SqlIdentifier for >> > star column. >> > Hsuan-Yi Chu 70fcbca [StarColumn] When group-by a column, projecting on >> a >> > star which cannot be expanded at planning time, use ITEM operator to >> wrap >> > this column >> > Jinfeng Ni b3dc5b5 [StarColumn] Reverse one change in CALCITE-356, which >> > regresses AggChecker logic, after * query in schema-less table is added. >> > Jinfeng Ni 05fea03 [StarColumn] Support select * from schema-less table >> in >> > execution engine like Drill. Include support for view, subquery, CTE. >> > >> > I have pushed at least 2 patches to Calcite in the timeframe of >> > 1.4.0-release. While working on the rebasing work, I also try to wrap >> some >> > patches (including the "famous" calcite Schema change!) and make they >> ready >> > for pushing back to Calcite. >> > >> > In general, I agree that we should set a target to remove the forked >> > version. For the time frame of >> > 1.2 release, I will try to push at least 2 patches to Calcite master. I >> > also would like to have others >> > listed above spend sometime to push their corresponding patches to >> > Calcite. For example, I talked >> > to Jason before, and he kindly agreed to push his two patches to >> Calcite ( >> > they are pretty ready to >> > merge to calcite master). There is also [CALCITE-827], which has been >> > already in Calcite next >> > release, but not in 1.4.0-release. >> > >> > However, I'm not sure if we could reduce the # of patches to 13 in 1.2, >> and >> > especially reduce to 2 >> > by 1.3 release. >> > - Many patches are to solve Drill's specific issues, for instance, >> Star >> > column in schema-less >> > system, AbstractConverter in VolcanoPlanner). That's why we even do >> not >> > create Calcite >> > JIRA for them yet. In other words, Calcite would work perfectly >> > without those patches, while >> > Drill would fail lots of queries. >> > - For some patches, we have tried to push to Calcite, but Calcite >> > community does not accept >> > the patches. What should we do in such scenarios? >> > >> > I would welcome any help, if someone else would like to jump in and >> offer >> > his/her help. >> > >> > Regards, >> > >> > Jinfeng >> > >> > >> > >> > >> > >> > On Sun, Aug 30, 2015 at 4:59 PM, Jacques Nadeau <[email protected]> >> > wrote: >> > >> > > Its great that we have rebased on top of Calcite 1.4. However, it >> looks >> > > like we're still on a forked version of Calcite. When I look here: >> > > https://github.com/mapr/incubator-calcite/commits/DrillCalcite1.4.0, >> I >> > > count ~26 patches that still need to be merged (many of them without >> > even a >> > > Calcite bug id). >> > > >> > > I think we need to have a target number of non-merged patches or we'll >> > > never get off the fork. What do people think about bounding the >> release >> > on >> > > merging at least half of the items listed on that branch before we >> > release >> > > Drill 1.2 (~13) and we get down to no more than 2 for 1.3? >> > > >> > >> > >
