[jira] [Created] (CALCITE-3968) Disable JoinPushThroughJoinRule.left by default

2020-05-02 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3968: -- Summary: Disable JoinPushThroughJoinRule.left by default Key: CALCITE-3968 URL: https://issues.apache.org/jira/browse/CALCITE-3968 Project: Calcite

Re: [ANNOUNCE] New committer: Forward Xu

2020-05-01 Thread Haisheng Yuan
Congrats, Forward. Haisheng On 2020/05/01 20:19:47, Vineet G wrote: > Congratulations! > > > On Apr 30, 2020, at 10:57 AM, Julian Hyde wrote: > > > > Congratulations and welcome, Forward! Thank you for your contributions. > > > > It would be great to add TBDS (and its logo) to the

Re: [ANNOUNCE] New committer: Wang Yanlin

2020-05-01 Thread Haisheng Yuan
Congrats, Yanlin. Looking forward to more. Haisheng On 2020/05/01 20:19:27, Vineet G wrote: > Congratulations! > > > On May 1, 2020, at 12:11 PM, Julian Hyde wrote: > > > > Yanlin, > > > > Please create a PR adding an Ant Financial paragraph to that page. in > > the PR, include the URL of

Re: [ANNOUNCE] New committer: Jin Xing

2020-05-01 Thread Haisheng Yuan
Welcome, Jin Xing. Looking forward to more. Haisheng On 2020/05/01 20:43:39, Xiening Dai wrote: > Congrats Jin Xing. Well deserved! > > > On Apr 30, 2020, at 7:54 PM, XING JIN wrote: > > > > Thanks a lot, Julian ~ > > I'm not from MaxCompute team, but from big data platform in Alibaba Ant >

[jira] [Created] (CALCITE-3966) Trigger rules for existing RelSubset when it becomes delivered

2020-05-01 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3966: -- Summary: Trigger rules for existing RelSubset when it becomes delivered Key: CALCITE-3966 URL: https://issues.apache.org/jira/browse/CALCITE-3966 Project

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-30 Thread Haisheng Yuan
ertain parts > >> of > >>>> he existing planner so we should make sure that we take this into > >> account. > >>>>>>>> > >>>>>>>> It's on my radar. I'm going to add these tests. >

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-26 Thread Haisheng Yuan
ner that makes us believe the > > new planner is stable and powerful enough. I mean obviously the current > > rule tests are far away from enough to support the new planner > > - We should give a more detailed design doc about the new planner, > > especially about the in

Re: [ANNOUNCE] New committer: Vineet Garg

2020-04-25 Thread Haisheng Yuan
Congrats, Vineet! On 2020/04/25 22:18:35, Forward Xu wrote: > Congratulations > > best, > Forward > > Francis Chuang 于2020年4月26日周日 上午6:04写道: > > > Congrats, Vineet! > > > > On 26/04/2020 7:52 am, Stamatis Zampetakis wrote: > > > Apache Calcite's Project Management Committee (PMC) has

[jira] [Created] (CALCITE-3949) RelDistributions.of() and RelCollations.of() should canonize trait instance

2020-04-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3949: -- Summary: RelDistributions.of() and RelCollations.of() should canonize trait instance Key: CALCITE-3949 URL: https://issues.apache.org/jira/browse/CALCITE-3949

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Haisheng Yuan
t; around and yelling "the end is nigh!". > > Thank you for taking these mumbled thoughts into account. > > Bestest Regards, > Andrii Tsvielodub > > On Tue, 21 Apr 2020 at 21:08, Haisheng Yuan wrote: > > > Hi Andrii, > > > > > I guess

Re: RelMetadataQuery.getRowCount stackoverflow

2020-04-21 Thread Haisheng Yuan
[2] https://issues.apache.org/jira/browse/CALCITE-3505 > [3] https://issues.apache.org/jira/browse/CALCITE-2223 > [4] https://issues.apache.org/jira/browse/CALCITE-3124 > > On Tue, Apr 21, 2020 at 6:43 AM Haisheng Yuan wrote: > > > Can you add a reproducible test case and lo

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-21 Thread Haisheng Yuan
> > planner would lower the risk. We don’t know what we don’t know. If we > > > introduced an issue while modifying the planner, most likely we would do > > > the same with new planner class. A new planner doesn’t necessarily > > prevent > > > the issue from happen

[jira] [Created] (CALCITE-3944) Move dumpSets and dumpGraphviz out of VolcanoPlanner

2020-04-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3944: -- Summary: Move dumpSets and dumpGraphviz out of VolcanoPlanner Key: CALCITE-3944 URL: https://issues.apache.org/jira/browse/CALCITE-3944 Project: Calcite

Re: RelMetadataQuery.getRowCount stackoverflow

2020-04-20 Thread Haisheng Yuan
Can you add a reproducible test case and log a JIRA? It may be different with CALCITE-2057. People who is interested will investigate the issue. On 2020/04/21 04:24:31, JiaTao Tao wrote: > Thanks > I didn't add any new rule, just these: > > CONSTANT_REDUCTION_RULES > ABSTRACT_RELATIONAL_RULES

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-20 Thread Haisheng Yuan
from happening, but just delay its surfacing, which is worse IMO. > > > > There’s one obvious benefit with new planner is that we can provide some > > sort of isolation so the change won’t cause test baseline updates, which > > could be painful at times. We should see if we

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-20 Thread Haisheng Yuan
it. Initially this upper bound for root is infinite - > > meaning that no feasible plan is available. Every time when we find a > > physical plan we update this upper bound, then start the search again. We > > could stop the search when the cost is less than a pre-defined thresh

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-19 Thread Haisheng Yuan
d Bound Space > Pruning it’s non-trivial task to get right search direction and most > promising plans in first planning iterations) > c) won’t be able to tune the good enough plan during several similar queries > are executed > > Regards, > Igor > > > > 19

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-19 Thread Haisheng Yuan
nse time) > > This mean we need all needed physical implementations after each logical > transformation is applied. > > Regards, > Igor > > > вс, 19 апр. 2020 г., 13:55 Haisheng Yuan : > > > Hi Igor, > > > > There will be no rule queue anymore

Re: [DISCUSS] Towards Cascades Optimizer

2020-04-19 Thread Haisheng Yuan
move its state > Optimized -> Optimizing and repeat described above operations. > > Does it look like true? > > Regards, > Igor > > > вс, 19 апр. 2020 г., 6:52 Haisheng Yuan : > > > Hi, > > > > In the past few months, we have discussed a lot abo

[DISCUSS] Towards Cascades Optimizer

2020-04-18 Thread Haisheng Yuan
Hi, In the past few months, we have discussed a lot about Cascades style top-down optimization, including on-demand trait derivation/request, rule apply, branch and bound space pruning. Now we think it is time to move towards these targets. We will separate it into several small issues, and

[jira] [Created] (CALCITE-3937) Only fire rule for RelSubset when it is derived

2020-04-18 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3937: -- Summary: Only fire rule for RelSubset when it is derived Key: CALCITE-3937 URL: https://issues.apache.org/jira/browse/CALCITE-3937 Project: Calcite

[jira] [Created] (CALCITE-3932) Make data type cache thread local, non-evictable

2020-04-16 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3932: -- Summary: Make data type cache thread local, non-evictable Key: CALCITE-3932 URL: https://issues.apache.org/jira/browse/CALCITE-3932 Project: Calcite

[jira] [Created] (CALCITE-3927) RelSubset is not fired for rule when set gets merged

2020-04-15 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3927: -- Summary: RelSubset is not fired for rule when set gets merged Key: CALCITE-3927 URL: https://issues.apache.org/jira/browse/CALCITE-3927 Project: Calcite

Re: Re: Flakey test on Linux (JDK 11), Avatica master

2020-04-14 Thread Haisheng Yuan
per your mail. If the analysis above makes sense, I can create an issue in JIRA and fix the problem to make digest & toString return the same value for TIMESTAMP. Best, Xu Haisheng Yuan 于2020年4月14日周二 上午10:00写道: > Hi, > > There are frequently flakey test on Linux(JDK 11) related

Flakey test on Linux (JDK 11), Avatica master

2020-04-13 Thread Haisheng Yuan
Hi, There are frequently flakey test on Linux(JDK 11) related with TIMESTAMP(0) vs TIMESTAMP. Below is failure test from [1]. FAILURE 0.0sec, org.apache.calcite.test.SqlToRelConverterExtendedTest > testTableValuedFunctionTumbleWithSubQueryParam() 1663 org.opentest4j.AssertionFailedError: plan

[jira] [Created] (CALCITE-3918) SubQueryFilterRemoveRule failed to decorrelate subquery for TPCH q17

2020-04-11 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3918: -- Summary: SubQueryFilterRemoveRule failed to decorrelate subquery for TPCH q17 Key: CALCITE-3918 URL: https://issues.apache.org/jira/browse/CALCITE-3918 Project

[jira] [Created] (CALCITE-3917) Revive pruned node when a rule generates RelNode that is already pruned

2020-04-11 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3917: -- Summary: Revive pruned node when a rule generates RelNode that is already pruned Key: CALCITE-3917 URL: https://issues.apache.org/jira/browse/CALCITE-3917

[jira] [Created] (CALCITE-3916) Apply rules bottom up by RelSet

2020-04-11 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3916: -- Summary: Apply rules bottom up by RelSet Key: CALCITE-3916 URL: https://issues.apache.org/jira/browse/CALCITE-3916 Project: Calcite Issue Type

Re: Cannot create physical plan with join

2020-04-11 Thread Haisheng Yuan
I have to admit that this is the most error-prone point when creating physical operator for first time user. Perhaps we can add some highlighted notes or examples to site docs to prevent people making similar mistake again. In Calcite, logical / physical operator is not only operator, but also

[jira] [Created] (CALCITE-3911) JoinCommuteRule may generate wrong plan for SEMI/ANTI join

2020-04-10 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3911: -- Summary: JoinCommuteRule may generate wrong plan for SEMI/ANTI join Key: CALCITE-3911 URL: https://issues.apache.org/jira/browse/CALCITE-3911 Project: Calcite

Re: Github autolinks to JIRA issues

2020-04-08 Thread Haisheng Yuan
l=project%20%3D%20INFRA%20AND%20text%20~%20autolinks > > On 9/04/2020 10:24 am, Haisheng Yuan wrote: > > Is it possible to add autolinks to JIRA issues as shown in [1]? > > > > [1] > > https://help.github.com/en/github/administering-a-repository/configuring-autolinks-to-reference-external-resources > > > > - Haisheng > > >

Github autolinks to JIRA issues

2020-04-08 Thread Haisheng Yuan
Is it possible to add autolinks to JIRA issues as shown in [1]? [1] https://help.github.com/en/github/administering-a-repository/configuring-autolinks-to-reference-external-resources - Haisheng

Re: [DICUSS] Support building physical RelNode in Calcite

2020-04-07 Thread Haisheng Yuan
Thanks Xiening for moving this to dev list. Unlike logical operators, with which many systems share the same structure, physical operators are pretty implementation dependent. RelBuilder provides the common interface to create aggregate, join. But what kind of physical aggregate? HashAgg,

Re: Re: Draft board report for April 2020

2020-04-05 Thread Haisheng Yuan
was Haisheng Yuan on 2019-11-11. - Feng Zhu was added as committer on 2020-02-29 ## Project Activity: Avatica 1.16.0 was released in the middle of December, including numerous bug fixes and security improvements while the build system has been migrated from Maven to gradle. Calcite 1.22.0 was released

Re: Re: Join Pruning/Rewrites

2020-04-01 Thread Haisheng Yuan
In many cases, they are not equivalent, unless the top level join is a semi-join, e.g. (T1⋈T2)⋉(T2⟖T3) on T1.a=T3.a can be optimized to (T1⋈T2)⋉T3 on T1.a=T3.a It is better to give a concrete example, so that we can discuss case by case. - Haisheng

[jira] [Created] (CALCITE-3891) Remove use of Pair.zip in RelTraitSet.satisfies

2020-04-01 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3891: -- Summary: Remove use of Pair.zip in RelTraitSet.satisfies Key: CALCITE-3891 URL: https://issues.apache.org/jira/browse/CALCITE-3891 Project: Calcite

Re: Query planning takes forever on TPC-H queries #5 and #7

2020-04-01 Thread Haisheng Yuan
They are known issues. Calcite isn't doing well on join reordering, there are multiple reasons contributing to the inefficiency. Try to turn off the rule that does join associativity. - Haisheng -- 发件人:Boyan Kolev 日 期:2020年04月01日

[jira] [Created] (CALCITE-3889) Add apply(Mappings.Mapping) to RelTrait and RelTraitSet

2020-03-31 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3889: -- Summary: Add apply(Mappings.Mapping) to RelTrait and RelTraitSet Key: CALCITE-3889 URL: https://issues.apache.org/jira/browse/CALCITE-3889 Project: Calcite

Re: Re: Re: Doubts about TPCH queries with volcano planner

2020-03-29 Thread Haisheng Yuan
uot;) > > TPCH19 May be timeout > > TPCH22 @Disabled("IllegalArgumentException during decorrelation") > > > And my statement may not be clear, what I wonder is that with calcite's > default rule set (RelOptUtil#registerDefaultRules), why there are so many > TP

[jira] [Created] (CALCITE-3886) Execute substitution rule according to the order they get matched

2020-03-28 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3886: -- Summary: Execute substitution rule according to the order they get matched Key: CALCITE-3886 URL: https://issues.apache.org/jira/browse/CALCITE-3886 Project

Re: Re: Doubts about TPCH queries with volcano planner

2020-03-26 Thread Haisheng Yuan
No one can see the picture. He has to paste the link to the picture. Calcite's volcano planner is production ready, without any question. You need to customize the adaptor and rules to fit into your system. - Haisheng --

[jira] [Created] (CALCITE-3879) Rel Id generator should not be static

2020-03-26 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3879: -- Summary: Rel Id generator should not be static Key: CALCITE-3879 URL: https://issues.apache.org/jira/browse/CALCITE-3879 Project: Calcite Issue Type

Re: Re: NPE at VolcanoPlanner.setRoot()

2020-03-24 Thread Haisheng Yuan
Hi João, Can you provide minimal reproducible test cases? You can log a JIRA if you believe this is a bug. - Haisheng -- 发件人:João Silva 日 期:2020年03月24日 23:40:12 收件人: 主 题:Re: NPE at VolcanoPlanner.setRoot() Currently on 1.21.0. And

[jira] [Created] (CALCITE-3868) Remove redundant ruleSet and ruleNames in VolcanoPlanner

2020-03-23 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3868: -- Summary: Remove redundant ruleSet and ruleNames in VolcanoPlanner Key: CALCITE-3868 URL: https://issues.apache.org/jira/browse/CALCITE-3868 Project: Calcite

[jira] [Created] (CALCITE-3865) RelCollationTraitDef.canConvert should always return true

2020-03-19 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3865: -- Summary: RelCollationTraitDef.canConvert should always return true Key: CALCITE-3865 URL: https://issues.apache.org/jira/browse/CALCITE-3865 Project: Calcite

Re: Re: Re: Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-03-14 Thread Haisheng Yuan
. - Haisheng -- 发件人:Vladimir Ozerov 日 期:2020年03月15日 04:18:53 收件人:Haisheng Yuan 抄 送:dev@calcite.apache.org (dev@calcite.apache.org) 主 题:Re: Re: Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels Hi Haisheng, You

Re: Re: [DISCUSS] Refactor how planner rules are parameterized

2020-03-14 Thread Haisheng Yuan
I don't think it is worth the refactoring. People who want to customize the rule, in most cases, won't be satisfied by a different parameter, they most likely still need to rewrite (copy & paste) the rule with some slightly their own logic. For many Calcite users, the rule is not reusable even

Re: Re: Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-03-14 Thread Haisheng Yuan
zation algorithm (e.g. see step 3), which might be a blocker for the current Calcite users. So maybe it is better to think of a new optimizer, rather than fixing VolcanoOptimizer. Regards, Vladimir. вт, 14 янв. 2020 г. в 23:52, Haisheng Yuan : > On the other hand, if we don't preprocess and

Re: [DISCUSS] Adapter's performance is not that fast

2020-02-26 Thread Haisheng Yuan
For the first time it is indeed slow, because it has startup cost and code generation. Can you post how you test it (by single run or multiple runs) and how does the flame graph look like? - Haisheng -- 发件人:Xiangwei Wei 日 

[jira] [Created] (CALCITE-3819) Prune parent RelNode when merging child RelSet with parent RelSet

2020-02-24 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3819: -- Summary: Prune parent RelNode when merging child RelSet with parent RelSet Key: CALCITE-3819 URL: https://issues.apache.org/jira/browse/CALCITE-3819 Project

Re: Re: [DISCUSS] Commit messages, again

2020-02-24 Thread Haisheng Yuan
> Should we expect at least one +1 from other committers before merging the > PR? Maybe yes, maybe no. > But at least we can leave more time to others to review the pr, rather than > merging it rapidly. Agreed. If the contributor or committer opens a PR (pull request), I view it as a request for

Re: Re: Translation of SQL EXISTS

2020-02-19 Thread Haisheng Yuan
-- 发件人:Christian Beikov 日 期:2020年02月20日 03:11:02 收件人:Haisheng Yuan; Apache Calcite dev list 主 题:Re: Translation of SQL EXISTS Hey Haisheng, it is nice to have a rule that detects such patterns but do you agree that it would be better to generate

Re: Translation of SQL EXISTS

2020-02-19 Thread Haisheng Yuan
Hi Christian, For the query in your example, Calcite first generates inner join plan with aggregate child, then through SemJoinRule transform the inner join to semi or antisemi join. The reason to have inner join is that it allows join commutativity, which is good for generating a potential

[jira] [Created] (CALCITE-3757) When merging sets, relnodes may be reregistered multiple times

2020-01-27 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3757: -- Summary: When merging sets, relnodes may be reregistered multiple times Key: CALCITE-3757 URL: https://issues.apache.org/jira/browse/CALCITE-3757 Project

[jira] [Created] (CALCITE-3756) RelSubset should not match operand(RelNode.class)

2020-01-23 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3756: -- Summary: RelSubset should not match operand(RelNode.class) Key: CALCITE-3756 URL: https://issues.apache.org/jira/browse/CALCITE-3756 Project: Calcite

[jira] [Created] (CALCITE-3755) Ascending rule match with RelSubset operand doesn't work

2020-01-23 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3755: -- Summary: Ascending rule match with RelSubset operand doesn't work Key: CALCITE-3755 URL: https://issues.apache.org/jira/browse/CALCITE-3755 Project: Calcite

[jira] [Created] (CALCITE-3753) Always try to match and execute substitution rule first and remove rulematch ordering

2020-01-21 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3753: -- Summary: Always try to match and execute substitution rule first and remove rulematch ordering Key: CALCITE-3753 URL: https://issues.apache.org/jira/browse/CALCITE-3753

[jira] [Created] (CALCITE-3744) Duplicate RuleMatches when RelSet gets merged

2020-01-16 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3744: -- Summary: Duplicate RuleMatches when RelSet gets merged Key: CALCITE-3744 URL: https://issues.apache.org/jira/browse/CALCITE-3744 Project: Calcite Issue

Re: Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-01-14 Thread Haisheng Yuan
-- 发件人:Haisheng Yuan 日 期:2020年01月14日 12:06:17 收件人:dev@calcite.apache.org 主 题:Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels The example is valid if Calcite doesn't do normalization or preprocessing before going

Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-01-13 Thread Haisheng Yuan
ntioned in the previous > e-mails should be solved in the first place. We need first to make > Volcano optimizer work right and only then make some improvements like > search space pruning. > > I would love to join this work to improve Volcano planner. Looking > forward for design d

Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-01-11 Thread Haisheng Yuan
- lack of group properties etc. I guess I'm not aware of these problems. -- Kind Regards Roman Kondakov On 08.01.2020 02:21, Haisheng Yuan wrote: >> @Haisheng, are you doing something like that? > Kind of, but not exactly. It is about on-demand trait propagation. > > @Roman seem

Re: Question about Volcano's planner root converters

2020-01-07 Thread Haisheng Yuan
> 1. Should we change the condition of root converters creation from the direct traitset difference to the difference with considering traits hierarchy (like [] includes [1])? I think so. Specifically non-empty required trait diff. Singleton is the only diff in your example. It doesn't care

Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-01-07 Thread Haisheng Yuan
> @Haisheng, are you doing something like that? Kind of, but not exactly. It is about on-demand trait propagation. @Roman seems to be keen on space pruning for Calcite. But IMHO, for now, the main reason of Calcite's poor performance is not lack of branch & bound space puning, but - rule

Re: Re: [DISCUSS] Proposal to add API to force rules matching specific rels

2020-01-07 Thread Haisheng Yuan
It is still on my radar. I have been busy these days, but will send out a design doc in a few days. But just a heads up, the change would be larger than everyone's expected. - Haisheng -- 发件人:Danny Chan 日 期:2020年01月07日 17:29:46

[jira] [Created] (CALCITE-3676) VolcanoPlanner. dumpGraphviz should handle exception gracefully

2020-01-03 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3676: -- Summary: VolcanoPlanner. dumpGraphviz should handle exception gracefully Key: CALCITE-3676 URL: https://issues.apache.org/jira/browse/CALCITE-3676 Project

[jira] [Created] (CALCITE-3668) VolcanoPlanner doesn't match all the RelSubSet in matchRecursive

2020-01-02 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3668: -- Summary: VolcanoPlanner doesn't match all the RelSubSet in matchRecursive Key: CALCITE-3668 URL: https://issues.apache.org/jira/browse/CALCITE-3668 Project

Re: Re: Draft board report for January 2020

2020-01-01 Thread Haisheng Yuan
s and 22 PMC members in this project. > The Committer-to-PMC ratio is roughly 2:1. > > Community changes, past quarter: > - Danny Chen was added to the PMC on 2019-10-30. > - Haisheng Yuan was added to the PMC on 2019-11-11. > - Stamatis Zampetakis was appointed as PMC chair on 20

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-30 Thread Haisheng Yuan
Normalizing scalar expressions is helpful, especially for deduplicating derived constraints that is pushed down to child relations. We should not stop making improvements just because there are many plan changes. How about adding optimizer config option to enable/disable the feature?

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Haisheng Yuan
> For instance, it thinks Join(A, B, $0=$1) and Join(A, B, $1=$0) are different > joins, however, they are equivalent. How is the alternative generated? I would rather check how to stop generating this alternative than reordering predicates. - Haisheng

Re: Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Haisheng Yuan
>> =(CAST(PREV(UP.$0, 0)):INTEGER NOT NULL, 100) I don't see the value of reordering this kind of expression. In GPDB, var vs constant comparison is reordered for input ref only, without additional function calls, like $1=5, because it is transformed into range constraints for rex

Re: Re: [DISCUSS] CALCITE-2450 reorder predicates to a canonical form

2019-12-29 Thread Haisheng Yuan
I recommend the way GPDB does. Normalize the the logical plan expression in the preprocessing phase: - variable always left, constant always right for applicable binary operators; - for join conditions, left operand always comes from left relation, right operand always comes from right relation

Re: Re: [ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis

2019-12-18 Thread Haisheng Yuan
Congratulations Stamatis! - Haisheng -- 发件人:Michael Mior 日 期:2019年12月19日 07:23:51 收件人: 主 题:Re: [ANNOUNCE] New Calcite PMC chair: Stamatis Zampetakis Congratulations Stamatis! -- Michael Mior mm...@apache.org Le mer. 18 déc. 2019 à

Re: [DISCUSS] Tests vs multiline strings

2019-12-14 Thread Haisheng Yuan
The plan becomes not so easier to read, which is more important to me. Unless we change $ to other symbol, I am inclined to keep using Java. I am not sure why schemas in MaterializationTest require column names to be quoted, If we can change that, queries will be much easier to read (even for

Re: Quicksql

2019-12-11 Thread Haisheng Yuan
ope Calcite move towards. > > > Give my best wishes to Calcite community. > > > Thanks, > Trista > > > Juan Pan > > > panj...@apache.org > Juan Pan(Trista), Apache ShardingSphere > > > On 12/11/2019 10:53,Haisheng Yuan wrote: &g

Re: Re: Quicksql

2019-12-10 Thread Haisheng Yuan
As far as I know, users still need to register tables from other data sources before querying it. QuickSQL uses Calcite for parsing queries and optimizing logical expressions with several transformation rules. The query on different data source will then be registered as temp spark tables (with

Re: Re: Re: Re: Volcano's problem with trait propagation: current state and future

2019-12-09 Thread Haisheng Yuan
e-66ab426494c1.h.y...@alibaba-inc.com%3e - Haisheng -- 发件人:Vladimir Ozerov 日 期:2019年12月06日 18:00:01 收件人:Haisheng Yuan 抄 送:dev@calcite.apache.org (dev@calcite.apache.org) 主 题:Re: Re: Re: Volcano's problem with trait propagation: current state and future "all we know is their collations"

[jira] [Created] (CALCITE-3576) Remove Enumerable convention check in FilterIntoJoinRule

2019-12-06 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3576: -- Summary: Remove Enumerable convention check in FilterIntoJoinRule Key: CALCITE-3576 URL: https://issues.apache.org/jira/browse/CALCITE-3576 Project: Calcite

Re: Re: Re: Volcano's problem with trait propagation: current state and future

2019-12-05 Thread Haisheng Yuan
-- 发件人:Haisheng Yuan 日 期:2019年12月06日 09:07:43 收件人:Vladimir Ozerov; dev@calcite.apache.org (dev@calcite.apache.org) 主 题:Re: Re: Volcano's problem with trait propagation: current state and future Generally agree with what Vladimir said. I

Re: Re: Volcano's problem with trait propagation: current state and future

2019-12-05 Thread Haisheng Yuan
Generally agree with what Vladimir said. I think what Calcite has is logical optimization or exploration, there are almost no physical optimization, Calcite leaves it to third party implementators. One of my friends at University of Wisconsin–Madison database research group told me that they

Re: Release managers

2019-11-29 Thread Haisheng Yuan
Hi Stamatis, I can volunteer to be the release manager for v1.23 or v1.24. - Haisheng -- 发件人:Stamatis Zampetakis 日 期:2019年11月30日 00:53:34 收件人: 主 题:Release managers Hi all, The pre-selection of release managers [1] has worked very

Re: Question about SortJoinTransposeRule and Inner Joins

2019-11-25 Thread Haisheng Yuan
Yes, we can. Currently the rule applies only on left/right outer join, because the cardinality of join will always be greater than or equal with the cardinality of outer relation. For inner join, the join cardinality may be much less, in which case, sorting on the join output might be cheaper.

Re: Re: Re: CALCITE-2905: Maven -> Gradle: any thoughts

2019-11-21 Thread Haisheng Yuan
est, Danny Chan 在 2019年11月21日 +0800 AM9:24,Haisheng Yuan ,写道: > Hi Vladimir, > > I had the same error: > Unable to load class 'org.gradle.api.internal.plugins.DefaultConvention'. > > IntelliJ IDEA 2018.3.4 (Community Edition) > Build #IC-183.5429.30, built on January 28, 2019 >

Re: Re: CALCITE-2905: Maven -> Gradle: any thoughts

2019-11-20 Thread Haisheng Yuan
Hi Vladimir, I had the same error: Unable to load class 'org.gradle.api.internal.plugins.DefaultConvention'. IntelliJ IDEA 2018.3.4 (Community Edition) Build #IC-183.5429.30, built on January 28, 2019 JRE: 1.8.0_152-release-1343-b26 x86_64 JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o macOS

[jira] [Created] (CALCITE-3521) CalciteSystemProperty can't load config file

2019-11-19 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3521: -- Summary: CalciteSystemProperty can't load config file Key: CALCITE-3521 URL: https://issues.apache.org/jira/browse/CALCITE-3521 Project: Calcite Issue

Re: Re: [DISCUSS] Disable Travis and Appveyor for Avatica and Avatica-Go

2019-11-18 Thread Haisheng Yuan
Agree, AppVeyor takes more time for queueing. We can remove AppVeyor and keep Travis for a while. - Haisheng -- 发件人:Vladimir Sitnikov 日 期:2019年11月19日 05:42:54 收件人:Apache Calcite dev list 主 题:Re: [DISCUSS] Disable Travis and Appveyor

Re: Top-down pass of an optimized plan

2019-11-12 Thread Haisheng Yuan
Hi Makis, You can use HepPlanner and your customized transoformation rule to rewrite the optimized plan. Top-down or Bottom-up can also be specified. Many systems have this kind of post processing phase. - Haisheng --

Re: Re: [ANNOUNCE] Haisheng Yuan joins Calcite PMC

2019-11-11 Thread Haisheng Yuan
] Haisheng Yuan joins Calcite PMC Congratulations Haisheng ~ You well deserved ! Kevin Risden 于2019年11月12日周二 上午3:13写道: > Congrats and welcome! > > Kevin Risden > > > On Mon, Nov 11, 2019 at 2:10 PM Chunhui Shi wrote: > > > Congratulations! > > > > On M

[jira] [Created] (CALCITE-3492) Exception thrown when terms has 1 RexNode in RexUtil.simplifyOrs()

2019-11-11 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3492: -- Summary: Exception thrown when terms has 1 RexNode in RexUtil.simplifyOrs() Key: CALCITE-3492 URL: https://issues.apache.org/jira/browse/CALCITE-3492 Project

Re: Re: Re: Optimizer: All the inputs have relevant nodes, however the cost is still infinite.

2019-11-08 Thread Haisheng Yuan
Yes, looks like EnumerableTableFunctionScan doesn't override computeSelfCost. - Haisheng -- 发件人:Haisheng Yuan 日 期:2019年11月09日 04:01:19 收件人:Apache Calcite dev list 主 题:Re: Re: Optimizer: All the inputs have relevant nodes, however

Re: Re: Optimizer: All the inputs have relevant nodes, however the cost is still infinite.

2019-11-08 Thread Haisheng Yuan
It is not surprising to get an infinitive cost, since the operators in the plan are logical operators, which need to be converted to physical operators to be costed. Did you try to add some implementation rules to the rule set, e.g. EnumerableProjectRule, EnumerableTableFunctionScanRule, etc..

Re: Re: Question about Interpreter and Corelations

2019-11-08 Thread Haisheng Yuan
I am afraid this query can't be easily decorrelated. - Haisheng -- 发件人:Zoltan Farkas 日 期:2019年11月08日 22:46:53 收件人: 主 题:Re: Question about Interpreter and Corelations Thanks Julian, Any idea how could I de-corelate the query?

[jira] [Created] (CALCITE-3483) Make RexLiteral member fields accessible by sub-class

2019-11-07 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3483: -- Summary: Make RexLiteral member fields accessible by sub-class Key: CALCITE-3483 URL: https://issues.apache.org/jira/browse/CALCITE-3483 Project: Calcite

Re: Re: Re: Problem with converters and possibly rule matching

2019-11-05 Thread Haisheng Yuan
Haisheng -- 发件人:Vladimir Ozerov 日 期:2019年11月05日 18:02:15 收件人:Haisheng Yuan 抄 送:dev@calcite.apache.org (dev@calcite.apache.org) 主 题:Re: Re: Problem with converters and possibly rule matching Hi Haisheng, think I already tried something very similar to what you explained, but

Re: Re: Problem with converters and possibly rule matching

2019-11-03 Thread Haisheng Yuan
Hi Vladimir, This is still can be done through top-down request approach. PhysicalFilter operator should request ANY distribution from child operator, unless there is outer reference in the filter condition, in which case, PhysicalFilter should request SINGLETON or BROADCAST distribution. So

Re: Re: [DISCUSS] On-demand traitset request

2019-11-03 Thread Haisheng Yuan
; MJ2: Insert a sort of (b, c, a) on LHS. This MJ operator now has > >> collation of (b, c, a) > >> 6) MJ1 and MJ2 could both satisfy permutation match request in step > >> 1, leading to two possible plans: > >> Agg1: with input of MJ1 > >> Agg2: with input

Re: Re: [ANNOUNCE] Danny Chan joins Calcite PMC

2019-10-30 Thread Haisheng Yuan
Congrats, Danny! - Haisheng -- 发件人:Peter Huang 日 期:2019年10月31日 05:55:17 收件人: 主 题:Re: [ANNOUNCE] Danny Chan joins Calcite PMC Congratulations Danny! On Wed, Oct 30, 2019 at 2:22 PM Francis Chuang wrote: > I'm pleased to announce

[jira] [Created] (CALCITE-3460) Poor performance in RexReplacer for large queries

2019-10-30 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3460: -- Summary: Poor performance in RexReplacer for large queries Key: CALCITE-3460 URL: https://issues.apache.org/jira/browse/CALCITE-3460 Project: Calcite

[jira] [Created] (CALCITE-3458) Remove desc in AbstractRelNode

2019-10-29 Thread Haisheng Yuan (Jira)
Haisheng Yuan created CALCITE-3458: -- Summary: Remove desc in AbstractRelNode Key: CALCITE-3458 URL: https://issues.apache.org/jira/browse/CALCITE-3458 Project: Calcite Issue Type

Re: Re: Problem with converters and possibly rule matching

2019-10-28 Thread Haisheng Yuan
Why do you need 2 conventions? I think 1 "Hazelcast" is enough. Usually logical operators are ones with NONE convention. Regarding with physical->logical, you can modify the rule instance to match logical operators or NONE convention only. - Haisheng

Re: [DISCUSSION] Cache Optimization in JdbcSchema

2019-10-28 Thread Haisheng Yuan
​I think this definitely will help. The code is already complicated, we can add more comments and doc to make it clear. Nothing is more attractive than saving 90% memory. - Haisheng -- 发件人:Feng Zhu 日 期:2019年10月29日 10:28:57 收件人: 主 

<    1   2   3   4   5   >