[
https://issues.apache.org/jira/browse/CALCITE-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenzhuang Zhu updated CALCITE-6907:
-----------------------------------
Description:
*HepPlanner:*
HepPlanner cannot quickly optimize plan with 10K+ rel nodes.
We can improve it by:
1.HepPlanner support graph reuse for different HepProgram.
2.Don't do eager graph GC, e.g. for listener and graph iterator.
3.Use large plan friendly graph iterator.
{*}RelDataTypeDigest{*}:
For nested Type with 1K+ inner field, computeDigest for composite type is too
heavy.
RelDataTypeDigest is similar to RelDigest. It will cache hash code and prevent
generate type string for composite type.
*RelOnceVisitor:*
Deep nested CTE can be expanded to more than 100 millions nodes if we don't do
dedup during visiting Rel tree.
BTW, explain has the same problem, needs a better string explain format for
duplicate RelNode.
was:
HepPlanner cannot quickly optimize plan with 10K+ rel nodes.
We can improve it by:
1.HepPlanner support graph reuse for different HepProgram.
2.Don't do eager graph gc, e.g. for listener and graph iterator.
3.Use large plan friendly graph iterator.
> Optimize latency for large plan
> -------------------------------
>
> Key: CALCITE-6907
> URL: https://issues.apache.org/jira/browse/CALCITE-6907
> Project: Calcite
> Issue Type: Improvement
> Reporter: Wenzhuang Zhu
> Priority: Major
>
> *HepPlanner:*
> HepPlanner cannot quickly optimize plan with 10K+ rel nodes.
> We can improve it by:
> 1.HepPlanner support graph reuse for different HepProgram.
> 2.Don't do eager graph GC, e.g. for listener and graph iterator.
> 3.Use large plan friendly graph iterator.
>
> {*}RelDataTypeDigest{*}:
> For nested Type with 1K+ inner field, computeDigest for composite type is too
> heavy.
> RelDataTypeDigest is similar to RelDigest. It will cache hash code and
> prevent generate type string for composite type.
>
> *RelOnceVisitor:*
> Deep nested CTE can be expanded to more than 100 millions nodes if we don't
> do dedup during visiting Rel tree.
> BTW, explain has the same problem, needs a better string explain format for
> duplicate RelNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
