[
https://issues.apache.org/jira/browse/DRILL-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jinfeng Ni resolved DRILL-3765.
---
Resolution: Won't Fix
I looked at the Calcite trace file. Seems the multiple rule firing is what
Calcite's rule framework is supposed to do.
The multiple rule firings are actually against different set of RelNode, which
are result of other rules firing. For instance, one scan could be the one
before Project push down, while the other one could be after the project push
down. In other words, there is no redundant rule firing, against the same set
of RelNode. Seems the multiple rule firing in this case is the way it should
be.
If we stop the subsequent rule's firing after the first rule firing, the query
planner may lose the chance to find the ultimate best plan.
Therefore, I'm going to mark this as "won't fix".
> Partition prune rule is unnecessary fired multiple times.
> --
>
> Key: DRILL-3765
> URL: https://issues.apache.org/jira/browse/DRILL-3765
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
>
> It seems that the partition prune rule may be fired multiple times, even
> after the first rule execution has pushed the filter into the scan operator.
> Since partition prune has to build the vectors to contain the partition /file
> / directory information, to invoke the partition prune rule unnecessary may
> lead to big memory overhead.
> Drill planner should avoid the un-necessary partition prune rule, in order to
> reduce the chance of hitting OOM exception, while the partition prune rule is
> executed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
