[
https://issues.apache.org/jira/browse/DRILL-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aman Sinha reopened DRILL-3765:
-------------------------------
Re-opening since this is becoming a performance issue. Discussed briefly with
[~jni]. The Volcano planner's invocations of the PruneScanRule multiple times
during the search exploration is too heavy-weight since the rule itself is
complex. We should ideally move the pruning rules to the Hep planner as long
as we ensure that associated rules such as filter push-down, projection
push-down are also applied before and after pruning.
> Partition prune rule is unnecessary fired multiple times.
> ----------------------------------------------------------
>
> Key: DRILL-3765
> URL: https://issues.apache.org/jira/browse/DRILL-3765
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Reporter: Jinfeng Ni
> Assignee: Jinfeng Ni
>
> It seems that the partition prune rule may be fired multiple times, even
> after the first rule execution has pushed the filter into the scan operator.
> Since partition prune has to build the vectors to contain the partition /file
> / directory information, to invoke the partition prune rule unnecessary may
> lead to big memory overhead.
> Drill planner should avoid the un-necessary partition prune rule, in order to
> reduce the chance of hitting OOM exception, while the partition prune rule is
> executed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
