[
https://issues.apache.org/jira/browse/DRILL-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987748#comment-14987748
]
Jinfeng Ni commented on DRILL-3765:
-----------------------------------
Caching the result will help. But I feel it probably will not help a lot in
case where intermediate filters are pushed down, and partition rule is fired
against the intermediate filters. Ideally, we only want to apply the partition
rule against the final filter.
> Partition prune rule is unnecessary fired multiple times.
> ----------------------------------------------------------
>
> Key: DRILL-3765
> URL: https://issues.apache.org/jira/browse/DRILL-3765
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Reporter: Jinfeng Ni
> Assignee: Jinfeng Ni
>
> It seems that the partition prune rule may be fired multiple times, even
> after the first rule execution has pushed the filter into the scan operator.
> Since partition prune has to build the vectors to contain the partition /file
> / directory information, to invoke the partition prune rule unnecessary may
> lead to big memory overhead.
> Drill planner should avoid the un-necessary partition prune rule, in order to
> reduce the chance of hitting OOM exception, while the partition prune rule is
> executed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
