[
https://issues.apache.org/jira/browse/DRILL-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dechang Gu closed DRILL-3765.
-----------------------------
Verified and tested. It looks fine.
alter session set `planner.enable_hep_partition_pruning` = false;
explain plan for select ss_sold_date_sk, ss_sold_time_sk, ss_item_sk,
ss_customer_sk from store_pb_store_sk where ss_store_sk in (11,15, 7, 202, 278,
100, 200, 300, 400, 500) and ss_customer_sk = 96479;
1 row selected (140.404 seconds)
alter session set `planner.enable_hep_partition_pruning` = true;
explain plan for select ss_sold_date_sk, ss_sold_time_sk, ss_item_sk,
ss_customer_sk from store_pb_store_sk where ss_store_sk in (11,15, 7, 202, 278,
100, 200, 300, 400, 500) and ss_customer_sk = 96479;
1 row selected (51.001 seconds)
> Partition prune rule is unnecessary fired multiple times.
> ----------------------------------------------------------
>
> Key: DRILL-3765
> URL: https://issues.apache.org/jira/browse/DRILL-3765
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Reporter: Jinfeng Ni
> Assignee: Jinfeng Ni
> Fix For: 1.4.0
>
>
> It seems that the partition prune rule may be fired multiple times, even
> after the first rule execution has pushed the filter into the scan operator.
> Since partition prune has to build the vectors to contain the partition /file
> / directory information, to invoke the partition prune rule unnecessary may
> lead to big memory overhead.
> Drill planner should avoid the un-necessary partition prune rule, in order to
> reduce the chance of hitting OOM exception, while the partition prune rule is
> executed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
