[
https://issues.apache.org/jira/browse/DRILL-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536032#comment-14536032
]
Jinfeng Ni commented on DRILL-2958:
-----------------------------------
Copy from review board about a summary of change in the patch:
Drill current use VolcanoPlanner in join planning. This planner has two known
issues:
1. The search space is increased exponentially with increased # of tables
joined. If query has more than > 10 tables join, the planning time itself could
be minutes, if not longer.
2. Drill did not enable a rule to swap both sides of join, due to the search
space problem. We only do a swap join afterwards. See DRILL-2236. This means
the join order chosen by Drill's VolcanoPlanner might not be optimal.
To address the above two issues, we are going to provide another planner for
the purpose of join ordering planning. This planner will use a different
optimization rules, and the search space is not increased exponentially with #
of table.
The main logic of this new planner:
1) Let VolcanoPlanner do all the rule transformations same as the current
planner's logical planning, except for the join permutation rule.
2) After that, pass to HepPlanner with Calcite LOPT optimization rule, to let
it do the join ordering. Feed with the HepPlanner with Drill's
RelMetaDataProvider, to leverage the statistics (rowcount) available in Drill's
table/files.
3) Continue with the same physical planning as before.
With the limited statistics available in Drill, the new planner seems to
produce better query plan than the current, for several TPCH queries.
Preliminary performance results show this planner run faster than the existing
one, and the join plan seems to be same or better than the plan chosen by the
existing planner.
> Move Drill to alternative cost-based planner for Join planning
> --------------------------------------------------------------
>
> Key: DRILL-2958
> URL: https://issues.apache.org/jira/browse/DRILL-2958
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Reporter: Jacques Nadeau
> Assignee: Jinfeng Ni
> Fix For: 1.0.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
