[
https://issues.apache.org/jira/browse/DRILL-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Hou closed DRILL-4743.
-----------------------------
Assignee: Robert Hou (was: Gautam Kumar Parai)
This fix has been verified.
> HashJoin's not fully parallelized in query plan
> -----------------------------------------------
>
> Key: DRILL-4743
> URL: https://issues.apache.org/jira/browse/DRILL-4743
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.5.0
> Reporter: Gautam Kumar Parai
> Assignee: Robert Hou
> Labels: doc-impacting
> Fix For: 1.8.0
>
>
> The underlying problem is filter selectivity under-estimate for a query with
> complicated predicates e.g. deeply nested and/or predicates. This leads to
> under parallelization of the major fragment doing the join.
> To really resolve this problem we need table/column statistics to correctly
> estimate the selectivity. However, in the absence of statistics OR even when
> existing statistics are insufficient to get a correct estimate of selectivity
> this will serve as a workaround.
> For now, the fix is to provide options for controlling the lower and upper
> bounds for filter selectivity. The user can use the following options. The
> selectivity can be varied between 0 and 1 with min selectivity always less
> than or equal to max selectivity.
> {code}planner.filter.min_selectivity_estimate_factor
> planner.filter.max_selectivity_estimate_factor
> {code}
> When using 'explain plan including all attributes for ' it should cap the
> estimated ROWCOUNT based on these options. Estimated ROWCOUNT of operators
> downstream is not directly controlled by these options. However, they may
> change as a result of dependency between different operators. The FILTER
> operator only operates on the input of its immediate upstream operator (e.g.
> SCAN, AGG). If two different filters are present in the same plan, they might
> have different selectivities based on their immediate upstream operators
> ROWCOUNT.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
