[
https://issues.apache.org/jira/browse/IMPALA-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18096981#comment-18096981
]
Aleksandr Efimov commented on IMPALA-7996:
------------------------------------------
IMPALA-7996 became directly observable after IMPALA-13203, starting with
4.4.1/4.5.0.
The existing PlannerTest/empty.test now expects a nested-loop outer join with a
FALSE predicate.
On large partitioned inputs this causes full scans and pathological runtime.
Versions before IMPALA-13203 retain the equi-conjunct and produce a hash join.
> Inefficient plan for pathological ON FALSE clause
> -------------------------------------------------
>
> Key: IMPALA-7996
> URL: https://issues.apache.org/jira/browse/IMPALA-7996
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.1.0
> Reporter: Paul Rogers
> Priority: Minor
>
> Bugs in the rewrite code prevented the ON clause from being subject to the
> full set of rewrite rules. See IMPALA-7997.
> Once that bug was fixed, it uncovered an inefficient plan for the
> pathological case of {{ON FALSE}}.
> {code:sql}
> # Constant conjunct in the ON-clause of an outer join is
> # assigned to the join.
> select *
> from functional.alltypessmall a
> right outer join functional.alltypestiny b
> on (a.id = b.id and !true)
> {code}
> Note the ON clause and its actual meaning:
> {noformat}
> a.id = b.id and !true --> a.id = b.id AND FALSE --> FALSE
> {noformat}
> (Note that a test should be added for the straight-up FALSE condition.)
> Once rewrites are applied applied, (or, presumably, if we have a test for
> {{ON FALSE}}), we get a new plan, which is actually worse:
> {noformat}
> PLAN-ROOT SINK
> |
> 02:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | join predicates: FALSE
> |
> |--01:SCAN HDFS [functional.alltypestiny b]
> | partitions=4/4 files=4 size=460B
> |
> 00:SCAN HDFS [functional.alltypessmall a]
> partitions=4/4 files=4 size=6.32KB
> {noformat}
> This says to loop over both tables, which is a very bad plan. The correct
> plan is to reduce the query to an empty result set.
> Similar case:
> {code:sql}
> # Constant conjunct in the ON-clause of an outer join is
> # assigned to the join.
> select *
> from functional.alltypessmall a
> left outer join functional.alltypestiny b
> on (a.id = b.id and 1 + 1 > 10)
> {code}
> Reasoning:
> {noformat}
> a.id = b.id and 1 + 1 > 10 --> a.id = b.id AND 2 > 10
> --> a.id = b.id AND FALSE --> FALSE
> {noformat}
> The plan after performing above rewrites, which is also an unnecessary nested
> loop join:
> {noformat}
> PLAN-ROOT SINK
> |
> 02:NESTED LOOP JOIN [LEFT OUTER JOIN]
> | join predicates: FALSE
> |
> |--01:SCAN HDFS [functional.alltypestiny b]
> | partitions=4/4 files=4 size=460B
> |
> 00:SCAN HDFS [functional.alltypessmall a]
> partitions=4/4 files=4 size=6.32KB
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
