[
https://issues.apache.org/jira/browse/HIVE-26524?focusedWorklogId=814019&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-814019
]
ASF GitHub Bot logged work on HIVE-26524:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Oct/22 20:02
Start Date: 05/Oct/22 20:02
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #3588:
URL: https://github.com/apache/hive/pull/3588#discussion_r985669610
##########
ql/src/test/results/clientpositive/llap/masking_10.q.out:
##########
@@ -137,9 +136,7 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Reducer 4 (CUSTOM_SIMPLE_EDGE)
- Reducer 3 <- Map 1 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
- Reducer 4 <- Map 1 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
Review Comment:
This is the query after applying the masking
```
select `alias01`.`key`, `alias01`.`value`, `alias02`.`a`, `alias02`.`value`,
`alias03`.`key`, `alias03`.`value` from
(SELECT `key`, CAST(reverse(value) AS string) AS `value`,
BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID, ROW__IS__DELETED FROM
`default`.`masking_test` WHERE key % 2 = 0 and key < 10)`alias01`
left join
(
select 2017 as `a`, `value` from (SELECT `key`, CAST(reverse(value) AS
string) AS `value`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID,
ROW__IS__DELETED FROM `default`.`masking_test` WHERE key % 2 = 0 and key <
10)`masking_test` group by 1, 2
) `alias02`
on `alias01`.key = `alias02`.`a`
left join
(SELECT `key`, CAST(reverse(value) AS string) AS `value`,
BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID, ROW__IS__DELETED FROM
`default`.`masking_test` WHERE key % 2 = 0 and key < 10)`alias03`
on `alias01`.key = `alias03`.key
```
The first join has a condition: `alias01.key = alias02.a`
In the left branch there is a Filter on `key`: `key % 2 = 0 and key < 10`
In the right branch `a` is constant `2017` so the join condition is going to
be evaluated always `false` and that join is replaced by its left branch
##########
ql/src/test/results/clientpositive/llap/ppd_udf_col.q.out:
##########
@@ -80,22 +80,9 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-0
Fetch Operator
- limit: -1
+ limit: 0
Processor Tree:
- TableScan
- alias: src
- filterExpr: (UDFToDouble(key) = 100.0D) (type: boolean)
- Filter Operator
- predicate: (UDFToDouble(key) = 100.0D) (type: boolean)
- Limit
- Number of rows: 0
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Select Operator
- expressions: _col0 (type: string), rand() (type: double),
'4' (type: string)
- outputColumnNames: _col0, _col1, _col2
- ListSink
+ ListSink
Review Comment:
This is the empty plan
```
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: 0
Processor Tree:
ListSink
```
Issue Time Tracking
-------------------
Worklog Id: (was: 814019)
Time Spent: 5h 40m (was: 5.5h)
> Use Calcite to remove sections of a query plan known never produces rows
> ------------------------------------------------------------------------
>
> Key: HIVE-26524
> URL: https://issues.apache.org/jira/browse/HIVE-26524
> Project: Hive
> Issue Type: Improvement
> Components: CBO
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Calcite has a set of rules to remove sections of a query plan known never
> produces any rows. In some cases the whole plan can be removed. Such plans
> are represented with a single {{Values}} operators with no tuples. ex.:
> {code:java}
> select y + 1 from (select a1 y, b1 z from t1 where b1 > 10) q WHERE 1=0
> {code}
> {code:java}
> HiveValues(tuples=[[]])
> {code}
> Other cases when plan has outer join or set operators some branches can be
> replaced with empty values moving forward in some cases the join/set operator
> can be removed
> {code:java}
> select a2, b2 from t2 where 1=0
> union
> select a1, b1 from t1
> {code}
> {code:java}
> HiveAggregate(group=[{0, 1}])
> HiveTableScan(table=[[default, t1]], table:alias=[t1])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
