[
https://issues.apache.org/jira/browse/HIVE-26452?focusedWorklogId=798398&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-798398
]
ASF GitHub Bot logged work on HIVE-26452:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Aug/22 11:11
Start Date: 05/Aug/22 11:11
Worklog Time Spent: 10m
Work Description: kasakrisz opened a new pull request, #3505:
URL: https://github.com/apache/hive/pull/3505
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA:
https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a
faster review.
-->
### What changes were proposed in this pull request?
When collecting columns for join's child RS operator's row schema search for
all the key column occurrences and add them to the row schema. Name the column
using the corresponding key index.
### Why are the changes needed?
See jira description.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Force calling `checkAndConvertSMBJoin` by setting `mapJoinConversion` to
`null` when Converting join to mapjoin by changing the line
https://github.com/apache/hive/blob/8b312d54ea206461cb50450ac95ec267aba60b21/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java#L158
to
```
MapJoinConversion mapJoinConversion = null;
```
```
set hive.auto.convert.join=true;
set hive.auto.convert.sortmerge.join=true;
set hive.optimize.dynamic.partition.hashjoin=true;
create table LU_CUSTOMER (CUSTOMER_ID int, value string);
create table ORDER_FACT (CUSTOMER_ID int, order_value string);
explain
select count(*)
from LU_CUSTOMER pa11
join ORDER_FACT a15
on (pa11.CUSTOMER_ID = a15.CUSTOMER_ID)
join LU_CUSTOMER a16
on (a15.CUSTOMER_ID = a16.CUSTOMER_ID and pa11.CUSTOMER_ID =
a16.CUSTOMER_ID)
;
explain
select count(*)
from LU_CUSTOMER pa11
join ORDER_FACT a15
on (pa11.CUSTOMER_ID = a15.CUSTOMER_ID)
join LU_CUSTOMER a16
on (a15.CUSTOMER_ID = a16.CUSTOMER_ID)
where pa11.CUSTOMER_ID = a16.CUSTOMER_ID
;
```
Issue Time Tracking
-------------------
Worklog Id: (was: 798398)
Remaining Estimate: 0h
Time Spent: 10m
> NPE when converting join to mapjoin and join column referenced more than once
> -----------------------------------------------------------------------------
>
> Key: HIVE-26452
> URL: https://issues.apache.org/jira/browse/HIVE-26452
> Project: Hive
> Issue Type: Bug
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> explain
> select count(*)
> from LU_CUSTOMER pa11
> join ORDER_FACT a15
> on (pa11.CUSTOMER_ID = a15.CUSTOMER_ID)
> join LU_CUSTOMER a16
> on (a15.CUSTOMER_ID = a16.CUSTOMER_ID and pa11.CUSTOMER_ID =
> a16.CUSTOMER_ID);
> {code}
> {{a16.CUSTOMER_ID}} is referenced more than once in the join condition.
> Hive generates Reduce sink operators for the join's children and one of the
> RS row schema contains only one instance of the join keys (customer_id).
> {code}
> RS[13]
> result = {HashMap@16092} size = 2
> "KEY.reducesinkkey0" -> {ExprNodeColumnDesc@16083} "Column[_col0]"
> "KEY.reducesinkkey1" -> {ExprNodeColumnDesc@16102} "Column[_col0]"
>
>
>
> result = {RowSchema@16104} "(KEY.reducesinkkey0: int|{$hdt$_2}customer_id)"
> signature = {ArrayList@16110} size = 1
> 0 = {ColumnInfo@16087} "KEY.reducesinkkey0: int"
> {code}
> {{KEY.reducesinkkey1}} is missing from the schema.
> When converting the join to mapjoin the converter algorithm fails looking up
> both join key column instances.
> https://github.com/apache/hive/blob/2aaba3c79e740ef27fc263b5a8ff33ad679c5a12/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java#L538
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
