deniskuzZ opened a new pull request, #6860:
URL: https://github.com/apache/hive/pull/6860
### What changes were proposed in this pull request?
In `SharedWorkOptimizer`, when two scans are merged by `SubtreeMerge` or
`RemoveSemijoin`, the discarded scan is now decomposed (`DecomposedTs modelD`)
**after** `replaceSemijoinExpressions(discardableTsOp, ...)`, not before it.
Adds `sharedwork_semijoin_ts_merge.q`: two scans of one fact table, each
with a semijoin filter from an identical dimension branch, but with different
filters.
### Why are the changes needed?
`modelD` was built from the discarded scan's filter while it still held its
own semijoin predicate. `replaceSemijoinExpressions` then swapped that
predicate for the retained scan's, and the discarded scan's semijoin branch was
removed. The merged scan's filter is assembled from `modelD`, though, so it
kept a dynamic value whose producer no longer exists. At runtime the scan fails:
```
Caused by: org.apache.hadoop.hive.common.NoDynamicValuesException: Value
does not exist in registry: RS_20_d_k_min
at
org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.getValue(DynamicValueRegistryTez.java:78)
at
org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:126)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FilterLongColumnBetweenDynamicValue.evaluate(...)
```
The ordering dates from HIVE-24295. It was found on TPC-DS q61 at 10 TB
(`RS_147_item_i_item_sk_min`), where `hive.transpose.aggr.join=true` produces
this plan shape. The new test hits it with no transpose settings.
### Does this PR introduce _any_ user-facing change?
No. Queries that failed with `NoDynamicValuesException` now run.
### How was this patch tested?
`TestMiniLlapLocalCliDriver -Dqfile=sharedwork_semijoin_ts_merge.q`, on
master `227d8da1b1`:
- with the fix: passes;
- with the fix reverted (master `SharedWorkOptimizer`): fails,
`NoDynamicValuesException` 22 times in hive.log;
- with the fix restored: passes.
`TestTezTPCDS30TBPerfCliDriver` (200 plans) passed with this change in a
combined image.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
