deniskuzZ commented on code in PR #6824:
URL: https://github.com/apache/hive/pull/6824#discussion_r4144570098
##########
ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPruningEventDesc.java:
##########
@@ -127,8 +123,7 @@ public boolean isSame(OperatorDesc other) {
DynamicPruningEventDesc otherDesc = (DynamicPruningEventDesc) other;
return Objects.equals(getTargetColumnName(),
otherDesc.getTargetColumnName()) &&
Objects.equals(getTargetColumnType(),
otherDesc.getTargetColumnType()) &&
- Objects.equals(getPartKeyString(), otherDesc.getPartKeyString()) &&
- Objects.equals(getPartPredicateString(),
otherDesc.getPartPredicateString());
Review Comment:
> Why can the predicate check be removed?
The predicate only exists for non-native tables. For them,
generateEventOperatorPlan receives ctx.parent, the `IN(partKey, <dynamic
list>)` expression, and stores it on the event
(DynamicPartitionPruningOptimization:178, 545). The dynamic list renders as its
producing operator (`RS[n]`), so two events that prune identically still render
different predicate strings, because each list comes from a different
ReduceSink.
isSame() therefore never matched them, and SWO merged only the table scans
and duplicated everything below. That's the q47/q57 regression.
The predicate adds nothing the other checks don't already cover:
- Which column is pruned: targetColumnName, targetColumnType and partKey,
which isSame() still compares.
- Where the values come from: the operators that feed the event. SWO
already compares them, including their expressions: compareOperator falls back
to logicalEquals → desc.isSame(), which compares the Select/GroupBy expressions
and the ReduceSink key/value/partition columns, all the way up the branch. If
the value-producing expressions differed, those operators wouldn't match and
the events wouldn't be merged.
So dropping the string comparison removes a false mismatch, not a real
distinction.
> How does it affect native tables?
It doesn't. For native partitioned tables the event is created with a null
predicate (DynamicPartitionPruningOptimization:197,
generateEventOperatorPlan(..., null)). getPartPredicateString() returned "-"
on both sides, so the removed condition was always true for native tables.
Their isSame() result is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
