mayankkunwar commented on code in PR #5707:
URL: https://github.com/apache/hive/pull/5707#discussion_r2015825040
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java:
##########
@@ -182,6 +187,30 @@ public static ReduceWork createReduceWork(
return reduceWork;
}
+ private static boolean hasBucketMapJoin(Operator<? extends OperatorDesc>
operator) {
+ if (operator == null) {
+ return false;
+ }
+
+ // Iterate over child operators
+ for (Operator<? extends OperatorDesc> childOp :
operator.getChildOperators()) {
+ // Check if this is a MapJoinOperator and is a Bucket Map Join
+ if (childOp instanceof MapJoinOperator) {
+ MapJoinOperator mjOp = (MapJoinOperator) childOp;
+ if (mjOp.getConf().isBucketMapJoin()) {
+ return true; // Found BMJ, no need to check further
+ }
+ }
+
+ // Recursively check children
+ if (hasBucketMapJoin(childOp)) {
+ return true;
+ }
Review Comment:
@okumin Thank you for your detailed explanation and the accompanying
diagram. It was extremely helpful in visualizing the concept, and I now have a
clear understanding of what you were trying to convey.
But in this query execution, when I try to debug, I see that
**createReduceWork** method is called only once, and for "**Reducer 4**"
according to the mentioned diagram. Why this method is not called for other
reducers?
So, if I remove recursion from the changes, the query will hang.
Seems like if I remove AUTOPARALLEL for all the reducer before
BucketMapJoin, the query execution do not hang.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
