dengzhhu653 commented on a change in pull request #2473:
URL: https://github.com/apache/hive/pull/2473#discussion_r675430051
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
##########
@@ -2572,18 +2569,24 @@ public static ContentSummary getInputSummary(final
Context ctx, MapWork work, Pa
int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
pathNeedProcess.size());
if (numExecutors > 1) {
Review comment:
The default value of `numExecutors` is 1 unless `pathNeedProcess.size()`
> 1 and `hive.exec.input.listing.max.threads` sets to the value bigger than 1.
Maybe we do not need the global lock when `numExecutors = 1`, and can be
converted to a finer-grained lock. In our cluster, we found that many handler
threads were blocked by the global lock when one of federation namespaces
failover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
