dengzhhu653 commented on a change in pull request #2473:
URL: https://github.com/apache/hive/pull/2473#discussion_r675430051
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
##########
@@ -2572,18 +2569,24 @@ public static ContentSummary getInputSummary(final
Context ctx, MapWork work, Pa
int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
pathNeedProcess.size());
if (numExecutors > 1) {
Review comment:
The default value of `numExecutors` is 1 unless `pathNeedProcess.size()`
> 1 and `hive.exec.input.listing.max.threads` sets to the value bigger than 1.
We do not need the global lock when `numExecutors = 1`, and can be converted
to a finer-grained lock. In our cluster, we found that many handler threads
were blocked by the global lock when one of federation namespaces failover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
