odraese commented on a change in pull request #721: HIVE-21988: Do not consider
nodes with 0 capacity when calculating host affinity
URL: https://github.com/apache/hive/pull/721#discussion_r303072707
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/Utils.java
##########
@@ -84,4 +71,31 @@ public static SplitLocationProvider
getSplitLocationProvider(Configuration conf,
}
return splitLocationProvider;
}
+
+ @VisibleForTesting
+ static SplitLocationProvider
getCustomSplitLocationProvider(LlapRegistryService serviceRegistry, Logger LOG)
throws
+ IOException {
+ LOG.info("Using LLAP instance " + serviceRegistry.getApplicationId());
+
+ Collection<LlapServiceInstance> serviceInstances =
+ serviceRegistry.getInstances().getAllInstancesOrdered(true);
+ Preconditions.checkArgument(!serviceInstances.isEmpty(),
+ "No running LLAP daemons! Please check LLAP service status and
zookeeper configuration");
+ ArrayList<String> locations = new ArrayList<>(serviceInstances.size());
+ for (LlapServiceInstance serviceInstance : serviceInstances) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Adding " + serviceInstance.getWorkerIdentity() + " with
hostname=" +
Review comment:
Maybe that LOG could be moved down, where we actually add to locations (or
better, where we don't add it if the capacity is zero). Otherwise the log might
be misleading because I would assume the daemon to be part of locations when
seeing it in the logs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
