[
https://issues.apache.org/jira/browse/HIVE-21988?focusedWorklogId=276036&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-276036
]
ASF GitHub Bot logged work on HIVE-21988:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Jul/19 17:21
Start Date: 12/Jul/19 17:21
Worklog Time Spent: 10m
Work Description: odraese commented on pull request #721: HIVE-21988: Do
not consider nodes with 0 capacity when calculating host affinity
URL: https://github.com/apache/hive/pull/721#discussion_r303072707
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/Utils.java
##########
@@ -84,4 +71,31 @@ public static SplitLocationProvider
getSplitLocationProvider(Configuration conf,
}
return splitLocationProvider;
}
+
+ @VisibleForTesting
+ static SplitLocationProvider
getCustomSplitLocationProvider(LlapRegistryService serviceRegistry, Logger LOG)
throws
+ IOException {
+ LOG.info("Using LLAP instance " + serviceRegistry.getApplicationId());
+
+ Collection<LlapServiceInstance> serviceInstances =
+ serviceRegistry.getInstances().getAllInstancesOrdered(true);
+ Preconditions.checkArgument(!serviceInstances.isEmpty(),
+ "No running LLAP daemons! Please check LLAP service status and
zookeeper configuration");
+ ArrayList<String> locations = new ArrayList<>(serviceInstances.size());
+ for (LlapServiceInstance serviceInstance : serviceInstances) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Adding " + serviceInstance.getWorkerIdentity() + " with
hostname=" +
Review comment:
Maybe that LOG could be moved down, where we actually add to locations (or
better, where we don't add it if the capacity is zero). Otherwise the log might
be misleading because I would assume the daemon to be part of locations when
seeing it in the logs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 276036)
Time Spent: 40m (was: 0.5h)
> Do not consider nodes with 0 capacity when calculating host affinity
> --------------------------------------------------------------------
>
> Key: HIVE-21988
> URL: https://issues.apache.org/jira/browse/HIVE-21988
> Project: Hive
> Issue Type: Sub-task
> Components: llap
> Reporter: Peter Vary
> Assignee: Peter Vary
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21988.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a node is blacklisted (capacity set to 0) then we should not assign any
> more tasks to it
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
