[
https://issues.apache.org/jira/browse/HIVE-2420?focusedWorklogId=590145&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-590145
]
ASF GitHub Bot logged work on HIVE-2420:
----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Apr/21 04:32
Start Date: 28/Apr/21 04:32
Worklog Time Spent: 10m
Work Description: Dawn2111 commented on a change in pull request #2065:
URL: https://github.com/apache/hive/pull/2065#discussion_r621709390
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java
##########
@@ -790,45 +842,72 @@ private void dumpPoolState(PoolState ps, List<String>
set) {
}
}
- private void handleMoveSessionOnMasterThread(final MoveSession moveSession,
- final WmThreadSyncWork syncWork,
- final HashSet<String> poolsToRedistribute,
- final Map<WmTezSession, GetRequest> toReuse,
- final Map<WmTezSession, WmEvent> recordMoveEvents) {
+ private static enum MoveSessionResult {
+ OK, // Normal case - the session was moved.
+ KILLED, // Killed because destination pool was full and delayed move is
false.
+ CONVERTED_TO_DELAYED_MOVE, // the move session was added to the pool's
delayed moves as the dest. pool was full
+ // and delayed move is true.
+ ERROR
+ }
+
+ private MoveSessionResult handleMoveSessionOnMasterThread(final MoveSession
moveSession,
+ final WmThreadSyncWork syncWork,
+ final HashSet<String> poolsToRedistribute,
+ final Map<WmTezSession, GetRequest> toReuse,
+ final Map<WmTezSession, WmEvent> recordMoveEvents,
+ final boolean convertToDelayedMove) {
String destPoolName = moveSession.destPool;
- LOG.info("Handling move session event: {}", moveSession);
+ LOG.info("Handling move session event: {}, Convert to Delayed Move: {}",
moveSession, convertToDelayedMove);
if (validMove(moveSession.srcSession, destPoolName)) {
+ String srcPoolName = moveSession.srcSession.getPoolName();
+ PoolState srcPool = pools.get(srcPoolName);
+ boolean capacityAvailableInDest = capacityAvailable(destPoolName);
+ // If delayed move is set to true and if destination pool doesn't have
enough capacity, don't kill the query.
+ // Let the query run in source pool. Add the session to the source
pool's delayed move sessions.
+ if (convertToDelayedMove && !capacityAvailableInDest) {
+ srcPool.delayedMoveSessions.add(moveSession);
+ moveSession.srcSession.setDelayedMove(true);
Review comment:
When a pool is updated or destroyed as a consequence of disabling WLM ,
all the sessions in the pool are removed and we remove the delayed move
sessions at that time as well - PoolState.extractAllSessionsToKill().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 590145)
Time Spent: 0.5h (was: 20m)
> partition pruner expr is not populated due to some bug in ppd
> -------------------------------------------------------------
>
> Key: HIVE-2420
> URL: https://issues.apache.org/jira/browse/HIVE-2420
> Project: Hive
> Issue Type: Bug
> Reporter: He Yongqiang
> Assignee: He Yongqiang
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-2420.reproduce.diff
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
