veghlaci05 commented on code in PR #3513:
URL: https://github.com/apache/hive/pull/3513#discussion_r951247036
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java:
##########
@@ -317,6 +318,15 @@ protected Boolean findNextCompactionAndExecute(boolean
collectGenericStats, bool
if ((runtimeVersion != null || ci.initiatorVersion != null) &&
!runtimeVersion.equals(ci.initiatorVersion)) {
LOG.warn("Worker and Initiator versions do not match. Worker: v{},
Initiator: v{}", runtimeVersion, ci.initiatorVersion);
}
+
+ if (StringUtils.isBlank(getPoolName()) &&
StringUtils.isNotBlank(ci.poolName)) {
+ LOG.warn("A timed out copmaction pool entry ({}) is picked up by one
of the default compaction pool workers.", ci);
+ }
+ if (StringUtils.isNotBlank(getPoolName()) &&
StringUtils.isNotBlank(ci.poolName) && !getPoolName().equals(ci.poolName)) {
Review Comment:
This normally should not happen at all, because the query filters the items
by pool name. However, I wanted to cover this case as well. If a labeled (not
the default) pool somehow gets a request assigned to another or the default
pool, I think it should not be processed.
- Simply skipping it could be problematic if the item gets returned by
`findNextCompact()` again and again. In this case this item will stuck in
`initiated` state, and if there's only one worker assigned to the pool, it will
even stuck the entire queue processing for that pool.
- Marking it as failed with a proper error message seemed to be more
convenient, which won't cause processing anomalies and is easier to track down
later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
