narendly commented on a change in pull request #1040:
URL: https://github.com/apache/helix/pull/1040#discussion_r433004938
##########
File path:
helix-core/src/main/java/org/apache/helix/task/AbstractTaskDispatcher.java
##########
@@ -449,43 +448,31 @@ private void
updatePartitionInformationInJobContext(CurrentStateOutput currentSt
* @param paMap
* @param assignedPartitions
*/
- private void processTaskWithPendingMessage(ResourceAssignment
prevAssignment, Integer pId,
- String pName, String instance, Message pendingMessage, TaskState
jobState,
- TaskPartitionState currState, Map<Integer, PartitionAssignment> paMap,
- Map<String, Set<Integer>> assignedPartitions) {
-
- // stateMap is a mapping of Instance -> TaskPartitionState (String)
- Map<String, String> stateMap = prevAssignment.getReplicaMap(new
Partition(pName));
- if (stateMap != null) {
- String prevState = stateMap.get(instance);
- if (!pendingMessage.getToState().equals(prevState)) {
- LOG.warn(String.format(
- "Task pending to-state is %s while previous assigned state is %s.
This should not"
- + "happen.",
- pendingMessage.getToState(), prevState));
+ private void processTaskWithPendingMessage(Integer pId, String pName, String
instance,
+ Message pendingMessage, TaskState jobState, TaskPartitionState currState,
+ Map<Integer, PartitionAssignment> paMap, Map<String, Set<Integer>>
assignedPartitions) {
+
+ if (jobState == TaskState.TIMING_OUT && currState ==
TaskPartitionState.INIT
+ &&
pendingMessage.getToState().equals(TaskPartitionState.RUNNING.name())) {
+ // While job is timing out, if the task is pending on INIT->RUNNING, set
it back to INIT,
+ // so that Helix will cancel the transition.
+ paMap.put(pId, new PartitionAssignment(instance,
TaskPartitionState.INIT.name()));
Review comment:
I'm just brainstorming here:
I think a better state to assign here might be DROPPED? Previously, we would
send things to INIT but that would be a long-winded way to cause this to be
dropped. And I'm not sure with the recent changes, the tasks that are timing
out that are in INIT state will indeed be dropped eventually.
If they don't, then there's a problem - init state takes up a quota in the
quota calculation, and we might end up seeing quota leaks.
Have you tried to drop here directly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
