[
https://issues.apache.org/jira/browse/GOBBLIN-1078?focusedWorklogId=401499&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401499
]
ASF GitHub Bot logged work on GOBBLIN-1078:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 11/Mar/20 15:05
Start Date: 11/Mar/20 15:05
Worklog Time Spent: 10m
Work Description: autumnust commented on pull request #2919:
[GOBBLIN-1078] Coordination between task cancel and initialization in Helix Task
URL: https://github.com/apache/incubator-gobblin/pull/2919#discussion_r391039245
##########
File path:
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/SingleTask.java
##########
@@ -160,16 +181,32 @@ protected JobState getJobState()
}
public void cancel() {
- if (_taskAttempt != null) {
+ int retryCount = 0 ;
+ int maxRetry = this._dynamicConfig.hasPath(MAX_RETRY_WAITING_FOR_INIT_KEY)
+ ? this._dynamicConfig.getInt(MAX_RETRY_WAITING_FOR_INIT_KEY) :
DEFAULT_MAX_RETRY_WAITING_FOR_INIT;
+
+ try {
+ _lock.lock();
try {
+ while (_taskAttempt == null) {
+ // await return false if timeout on this around
+ if (!_taskAttemptBuilt.await(5, TimeUnit.SECONDS) && ++retryCount >
maxRetry) {
+ throw new IllegalStateException("Failed to initialize taskAttempt
object before cancel");
Review comment:
It will fail the cancel method and propagate exception back to Helix.
The reason of doing so, is because it is potentially dangerous to proceed in
this case where the cancel was issue but the `taskAttempt` has not been
initialized. For example it could result in two task running in the same helix
instance in our streaming-ingestion pipeline where both task not falling behind
due to the lack of enough resources
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 401499)
Time Spent: 1h (was: 50m)
> Coordination between task cancel and task initialization in Helix
> -----------------------------------------------------------------
>
> Key: GOBBLIN-1078
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1078
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Lei Sun
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
