leventov commented on a change in pull request #8236: Add TaskResourceCleaner;
fix a couple of concurrency bugs in batch tasks
URL: https://github.com/apache/incubator-druid/pull/8236#discussion_r311608563
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java
##########
@@ -146,10 +146,13 @@
private final RetryPolicyFactory retryPolicyFactory;
@JsonIgnore
- private List<IndexTask> indexTaskSpecs;
+ private final AppenderatorsManager appenderatorsManager;
@JsonIgnore
- private AppenderatorsManager appenderatorsManager;
+ private List<IndexTask> indexTaskSpecs;
+
+ @Nullable
+ private volatile IndexTask currentRunningTaskSpec = null;
Review comment:
So if `stopGracefully()` may be called at any time concurrently with
`run()`, I think the concurrency property that is worth ensuring is not
starting running the next task if `stopGracefully()` is already stopping the
previous task. To achieve this, `volatile` is not enough. This property could
be achieved the following way:
- `currentRunningTaskSpec` is `AtomicReference<Object>`
- The stopping callback:
```java
Object currentRunningTask =
currentRunningTaskSpec.getAndSet(SPECIAL_VALUE_STOPPED);
if (currentRunningTask != null) {
((IndexTask) currentRunningTask).stopGracefull(config);
}
```
- The code in the loop in `runTask()` does something like
```java
Object prevSpec = currentRunningTaskSpec.get();
if (prevTask == SPECIAL_VALUE_STOPPED ||
!currentRunningTaskSpec.compareAndSet(prevSpec, eachSpec)) {
log.info("Stopped concurrently");
...
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
