Caideyipi commented on code in PR #19140:
URL: https://github.com/apache/iotdb/pull/19140#discussion_r4509245818
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/subtask/processor/PipeProcessorSubtask.java:
##########
@@ -119,17 +119,21 @@ public void bindExecutors(
this.subtaskWorkerThreadPoolExecutor = subtaskWorkerThreadPoolExecutor;
this.subtaskScheduler = subtaskScheduler;
- // double check locking for constructing PipeProcessorSubtaskWorkerManager
- if (subtaskWorkerManager.get() == null) {
+ // Double-checked locking. Recreate the manager if the cached one is bound
to a shut-down
+ // executor, so subtasks are not scheduled to workers that can never run.
+ PipeProcessorSubtaskWorkerManager manager = subtaskWorkerManager.get();
+ if (manager == null || manager.isShutdown()) {
synchronized (PipeProcessorSubtaskWorkerManager.class) {
- if (subtaskWorkerManager.get() == null) {
- subtaskWorkerManager.set(
+ manager = subtaskWorkerManager.get();
+ if (manager == null || manager.isShutdown()) {
+ manager =
new PipeProcessorSubtaskWorkerManager(
- subtaskWorkerThreadPoolExecutor,
subtaskWorkerScheduledExecutor));
+ subtaskWorkerThreadPoolExecutor,
subtaskWorkerScheduledExecutor);
+ subtaskWorkerManager.set(manager);
}
}
}
- subtaskWorkerManager.get().schedule(this);
+ manager.schedule(this);
Review Comment:
[P2] Retry if this manager shuts down before scheduling. The cached check
can race with executor shutdown: after `isShutdown()` returns false, another
thread can shut down the manager pool before this call. `schedule()` then adds
the subtask to the old worker, but `startLockedIfNecessary()` sees shutdown and
returns; registration succeeds while no worker can run the subtask, and a later
manager recreation does not migrate it. Please revalidate/retry using the
current manager or otherwise make selection and scheduling atomic, and cover
this shutdown/bind race.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]