FrankChen021 commented on code in PR #18226:
URL: https://github.com/apache/druid/pull/18226#discussion_r4429484108
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/SeekableStreamIndexTaskRunner.java:
##########
@@ -1015,6 +1030,11 @@ public void onFailure(Throwable t)
}
ingestionState = IngestionState.COMPLETED;
+ if (unrecoverableReadException != null) {
+ final String errorMsg = unrecoverableReadException.getMessage();
+ toolbox.getTaskReportFileWriter().write(task.getId(),
getTaskCompletionReports(errorMsg, handoffWaitMs));
+ return TaskStatus.failure(task.getId(), errorMsg);
Review Comment:
[P1] Failed tasks can be recreated from stale offsets after publishing
**Finding:** When this new path catches an unrecoverable read error, it
commits and publishes the partial sequence and then reports FAILED. A
seekable-stream supervisor removes the failed task but can retain the active
task group, which is then topped up using its original starting/checkpoint
sequences rather than the newly published metadata offsets. A replacement task
can therefore reread data already committed or repeatedly hit the same expired
offset, making the fail-after-publish behavior unsafe under supervisor-managed
ingestion.
**Suggestion:** Ensure this terminal read failure also clears or advances
the supervisor task group from the committed metadata before replacement tasks
are created, and add a supervisor-level regression test verifying that the next
task starts at the published offset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
