sunchao commented on PR #58402: URL: https://github.com/apache/spark/pull/58402#issuecomment-5460234306
Thanks, @dongjoon-hyun. I pushed the cleanup in [d7759d0](https://github.com/sunchao/spark/commit/d7759d0c4a7ab4460a6f403c8dd8a6e0c285a882): - Restored the original `maxAttemptIdToIgnore` scaladoc and both original output-clearing comments. - Renamed the barrier marker to `latestFailedBarrierAttemptId` and made the failure comment explain why late completions are ignored. - Added `SPARK-59094:` to all eleven cases, fixed the singular/plural wording, and renamed the producer control to describe accepting successes after a late consumer fetch failure. - Kept the direct completion-handler calls and their ordering comments: the tests intentionally avoid draining a timed resubmission event between the events under test. - Shortened the PR description to the fix, scope, test coverage, and suite results. Both suites were rerun: `DAGSchedulerSuite` 233/233 and `TaskSchedulerImplSuite` 121/121 passed. For the producer-side follow-up, I confirmed a concrete ordering with a partially completed retry: accept one healthy producer-retry output, process the old consumer's second fetch failure, resubmit while the producer is still running, then complete its remaining task. An isolated scheduler characterization confirms that both the public baseline and this revision submit a one-task retry that still declares two barrier participants. This is scheduler-state evidence, not a native timeout reproduction. The follow-up report is prepared, but its JIRA has not yet been filed; that is the remaining item. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
