Boyang Jerry Peng created SPARK-59844:
-----------------------------------------
Summary: Preserve root causes for RTM async operation failures
Key: SPARK-59844
URL: https://issues.apache.org/jira/browse/SPARK-59844
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 4.4.0
Reporter: Boyang Jerry Peng
Async progress tracking writes checkpoint offset and commit logs on a
background thread. In Real-Time Mode, a failure in that thread interrupts the
query execution thread.
ErrorNotifier retains the first asynchronous error it observes. If it had
already retained an earlier, non-interrupting error, the query could report
that earlier error instead of the
checkpoint-log failure that caused the interruption. A race with the foreground
error check could also make the reported exception shape depend on timing.
Additionally, asynchronous futures can introduce nested CompletionException or
ExecutionException wrappers that obscure an already-categorized Spark error.
Attaching related exceptions
as suppressed errors could create cyclic or overlapping throwable graphs,
producing [CIRCULAR REFERENCE] in rendered stack traces.
This change makes the primary cause deterministic and exposes the actionable
checkpoint-storage exception while retaining independent diagnostic context.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
