[
https://issues.apache.org/jira/browse/FLINK-23843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472136#comment-17472136
]
Etienne Chauchot edited comment on FLINK-23843 at 1/10/22, 4:19 PM:
--------------------------------------------------------------------
What is the expected behavior in case an exception is raised in the Runnable?
Should we catch it, log it and ignore it ? I guess not or more likely that
depends on the exception. So we should provide a way for the API user (source
connector authors) to define a callback to react depending on the exception.
Catching the exception at the executor level does not prevent the executor to
try to instanciate a new thread which is prohibited by the factory. So that
leaves:
- either catch in the Runnable: so in that case there is not much more we can
do than simply write a javadoc for source authors to tell them to catch their
exceptions in their Runnable when using
_SplitEnumeratorContext.runInCoordinatorThread(Runnable)_.
- or we allow the API user to provide his own _Thread.UncaughtExceptionHandler_
to handle exceptions. This is better IMHO but this could be dangerous because
ignoring an exception could lead to incoherent state so adding a strong javadoc
on the exception handle seems mandatory.
=> WDYT ?
was (Author: echauchot):
What is the expected behavior in case an exception is raised in the Runnable?
Should we catch it, log it and ignore it ? I guess not or more likely that
depends on the exception. So we should provide a way for the API user (source
connector authors) to define a callback to react depending on the exception.
Catching the exception at the executor level does not prevent the executor to
try to instanciate a new thread which is prohibited by the factory. So that
leaves:
- either catch in the Runnable: so in that case there is not much more we can
do than simply write a javadoc for source authors to tell them to catch their
exceptions in their Runnable when using
_SplitEnumeratorContext.runInCoordinatorThread(Runnable)_.
- or we allow the API user to provide his own _Thread.UncaughtExceptionHandler_
to handle exceptions. This is better IMHO but this could be dangerous because
ignoring an exception could lead to incoherent state so adding a proper javadoc
on the exception handle seems mandatory.
=> WDYT ?
> Exceptions during "SplitEnumeratorContext.runInCoordinatorThread()" should
> cause Global Failure instead of Process Kill
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-23843
> URL: https://issues.apache.org/jira/browse/FLINK-23843
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.13.2
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Priority: Critical
> Fix For: 1.15.0
>
>
> Currently, when a the method
> "SplitEnumeratorContext.runInCoordinatorThread()" throws an exception, the
> effect is a process kill of the JobManager process.
> The chain how the process kill happens is:
> * An exception bubbling up in the executor, killing the executor thread
> * The executor starts a replacement thread, which is forbidden by the thread
> factory (as a safety net) and causes a process kill.
> We should prevent such exceptions from bubbling up in the coordinator
> executor.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
