[ 
https://issues.apache.org/jira/browse/FLINK-20992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267826#comment-17267826
 ] 

Roman Khachatryan commented on FLINK-20992:
-------------------------------------------

{quote}one could use a handle call in which one first checks whether the 
CheckepointCoordinator is still running
{quote}
But checking *inside* handle(Async) means checking inside an already 
*submitted* action - i.e. too late:
{code:java}
coordinatorCheckpointsComplete.handleAsync(x -> {
    if (executor.isShutdown()) { // too late
        // ...
    }
} , timer);
{code}
 Or am I missing something?

> Checkpoint cleanup can kill JobMaster
> -------------------------------------
>
>                 Key: FLINK-20992
>                 URL: https://issues.apache.org/jira/browse/FLINK-20992
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Checkpointing
>    Affects Versions: 1.12.0
>            Reporter: Till Rohrmann
>            Assignee: Roman Khachatryan
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 1.13.0, 1.12.2
>
>
> A user reported that cancelling a job can lead to an uncaught exception which 
> kills the {{JobMaster}}. The problem seems to be that the 
> {{CheckpointsCleaner}} might trigger {{CheckpointCoordinator}} actions after 
> the job has reached a terminal state and, thus, is shut down. Apparently, we 
> do not properly manage the lifecycles of {{CheckpointCoordinator}} and 
> checkpoint post clean up actions.
> The uncaught exception is 
> {code}
> java.util.concurrent.RejectedExecutionException: Task 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@41554407 
> rejected from 
> java.util.concurrent.ScheduledThreadPoolExecutor@5d0ec6f7[Terminated, pool 
> size = 0, active threads = 0, queued tasks = 0, completed tasks = 25977] at 
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063
>  at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:326
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:622
>  at 
> java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:668
>  at 
> org.apache.flink.runtime.concurrent.ScheduledExecutorServiceAdapter.execute(ScheduledExecutorServiceAdapter.java:62
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.scheduleTriggerRequest(CheckpointCoordinator.java:1152
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointsCleaner.lambda$cleanCheckpoint$0(CheckpointsCleaner.java:58
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624
>  at java.lang.Thread.run(Thread.java:748 undefined)
> {code}
> cc [~roman_khachatryan].
> https://lists.apache.org/thread.html/r75901008d88163560aabb8ab6fc458cd9d4f19076e03ae85e00f9a3a%40%3Cuser.flink.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to