[
https://issues.apache.org/jira/browse/FLINK-20886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761591#comment-17761591
]
Zakelly Lan commented on FLINK-20886:
-------------------------------------
Hi [~yunta], I finished my PR for this issue. In current implementation, when
checkpoint expires, the JM will notify TM(tasks) to do a thread dump via the
mailbox (the same way JM notify tasks about a checkpoint abortion). And the
task will log a thread dump {*}only when the checkpoint is currently
running{*}. So if the task cannot receive the checkpoint barrier, it have not
started the checkpoint yet, thus no thread dump will be logged.
I think logging a thread dump when a task has not started the checkpoint is
also useful, and I can create another ticket for it. WDYT? Besides, would you
like review my PR? Thanks!
> Add the option to get a threaddump on checkpoint timeouts
> ---------------------------------------------------------
>
> Key: FLINK-20886
> URL: https://issues.apache.org/jira/browse/FLINK-20886
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing
> Reporter: Nico Kruber
> Assignee: Zakelly Lan
> Priority: Minor
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> usability
>
> For debugging checkpoint timeouts, I was thinking about the following
> addition to Flink:
> When a checkpoint times out and the async thread is still running, create a
> thread dump [1] and either add this to the checkpoint stats, log it, or write
> it out.
> This may help identifying where the checkpoint is stuck (maybe a lock, could
> also be in a third party lib like the FS connectors,...). It would give us
> some insights into what the thread is currently doing.
> Limiting the scope of the threads would be nice but may not be possible in
> the general case since additional threads (spawned by the FS connector lib,
> or otherwise connected) may interact with the async thread(s) by e.g. going
> through the same locks. Maybe we can reduce the thread dumps to all async
> threads of the failed checkpoint + all thready that interact with it, e.g.
> via locks?
> I'm also not sure whether the ability to have thread dumps or not should be
> user-configurable (Could it contain sensitive information from other jobs if
> you run a session cluster? Is that even relevant since we don't give
> isolation guarantees anyway?). If it is configurable, it should be on by
> default.
> [1] https://crunchify.com/how-to-generate-java-thread-dump-programmatically/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
