Till Rohrmann created FLINK-15924:
-------------------------------------
Summary: Detect and log blocking main thread operations
Key: FLINK-15924
URL: https://issues.apache.org/jira/browse/FLINK-15924
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.10.0
Reporter: Till Rohrmann
Fix For: 1.11.0
When using the {{RpcEndpoint}} it is important that all operations which run on
the main thread are never blocking. We have seen in the past that it is quite
hard to always catch blocking operations in reviews and sometimes these changes
caused instabilities in Flink. Once this happens it is not trivial to find the
culprit which is responsible for the blocking operation.
One way to make debugging easier is to add a monitor which detects and logs if
a {{RpcEndpoint}} operation takes longer than {{n}} seconds for example.
Depending on the overhead of this monitor one could even think about enabling
it only via a special configuration (e.g. debug mode).
A proper class to introduce this monitor could be the {{AkkRpcActor}} which is
responsible for executing main thread operations. Whenever we schedule an
operation, we could start a timeout which if triggered and the operation has
not been completed will log a warning.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
