[
https://issues.apache.org/jira/browse/FLINK-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653584#comment-16653584
]
Till Rohrmann commented on FLINK-10564:
---------------------------------------
Thanks for reporting this issue [~chenlf]. It looks as if the JM sometimes does
not answer before the {{akka.ask.timeout}} or {{akka.client.timeout}} has been
reached. This could be cause by a busy {{JobManager}}. I would try to increase
these timeouts and see whether the problem still persists.
In general, the Flink community no longer actively supports the version {{<
1.5.x}}. Therefore, I would recommend considering to upgrade to a new Flink
version, eventually.
> tm costs too much time when communicating with jm
> --------------------------------------------------
>
> Key: FLINK-10564
> URL: https://issues.apache.org/jira/browse/FLINK-10564
> Project: Flink
> Issue Type: Bug
> Components: Core, JobManager, TaskManager
> Environment: configs are following:
> jm
> high-availability zookeeper
> taskmanager.heap.mb 16384
> taskmanager.memory.preallocate false
> taskmanager.numberOfTaskSlots 64
> tm
> slots 128
> free slots 0-128
> cpu core 40
> Physical Memory 95gb
> free Memory 32gb-50gb
> Flink Managed Memory 22gb-35gb
> Reporter: chenlf
> Priority: Major
> Attachments: timeout.log
>
>
> it works fine until the number of tasks is above about 400.
> There are 600+ tasks(each task handles billion data) running in our cluster
> now,and the problem is it costs too much time (even time out)when
> submiting/canceling/querying a task.
> Recouses like memory,cpu are on normal level.
> after debuging,we found this method is the culprit:
>
> org.apache.flink.runtime.util.LeaderRetrievalUtils.LeaderGatewayListener.notifyLeaderAddress(String,
> UUID)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
