[
https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039191#comment-17039191
]
huweihua edited comment on FLINK-16069 at 2/18/20 4:07 PM:
-----------------------------------------------------------
Hi, [~trohrmann], thanks for your time. Yes, The iteration over the input edges
taking so long.
I didn't think too much about race condition, and i tried create the
TaskDeploymentDescriptorFactory in the main thread, then put
createDeploymentDescriptor into a future. This reduces the time taken in the
main thread to 2s.
Glad to receive any suggestions.
was (Author: huwh):
Hi, [~trohrmann], thanks for your time. Yes, The iteration over the input edges
taking so long.
I didn't think too much about race condition, and i tried create the
TaskDeploymentDescriptorFactory in the main thread, then put
createDeploymentDescriptor into a future. This reduces the time taken in the
main thread to 2s.
Glad to receive any suggestions.
[有道词典|http://fanyi.youdao.com/translate?i=Sorry%20for%20not%20thinking%20about%20race%20condition.%20I%20am%20glad%20to&keyfrom=chrome]
Sorry for not t
...[详细|http://fanyi.youdao.com/translate?i=Sorry%20for%20not%20thinking%20about%20race%20condition.%20I%20am%20glad%20to&smartresult=dict&keyfrom=chrome.extension]X
对不起没有考虑竞争条件。我很高兴
> Creation of TaskDeploymentDescriptor can block main thread for long time
> ------------------------------------------------------------------------
>
> Key: FLINK-16069
> URL: https://issues.apache.org/jira/browse/FLINK-16069
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: huweihua
> Priority: Major
>
> The deploy of tasks will take long time when we submit a high parallelism
> job. And Execution#deploy run in mainThread, so it will block JobMaster
> process other akka messages, such as Heartbeat. The creation of
> TaskDeploymentDescriptor take most of time. We can put the creation in future.
> For example, A job [source(8000)->sink(8000)], the total 16000 tasks from
> SCHEDULED to DEPLOYING took more than 1mins. This caused the heartbeat of
> TaskManager timeout and job never success.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
