[
https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17045135#comment-17045135
]
Zhu Zhu commented on FLINK-16069:
---------------------------------
Making building deployment descriptor more static seems not to be an easy work.
The critical path is the
{{TaskDeploymentDescriptorFactory#createInputGateDeploymentDescriptors() ->
getConsumedPartitionShuffleDescriptors()}} which generates
{{InputGateDeploymentDescriptor}}s for a task. This relies on
{{ShuffleDescriptor}}s which are created from the producer tasks's
{{ResultPartitionDeploymentDescriptor}}s.
{{ResultPartitionDeploymentDescriptor}}, however, is created lazily, i.e. only
after its producer has acquired the slot to determine the location. Besides
that, even if we can create the {{InputGateDeploymentDescriptor}} in a static
way and update it with the {{ResultPartitionDeploymentDescriptor}} when
scheduling, seems we do not avoid the 8000x8000 complexity to compute the
{{ShuffleDescriptor}}s.
I'm thinking whether we can add a shortcut for the ALL-to-ALL pattern by
caching generated {{ShuffleDescriptor}}s. e.g. In the case that
A(parallelism=8000) -> B(parallelism=8000), each
{{InputGateDeploymentDescriptor}} of B instances should contain the same 8000
{{ShuffleDescriptor}}s. So it is not needed to generate these 8000
shuffleDescriptors for 8000 times.
> Creation of TaskDeploymentDescriptor can block main thread for long time
> ------------------------------------------------------------------------
>
> Key: FLINK-16069
> URL: https://issues.apache.org/jira/browse/FLINK-16069
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: huweihua
> Priority: Major
>
> The deploy of tasks will take long time when we submit a high parallelism
> job. And Execution#deploy run in mainThread, so it will block JobMaster
> process other akka messages, such as Heartbeat. The creation of
> TaskDeploymentDescriptor take most of time. We can put the creation in future.
> For example, A job [source(8000)->sink(8000)], the total 16000 tasks from
> SCHEDULED to DEPLOYING took more than 1mins. This caused the heartbeat of
> TaskManager timeout and job never success.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
