[
https://issues.apache.org/jira/browse/FLINK-36072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junrui Lee reassigned FLINK-36072:
----------------------------------
Assignee: Junrui Lee
> Optimizing the Overhead of the Network Layer in Adaptive Execution Scenarios
> ----------------------------------------------------------------------------
>
> Key: FLINK-36072
> URL: https://issues.apache.org/jira/browse/FLINK-36072
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Junrui Li
> Assignee: Junrui Lee
> Priority: Major
>
> In adaptive execution scenarios, hash edges may transition to broadcast
> edges; however, at that point, the upstream may have already produced data
> based on hashes.
> For example, in the adaptive broadcast join case, this results in each
> downstream task needing to connect to all upstream tasks and create a
> partition reader for each subpartition, leading to significant overhead
> (O(N²)). To optimize the overhead of the network layer, we need to enable
> downstream tasks to consume all subpartitions from a specific Task Manager
> using a single channel and a single partition reader in such situations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
