[
https://issues.apache.org/jira/browse/FLINK-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Garrett Li updated FLINK-8784:
------------------------------
Description:
A logical network connection exists for each point-to-point exchange of data
over the network, which typically happens at repartitioning- or broadcasting
steps (shuffle phase). et
Currently Flink divide the network buffer pool evenly for every partition
operators. Sometimes it could waste too many network buffers since we might
have a lot of repartitioning and broadcasting in the one Flink job, and NOT all
of the partition operators in the same job have the same traffic.
was:
A logical network connection exists for each point-to-point exchange of data
over the network, which typically happens at repartitioning- or broadcasting
steps (shuffle phase). et
Sometimes we have a lot of repartitioning and broadcasting in the Flink job,
and currently Flink divide the network buffer pool evenly for every partition
operators, sometimes it could waste too many network buffers since NOT all of
the partition operators in the same job have the same traffic.
> Share network buffer pool among different partition operators
> -------------------------------------------------------------
>
> Key: FLINK-8784
> URL: https://issues.apache.org/jira/browse/FLINK-8784
> Project: Flink
> Issue Type: Improvement
> Components: TaskManager
> Reporter: Garrett Li
> Priority: Major
>
> A logical network connection exists for each point-to-point exchange of data
> over the network, which typically happens at repartitioning- or broadcasting
> steps (shuffle phase). et
> Currently Flink divide the network buffer pool evenly for every partition
> operators. Sometimes it could waste too many network buffers since we might
> have a lot of repartitioning and broadcasting in the one Flink job, and NOT
> all of the partition operators in the same job have the same traffic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
