[
https://issues.apache.org/jira/browse/FLINK-23453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski closed FLINK-23453.
----------------------------------
Fix Version/s: 1.14.0
Resolution: Fixed
merged to master as 088d1651712..d8477bda394
> Dynamic calculation of the buffer size
> --------------------------------------
>
> Key: FLINK-23453
> URL: https://issues.apache.org/jira/browse/FLINK-23453
> Project: Flink
> Issue Type: Sub-task
> Reporter: Anton Kalashnikov
> Assignee: Anton Kalashnikov
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> To calculate the desired buffer size we need to take into account the
> throughput, configuration(timeInInputBuffer), and the actual number of
> buffers in use. It makes sense to use EMA for this calculation to smoothen
> out intermittent spikes.
> The calculation based on the actual number of buffers in use helps to avoid
> problems with the data skew (when only a couple of channels out of thousands
> have any data). So the solution needs to reliably and efficiently calculate
> either the estimated or an average number of buffers in use.
> Buffer size can be erratic if it’s not trivial to make it stable in the MVP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
