[
https://issues.apache.org/jira/browse/FLINK-23453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski reassigned FLINK-23453:
--------------------------------------
Assignee: Anton Kalashnikov
> Dynamic calculation of the buffer size
> --------------------------------------
>
> Key: FLINK-23453
> URL: https://issues.apache.org/jira/browse/FLINK-23453
> Project: Flink
> Issue Type: Sub-task
> Reporter: Anton Kalashnikov
> Assignee: Anton Kalashnikov
> Priority: Major
> Labels: pull-request-available
>
> To calculate the desired buffer size we need to take into account the
> throughput, configuration(timeInInputBuffer), and the actual number of
> buffers in use. It makes sense to use EMA for this calculation to smoothen
> out intermittent spikes.
> The calculation based on the actual number of buffers in use helps to avoid
> problems with the data skew (when only a couple of channels out of thousands
> have any data). So the solution needs to reliably and efficiently calculate
> either the estimated or an average number of buffers in use.
> Buffer size can be erratic if it’s not trivial to make it stable in the MVP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
