[
https://issues.apache.org/jira/browse/FLINK-24191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413014#comment-17413014
]
Piotr Nowojski commented on FLINK-24191:
----------------------------------------
Yes, something like that. The {{newBufferCount}} on the input should also take
into account the number of "channels with data". {{newBufferCount}} should be
divided somehow between input and output.
There are a couple of things that I'm not clear about, that should be still
designed:
# how to distribute buffers on the input side between exclusive/floating. What
should be the minimal value for floating buffers? Should we first shrink number
of exclusive buffers, or floating? This might depend if number of "channels
with data" > floating buffers or not?
# what to do exactly on the outputs side?
# should we be shrinking/growing the buffer pools?
> Adjusting number of buffers besides buffer size
> -----------------------------------------------
>
> Key: FLINK-24191
> URL: https://issues.apache.org/jira/browse/FLINK-24191
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Affects Versions: 1.14.0
> Reporter: Anton Kalashnikov
> Priority: Major
> Fix For: 1.15.0
>
>
> "Buffer debloat" adjusts only the buffer size but it also makes sense to
> adjust the number of buffers. It is not clear for now what should be adjusted
> and in which proportions so it needs to think about how to figure this out.
> The main idea of this ticket is to understand what is better to have one
> buffer of 10 records or 10 buffers of 1 record. On one hand handling of each
> buffer has an overhead on another hand, in case of low load it doesn't
> really make sense and it is ok to have many buffers with small sizes.
> Perhaps, we need benchmarks(microbenchamrk) to understand correlation between
> performance / buffers number / buffer size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
