[
https://issues.apache.org/jira/browse/FLINK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17819578#comment-17819578
]
Maximilian Michels edited comment on FLINK-34471 at 2/22/24 10:18 AM:
----------------------------------------------------------------------
Thanks! That would be very helpful – I don’t mind at all. I realized point to
point connections might be a bit tricky when the parallelism between tasks
changes from being equal to being different because at runtime we then switch
to a different partitioner. So worst case we could run out of network buffers
in this scenario. I’m curious how you want to solve this. We probably need to
add extra buffers to account for this edge case.
was (Author: mxm):
Thanks! That would be very helpful – I don’t mind at all. I realized point to
point connections might be a bit tricky when the parallelism between tasks
changes from being equal to changing because at runtime we then switch to a
different partitioner. So worst case we could run out of network buffers in
this scenario. I’m curious how you want to solve this. We probably need to add
extra buffers to account for this edge case.
> Tune network memory as part of Autoscaler Memory Tuning
> -------------------------------------------------------
>
> Key: FLINK-34471
> URL: https://issues.apache.org/jira/browse/FLINK-34471
> Project: Flink
> Issue Type: Improvement
> Components: Autoscaler
> Reporter: Rui Fan
> Assignee: Maximilian Michels
> Priority: Major
>
> Design doc:
> https://docs.google.com/document/d/19HYamwMaYYYOeH3NRbk6l9P-bBLBfgzMYjfGEPWEbeo/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
