[
https://issues.apache.org/jira/browse/FLINK-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992338#comment-16992338
]
Xintong Song commented on FLINK-15103:
--------------------------------------
Sure. I can configure them to use the same amount of network buffers as before.
I'm just in progress of verifying whether that also works (eliminates the
regressions) after FLIP-49 is activated.
I think we have more network buffers mostly in local execution environment.
That's where the JVM is already started and we use the JVM heap size to
estimate the "total memory" of a task executor (because we need that total
memory to calculate memory sizes for task executor components, e.g., memory
manager, shuffle environment).
For other scenarios (standalone / yarn / mesos / k8s), the total memory is
given before the task executor JVM is started, and we set JVM xmx/xms
parameters to decide JVM heap size with respect to whether managed memory is
on-heap or off-heap, so that other things should not be affected. For those
scenarios, I cannot guarantee that they have exactly same amount of network
memory as before (since we also replaced the container cut-off and replaced
with something different), but in general we expect they to be about the same.
Efforts are still needed to tune the default configuration values with various
real production jobs.
> Performance regression on 3.12.2019 in various benchmarks
> ---------------------------------------------------------
>
> Key: FLINK-15103
> URL: https://issues.apache.org/jira/browse/FLINK-15103
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Reporter: Piotr Nowojski
> Priority: Blocker
> Fix For: 1.10.0
>
>
> Various benchmarks show a performance regression that happened on December
> 3rd:
> [arrayKeyBy (probably the most easily
> visible)|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=arrayKeyBy&env=2&revs=200&equid=off&quarts=on&extr=on]
>
> [tupleKeyBy|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=tupleKeyBy&env=2&revs=200&equid=off&quarts=on&extr=on]
>
> [twoInputMapSink|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=twoInputMapSink&env=2&revs=200&equid=off&quarts=on&extr=on]
> [globalWindow (small
> one)|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=globalWindow&env=2&revs=200&equid=off&quarts=on&extr=on]
> and possible others.
> Probably somewhere between those commits: -8403fd4- 2d67ee0..60b3f2f
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
