[
https://issues.apache.org/jira/browse/FLINK-15103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992317#comment-16992317
]
Piotr Nowojski commented on FLINK-15103:
----------------------------------------
[~xintongsong] thanks for analysing this. I was actually suspecting it the
other way around, that somehow we decreased the number of network buffers, but
you are right. That's probably it. And yes, benchmarking takes (unfortunately)
into account setup times.
Those benchmarks are running on a single machine with only very small
parallelism and the jobs are quite simple, so they even do not need 12945
buffers. Something like 1000 should be more than enough, but that's for another
story. Could you [~xintongsong] for now configure the benchmarks to always use
the same number of buffers as before? After the release, we can decrease the
number of buffers to something more sane (that would decrease the startup
overhead costs, but would also present as "false" performance improvement, so I
don't want this to happen while we are investigating this and FLINK-15104 ).
Another issue is, In what scenarios there will be more network buffers compared
to now? If Flink is using more network buffers, there is fewer memory for other
things, which can cause memory issues for currently working setups.
> Performance regression on 3.12.2019 in various benchmarks
> ---------------------------------------------------------
>
> Key: FLINK-15103
> URL: https://issues.apache.org/jira/browse/FLINK-15103
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Reporter: Piotr Nowojski
> Priority: Blocker
> Fix For: 1.10.0
>
>
> Various benchmarks show a performance regression that happened on December
> 3rd:
> [arrayKeyBy (probably the most easily
> visible)|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=arrayKeyBy&env=2&revs=200&equid=off&quarts=on&extr=on]
>
> [tupleKeyBy|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=tupleKeyBy&env=2&revs=200&equid=off&quarts=on&extr=on]
>
> [twoInputMapSink|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=twoInputMapSink&env=2&revs=200&equid=off&quarts=on&extr=on]
> [globalWindow (small
> one)|http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=globalWindow&env=2&revs=200&equid=off&quarts=on&extr=on]
> and possible others.
> Probably somewhere between those commits: -8403fd4- 2d67ee0..60b3f2f
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
