fredia commented on PR #19448:
URL: https://github.com/apache/flink/pull/19448#issuecomment-1171001821
I re-run the benchmark of this PR and master, and filter out the item that
regression>5%, here are top regressions from jenkins's results:
<google-sheets-html-origin><!--td {border: 1px solid #ccc;}br
{mso-data-placement:same-cell;}-->
Benchmark | Unit | [PR
run#1](http://codespeed.dak8s.net:8080/job/flink-benchmark-request/159/) |
[PR run#2](http://codespeed.dak8s.net:8080/job/flink-benchmark-request/160/) |
[master
run#1](http://codespeed.dak8s.net:8080/job/flink-master-benchmarks-java8/448/)
| [master
run#2](http://codespeed.dak8s.net:8080/job/flink-benchmark-request/161/) | param
-- | -- | -- | -- | -- | -- | --
org.apache.flink.benchmark.BlockingPartitionBenchmark.uncompressedMmapPartition
| ops/ms | 8666.328735 | 9042.724129 | 9446.251683 | 9157.253861 |
org.apache.flink.benchmark.InputBenchmark.mapRebalanceMapSink | ops/ms |
15071.44224 | 13998.4278 | 13563.52986 | 14826.58196 |
org.apache.flink.benchmark.MultipleInputBenchmark.multiInputOneIdleMapSink |
ops/ms | 9849.316859 | 10965.11345 | 10850.4832 | 10897.66905 |
org.apache.flink.benchmark.RemoteChannelThroughputBenchmark.remoteRebalance
| ops/ms | 16325.58916 | 16523.94144 | 16774.34937 | 17514.40819 | DEBLOAT
org.apache.flink.benchmark.StreamNetworkThroughputBenchmarkExecutor.networkThroughput
| ops/ms | 65898.42896 | 61612.85227 | 68250.9443 | 68369.23892 | 100,100ms
org.apache.flink.benchmark.TwoInputBenchmark.twoInputMapSink | ops/ms |
17380.14833 | 17777.86862 | 17506.91314 | 18379.76836 |
org.apache.flink.benchmark.WindowBenchmarks.globalWindow | ops/ms |
6066.933889 | 6326.320592 | 6621.917999 | 6370.900111 |
org.apache.flink.benchmark.WindowBenchmarks.slidingWindow | ops/ms |
660.021689 | 685.552758 | 701.218769 | 663.998936 |
and I also re-run `remoteRebalance`, `networkThroughput`,` globalWindow` on
my local machine:
`org.apache.flink.benchmark.RemoteChannelThroughputBenchmark.remoteRebalance`:
<google-sheets-html-origin><!--td {border: 1px solid #ccc;}br
{mso-data-placement:same-cell;}-->
| run#1 | run#2 | run#3 | run#4
-- | -- | -- | -- | --
master | 14376.388 ± 916.003 | 14346.302 ± 1222.127 | 16307.239 ± 532.418 |
12082.863 ± 1558.623
PR19448 | 14503.878 ± 1245.878 | 13375.936 ± 1423.362 | 13242.821 ± 1636.507
| 13585.857 ± 1020.836
`org.apache.flink.benchmark.StreamNetworkThroughputBenchmarkExecutor.networkThroughput`:
<google-sheets-html-origin><!--td {border: 1px solid #ccc;}br
{mso-data-placement:same-cell;}-->
| run#1 | run#2 | run#3 | run#4
-- | -- | -- | -- | --
master | 60693.460 ± 3910.983 | 67485.945 ± 2368.999 | 67428.326 ± 2640.449
| 65570.558 ± 4839.778
PR19448 | 62581.403 ± 3676.553 | 68797.921 ± 2080.682 | 59300.429 ± 9673.195
| 70301.194 ± 1727.879
`org.apache.flink.benchmark.WindowBenchmarks.globalWindow`:
<google-sheets-html-origin><!--td {border: 1px solid #ccc;}br
{mso-data-placement:same-cell;}-->
| run#1 | run#2 | run#3 | run#4
-- | -- | -- | -- | --
master | 5528.938 ± 1420.702 | 6685.339 ± 350.645 | 6678.014 ± 353.418 |
6374.245 ± 947.530
PR19448 | 6536.733 ± 571.271 | 6789.017 ± 247.339 | 6836.893 ± 369.841 |
6973.793 ± 240.786
According to the above data, this PR is **not always worse than** master,
so I think that the regressions are caused by noise. @rkhachatryan WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
