[
https://issues.apache.org/jira/browse/FLINK-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170876#comment-17170876
]
Roman Khachatryan commented on FLINK-18786:
-------------------------------------------
The same symptoms as in FLINK-18684: regression in some tests observed over
time, but can't be reproduced retroactively:
{code:java}
(older commit) 4b25ba27922e1e7c4a17067fc94b69c5eac4ad2d
"MemoryStateBackendBenchmark.stateBackends","thrpt",1,30,4562.465426,100.369594,"ops/ms",,,MEMORY
"MemoryStateBackendBenchmark.stateBackends","thrpt",1,30,4182.523419,79.437423,"ops/ms",,,FS
"MemoryStateBackendBenchmark.stateBackends","thrpt",1,30,4584.939673,152.263596,"ops/ms",,,FS_ASYNC{code}
{code:java}
(newer commit) 27ecf02fb2a8e61b68e93190080f540e53aca171
"MemoryStateBackendBenchmark.stateBackends","thrpt",1,30,4570.875302,141.128996,"ops/ms",,,MEMORY
"MemoryStateBackendBenchmark.stateBackends","thrpt",1,30,4221.538372,93.620711,"ops/ms",,,FS
"MemoryStateBackendBenchmark.stateBackends","thrpt",1,30,4575.985146,115.443546,"ops/ms",,,FS_ASYNC{code}
Running locally:
{code:java}
(older commit) 4b25ba2
Benchmark (stateBackend) Mode Cnt Score Error Units
MemoryStateBackendBenchmark.stateBackends MEMORY thrpt 30 3072.503 ± 58.403
ops/ms
MemoryStateBackendBenchmark.stateBackends FS thrpt 30 2910.727 ± 32.738 ops/ms
MemoryStateBackendBenchmark.stateBackends FS_ASYNC thrpt 30 3145.262 ± 41.423
ops/ms{code}
{code:java}
(newer commit) 3ab1a1c66772320a0901ea085cfeaa6bf161bf9a
Benchmark (stateBackend) Mode Cnt Score Error Units
MemoryStateBackendBenchmark.stateBackends MEMORY thrpt 30 3117.415 ± 36.979
ops/ms
MemoryStateBackendBenchmark.stateBackends FS thrpt 30 2917.314 ± 40.304 ops/ms
MemoryStateBackendBenchmark.stateBackends FS_ASYNC thrpt 30 3062.132 ± 102.320
ops/ms{code}
> Performance regresssion on 2020.07.29 in some state backend benchmarks
> ----------------------------------------------------------------------
>
> Key: FLINK-18786
> URL: https://issues.apache.org/jira/browse/FLINK-18786
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks, Runtime / State Backends
> Affects Versions: 1.12.0
> Reporter: Roman Khachatryan
> Assignee: Roman Khachatryan
> Priority: Major
> Attachments: statebackend-fs-maybe-regression.png
>
>
> E.g.
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=1,3&ben=stateBackends.FS&env=2&revs=200&equid=off&quarts=on&extr=on]
> [http://codespeed.dak8s.net:8000/timeline/?ben=stateBackends.ROCKS&env=2]
> [http://codespeed.dak8s.net:8000/timeline/?ben=stateBackends.ROCKS&env=2#/?exe=1,3&ben=stateBackends.ROCKS_INC&env=2&revs=200&equid=off&quarts=on&extr=on]
> [http://codespeed.dak8s.net:8000/timeline/?ben=slidingWindow&env=2]
>
> See attachment for some hints
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
