[
https://issues.apache.org/jira/browse/FLINK-27571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski closed FLINK-27571.
----------------------------------
Fix Version/s: 1.17.0
Resolution: Fixed
merged commit b21a946 into apache:master
> Recognize "less is better" benchmarks in regression detection script
> --------------------------------------------------------------------
>
> Key: FLINK-27571
> URL: https://issues.apache.org/jira/browse/FLINK-27571
> Project: Flink
> Issue Type: Bug
> Components: Benchmarks
> Affects Versions: 1.16.0
> Reporter: Roman Khachatryan
> Assignee: Yanfei Lei
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.17.0
>
> Attachments: Screenshot_2022-05-09_10-33-11.png,
> image-2022-12-29-14-39-59-976.png
>
>
> Example benchmark:
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=5&ben=schedulingDownstreamTasks.BATCH&extr=on&quarts=on&equid=off&env=2&revs=200]
>
> [Proposed
> solution|https://issues.apache.org/jira/browse/FLINK-27555?focusedCommentId=17534423&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17534423]:
> {quote}
> I think #2 is the correct way.
> Maybe we can modify the save_jmh_result.py to correctly set the 'units' and
> the 'lessisbetter' fields of benchmark results. The 'units' is already
> contained in the jmh result and the 'lessisbetter' can be derived from the
> mode(false if it is 'thrpt' mode, otherwise true). An example of the jmh
> result format can be found at https://i.stack.imgur.com/vB3fV.png.
> This can fix the web UI as well as the REST result, and then the
> regression_report.py will be able to identify which benchmarks are "less is
> better" and treat them differently.
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
