Valentyn Tymofieiev created BEAM-8199:
-----------------------------------------
Summary: Improve tooling to enable alerts on performance
regression based on data collected from Beam benchmarks
Key: BEAM-8199
URL: https://issues.apache.org/jira/browse/BEAM-8199
Project: Beam
Issue Type: Bug
Components: testing
Reporter: Valentyn Tymofieiev
Currently we have some performance benchmarks in Beam, however they are not
always monitored and we may not notice when a regression happens, see [1].
Filing this issue to improve the tooling. Note that if automation becomes too
noisy (too many regressions) people likely won't be paying attention either.
As a first step, we could monitor regression in 50th percentile of performance
metrics we collect for benchmarks that we run repeatedly.
cc: [~kamilwu] [~kasiak] [~markflyhigh]
https://lists.apache.org/thread.html/51e000f16481451c207c00ac5e881aa4a46fa020922eddffd00ad527@%3Cdev.beam.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
