aaltay commented on a change in pull request #12455: URL: https://github.com/apache/beam/pull/12455#discussion_r464570198
########## File path: website/www/site/content/en/contribute/release-guide.md ########## @@ -244,7 +247,21 @@ __Attention__: Only PMC has permission to perform this. If you are not a PMC, pl ********** -## 2. Create a release branch in apache/beam repository +## 3. Investigate performance regressions + +Check the Beam load tests for possible performance regressions. Measurements are available on [metrics.beam.apache.org](http://metrics.beam.apache.org). + +All Runners which publish data should be checked for the following, in both *batch* and *streaming* mode: + +- [ParDo](http://metrics.beam.apache.org/d/MOi-kf3Zk/pardo-load-tests) and [GBK](http://metrics.beam.apache.org/d/UYZ-oJ3Zk/gbk-load-test): Runtime, latency, checkpoint duration +- [Nexmark](http://metrics.beam.apache.org/d/ahuaA_zGz/nexmark): Query runtime for all queries +- [IO](http://metrics.beam.apache.org/d/bnlHKP3Wz/java-io-it-tests-dataflow): Runtime + +If regressions are found, the release branch can still be created, but the regressions should be investigated and fixed as part of the release process. +JIRA issues should be created for each regression with the 'Fix Version' set to the to-be-released version. +Next, the mailing list should be informed to allow fixing the regressions in the course of the release. Review comment: Do we want to spell out what to do after filing the issues? Or do we want to use the same process as any other jira issues tagged for that release (i.e. triage then decide blocker or not)? ########## File path: website/www/site/content/en/contribute/release-guide.md ########## @@ -244,7 +247,21 @@ __Attention__: Only PMC has permission to perform this. If you are not a PMC, pl ********** -## 2. Create a release branch in apache/beam repository +## 3. Investigate performance regressions + +Check the Beam load tests for possible performance regressions. Measurements are available on [metrics.beam.apache.org](http://metrics.beam.apache.org). + +All Runners which publish data should be checked for the following, in both *batch* and *streaming* mode: + +- [ParDo](http://metrics.beam.apache.org/d/MOi-kf3Zk/pardo-load-tests) and [GBK](http://metrics.beam.apache.org/d/UYZ-oJ3Zk/gbk-load-test): Runtime, latency, checkpoint duration +- [Nexmark](http://metrics.beam.apache.org/d/ahuaA_zGz/nexmark): Query runtime for all queries +- [IO](http://metrics.beam.apache.org/d/bnlHKP3Wz/java-io-it-tests-dataflow): Runtime + +If regressions are found, the release branch can still be created, but the regressions should be investigated and fixed as part of the release process. +JIRA issues should be created for each regression with the 'Fix Version' set to the to-be-released version. Review comment: Who could be the owner of doing a first investigation? Release manager could file issues but may not be in a position to understand the cause of the regressions. One suggestion for the list of all benchmarks above we add a first point of contact? Would that be reasonable? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
