damccorm commented on code in PR #29595: URL: https://github.com/apache/beam/pull/29595#discussion_r1414484131
########## contributor-docs/rc-testing-guide.md: ########## @@ -0,0 +1,39 @@ +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Release Candidate (RC) Testing Guide + +This guide is meant for anybody who is interested in testing Beam Release Candidates against downstream projects. Note +that one need not have any status on the Apache Beam project (eg. PMC Member, Committer) to vote; all are welcome. +Please subscribe to the [dev list](https://lists.apache.org/[email protected]), and vote on the RC Vote email thread. + + +## RC Testing Objectives + +The RC testing process aims to: + + - Test new release candidates against existing code bases utilizing Apache Beam, to ensure there are no unexpected behaviors downstream. + - Incorporate a breadth of perspectives (including validation on multiple SDKs and multiple runners), before releasing a new version. + - Allow Beam Contributors to dogfood their changes and verify that they work as intended. + + Review Comment: I think we should at least have parity with https://beam.apache.org/blog/validate-beam-release/ before merging this, so we should add a java/go section. With that said, I do think a markdown doc is better for this kind of thing and this has value ########## contributor-docs/rc-testing-guide.md: ########## @@ -0,0 +1,39 @@ +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Release Candidate (RC) Testing Guide Review Comment: Probably instead of the link to the release blog, but more prominently called out ########## contributor-docs/rc-testing-guide.md: ########## @@ -0,0 +1,39 @@ +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Release Candidate (RC) Testing Guide Review Comment: We should link to this from https://github.com/apache/beam/blob/master/contributor-docs/release-guide.md#vote-and-validate-the-release-candidate ########## contributor-docs/rc-testing-guide.md: ########## @@ -0,0 +1,39 @@ +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Release Candidate (RC) Testing Guide + +This guide is meant for anybody who is interested in testing Beam Release Candidates against downstream projects. Note +that one need not have any status on the Apache Beam project (eg. PMC Member, Committer) to vote; all are welcome. +Please subscribe to the [dev list](https://lists.apache.org/[email protected]), and vote on the RC Vote email thread. + Review Comment: We should probably have a brief overview of the Beam release process (and especially the voting process), with links to https://github.com/apache/beam/blob/master/contributor-docs/release-guide.md ########## contributor-docs/rc-testing-guide.md: ########## @@ -0,0 +1,39 @@ +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Release Candidate (RC) Testing Guide + +This guide is meant for anybody who is interested in testing Beam Release Candidates against downstream projects. Note +that one need not have any status on the Apache Beam project (eg. PMC Member, Committer) to vote; all are welcome. +Please subscribe to the [dev list](https://lists.apache.org/[email protected]), and vote on the RC Vote email thread. + + +## RC Testing Objectives + +The RC testing process aims to: + + - Test new release candidates against existing code bases utilizing Apache Beam, to ensure there are no unexpected behaviors downstream. + - Incorporate a breadth of perspectives (including validation on multiple SDKs and multiple runners), before releasing a new version. + - Allow Beam Contributors to dogfood their changes and verify that they work as intended. + + + +## Ideas for Python SDK Validators + +_Note: Do the following in a dev-like environment._ +- If you are a Python SDK user that utilizes notebooks (eg. Jupyter Notebooks, or Colab Notebooks), change `pip install` +to point to the new RC (e.g. `pip install apache_beam[gcp]==2.52.0rc1`). Re-execute the workflow to ensure everything +works as intended. +- If your workflow utilizes [Dataflow Templates](https://github.com/GoogleCloudPlatform/DataflowTemplates), or another way of launching your job, modify your `requirements.txt` file, `setup.py` file, or `DockerFile` to point to the new Beam RC. +- _Tip_: Run your pipeline both against Direct Runner, and another runner of your choice by modifying your job's `PipelineOptions`. Review Comment: Maybe we could add a section: ``` ## After validation - Reply to Beam dev list vote thread with your vote (either +1 or -1) and an explanation of the use case you tested. - (optional) If part of your use case can be well represented by a test, consider contributing that back to Beam. Note that part of the value of validation is manual testing outside of the beam development environment, so it is still recommended to validate these test cases in the future. ``` (feel free to wordsmith) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
