Thanks Suneet. All look good to me except the ones below. > - Verify release notes contains a note about all issues marked with release notes label
I personally treat the "release notes" label as a mark for the candidate PRs that should be noted in the release notes. I sometimes intentionally drop some notes if the release notes grow too large when I'm the release manager. I'm not sure what others do when they are the release manager but think this is OK. > - [Druid requirement] Vote on whether any features are being released that are incomplete - missing integration tests, un-expected change in behavior, etc. Can you elaborate more on this suggestion? Are you suggesting making it a part of the release vote process? Or is it voting for each feature before starting the release vote? To me, making it a part of the release vote process seems too expensive because the release vote can start only when the tag is ready. I would like to have such discussions before the release vote is started. Today, we have been having such discussions in each PR when someone thinks a particular PR should be or should not be in the release. These discussions usually do not involve a vote because we usually reach a consensus after discussion. > Does a person voting need to run these integration tests today, since they are not part of the automated pipeline? This is a good point. I think we should run them manually for now. In the long term, we should find a way to automate those integration tests. On Wed, Apr 28, 2021 at 10:18 PM Suneet Saldanha <suneet.salda...@imply.io> wrote: > > Hi Jihoon, > > Thanks for pointing that out. Then, I'd update my proposal to include this > step as something that always needs to be done manually. > > > What I think should always be done manually: > > > > - [ASF requirement] download all signed source code packages, validate > checksums + signatures, compile without tests. start a Druid cluster and > ingest wikipedia data onto a micro-quickstart cluster and run a > simple query against it. > > - [Druid requirement] Vote on whether any features are being released > that are incomplete - missing integration tests, un-expected change in > behavior, etc. > > - [Druid requirement] Verify that all the automated tests that run for > the tag have actually passed (maybe this can be automated too?). I imagine > this step is to look at travis for the tag to ensure that all the tests > actually passed. https://travis-ci.com/github/apache/druid/builds/224182054 > - this is the travis build for the 0.21.0 tag (which appears to be failing) > > If we get agreement on this, I'll be happy to add / update a doc that talks > about what specifically is expected from someone voting on an Apache Druid > release. > > While writing out this proposal below, I've realized we have a set of > integration tests that do not run as part of the CI pipeline because they > require credentials to some third party systems (eg. Kinesis ITs) or we > can't get the tests to consistently run in the Travis pipeline without > intermittent failures (eg. Hadoop ITs). Does a person voting need to run > these integration tests today, since they are not part of the automated > pipeline? > > > > > > > On Mon, Apr 26, 2021 at 12:30 PM Jihoon Son <jihoon...@apache.org> wrote: > > > Hi Suneet, thanks for your suggestion. > > > > However, the Apache release policy [1] explicitly says the below. > > > > > Before casting +1 binding votes, individuals are REQUIRED to download > > all signed source code packages onto their own hardware, verify that they > > meet all requirements of ASF policy on releases as described below, > > validate all cryptographic signatures, compile as provided, and test the > > result on their own platform. > > > > Because only the source release is our official release, PMCs have to > > download at least the source code package and verify it manually per > > this requirement. > > That being said, we can automate only verifying the binary package and > > docker image at this moment. > > > > Jihoon > > > > [1] https://www.apache.org/legal/release-policy.html#policy > > > > On Fri, Apr 23, 2021 at 6:26 PM Suneet Saldanha > > <suneet.salda...@imply.io> wrote: > > > > > > Hi Druids, > > > > > > I was thinking about the 0.21.0 release vote that was just happening, and > > > found it tedious to help with the vote because I needed to build the code > > > from source as well as run all the tests on my local machine. This got me > > > thinking about what are the key steps that we should check manually in > > the > > > release process and what steps can we rely on automation that exists in > > > Travis today. I don't have full context on what steps are needed, but I > > > thought I'd start a thread to hear from everyone about what steps should > > be > > > done before a release, and whether any of them need to be done manually, > > or > > > if we could leverage our CI pipelines to replace these steps. > > > > > > What I think should be done before the release? > > > > > > - Run all unit tests > > > - RAT License checks > > > - Run all integration tests > > > - Verify checksums > > > - Verify signatures > > > - Verify release notes contain > > > > > > What I think should be done manually (because automation doesn't exist > > > today): > > > > > > - Verify docker pull works > > > - Verify Druid can start from the pulled docker image > > > - Verify no open issues on github with the milestone for the version > > > being released > > > - Verify release notes contains a note about all issues marked with > > > release notes label > > > > > > > > > What I think should always be done manually: > > > > > > - Vote on whether any features are being released that are incomplete > > > - Verify that all the automated tests that run for the tag have > > actually > > > passed (maybe this can be automated too?) > > > > > > Look forward to hearing thoughts on this :) > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org > > For additional commands, e-mail: dev-h...@druid.apache.org > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@druid.apache.org For additional commands, e-mail: dev-h...@druid.apache.org
