On Fri, Oct 12, 2012 at 12:28 PM, Will Chan <will.c...@citrix.com> wrote: > I realize just blindly taking QA's test as a proxy is not actual doing > anything but what if you are actually sitting down with a QA engineer, and > executing the same steps with them albeit doing it on the same setup to save > time? Would that not count? > > The second question is if the QA engineer has verified the build before the > vote, would he/she need to redo the exact same steps afterwards to vote? If > the answer to this is "Yes", then I suppose my first claim is useless. > > Sorry if I am asking these questions but I wasn't sure how this voting > process goes because I can assure you I've spent huge amounts of time > QA/testing the build with the QA engineers and sometimes do not have the time > to do them again. However, if the vote cast is about individuals doing it > themselves after the vote cast, I can abstain from voting and simply just > comment for the QA. > > Will
I think that we are mixing things here. The Citrix QA engineers have done fantastic work as part of the community effort to get to the voting process phase. Now that we are cutting release candidates, this moves from a period where we are relying on the QA team's skills to test features / functions / docs, and into a period where individual responsibility kicks in. The release testing process [1] includes three major sets of checks: First, it asks the voter to verify things like "Is the downloaded source archive exactly the same as what we know to be in the git repo at the commit-ish that was announced as the RC source?" and "Do the signatures and hash'es match up as expected?". Second, it asks the voter to review the source code package itself for legal issues. Third, it includes a set of *minimum* build / functional tests that are useful in determining that the functionality is as expected. This is the step that starts to repeat the previous QA testing process. Some individuals may expand or alter what they do in this phase... for example, Wido obviously tested the release candidate in an environment other than DevCloud. That's OK! I look at this phase as being one where people validate the core functions that they expect to work. So if I was in your shoes, I wouldn't abstain from voting. I'd spend time looking at phase 1 and 2, and then run through the basic devcloud tests. For QA engineers that want to participate in the RC testing, the same thing applies! Just like others, they might choose to double check things in an environment of their choosing. As long as people are testing the exact software that's in the release candidate artifacts, any amount of functional re-test is good. The point about one's vote being as an individual is that Apache relies on these individual verification steps to be executed by multiple individuals. That's also the reason why there are rules about a vote only passing if there are at least 3 PPMC/IPMC votes on the dev list, and then a second round of voting within the incubator general list where at least 3 IPMC votes need to be recorded. This, combined with the community-wide vote having to have more +1's than -1's, is basically how ASF has delegated responsibility for verifying a release. [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+4.0+test+procedure > From: Noah Slater [nsla...@tumbolia.org] > Sent: Thursday, October 11, 2012 10:40 PM > To: cloudstack-dev@incubator.apache.org > Subject: Re: [VOTE] Apache Cloudstack 4.0.0-incubating Release, first round > > On Fri, Oct 12, 2012 at 4:48 AM, Will Chan <will.c...@citrix.com> wrote: > >> Go ahead with the recast of votes then if you don't want QA to re-test. >> My binding vote is based on Citrix QA test. >> > > It is wise to consider the QA testing, but your vote must be based on your > testing of the artefact itself. When you vote, you are voting as an > individual. It is your personal mark of "I have downloaded this artefact > and I have verified that we should release this personally." > > Please do not rely on other people's testing as a proxy for your vote. All > that does is add noise. If we want the QA team's efforts to impact our vote > (of course we do) then the QA team should vote. (And I am not suggesting > they run the full battery of tests. Just that they use their experience to > verify the release candidate like everyone else.) We don't need people > voting +1 to signal that they are pleased with the QA team results. > > > -- > NS
