Agreed, the more bugs found before a release (and by us) the better. Expecting or even desiring that our users help in finding the bugs before a release is IMHO unrealistic.
> On Apr 24, 2016, at 4:00 PM, Jed Brown <[email protected]> wrote: > > Barry Smith <[email protected]> writes: > >>> On Apr 24, 2016, at 1:48 PM, Karl Rupp <[email protected]> wrote: >>> On the other hand, a feature freeze for a week is a perfect >>> opportunity for improving documentation and other work that will not >>> break the code. And even if one prefers to keep coding, new features >>> can still be put in next for testing. Particularly if communicated in >>> advance ("Hey, we will have a feature freeze in 7 days and plan to >>> release in 10 days"), people can easily arrange things. I think >>> Lisandro's feature would have been ready on time if he had known 7 >>> days in advance. >> >> Nonsense, he knew weeks ahead. > > He knew that a release was coming, but no particular date. > >> The option is ALWAYS there. We even have users who run Jenkins or >> buildbot against master everyday. It's not like there was not a >> warning that a release was coming out soon. People who want to do >> the pretesting (and there are very very few of them) had the >> opportunity. > > However, there was no guarantee that such testing would be relevant to > the release. Someone could be working on some feature that is going > into the release. > >> Some people like to have a pre-annouced day for a release (thinking >> if people are told that day, they will actually finish their >> feature on time or do some testing before then), I've avoided this >> because we are reliant on people (both in and outside PETSc) whose >> schedules are erratic, I think people do crappy work with fixed >> deadlines, and when we've tried it in the past we've always pushed >> past it anyways so what meaning did it have? > > You have announced specific dates in the past. Every conference has a > deadline for paper submission and many/most extend it by a week or two. > I think there would be mass complaints if the organizers instead > announced that they were going to close submissions at some point in the > coming weeks, let some time go by, then said today's the last day for > submissions. > >> Actually he is proposing something that induces more effort. For >> example for a whole freaking week my work-flow of moving stuff that >> passes the tests in next into master is broken and this induces >> later more manual merges because other developers started their >> features/fixes with outdated code. Plus I am suppose to be running >> around for a week fixing documentation and testing on odd-ball >> machines. > > Our features are rarely so interdependent that the feature you implement > today breaks the feature some other developer starts on Tuesday. That > merge isn't really urgent and the marginal cost of waiting a week is > really low. > > But I think the reminder about fixing documentation and any outstanding > bugs/reports is well worth it in terms of improving the quality of the > release. > > Not every user that encounters a bug reports it. While the cost to us > in fixing a bug is about the same if it is discovered and fixed in > 'master' versus 'maint', it is significantly higher for users. > Moreover, when we ship a release with bugs, users encounter that bug for > months, even after we have patched it (they don't instantly and > effortlessly receive the patch release when we tag it), and then we get > the support mail. So bugs that affect multiple users are much more > costly when discovered after the release. > >> If you have a staff of programmers sitting in an office everyday > > [...] > > I have no expectation that everyone participate actively in the release. > But I also don't think it's true that everyone ignores the release > either.
