On Fri, 2009-09-11 at 20:24 -0400, Joe Atzberger wrote: > I'm looking to formally record in Trac the various bugs encountered > during development and testing. Right now, the number of EG Versions > to file against are a small subset of the actual number of available > versions (plus 2.0 that doesn't yet exist). So I have a couple > questions: > * How should the 2.0 Version should be used? > * What level of granularity do we want: each increment, > major.minor only, something else? > * What about trunk? > I suggest we probably want each increment, but more important than > that, I think we need a version for trunk (or HEAD) where the majority > of bugs and enhancements will be filed... and some hopefully fixed. > It would be counterproductive to file enhancements against a stable > version, since all the new features should be going against trunk when > possible. And you don't want it against whatever number version > happens to be current now, because if the ticket sits a while or the > feature takes more time to develop, it needs to keep up w/ the > progressing version numbers and not look like a piece of history. > > Any thoughts on this?
Hey Joe: A simple reply would be that one can and should modify tickets if a feature targeted for a given release gets bumped to a subsequent release. However, I think it would be worth revisiting how releases work in general; this is just one aspect (but an important one, I would love to see more formal use of Trac for bugs and features and I think it would be very healthy for the community). Our development and release process is a work in progress, as http://open-ils.org/documentation/contributing.html states. I'm going to suggest a few changes of direction that might help us move towards a more agile release process (and hopefully get a conversation going about other changes that we might want to consider). * Time-based releases: We've seen projected release dates slip because we were trying to fit a feature into a given release and waiting on that feature to be finalized. Meanwhile, other enhancements get stuck waiting on that release. Generating a regular (monthly? weekly? daily?) release tarball would give us the opportunity to have regular, objective feedback on the quality of our releases, get new features into the hands of interested parties with less hassle, and still give us the ability to anoint a given tarball an official release. But this won't be possible without a few other wishlist items: * Automated releases: Right now, the effort required to cut a release comes down to a whole lot of (rather poorly documented) manual work that doesn't really encourage a "release often" approach. If we were to automate the process of creating a release tarball & corresponding staff client, we could start pumping out time-based releases that could get new features into the hands of testers or adopters earlier. This would require us to be disciplined about maintaining a database upgrade script each time the database schema is modified, rather than the heroic effort that miker has put into each and every database upgrade script to date. We would find out right away if fundamental pieces of the build break due to dependencies (such as the translations, which frustrated miker a while back when some of the Python modules used by the build were updated and broke our existing translation build scripts). * Automated testing: This is one area where Evergreen, a complex beast, is currently frighteningly weak - but I hear that there's some work underway to build a sample database, which would be a great base for a suite of regression tests (at least at the OpenSRF level; automated testing of XUL / Dojo interfaces / catalogue is another more complex kettle of fish, but it has been done by others...) Once we have that base available to us, I think it should be a condition that each new feature should come with a suite of tests to ensure that changes in another part of the stack (whether our code changes, or changes to external dependencies - hello, PostgreSQL 8.4!) don't unintentionally affect how this particular feature works. There's loads of room for adding test coverage of existing functionality, of course. When a bug is found, a test should be added that triggers the bug (and when the bug is fixed, the test will start passing). * Divide and conquer responsibilities: I think there are a number of responsibilities that could be picked up by more members of the community. Sharing responsibilities across a broader team should make everyone more efficient (at the cost of some communication and coordination overhead). Some ideas about possible roles: * Translation management, currently rather poorly done by me, could be relatively easily picked up by someone else (particularly if Pootle is adopted as an official tool). * Platform owners - people responsible for testing Evergreen releases on a particular distribution & maintaining installation documentation / install scripts for that distribution & dependency Makefiles & possibly generating virtual images. Currently there's a rough focus on Debian and Ubuntu, but there are members of the community who could cover RHEL, and we could have better defined coverage for specific versions of Debian and Ubuntu. * Tests: once we have a test framework and some good sample tests in place, I'm sure there are a number of community members who could start writing tests for existing functionality. For that matter, perhaps there are community members who can help build that framework. * Patch management: I've seen a few contributed patches get sent to the list with little feedback from those of us with commit bits. Some other projects more formally track the status of patches to ensure that a given patch gets reviewed and that it is either accepted, rejected, or that feedback is given and that a revised patch has been requested - either via a wiki page, or via a regular list of patches sent to the development mailing list. Someone, or someones, could be responsible for tracking the disposition of all patches (and helping us ensure that we give feedback to would-be contributors in a timely fashion). On another track, versioning, right now I think there are three or four active release branches: 1.2 - hopefully limited to just bugfixes (but, last I heard, there were still a lot of sites on 1.2) 1.4 - hopefully limited to just bugfixes (now that we have a usable release in 1.4.0.6, by all accounts) 1.6 - holding tank for a ton of new features, with acquisitions and serials and the event-driven infrastructure 2.0 currently aka trunk - holding tank for everything else in development Each release since 1.2 has an a.b.c.d numbering scheme, which doesn't jive with our documented versioning scheme (http://evergreen-ils.org/dokuwiki/doku.php?id=versioning). Going forward, I suggest we toss the a.b.c.d scheme that crept in with the 1.2 release and stick with the documented approach of features only going into the "a" or "b" iterations and bugfixes only going into the "c" iterations. Also, perhaps we should just use increments of 1, rather than suggesting that an odd release number is considered unstable. We haven't pushed out an odd release since 1.1 (May 2007), so we're not really making use of this distinction. If we went with time-based releases, then these could be date-stamped tags in SVN, and we could just tag the one that we pick for the next official release as a.b+1.0. We could also adopt a support policy of something like "current a release + previous a release" (picking numbers out of thin air, assuming that we did something like bump "a" once a year and "b" approximately every three months) to give users an explicit target for how long a given release will be supported. Of course, there's an opportunity for commercial support vendors to offer extended support :) There would be some work required to adopt these ideas, but I think it would result in a healthier development community that would be able to focus on development and deliver enhancements to its users at a more rapid pace. Hopefully someone is still awake out there. I'd love to hear some feedback.