In mid-February, Brad sent an email[1] to these mailing lists announcing Equinox’s desire to push systematic QA for Evergreen into gear. After much internal discussion about what we can reasonably attempt to accomplish in a three month plan, and talking to interested parties about their highest priority concerns with new releases, we have identified what we believe is a solid, accomplishable set of goals. The result of this project will be to provide some immediate and measurable improvement to code coverage and testing of specific areas of Evergreen, while providing a broad set of templates for others to follow when implementing future development, testing and documentation.
A key to the sustainability of this community-wide effort will be non-intrusive procedures, and this is something we, as a community, have attempted to focus on through build and testing automation. For those areas where some more work on the part of a developer would be beneficial, examples that can be copied and extended are meant as a means of lowering the barrier to entry. For instance, providing developer documentation is important for long-term maintenance of the code base; in addition to the common practice of describing an API and the concepts it embodies, detailing the design decisions that were made, and why, can be the difference between the introduction of bugs and the long-term stability of a feature. It is important to understand that what is offered here is a beginning for sustainable QA, not the end. QA is a set of ongoing processes that are undertaken consistently and systematically; it is not the individual tasks that inform us of bugs or performance regressions, as important as those are. Many of the tasks we plan to perform are those already identified by the general Evergreen development community, though many only proposed or suggested and some others implemented only sporadically. The structure of this three month plan is meant to help build momentum and buy-in from the broader Evergreen development community, so that the most successful and effective actions can be repeated and those found to be less effective can be either discarded or improved. This effort is complementary to that already underway by MassLNC[2]. The goal of that project is to identify specific areas of the Evergreen code that can be improved for performance, and provide suggestions for that improvement. What we aim for here is to create a set of procedures and methodologies that will support and facilitate any ongoing improvement to the code. Put another way, the MassLNC project it tactical where this proposal aims to be strategic. With that introduction out of the way, the main point of this note is to deliver the plan we’ve described so far. You can download the PDF[3] of it from our site and see exactly what we want to accomplish over the next few months. We’ve received significant positive feedback so far, and we are nearing the commitment goal that will allow us to proceed. We will not begin until we have commitments to fully fund the entire project because we feel that an underfunded measure is handicapped for failure from the start. We are close, however, and confident that those details can be nailed down over the next few weeks. If you have question or concerns, please feel free to contact me ( [email protected]) directly, or respond here on the list. Equinox as a whole is, and each of us individually are, committed to improving the state of QA for Evergreen and welcome any advice, constructive criticism or kudos :) you have to offer. Thanks, folks! [1] http://markmail.org/message/i3x7nsd3eygau4nb [2] http://masslnc.cwmars.org/node/2778 [3] http://esilibrary.com/esi/sustainable-qa.pdf -- Mike Rylander | VP, Research and Design | Equinox Software, Inc. / The Evergreen Experts | phone: 1-877-OPEN-ILS (673-6457) | email: [email protected] | web: http://www.esilibrary.com
