On 05/10/10 01:11 PM, [email protected] wrote:
On 05/ 7/10 06:17 PM, Keith Mitchell wrote:
Hi all,
In the interest of improving the stability of the slim_source gate,
and providing tools for developers to more rapidly verify that code
changes don't cause regressions in related components, the testing
requirements that must be met prior to be pushing have been updated:
pushes to slim_source will now require unit tests to be included.
Previously, bug fixes required the fixer to perform "some form" of
testing, at the discretion of the fixer and requiring validation
during code review. With this change, the testing requirement is more
formal[1]. In general, the testing requirements to push to
slim_source have three components. First, existing unit tests must
not fail as a result of any changes. Second, new unit tests should be
delivered that demonstrate the bug being fixed is indeed fixed - the
new tests will ensure no regressions occur down the road. Third,
basic sanity checking is still expected (make sure an image still
boots, etc.) - however, as the gate gets more and more tests,
extensive sanity checking becomes unnecessary.
As a general guideline, we are aiming for 80% unit test coverage.
Note that bug fixes to areas of the gate that are intended to be
replaced by future project work are excluded from this requirement,
as the projects that deliver replacement code will be responsible for
supplying the initial set of unit tests to the new code. See the Unit
Testing section in the link[1] below for details on which areas will
not immediately require tests. (Be aware of whether or not a code
change will require unit tests *before* starting, and if it's
unclear, send questions to this alias to clarify ahead of time).
Please let me know if you have any questions or concerns about this
change.
Thanks,
Keith
[1] See item 4 in the following link, as well as the subsection on
Unit Testing (immediately follows item 9):
http://hub.opensolaris.org/bin/view/Project+caiman/Developers
This page was updated to reflect the new requirement.
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
Hey Keith,
I strongly believe in the value of unit testing. However it's not
clear to me if requiring unit tests for every bug fix is a good idea.
Can we please discuss this at our next group meeting.
Thank you. Joe
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
Hi Joe,
Pardon the delayed response - I've not been ignoring you, just trying to
cut through a jungle of other things grabbing for my attention.
As Dave mentioned, we've slotted some time for further discussion. I
will add that the requirement, like all requirements, is certainly
subject to exceptions, depending on circumstances. The idea, though, is
that for cases where exceptions are needed, they're sought out prior to
working on a bug, instead of after posting the code review - exceptions
would be deliberate. The overall goal, as indicated by the subject line,
is to have a more structured set of guidelines and requirements for what
testing should accompany a push, and to have the requirements be such
that the tests are easily re-usable by other developers and the
gatekeeper to monitor and prevent regression - with the end goal of
making pre-push fix verification an easier and stabler task.
I'm oversimplifying some of the above statements, but can clarify as needed.
- Keith
_______________________________________________
caiman-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
