> Micheal J wrote:
> > Juha-P Lindfors wrote:
> > > On Fri, 23 Feb 2001, Phan Anh Tran wrote:
> > > > 1. Are developers required to pass existing regression test
> > > > before committing changes?
> > >
> > > Yes.
> > >
> > > > 2. Are developers required to develop a regression test
> > > > (if one does not exist) and pass it before committing changes?
> > >
> > > No.
> >
> > Aha. A QA loop-hole.
> > We can't have that now can we?
>
> That depends.
>
> What matters for the quality of the
> product is end-to-end functionality
> testing.
> For this we have jbosstest.
Disagree. It looks good is not the same as it *is* good.
> Some regression tests belong in the
> end-to-end functionality testing
> category, and such should be added to
> jbosstest in case it wasn't there
> already. Here we may have a loop-hole,
> but I haven't seen any problems
> arising from it yet.
The constant stream of commits, followed by an undo when something breaks
and then a later re-commit along with a "I have tested it on blah, blah,
blah and it shouldn't break anything else but..." comment is proof of that.
> Other tests (like unit tests) are not
> really done to improve the quality of
> the final product, but to improve the
> quality of the development process.
Unit-testing (when done to properly) will help by improving our confidence
in the quality of the code. If the tests define the contract the code is
supposed to fill accurately then, we know that come rain or shine (or
commits by whoever) if the any of the tests fail then something has been
broken (or some test has been badly written).
Long term it improves our collective ability to improve the code. Just do it
and run the tests...
The only problem I see is that the tests *have* to be written. And on an
open source project it can be difficult to get people unfamilliar with the
unit0testing process to get involved.
> But tests do not prove that a product
> is bug-free. It only gives you a
> certain confidence that whatever is
> tested works as tested.
Which is *infinitely* better than not testing at all...
> IMHO, what *really* matters for the
> quality of JBoss is the large number
> of users that are using it in a lot
> of different situations, with
> different configurations and different
> beans. A large part of these users
> report when they have problems that
> could be bugs in JBoss, and this
> enables developers to fix things that
> do not work in situations they have
> not even thought about.
> That does not only help to reduce the
> number of bugs in JBoss, but also
> makes JBoss more robust. And IMHO
> robustness is just as important as
> functionality for JBoss.
This sound like a "write the code, don't test it, just release it" stance.
If there are any bugs, the users would find them soon enough when they try
to use it. We kind of practise this now and it has served us so far but, I
suspect we would have been much farther along if we didn't have the regular
backtracks to (re-)fix something that a new commit has broken, then re-apply
all the new stuff and hope nothing breaks this time...
IMHO, functional tests and end-user acceptance tests compliment unit-tests.
They do not replace them. In fact when a bug is found, we ask for a script
or series of steps that lets us re-create the bug. A bug signature if you
will.
The reverse should also be true. A bug is due to the code breaking one or
more contracts that it is supposed to fill. When we find the broken
contracts, we should produce unit tests to ensure that later fixes/upgrades
do not break that contract again. Every bug-fix should be accompanied by
unit tests that can detect the (underlying cause of the) bug were it to
reappear. A bug-fix signature if you will.
Cheers!,
Micheal
--
--------------------------------------------------------------
To subscribe: [EMAIL PROTECTED]
To unsubscribe: [EMAIL PROTECTED]
List Help?: [EMAIL PROTECTED]
