Thanks Jason for showing me the bigger picture. I agree with you now,
by and large.
However in your build flow for each *-testsuite, you begin with
generating a build ticket. I wonder if we should generate the build
ticket at the geronimo/testsuite level and not one level down.
Yes, we need some real tests now. I'll start working on one so that it
could serve as an example for others to write their own.
Cheers
Prasad
On 9/28/06, Jason Dillon <[EMAIL PROTECTED]> wrote:
On Sep 27, 2006, at 8:40 PM, Prasad Kashyap wrote:
Build flow (for a given *-testsuite), probably ends up something like:
generate build ticket (used to track all child module reports)
do pre-integration-test (start-server, blah)
for each module {
run tests
run plugins (like pmd, clover, etc)
collect reports
}
do post-integration-test
publish reports
Published reports for a given run will all be indexed together by the
build ticket, so it will be easy to see what results are for which
build. In this example "generate build ticket", "collect reports"
and "publish reports" would all be handled by the reporting plugin I
am talking about, the rest is the existing testsuite bits.
I think this will work out quite well... giving us flexibility w/o
making our exiting plugins more complicated or waiting for other
plugin developers to make modifications for us (which is not very
likely to happen anyways).
Anyways...
I think that the reporters we have now (in g-m-p) is fine for a while
though... but as soon as we need to start adding similar reporter-
like functionality to other plugins, then we should really consider
implementing a more generic reporting system that can function on
plugin outputs alone (or using a maven reporting api if/when it
becomes available).
* * *
Overall I think the testsuite work of late is positive movement for
our community to be able to start building a more comprehensive set
of integration and system tests... NOW.
I believe that we do need to start working on making some real
tests... so that we can validate that the framework we have now is
sufficient, and if not refactor it now before it sits for too long.
--jason
