It's probably going to be you doing this work, so ultimately, you are
the one who decides what you want to write. I will just say that you
and I share the same goals, but your proposal goes against the grain
of maven which fundamentally combines build/test. Anytime you do
that, it's painful.
So let me just stop proposing and do my best to provide the
information you'd need to get something done.
On Jan 30, 2006, at 11:09 AM, Prasad Kashyap wrote:
1. Build and Test will be kept separate functionally.
2. The itests 'subproject' can scale better. The testbucket could
become potentially large as more modules start adding their own itests
or increasing the number of test scenarios.
A point of clarification. There is no technical limit to
modularizing and building a set of tests in one area and reusing the
artifacts produced by them elsewhere. In other words, where you keep
the itests does not dictate where you run them. It is possible to
build tests in maven 2 and run them in maven 1. The reverse could be
true if a maven 2 version of the deployment plugin was created.
3. The throughput of the itests will not determine the throughput of
the G build. Even if we do fix the openejb itests in G now to make it
very fast, when we have a potentially large number of itests
contributed by different folks for different modules, they tend to add
up.
There are logically two different "build times" to consider. I would
consider Developer build time to be more precious, and Automated
built time to be less precious. Making the running of itests
optional to developers is mandatory. Making the itests off by
default may be a good idea.
Either way, there is no technical limitation to achieving the above
goals and running itest "on" or "during" the assemblies.
4. It will be difficult to selectively include/exclude a few tests.
This is where you pick your poison so to speak. Not all itest will
apply to each assembly and vice versa. Meaning, there are only two
ways to skin this cat.
1. You make each assembly say which tests apply to it
2. You make each set of tests say which assemblies apply to it
With #1 you will have challenges specifying the tests to run. You
will have to deploy several apps and run distinct sets of tests.
With three assemblies and four distinct sets of tests (for example),
you will unpack and start each server once and deploy into it four
times (essentially). A total time of 3 unpacks and startups and 12
deploys.
With #2 you will have challenges specifying on which assemblies the
distinct set of tests should run against. The distinct set of tests
will have to unpack and start each assembly to run its set of tests.
With four distinct sets of tests and three assemblies (same example),
you will unpack, start and deploy on each assembly once for that set
of tests. A total of 12 unpacks, starts and deploys will be required.
This is how I'd imagined it, at a very high level.
- The itests subproject will have it's own set of modules. These are
the modules in G that have dropped their itests into this subproject.
Totally possible.
- The tests will be grouped logically into suites or categories. Each
individual test will fall in one of these suites.
Not sure of your usage of the word "test" in this sentence. In junit
terms is that TestSuite, TestCase, or individual test method. Can
you clarify?
- The tests can all be run or only a subset of suites can be run
quite easily.
When you say "easily" do you mean by virtue of them being separate or
some other idea?
- Continuum will continue to build G first and then run itests on the
assemblies.
- If the assembly passes a certain set of tests, then it can be
certified good and then be uploaded to the repository.
The maven 2 lifecycle supports this flow on a per module basis
(truncated, but still in order):
compile
test-compile
test
package
before-integration-test
integration-test
after-integration-test
verify
install
deploy
If there is a failure the next phase of the lifecycle is not called.
So if "test" fails, "package" is not called. If "integration-test"
fails then "install" or "deploy" are not called.
Note, there are no "integration-test-complile" or "integration-
test-package"
phases as it is expected these exist in other modules.
The life cycle of assembly SuperServer could work in maven 2 as follows:
Geronimo :: Assembly :: Super Server
compile (no op)
test-compile (no op)
test (no op)
package build the assembly
before-integration-test unpack and start the server, deploy
required itest components
integration-test run all itests applicable for this
assembly
after-integration-test stop server
verify run any additional checks to verify the package is
valid and meets quality criteria.
install put in local maven repo
deploy upload to public repo
If you wish to split out integration-test from the maven lifecycle
and delay publishing the SuperServer binary, you would need at least
three modules to achieve the same.
Geronimo :: Assembly :: Super Server
compile (no op)
test-compile (no op)
test (no op)
package build the assembly
before-integration-test (no op)
integration-test (no op)
after-integration-test (no op)
verify (no op)
install put in local maven repo
deploy WARNING: don't call this our it will be upload to
public repo without testing
Geronimo :: iTest :: Super Server
compile (no op)
test-compile (no op)
test (no op)
package (no op)
before-integration-test unpack and start the SuperServer,
deploy required itest components
integration-test run all itests applicable for this
assembly
after-integration-test stop the SuperServer
verify run any additional checks to verify the package is
valid and meets quality criteria.
install (no op)
deploy (no op)
Geronimo :: Deploy :: Super Server
compile (no op)
test-compile (no op)
test (no op)
package (no op)
before-integration-test (no op)
integration-test (no op)
after-integration-test (no op)
verify devise some way to verify the itests have run
successfully on this exact SuperServer binary
install (no op)
deploy upload to public repo
You will have to do a lot of tricking on maven to do things like
this, but we've proven that with enough hacking you can push maven
against it's grain.
- a download page will be available to download assemblies and
binaries from every single build. The builds that passed itests will
be clearly marked so.
Great.
-David
