As I said in another thread, we just ended 3. We named the profile integration-tests but to have the same name as in the core we'll rename it run-its
It seems that there is 2 point of view about ITs activation : -1) ITs are part of the build and always launched -2) ITs are launched on demand (because they are consumming a lot of time and ressources) What do you prefer ? For me it's (2). Another question : What is an integration test ? For me unit tests don't handle resources like files. In unit tests we are using mocks to be able to use only in-memory tests. Everything else which is reading poms, settings or other resources are integration tests (whatever the framework used : maven-embedder, maven-verifier, ...). The problem actually is that our APIs aren't often developped to have unit tests. For example, we often have in parameters InputStream or something like than instead of a Project object which can be mocked. We should have to review them to be able to have many more UTs. Like you can see in [1] we don't have an important coverage of UTs. ITs are needed for example in plugins when we wrap another tool, but otherwise, and particularly in the core we should have a good unit tests suite. [1] http://nemo.sonar.codehaus.org/project/dashboard/19084?sid=669108 Arnaud On Wed, Aug 6, 2008 at 5:24 PM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > Hi, > > I think we've gotten to the point where we need to decide how we are going > to test plugins. We need to pick one of the frameworks, settle on a pattern, > and use that in the plugins otherwise there will be no sane way to validate > a set of plugins works against a given version of Maven. What I'm thinking > about here concretely is testing all the plugins that we have here against > Maven 2.1 to know that we have not screwed something up so terribly that > things like the deploy plugin doesn't work, or whatever. > > I think how this starts is that we: > > 1) Pick one of the tools > 2) Create a touchstone project that can be expanded where necessary for any > given plugin so that we have a baseline project against which to test > 3) Pick a standard profile name for invoking this test > > This way we create a standard hook point for a larger harness to get hold > off. We can check out sources and create an aggregator POM with the given > profile activated to test a set of plugins. I don't know yet what the best > way would be to share a touchstone project (and that is not to say we won't > need different projects but we have to start with a baseline), but once we > start this we can also start plugging in other things like integration > testing that includes things like coverage or whatever else. > > I think the key in moving forward is getting 1-3 sorted out so we're not > using 5 frameworks and testing plugins with N different patterns where it's > impossible to hook into for larger scale testing. I think this is the only > way forward to validate that a set of plugins work against a given version > of Maven which is vital information to know before releasing 2.1. > > For integration testing I have found the SHITTY plugin (we would simply > have to change that name, sorry Jason Dillon) to be the most useful and > feature rich. Should be relatively simple to create a test project, and a > profile name (run-its like the core ITs). Then we figure out how to share > and version the test project to create a stable baseline. I chatted about > this briefly in IRC with Benjamin and wanted to get the information out. I > think it's vital to get this rolling if we want to roll out a 2.1-alpha-1 > with some degree of confidence we have toasted a bunch of plugins due to > incompatibilities in the core. > > Thanks, > > Jason > > ---------------------------------------------------------- > Jason van Zyl > Founder, Apache Maven > jason at sonatype dot com > ---------------------------------------------------------- > > What matters is not ideas, but the people who have them. Good people can > fix bad ideas, but good ideas can't save bad people. > > -- Paul Graham > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- .......................................................... Arnaud HERITIER .......................................................... OCTO Technology - aheritier AT octo DOT com www.octo.com | blog.octo.com .......................................................... ASF - aheritier AT apache DOT org www.apache.org | maven.apache.org ...........................................................
