I've used (and worked on) all the frameworks and also think SHITTY is the closest. It needs a few improvements before it can be mainstream: 1. It freaks out 2.1 because it circumnavigates the packaging and installs the plugin that was packaged by the main lifecycle. The problem is the data inside has a version that doesn't match what is expected (MSHITTY-10) 2. It needs local repo isolation (MSHITTY-12) 3. It needs to copy the tests to target before running them, just to avoid leaving turds in the source tree (MSHITTY-14) 4. It needs a way to call back to java code in /test-classes/... I don't want to be required to write everything in groovy...sure groovy might be java but if I have some existing classes that do what I need, then I want to use them directly. (I think this is already fixed but haven't tried it)
-----Original Message----- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 06, 2008 12:15 PM To: Maven Developers List Subject: Re: Sane plugin testing My pick for the tool is STY. I think Brian has used it, and Jason Dillon definitely has his opinion. The unit testing is different and the plugin-testing-harness is for unit testing and I'm not concerned about that in this context. If you look at the way Jason Dillon tests his plugins I think it's the best example of how to do it. It's got some groovy bits but that's fine with me. If I was to pick something today to move forward with it would be STY and I would rename that now :-) On 6-Aug-08, at 8:40 AM, Brett Porter wrote: > +1 to all below. > > All the information I could find in January is here: > http://docs.codehaus.org/display/MAVENUSER/Review+of+Plugin+Testing+Stra tegies > > Please use that as a starting point. There has probably been stuff > added to STY since. It generally seemed the best, but I would like > to see it get some of the verifier functionality and the ability to > trigger via a junit test. > > Thanks! > > - Brett > > On 07/08/2008, at 1:24 AM, Jason van Zyl wrote: > >> Hi, >> >> I think we've gotten to the point where we need to decide how we >> are going to test plugins. We need to pick one of the frameworks, >> settle on a pattern, and use that in the plugins otherwise there >> will be no sane way to validate a set of plugins works against a >> given version of Maven. What I'm thinking about here concretely is >> testing all the plugins that we have here against Maven 2.1 to know >> that we have not screwed something up so terribly that things like >> the deploy plugin doesn't work, or whatever. >> >> I think how this starts is that we: >> >> 1) Pick one of the tools >> 2) Create a touchstone project that can be expanded where necessary >> for any given plugin so that we have a baseline project against >> which to test >> 3) Pick a standard profile name for invoking this test >> >> This way we create a standard hook point for a larger harness to >> get hold off. We can check out sources and create an aggregator POM >> with the given profile activated to test a set of plugins. I don't >> know yet what the best way would be to share a touchstone project >> (and that is not to say we won't need different projects but we >> have to start with a baseline), but once we start this we can also >> start plugging in other things like integration testing that >> includes things like coverage or whatever else. >> >> I think the key in moving forward is getting 1-3 sorted out so >> we're not using 5 frameworks and testing plugins with N different >> patterns where it's impossible to hook into for larger scale >> testing. I think this is the only way forward to validate that a >> set of plugins work against a given version of Maven which is vital >> information to know before releasing 2.1. >> >> For integration testing I have found the SHITTY plugin (we would >> simply have to change that name, sorry Jason Dillon) to be the most >> useful and feature rich. Should be relatively simple to create a >> test project, and a profile name (run-its like the core ITs). Then >> we figure out how to share and version the test project to create a >> stable baseline. I chatted about this briefly in IRC with Benjamin >> and wanted to get the information out. I think it's vital to get >> this rolling if we want to roll out a 2.1-alpha-1 with some degree >> of confidence we have toasted a bunch of plugins due to >> incompatibilities in the core. >> >> Thanks, >> >> Jason >> >> ---------------------------------------------------------- >> Jason van Zyl >> Founder, Apache Maven >> jason at sonatype dot com >> ---------------------------------------------------------- >> >> What matters is not ideas, but the people who have them. Good >> people can fix bad ideas, but good ideas can't save bad people. >> >> -- Paul Graham >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> > > -- > Brett Porter > [EMAIL PROTECTED] > http://blogs.exist.com/bporter/ > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > Thanks, Jason ---------------------------------------------------------- Jason van Zyl Founder, Apache Maven jason at sonatype dot com ---------------------------------------------------------- Selfish deeds are the shortest path to self destruction. -- The Seven Samuari, Akira Kirosawa --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
