dfabulich wrote:
> 
> As I said, I don't think you're using the right tool for the job here. 
> 
Hm, I am not really in the mood to start off a discussion with my project
manager ;-)


dfabulich wrote:
> 
> You can get the exact behavior you want from JUnit.
> 
And I could get it from TestNG if I passed each class in as an individual
suite, couldn't I?


dfabulich wrote:
> 
> Again, if you want class-level logging, I encourage you to take up the 
> matter with the TestNG team.
> 
Might be a good idea. As some kind of explanation why I started this
discussion on the Surefire corner at all: Surefire 2.3.1 did what I wanted
so the first impression after updating to Surefire 2.4 was that it got
messed up.


dfabulich wrote:
> 
> Doesn't this option smell bad to you?
> 

It's a rather technical/sophisticated option, I agree about that. But aren't
"useSystemClassLoader", "childDelegation" and "forkMode" likewise? Whoever
develops tests needs to have proper knowledge about their requirements on
the execution runtime and hence, needs to configure the test runner
appropriately. The same would apply for this option, default it to "all",
let the experts change it to meet their special needs.


dfabulich wrote:
> 
> Instead of making this option, wouldn't it be nice if it just logged 
> correctly?  If TestNG notified us when to log, we could log anything we 
> wanted.
> 
I fear having just another lister in the TestNG framework is not enough. The
problem I see here is the following: If Sufefire passes all test classes
(say A and B) as a single suite to TestNG, what would TestNG prevent from
running them like this:
  A.testOne
  B.testOne
  A.testTwo
  B.testTwo
  A.testThree
  B.testThree
Our test methods have no dependencies that would require a deterministic
order. Even if TestNG exposed events for starting/stopping a test class, I
still would not get a guarantee that all methods from a single class are run
without inter-leaving with methods from other classes to give me that
class-level testing feeling. I hardly can see an argument how one could
request a change on this behavior. A suite is the outer-most group of tests
and as long as I do not specify dependencies, TestNG is free in ordering
them.


dfabulich wrote:
> 
> Surefire is meant to be a wrapper around the underlying test framework; 
> 
No problems with that. But it's Surefire and not the test framework that
aggregates my test classes into a single suite. I am basically asking for an
option to control the way how Surefire assembles the suites *before* it
passes control to the test framework.


Benjamin Bentmann
-- 
View this message in context: 
http://www.nabble.com/Test-Suites%2C-Ant%2C-Surefire%2C-and-JunitReport-tp15076378p15138325.html
Sent from the Surefire - Developer mailing list archive at Nabble.com.

Reply via email to