On 5/1/14, 2:28 PM, Jason Spencer wrote:
On Thursday, 1 May 2014 at 17:57:05 UTC, Andrei Alexandrescu wrote:
Well how complicated can we make it all? -- Andrei
As simple as possible, but no simpler :)
I've seen you favor this or that feature because it would make unit
testing easier and more accessible, and eschew features that would cause
folks to not bother. In truth, we could leave it how it is. But I
surmise you started this thread to improve the feature and encourage
more use of unit test. So we're looking for the sweet spot.
I don't think it's important to support the sharing of state between
unit tests. But I do see value in being able to influence the order of
test execution, largely for debugging reasons. It's important for a
module's tests to be able to depend on other modules--otherwise,
unittest is not very enticing. If it does and there's a failure, it's
hugely helpful to know the failure is caused by the unit-under-test, and
not the dependency(s). The common way to do that is to run the tests in
reverse order of dependency--i.e. levelize the design and test from the
bottom up. See "Large Scale C++ SW Design", Lakos, Chp. 3-4.
I imagine there are other niche reasons for order, but for me, this is
the driving reason. So it seems a nice middle ground.
If order is important, it might be a workable approach to run unittest
in the reverse module dependency order by default. A careful programmer
could arrange those classes/functions in modules to take advantage of
that order if it were important. Seems like we'd have the dependency
information--building and traversing a tree shouldn't be that tough....
To preserve it, you'd only be able to parallelize the UTs within a
module at a time (unless there's a different flag or something.)
But it seems the key question is whether order can EVER be important for
any reason. I for one would be willing to give up parallelization to
get levelized tests. What are you seeing on your project? How do you
allow tests to have dependencies and avoid order issues? Why is
parallelization more important than that?
I'll be blunt. What you say is technically sound (which is probably why
you believe it is notable) but seems to me an unnecessarily complex
engineering contraption that in all likelihood has more misuses than
good uses. I fully understand you may think I'm a complete chowderhead
for saying this; in the past I've been in your place and others have
been in mine, and it took me years to appreciate both positions. -- Andrei
