On Mon, Jul 13, 2009 at 4:56 PM, Ovid<publiustemp-perl...@yahoo.com> wrote: > > We currently have over 30,000 tests in our system. It's getting harder to > manage them. In particular, it's getting harder to find out which TODO tests > are unexpectedly passing. It would be handy have to some option to force > TODO tests to die or bailout if they pass (note that this behavior MUST be > optional). > > Now one might think that it would be easy to track down missing TODOs, but > with 15,000 tests aggregated via Test::Aggregate, I find the following > unhelpful: > > > TODO passed: 2390, 2413 > > If those were in individual tests, it would be a piece of cake to track them > down, aggregated tests get lumped together. Lacking proper subtest support > (which might not mitigate the problem) or structured diagnostics (which could > allow me to attach a lot more information to TODO tests) at the end of the > day, I need an easier way of tracking this.
I think it would be better to have a tool (Smolder) be able to display various drill-downs from the aggregated test report. e.g. list of all the TODOs list of all the TODOs that pass etc... That way you don't need to run your test suit again with an option but can get this information from the report of the regular run. Gabor
