Re: Making TODO Tests Fail

2009-07-15 Thread Steffen Schwigon
Ovid writes: > it's getting harder to find out which TODO tests are unexpectedly > passing. > [...] > Suggestions? I am spoiling my own YAPC::EU 2009 talk here but try this toolchain: # the tool chain $ cpan App::DPath # prepare example project which contains passing TODOs $ cd /tmp/

Re: Making TODO Tests Fail

2009-07-14 Thread Michael G Schwern
Ovid wrote: > - Original Message >> From: chromatic > >> Add diagnostics to TODO tests and let your test harness do what it's >> supposed >> to do. Shoving yet more optional behavior in the test process continues to >> violate the reasons for having separate test processes and TAP an

Re: Making TODO Tests Fail

2009-07-14 Thread Eric Wilhelm
# from Ovid # on Tuesday 14 July 2009 03:33: >> Fork/branch Test::Builder and make it work yourself. When it's ready >> and usable, ask Schwern to evaluate, improve and merge. >> >> Code = Conversation. :) > >I know.  I've thought about that, but truth be told, I'm really > getting burnt out with

Re: Making TODO Tests Fail

2009-07-14 Thread David Golden
On Tue, Jul 14, 2009 at 5:43 AM, Ovid wrote: > We have no diagnostics. We've never had diagnostics (the ad-hoc things > going to STDERR don't count because they can't be synched or reliably > parsed). Thus, I can't add diagnostics to the TODO tests until Schwern puts > diagnostics in Test::Builde

Re: Making TODO Tests Fail

2009-07-14 Thread chromatic
On Tuesday 14 July 2009 02:43:44 Ovid wrote: > Thus, I'm trying to think of a way of solving my problem now, not at some > hypothetical date in the future. Next option: write your own test harness which dies when it encounters a bonus test. This should take you less than an afternoon. If that

Re: Making TODO Tests Fail

2009-07-14 Thread Nicholas Clark
On Tue, Jul 14, 2009 at 03:33:29AM -0700, Ovid wrote: > I know. I've thought about that, but truth be told, I'm really getting burnt > out with the Perl community right now. Lots of people are being rude, > thinking that being "right" is all they need to justify being arrogant and > it's sapp

Re: Making TODO Tests Fail

2009-07-14 Thread Ovid
- Original Message > From: Salve J Nilsen > > Fork/branch Test::Builder and make it work yourself. When it's ready and > usable, > ask Schwern to evaluate, improve and merge. > > Code = Conversation. :) I know. I've thought about that, but truth be told, I'm really getting burnt o

Re: Making TODO Tests Fail

2009-07-14 Thread Salve J Nilsen
Ovid said: - Original Message From: chromatic Add diagnostics to TODO tests and let your test harness do what it's supposed to do. Shoving yet more optional behavior in the test process continues to violate the reasons for having separate test processes and TAP analyzers. We ha

Re: Making TODO Tests Fail

2009-07-14 Thread Ovid
- Original Message > From: chromatic > Add diagnostics to TODO tests and let your test harness do what it's supposed > to do. Shoving yet more optional behavior in the test process continues to > violate the reasons for having separate test processes and TAP analyzers. We have no di

Re: Making TODO Tests Fail

2009-07-13 Thread chromatic
On Monday 13 July 2009 06:56:15 Ovid wrote: > We currently have over 30,000 tests in our system. It's getting harder to > manage them. In particular, it's getting harder to find out which TODO > tests are unexpectedly passing. It would be handy have to some option to > force TODO tests to die o

Re: Making TODO Tests Fail

2009-07-13 Thread Jonathan Rockway
* On Mon, Jul 13 2009, Ovid wrote: > How would Smolder (which we're not using since we use Hudson) help > with this? With over 15,000 tests being reported for t/aggregate.t, I > think a drill-down would be problematic here. Plus, tying the TODO to > the appropriate test file being aggregated is ne

Re: Making TODO Tests Fail

2009-07-13 Thread Michael Peters
Gabor Szabo wrote: AFAIK due to the number of tests it won't work well in Smolder - but I have not tried it. I was referencing to a future version of it ;-) It's worth a try. Our main test suite at $work has 23,000+ tests and Smolder handles it just fine. -- Michael Peters Plus Three, LP

Re: Making TODO Tests Fail

2009-07-13 Thread Gabor Szabo
On Mon, Jul 13, 2009 at 5:10 PM, Michael Peters wrote: > Gabor Szabo wrote: > >> I think it would be better to have a tool (Smolder) be able to display >> various drill-downs from the aggregated test report. > > If you want to see what Smolder would do to your tests, create a TAP archive > and then

Re: Making TODO Tests Fail

2009-07-13 Thread Michael Peters
Gabor Szabo wrote: I think it would be better to have a tool (Smolder) be able to display various drill-downs from the aggregated test report. If you want to see what Smolder would do to your tests, create a TAP archive and then you can upload it to the "Junk" project at http://smolder.plusth

Re: Making TODO Tests Fail

2009-07-13 Thread Ovid
- Original Message > From: Gabor Szabo > > I think it would be better to have a tool (Smolder) be able to display > various drill-downs from the aggregated test report. > e.g. list of all the TODOs > list of all the TODOs that pass > etc... How would Smolder (which we're not using since

Re: Making TODO Tests Fail

2009-07-13 Thread Gabor Szabo
On Mon, Jul 13, 2009 at 4:56 PM, Ovid wrote: > > We currently have over 30,000 tests in our system. It's getting harder to > manage them. In particular, it's getting harder to find out which TODO tests > are unexpectedly passing. It would be handy have to some option to force > TODO tests to