I don't have much comment on the functionality you want, seems reasonable enough...
I do have implementation commentary however: * You should not be obtaining a context inside your subtest (specifically line 18 https://github.com/rjbs/Test-Abortable/blob/master/lib/Test/Abortable.pm#L18). Obtaining that context means that tests run inside your eval are likely to report errors to the wrong files+lines. If you need a context that is within the subtest you should obtain it after your eval. * https://github.com/rjbs/Test-Abortable/blob/master/lib/Test/Abortable.pm#L30 this should probably be $ctx->throw() which is essentially a die, but it "aborts" the context so that it is released properly (not the die you use prevents context->release from being called). Though on second thought, throw() will append a file+line number to your exception, so you should probably just add a $ctx->release right before that die. (and on further reading your second sub asks if release should be done, the answer is yes. -Chad On Tue, Nov 29, 2016 at 5:52 PM, Ricardo Signes <perl...@rjbs.manxome.org> wrote: > Often, I have a test like this: > > subtest "do things with an api" => sub { > my $result = $api_client->do_first_thing; > > is( > $result->documents->first->title, > "The Best Thing", > ); > > ... > }; > > Sometimes, the result comes back with zero documents. ->first throws an > exception and then my whole test program comes crashing down and it's > miserable. For a while, I've been meaning to make it possible for some > exceptions to be recognized by my test programs as instructions to emit a > failure, stop this subtest, and move on. > > It's important to note that I'm talking, in the code above, about an > exception > that would be thrown by ($result->documents) when ->first is called on it. > This kind of flow control eliminates needing to write: > > my $result = $api_client->do_first_thing; > my $docs = $result->documents; > fail("no docs"), return unless $docs->has_entries; > my $first = $docs->first; > fail("no title"), return unless $first->has_title; > > is(...); > > In my case, I'm working with an API client that's specifically designed to > be > used for testing, so this kind of loose coupling between thrown exceptions > and > the test code is a good fit. Not every exception should be caught this > way. > Truly unexpected ones should still die. > > So, I've written something to do this, and I'm sharing it here before going > further with it. In my code, if an exception is meant to be caught and > used as > a local abort instruction, it has a method called as_test_abort_events. > This > method returns a reference to an array of Test2 event descriptions. For > example, maybe: > > sub as_test_abort_events { > return [ > [ Ok => (pass => 0, name => "no documents, but ->first called") ], > [ Diag => (message => "collection state: ....") ], > ]; > } > > This method is easy to add to any exception you want, and you don't need to > change your exception class hierarchy in any way. > > Next, you need something to run the tests and look for these exceptions > being > thrown. I have written a library for this, called Test::Abortable. > > https://github.com/rjbs/Test-Abortable > > Test::Abortable provides two subroutines: subtest and testeval. subtest > acts > just like Test::More's subtest, but catches abort exceptions, emits their > events, and returns normally. (I've also updated my library > Test::Routine, in > a branch, to behave this way, as I use it in place of subtest for many > things.) > > testeval acts like eval, but only catches abort exceptions. > > ok(1); > testeval { > ok(2); > this_throws_an_abort; > ok(3); > }; > ok(4); > > This ends up emitting ok 1, ok 2, whatever the abort wants, and ok 4. > testeval > returns the return value of the code block if it succeeds. If it fails > due to > abort, it returns false, emits the abort events, and puts the abort in > $@. If > it fails because of any other exception, the exception is re-thrown. > > Let me know if you have any thoughts before I begin using this in anger. > :-) > > -- > rjbs >
