[Top-posting]
Chad, I think I understand what you mean now. You were referring to
whether the underlying pinnings should take care of it (Test2) or
whether the chrome (testing functions) around that should do so. Yes?
If so, I think you should probably clarify what Test2 *does* do. It
doesn't provide the functions - alright. What *does* it provide then?
Also, since the discussion was "Should I - the testing functions' user
- write code around my testing functions to accommodate for the
testing framework not preserving `$!` and `$@`?" instead of "Should
the testing functions take care of it rather than the gory details
underneath them?", it might be useful (other than explaining what the
difference between the testing functions and the gory details,
recommended in previous paragraph) explaining how such an
implementation would look like, or how it would be different, in the
testing functions vs. in the gory details.
I think then it would be simple to understand your intent and to help
you with some useful commentary.
If I didn't understand what you meant, then... this might needs
additional clarification.
S. :)
On Tue, Jan 12, 2016 at 8:56 PM, Chad Granum <exodi...@gmail.com> wrote:
> Thanks for the input.
>
> My question was not very well formed.
>
> What I should have asked is this:
>
> Preserving $! and $@ is a valuable behavior, and one I won't get rid of.
> However, I am questioning the decision to make the test library jump through
> hoops to preserve them, as opposed to having the test functions be
> responsible for doing it.
>
> Now then, having the test library do it means the tools usually do not have
> to worry about it (though it would be easy for them to still damage $! or
> $@). On the other hand, the protection in-library gets very complicated, and
> hard to maintain, and will not solve all cases.
>
> It was not a question of if we should do it, but how we should do it. Test2
> is just the back-end, but asking if Test2 should do it sounded like I was
> asking if we should do it at all, which was not the intent, the front end
> was always gonna do it.
>
> That said, the discussion served as rubber ducking and I was able to find a
> solution that lets the library do the heavy lifting, but in a single spot
> that avoids the complexity and un-maintainability of the way it has done it
> so far. So the discussion is no longer necessary, the library will continue
> to do the protection, and it will do it in a better way. New tools will not
> need to start concerning themselves with this. Old tools never had to worry
> as I was not going to break backcompat.
>
> -Chad
>
> On Tue, Jan 12, 2016 at 11:41 AM, Michael G Schwern <schw...@pobox.com>
> wrote:
>>
>> On 1/11/16 4:53 PM, Chad Granum wrote:
>> > Test::More/Test::Builder work VERY hard to ensure nothing inside them
>> > alters $! or $@. This is for
>> > thing like this:
>> >
>> > ok(do_something_scary());
>> > is($!, 0, "expected $! val");
>> > is($@, undef, '$@ not changed');
>> >
>> > Without Test::More/Builder being careful to support this, the second 2
>> > assertions could fail because
>> > something inside ok() modifies $! or $@.
>>
>> If your test functions modify the really common parts of your global
>> environment then it becomes
>> very difficult to test them... and testing error reporting is a very
>> common thing you'd want to do!
>>
>> Kent already pointed out why changing this makes testing $! and $@ very,
>> very awkward. Let's see
>> that again.
>>
>> my ( $error, $exception );
>> ok(do {
>> local $@;
>> local $!;
>> my $ret = do_something_scary());
>> ( $error, $exception ) = ($!, $@);
>> $ret
>> });
>> is($error, 0, "expected $! val");
>> is($exception, undef, '$@ not changed);
>>
>> Gross. I don't know how many times I've written code like this. I hate
>> it. I always encapsulate
>> it somehow. And when a library doesn't have good global discipline it
>> makes it even harder for me
>> to have good global discipline.
>>
>> We tell the users that $! and $@ are only safe per function call. Then we
>> encourage them all over
>> the docs and interface to pass function return values directly into test
>> functions. Then we tell
>> them they should be testing their error cases... but to do safely requires
>> gross scaffolding.
>> That's not fair to the user. The result will be that people won't test
>> their error conditions,
>> library quality will drop, and you'll waste a lot of time on a bug that
>> should have been tested.
>>
>> The argument that $! is only reliable per function call, that's a lowest
>> common denominator
>> thinking. One of the fundamental design principles of Test::Builder was
>> that it had to be the
>> GREATEST common denominator!
>>
>> I don't write libraries to the lowest common denominator. I write
>> libraries that raise the bar and
>> encourage others to do so as well. Perl 5's error handling is bad, but
>> that doesn't mean my
>> library's error handling also has to be bad. As library authors we've
>> been spending decades working
>> around Perl 5's bad parts. We know how to do it better. This is
>> extraordinarily important for a
>> language's testing library: if you can't test it (or it's gross and
>> annoying) then it won't happen.
>> If you want to see better global discipline in Perl libraries, then the
>> testing library has to
>> start first, because everything will use it.
>>
>> Test libraries are first and foremost about the user. They encourage the
>> user to more and better
>> testing! Implementation effort is a much, much lesser concern. And all
>> the positives reasons below
>> not to do it are implementation reasons. There are no positives for the
>> Test2 user.
>>
>> > *Reasons for dropping this promise from Test2:*
>> >
>> > * Simplifies code
>> > * $! and $@ are altered by many many things, adding { local ... }
>> > around all of them is a pain
>> > * Sometimes internals you don't expect to set $! will
>> > * Perl itself documents that you cannot depend on $! and $@ being
>> > unchanged past the immediate
>> > line after you set it.
>> > * Test::Builder will continue to protect $! and $@, so nothing will
>> > break
>> > * Test2 is new, nothing depends on it preserving these
>>
>> The one below is the only reason that talks about making Test2 better for
>> the user.
>>
>> > *Reasons not to drop it:*
>> >
>> > * It is helpful to people who might not realize $! and $@ are often
>> > altered unexpectedly.
>> >
>> > I am asking for input in case I missed any reasons/arguments for either
>> > side. In either case
>> > backwards compatibility will be preserved.
>> >
>> > -Chad
>> >
>> >
>
>
