Re: expectations 0.1.0
On Tuesday, 4 September 2018 at 22:08:48 UTC, Nick Sabalausky (Abscissa) wrote: I think you may be getting hung up on a certain particular detail of Vladimir's exact "draft" implementation of Success, whereas I'm focusing more on Success's more general point of "Once the object is no longer around, guarantee the error doesn't get implicitly squelched." You're right that, *in the draft implementation as-is*, it can be awkward for the caller to then pass the Success along to some other code (another function call, or something higher up the stack). *Although*, still not impossible. So #3 still isn't eliminated, it's simply made awkward... But reference counting would be enough to fix that. (Or a compiler-supported custom datatype that's automatically pass-by-moving, but that's of course not something D has). Ok, I think I understand what you're proposing now--basically, something comparable to Rust's `#[must_use]` attribute. Thanks for taking the time to explain. I agree that that would be a nice feature for `Expected` to have. The thing is, D already has a mechanism for signalling failures that can't be ignored: exceptions. So adding that functionality to `Expected`, while convenient, doesn't actually let you accomplish anything you couldn't already. Now, if it were easy to implement, then sure, no problem. But it's not. Reference counting in particular is so problematic that Walter and Andrei have proposed *multiple* new language features (copy constructors, __mutable) to make it work cleanly. As things currently stand, making `Expected` reference-counted would mean at the very least giving up compatibility with `const` and `immutable`, which makes `Expected` a worse fit for strongly-pure functions (currently its *best* use-case). It's a shame that D forces us to make this tradeoff, but given the options in front of me, I would rather have `Expected` shine in the area where it has a comparative advantage, even if that means making it less universally-applicable as an error-handling mechanism.
Re: expectations 0.1.0
On 09/04/2018 12:05 AM, Paul Backus wrote: On Monday, 3 September 2018 at 21:55:57 UTC, Nick Sabalausky (Abscissa) wrote: By contrast, a function that returns an `Expected!T` does *not* force its caller to acknowledge it. If an error occurs, and the caller never checks value or hasValue...nothing happens. That's called squelching an error, and its EXACTLY the same problem as using non-Expect return values to indicate errors. I'd regard that as very notable hole in any Expected design as it breaks one of the core points of using Expect vs returning plain error codes: The user can still accidentally (or deliberately) squelch an error. If you receive an `Expected!T`, you have the following choices available to you: 1. Handle the success case locally, and the failure case non-locally (i.e. use `value` directly). 2. Handle both the success case and the failure case locally (i.e. check `hasValue`). 3. Handle both the success case and the failure case non-locally (i.e., pass the `Expected!T` along untouched). The difference between `Expected`, on the one hand, and both `Success` and plain-old exceptions, on the other, is that `Expected` gives you choice #3, and the other two don't. I think you may be getting hung up on a certain particular detail of Vladimir's exact "draft" implementation of Success, whereas I'm focusing more on Success's more general point of "Once the object is no longer around, guarantee the error doesn't get implicitly squelched." You're right that, *in the draft implementation as-is*, it can be awkward for the caller to then pass the Success along to some other code (another function call, or something higher up the stack). *Although*, still not impossible. So #3 still isn't eliminated, it's simply made awkward... But reference counting would be enough to fix that. (Or a compiler-supported custom datatype that's automatically pass-by-moving, but that's of course not something D has). And you haven't actually directly addressed the issue I've raised about failing to guarantee errors aren't implicitly squelched. Why is choice #3 important? Because it doesn't branch. Both success and failure follow the same code path. That makes functions that use `Expected` much easier to compose than ones that throw exceptions. For example, if you throw an exception in the middle of a range pipeline, the entire thing comes crashing down--but an `Expected!T` will pass right through, and let you handle it when it comes out the other end. Right. And as described above, I'm advocating an approach that preserves that (even for void) while *also* improving Expect so it can not *merely* improve things "in most cases", but would actually *guarantee* errors are not implicitly squelched in ALL cases where Expect!whatever is used. I don't see the laziness here as being the core point. The laziness is the "how", not the raison d'etre. The laziness is simply a tool being used to achieve the real goals of: - Allowing the caller the decide between foo/tryFoo versions without the API duplication. - Decreasing exception-related overhead and increasing utility of nothrow. The laziness (on the part of the caller, i.e., the code that *receives* the `Expected!T`) is important because it's what makes choice #3 possible. It's an essential part of the design. Again, what I'm proposing still preserves that.
Re: expectations 0.1.0
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote: expectations is an error-handling library that lets you bundle exceptions together with return values. It is based on Rust's Result [1] and C++'s proposed std::expected. [2] If you're not familiar with those, Andrei's NDC Oslo talk, "Expect the Expected" [3], explains the advantages of this approach to error handling in considerable detail. expectations 0.2.0 is now available, with the following updates - `hasValue`, `value`, and `exception` now work for const and immutable `Expected` objects. - `Expected!void` has been removed. - `map` and `andThen` can now be partially applied to functions, "lifting" them into the Expected monad. - The documentation has been improved based on the feedback given in this thread.
Re: expectations 0.1.0
On Monday, 3 September 2018 at 13:00:05 UTC, aliak wrote: This would be great to have in D. Indeed, if it's really going into C++ D needs to think about how to handle that anyway if it wants to offer C++ ABI interfacing. Swift [0] has something similar, and personally after using it for a few years, I can say that I've seen next to no unhandled exception errors in iOS code at least. Thanks, didn't know that Swift is already using something like this.
Re: expectations 0.1.0
On Monday, 3 September 2018 at 21:55:57 UTC, Nick Sabalausky (Abscissa) wrote: By contrast, a function that returns an `Expected!T` does *not* force its caller to acknowledge it. If an error occurs, and the caller never checks value or hasValue...nothing happens. That's called squelching an error, and its EXACTLY the same problem as using non-Expect return values to indicate errors. I'd regard that as very notable hole in any Expected design as it breaks one of the core points of using Expect vs returning plain error codes: The user can still accidentally (or deliberately) squelch an error. If you receive an `Expected!T`, you have the following choices available to you: 1. Handle the success case locally, and the failure case non-locally (i.e. use `value` directly). 2. Handle both the success case and the failure case locally (i.e. check `hasValue`). 3. Handle both the success case and the failure case non-locally (i.e., pass the `Expected!T` along untouched). The difference between `Expected`, on the one hand, and both `Success` and plain-old exceptions, on the other, is that `Expected` gives you choice #3, and the other two don't. Why is choice #3 important? Because it doesn't branch. Both success and failure follow the same code path. That makes functions that use `Expected` much easier to compose than ones that throw exceptions. For example, if you throw an exception in the middle of a range pipeline, the entire thing comes crashing down--but an `Expected!T` will pass right through, and let you handle it when it comes out the other end. Now, you will probably object--and rightly so--that there is an implicit assumption being made here, which is that "handle the success case" is equivalent to "use the return value." Clearly, this equivalence does not always hold in the presence of side effects. That's why `Expected!void` is so problematic. Nevertheless, I think it holds in enough cases to make `Expected` useful in practice. In particular, it is guaranteed to hold for strongly-pure functions, and will also hold for functions whose side effects are visible only through the return value (e.g., `readln`). I don't see the laziness here as being the core point. The laziness is the "how", not the raison d'etre. The laziness is simply a tool being used to achieve the real goals of: - Allowing the caller the decide between foo/tryFoo versions without the API duplication. - Decreasing exception-related overhead and increasing utility of nothrow. The laziness (on the part of the caller, i.e., the code that *receives* the `Expected!T`) is important because it's what makes choice #3 possible. It's an essential part of the design.
Re: expectations 0.1.0
On 09/03/2018 02:49 AM, Paul Backus wrote:
On Monday, 3 September 2018 at 04:49:40 UTC, Nick Sabalausky (Abscissa)
wrote:
Note that the above has *nothing* to do with retrieving a value.
Retrieving a value is merely used by the implementation as a trigger
to lazily decide whether the caller wants `foo` or `tryFoo`. Going out
of scope without making the choice could also be considered another
trigger point. In fact, this "out-of-scope without being checked"
could even be used as an additional trigger for even the non-void
variety. After all: what if an error occurs, but the caller checks
*neither* value nor hasValue?
The thing is, triggering on explicit access lets you handle errors
lazily, whereas triggering at the end of the scope forces you to handle
them eagerly.
Vladimir's `Success` type is, essentially, a way for a function to send
something back up the stack that its caller is forced to acknowledge.
Yes, that's correct.
Throwing an exception is *also* a way for a function to send something
back up the stack that its caller is forced to acknowledge.
Yes, but it's heavier-weight AND prevents the callee from being nothrow.
but when it comes to overall control-flow
semantics, they are basically equivalent.
Control-flow semantics, sure, but as I pointed out in my previous
sentence, there's more relevant things involved here than just control
flow semantics.
By contrast, a function that returns an `Expected!T` does *not* force
its caller to acknowledge it. If an error occurs, and the caller never
checks value or hasValue...nothing happens.
That's called squelching an error, and its EXACTLY the same problem as
using non-Expect return values to indicate errors. I'd regard that as
very notable hole in any Expected design as it breaks one of the core
points of using Expect vs returning plain error codes: The user can
still accidentally (or deliberately) squelch an error.
To clarify: If the caller never checks value or hasValue, that does NOT
mean the caller has carefully and deliberately chosen to disregard the
error. It *could* mean that, but it could also mean they simply messed
up. Deliberately squeching an error should NEVER be implicit, it should
always require something like:
catch(...) { /+ Do nothing +/ }
or
if(!x.hasValue) { /+ Do nothing +/ }
That's what being lazy
means: if you never open the box, it doesn't matter whether the cat is
alive or dead.
I don't see the laziness here as being the core point. The laziness is
the "how", not the raison d'etre. The laziness is simply a tool being
used to achieve the real goals of:
- Allowing the caller the decide between foo/tryFoo versions without the
API duplication.
- Decreasing exception-related overhead and increasing utility of nothrow.
Having one specialization be lazy and one be eager
would be a nightmare for anyone trying to use the library.
Vladimir's Success vs Expect!T is NOT an example of "eager vs lazy". In
BOTH cases, the callee treats errors lazily. And in BOTH cases, the
caller (or whomever the caller passes it off to) is expected to, at some
point, make a deliberate, explicit choice between "handle or throw". And
as I said before, allowing the caller to accidentally (or implicitly)
squelch the error is a fundamental breakage in the whole point behind
Except.
Re: expectations 0.1.0
On Monday, 3 September 2018 at 06:00:06 UTC, Thomas Mader wrote: On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev wrote: There are generally two classic approaches to error handling: std::expected is not the only thing on this topic going on in C++. There is also the proposal from Herb Sutter [1]. It's not a library solution and changes even the ABI but it's an interesting approach. He also tries to get compatibility into C via an extension. (See 4.6.11 in [1]) [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf This would be great to have in D. Swift [0] has something similar, and personally after using it for a few years, I can say that I've seen next to no unhandled exception errors in iOS code at least. [0] https://www.mikeash.com/pyblog/friday-qa-2017-08-25-swift-error-handling-implementation.html
Re: expectations 0.1.0
On Monday, 3 September 2018 at 06:49:41 UTC, Paul Backus wrote: To me, the only acceptable choices are for `Expected!void` to have the same lazy semantics as `Expected!T`, or for `Expected!void` to be removed altogether. Having one specialization be lazy and one be eager would be a nightmare for anyone trying to use the library. For the reasons Vladimir brought up, I'm leaning toward removal--without something like Rust's `#[must_use]` attribute, it seems like `Expected!void` is likely to do more harm than good. I'm leaning on agreeing with removal of Expected!void as well When we get opPostMove then maybe Expected!void can throw on a move or a copy if the result was a failure. This would also not allow the error to be ignored as it'd throw. Or... can it throw in ~this() if it was not checked?
Re: expectations 0.1.0
On Monday, 3 September 2018 at 04:49:40 UTC, Nick Sabalausky (Abscissa) wrote: Note that the above has *nothing* to do with retrieving a value. Retrieving a value is merely used by the implementation as a trigger to lazily decide whether the caller wants `foo` or `tryFoo`. Going out of scope without making the choice could also be considered another trigger point. In fact, this "out-of-scope without being checked" could even be used as an additional trigger for even the non-void variety. After all: what if an error occurs, but the caller checks *neither* value nor hasValue? The thing is, triggering on explicit access lets you handle errors lazily, whereas triggering at the end of the scope forces you to handle them eagerly. Vladimir's `Success` type is, essentially, a way for a function to send something back up the stack that its caller is forced to acknowledge. Throwing an exception is *also* a way for a function to send something back up the stack that its caller is forced to acknowledge. The exact details are different, but when it comes to overall control-flow semantics, they are basically equivalent. By contrast, a function that returns an `Expected!T` does *not* force its caller to acknowledge it. If an error occurs, and the caller never checks value or hasValue...nothing happens. That's what being lazy means: if you never open the box, it doesn't matter whether the cat is alive or dead. The problem, when it comes to `Expected!void`, is that there's no good way to express what we *actually* care about--the function's side effect--as a value. If we could write `Expected!(IO!Unit)` like in Haskell, everything would be fine. To me, the only acceptable choices are for `Expected!void` to have the same lazy semantics as `Expected!T`, or for `Expected!void` to be removed altogether. Having one specialization be lazy and one be eager would be a nightmare for anyone trying to use the library. For the reasons Vladimir brought up, I'm leaning toward removal--without something like Rust's `#[must_use]` attribute, it seems like `Expected!void` is likely to do more harm than good.
Re: expectations 0.1.0
On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev wrote: There are generally two classic approaches to error handling: std::expected is not the only thing on this topic going on in C++. There is also the proposal from Herb Sutter [1]. It's not a library solution and changes even the ABI but it's an interesting approach. He also tries to get compatibility into C via an extension. (See 4.6.11 in [1]) [1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf
Re: expectations 0.1.0
On 09/02/2018 11:23 PM, Paul Backus wrote: This is a really clever technique. As you said, hard to say whether it's worth it compared to just throwing an exception normally, but still, really clever. IMO, it's worth it. First of all, it decreases the asymmetry between `Expected!void` and other `Expected!T`. But more than that, there's one of the core benefits of of expected: What's awesome about expected is that by providing only one function, the caller can decide whether they want a `foo()` that throws, or a `tryFoo()` that lets them manually handle the case where it doesn't work (and is potentially nothrow). Note that the above has *nothing* to do with retrieving a value. Retrieving a value is merely used by the implementation as a trigger to lazily decide whether the caller wants `foo` or `tryFoo`. Going out of scope without making the choice could also be considered another trigger point. In fact, this "out-of-scope without being checked" could even be used as an additional trigger for even the non-void variety. After all: what if an error occurs, but the caller checks *neither* value nor hasValue? There's only one possible downside I see: What if the caller *intentionally* wants to ignore the error condition? Yes, that's generally bad practice, and signifies maybe it shouldn't be an exception in the first place. But consider Scriptlike: It has functions like `tryMkdir` and `tryRmdir` with the deliberate purpose letting people say "Unlike Phobos's mkdir/rmdir, I don't care whether the directory already exists or not, just MAKE SURE it exists (or doesn't) and don't bother me with the details!" I suppose for cases like those, it's perfectly worth leaving it up to expectation's user to design, create and document a "Don't worry about the failure" variant, should they so choose. Probably safer that way, anyway.
Re: expectations 0.1.0
On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev
wrote:
Please correct me if I'm wrong, but from looking at the code,
given e.g.:
Expected!void copyFile(string from, string to);
nothing prevents me from writing:
void main() { copyFile("nonexistent", "target"); }
The success value is silently discarded, so we end up with a
"ON ERROR RESUME NEXT" situation again, like badly written C
code.
This is definitely a big weakness of `Expected!void`, and one I
hadn't considered when writing the code. With a normal
`Expected!T`, the fact that you care about the return value is
what forces you to check for the error, but that doesn't apply
when the return value is `void`.
I'm not sure at this point if it's better to leave
`Expected!void` in for the sake of completeness, or remove it so
that nobody's tempted to shoot themself in the foot. Definitely
something to think about.
One way we could improve on this in theory is to let functions
return a successfulness value, which is converted into a thrown
exception IFF the function failed AND the caller didn't check
if an error occurred.
Draft implementation:
struct Success(E : Exception)
{
private E _exception;
private bool checked = false;
@property E exception() { checked = true; return _exception; }
@property ok() { return exception is null; }
@disable this(this);
~this() { if (_exception && !checked) throw _exception; }
}
This is a really clever technique. As you said, hard to say
whether it's worth it compared to just throwing an exception
normally, but still, really clever.
Re: expectations 0.1.0
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:
expectations is an error-handling library that lets you bundle
exceptions together with return values. It is based on Rust's
Result [1] and C++'s proposed std::expected. [2] If
you're not familiar with those, Andrei's NDC Oslo talk, "Expect
the Expected" [3], explains the advantages of this approach to
error handling in considerable detail.
Sorry, I didn't watch the talk, but this sounds like something
that's been on my mind for a while.
There are generally two classic approaches to error handling:
- Error codes. Plus: no overhead. Minus: you need to remember to
check them. Some languages force you to check them, but it
results in very noisy code in some cases (e.g.
https://stackoverflow.com/a/3539342/21501).
- Exceptions. Plus: simple to use. Minus: unnecessary (and
sometimes considerable) overhead when failure is not exceptional.
Now, Rust's Result works because it forces you to check the error
code, and provides some syntax sugar to pass the result up the
stack or abort if an error occurred. D, however, has nothing to
force checking the return value of a function (except for pure
functions, which is inapplicable for things like I/O).
Please correct me if I'm wrong, but from looking at the code,
given e.g.:
Expected!void copyFile(string from, string to);
nothing prevents me from writing:
void main() { copyFile("nonexistent", "target"); }
The success value is silently discarded, so we end up with a "ON
ERROR RESUME NEXT" situation again, like badly written C code.
One way we could improve on this in theory is to let functions
return a successfulness value, which is converted into a thrown
exception IFF the function failed AND the caller didn't check if
an error occurred.
Draft implementation:
struct Success(E : Exception)
{
private E _exception;
private bool checked = false;
@property E exception() { checked = true; return _exception; }
@property ok() { return exception is null; }
@disable this(this);
~this() { if (_exception && !checked) throw _exception; }
}
Success!E failure(E)(E e) { return Success!E(e); }
Success!Exception copyFile(string from, string to)
{
// dummy
if (from == "nonexistent")
return failure(new Exception("EEXIST"));
else
return typeof(return)();
}
void main()
{
import std.exception;
copyFile("existent", "target");
assert(!copyFile("nonexistent", "target").ok);
assertThrown!Exception({ copyFile("nonexistent", "target"); }());
}
This combines some of the advantages of the two approaches above.
In the above draft I used a real exception object for the
payload, constructing which still has a significant overhead (at
least over an error code), though we do get rid of a try/catch if
an error is not an exceptional situation. The advantage of using
a real exception object is that its stack trace is generated when
the exception is instantiated, and not when it's thrown, which
means that the error location inside the copyFile implementation
is recorded; but the same general idea would work with a
numerical error code payload too.
Any thoughts?
Re: expectations 0.1.0
On Sunday, 2 September 2018 at 23:38:41 UTC, Per Nordlöw wrote: On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote: expectations is an error-handling library that lets you bundle exceptions together with return values. It is based on Rust's Result [1] and C++'s proposed std::expected. [2] If you're not familiar with those, Andrei's NDC Oslo talk, "Expect the Expected" [3], explains the advantages of this approach to error handling in considerable detail. Incidentally, I've already proposed `Expected` into Phobos std.experimental.typecons here https://github.com/dlang/phobos/pull/6686 Is it ok if I try to merge your effort into this pull request? I don't think expectations has reached a high enough level of quality yet to be ready for inclusion in Phobos. However, if you'd like to submit it for comments as a WIP, or use it as a reference for your own implementation, that's completely fine, and I'd be happy to help you.
Re: expectations 0.1.0
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote: expectations is an error-handling library that lets you bundle exceptions together with return values. It is based on Rust's Result [1] and C++'s proposed std::expected. [2] If you're not familiar with those, Andrei's NDC Oslo talk, "Expect the Expected" [3], explains the advantages of this approach to error handling in considerable detail. Incidentally, I've already proposed `Expected` into Phobos std.experimental.typecons here https://github.com/dlang/phobos/pull/6686 Is it ok if I try to merge your effort into this pull request?
expectations 0.1.0
expectations is an error-handling library that lets you bundle exceptions together with return values. It is based on Rust's Result [1] and C++'s proposed std::expected. [2] If you're not familiar with those, Andrei's NDC Oslo talk, "Expect the Expected" [3], explains the advantages of this approach to error handling in considerable detail. Features: - `Expected` values can be treated as either return codes or exceptions. - Functions that return `Expected` values can be composed easily using a monadic interface (`andThen`). - `Expected!void` is valid and (hopefully) works the way you'd expect. - Everything, except for `opEquals` (which depends on `Object.opEquals`), works in @safe code. This is very much a work in progress; all comments and feedback are welcome. Documentation: https://pbackus.github.io/expectations/expectations.html DUB: https://code.dlang.org/packages/expectations Code: https://github.com/pbackus/expectations [1] https://doc.rust-lang.org/std/result/enum.Result.html [2] https://wg21.link/p0323r7 [3] https://www.youtube.com/watch?v=nVzgkepAg5Y
