Re: Comparing Exceptions and Errors
On Thursday, 16 June 2022 at 13:54:52 UTC, Steven Schveighoffer wrote: [scope (success) lowered to finally] [...] Furthermore I always thought of scope guards as a means for cleanup. Cleanup implies in my eyes removing things which have been used in a previous task. This intended use is documented here: https://tour.dlang.org/tour/en/gems/scope-guards Using scope guards makes code much cleaner and allows resource allocation and clean up code to be placed next to each other. These little helpers also improve safety because they make sure certain cleanup code is always called independent of which paths are actually taken at runtime. Performing a COMMIT is rather the opposite of cleanup: It makes (writes) changes to the database persistent. Semantically, you are just not undoing (rollback) the previous steps. Objection! Not doing a rollback is not the same as doing a COMMIT. Even not "semantically". Until the COMMIT has succeeded none of the changed data of the transactions is visible in other DMBS sessions.
Re: Comparing Exceptions and Errors
On 6/16/22 6:07 AM, kdevel wrote: On Wednesday, 15 June 2022 at 20:46:56 UTC, Steven Schveighoffer wrote: [...] It has not harmed my code though. I tried throwing inside a scope guard, and it just works, I'm not sure why you can't throw in those? You can but that is not acceptable for the spec explicitly forbids that: https://dlang.org/spec/statement.html#scope-guard-statement Quote (again): "A [...] scope(success) statement may not exit with a throw [...]." I know. I'm not saying it's valid, I'm wondering *why* it's not valid, since it's trivial for the compiler to detect that code might throw (yet doesn't in this case), and the construct it lowers to (the finally clause) allows throwing. Note that the finally clause has all the same restrictions as scope(exit) and scope(success) *except* throwing. It might be an oversight in the spec. Furthermore I always thought of scope guards as a means for cleanup. Cleanup implies in my eyes removing things which have been used in a previous task. This intended use is documented here: https://tour.dlang.org/tour/en/gems/scope-guards Using scope guards makes code much cleaner and allows resource allocation and clean up code to be placed next to each other. These little helpers also improve safety because they make sure certain cleanup code is always called independent of which paths are actually taken at runtime. Performing a COMMIT is rather the opposite of cleanup: It makes (writes) changes to the database persistent. Semantically, you are just not undoing (rollback) the previous steps. How it's implemented in the database is up to them. -Steve
Re: Comparing Exceptions and Errors
On Thursday, 16 June 2022 at 11:38:40 UTC, kdevel wrote: On Thursday, 16 June 2022 at 11:28:32 UTC, bauss wrote: [...] https://dlang.org/spec/statement.html#scope-guard-statement Quote (again): "A [...] scope(success) statement may not exit with a throw [...]." [...] If the spec forbids it, but the compiler allows it, wouldn't it then be a bug? What does "it" referer to? The code, throwing in scope guard? The spec? The compiler? [yes, no, no] Throwing in a scope guard.
Re: Comparing Exceptions and Errors
On Thursday, 16 June 2022 at 11:28:32 UTC, bauss wrote: [...] https://dlang.org/spec/statement.html#scope-guard-statement Quote (again): "A [...] scope(success) statement may not exit with a throw [...]." [...] If the spec forbids it, but the compiler allows it, wouldn't it then be a bug? What does "it" referer to? The code, throwing in scope guard? The spec? The compiler? [yes, no, no]
Re: Comparing Exceptions and Errors
On Thursday, 16 June 2022 at 10:07:23 UTC, kdevel wrote: On Wednesday, 15 June 2022 at 20:46:56 UTC, Steven Schveighoffer wrote: [...] It has not harmed my code though. I tried throwing inside a scope guard, and it just works, I'm not sure why you can't throw in those? You can but that is not acceptable for the spec explicitly forbids that: https://dlang.org/spec/statement.html#scope-guard-statement Quote (again): "A [...] scope(success) statement may not exit with a throw [...]." Furthermore I always thought of scope guards as a means for cleanup. Cleanup implies in my eyes removing things which have been used in a previous task. This intended use is documented here: https://tour.dlang.org/tour/en/gems/scope-guards Using scope guards makes code much cleaner and allows resource allocation and clean up code to be placed next to each other. These little helpers also improve safety because they make sure certain cleanup code is always called independent of which paths are actually taken at runtime. Performing a COMMIT is rather the opposite of cleanup: It makes (writes) changes to the database persistent. If the spec forbids it, but the compiler allows it, wouldn't it then be a bug?
Re: Comparing Exceptions and Errors
On Wednesday, 15 June 2022 at 20:46:56 UTC, Steven Schveighoffer wrote: [...] It has not harmed my code though. I tried throwing inside a scope guard, and it just works, I'm not sure why you can't throw in those? You can but that is not acceptable for the spec explicitly forbids that: https://dlang.org/spec/statement.html#scope-guard-statement Quote (again): "A [...] scope(success) statement may not exit with a throw [...]." Furthermore I always thought of scope guards as a means for cleanup. Cleanup implies in my eyes removing things which have been used in a previous task. This intended use is documented here: https://tour.dlang.org/tour/en/gems/scope-guards Using scope guards makes code much cleaner and allows resource allocation and clean up code to be placed next to each other. These little helpers also improve safety because they make sure certain cleanup code is always called independent of which paths are actually taken at runtime. Performing a COMMIT is rather the opposite of cleanup: It makes (writes) changes to the database persistent.
Re: Comparing Exceptions and Errors
On 6/15/22 3:51 PM, kdevel wrote:
On Wednesday, 15 June 2022 at 03:09:56 UTC, Steven Schveighoffer wrote:
I don't see what you see wrong with the code I wrote. It's
straightforward, obvious, and does the job I need it to do, in a way
that's not prone to future mistakes.
Sometimes it is not easy to explain why code "feels" wrong and in the
case of
```
scope(success) conn.exec("COMMIT");
```
it was not that clear to me some days ago. The reason why I would not write
it is: `conn.exec("COMMIT")` may throw! Think of deferred constraints
which are checked not before the commit. But "[a] [...] scope(success)
statement may not exit with a throw [...]" [1]
[1] https://dlang.org/spec/statement.html#ScopeGuardStatement
I do depend on the commit (and the rollback) not throwing. I probably
should swallow any exceptions at that point, and close the connection.
It has not harmed my code though. I tried throwing inside a scope guard,
and it just works, I'm not sure why you can't throw in those?
I've been meaning to add a reentrant transaction system to mysql-native,
because I hate that I can't do nested transactions. That would solve
that problem and this problem, by giving me a nothrow function to call
in the scope guard.
-Steve
Re: Comparing Exceptions and Errors
On Wednesday, 15 June 2022 at 03:09:56 UTC, Steven Schveighoffer
wrote:
I don't see what you see wrong with the code I wrote. It's
straightforward, obvious, and does the job I need it to do, in
a way that's not prone to future mistakes.
Sometimes it is not easy to explain why code "feels" wrong and in
the case of
```
scope(success) conn.exec("COMMIT");
```
it was not that clear to me some days ago. The reason why I would
not write
it is: `conn.exec("COMMIT")` may throw! Think of deferred
constraints which are checked not before the commit. But "[a]
[...] scope(success) statement may not exit with a throw [...]"
[1]
[1] https://dlang.org/spec/statement.html#ScopeGuardStatement
Re: Comparing Exceptions and Errors
I don't see what you see wrong with the code I wrote. It's straightforward, obvious, and does the job I need it to do, in a way that's not prone to future mistakes. I explained why, but you don't agree with the explanation. That's OK, we don't all have to write the same exact systems. D is a language that can satisfy many needs and philosophies! If you find it doesn't do it quite the way you want, and C++ does, C++ is also a fine language to use. -Steve
Re: Comparing Exceptions and Errors
On Monday, 13 June 2022 at 13:15:42 UTC, Steven Schveighoffer wrote: Possibly, but I don't close the handle. It goes back to a pool to get reused. Um. I need a (fresh) connection per CGI process. I wonder if nowadays the TLS startup between the browser and the webserver isn't at least one or two magnitudes more expensive than the (unencrypted) connection to the domain socket of the DB engine. May pooling DB connections instead of closing them be optimizing on the wrong end? Why not have a Transaction class (struct)? A transaction class/struct cannot hook normal return and throwing differently (the destructor has no way of knowing whether an exception is in flight). Regarding C++: “[...] In a proper robust design, commits should always be explicit and destructors only rollback uncommitted transactions. [...] There is now enough introspection in the language to actually implement implicit commit in a non exceptional path, but I think it would be a mistake." [1] https://news.ycombinator.com/item?id=24648406
Re: Comparing Exceptions and Errors
On 6/13/22 9:15 AM, Steven Schveighoffer wrote: Yes. If you don't execute the rollback and start executing more DB calls, they all get included in the transaction (and might be expected to be). Should have said "might *not* be expected to be" -Steve
Re: Comparing Exceptions and Errors
On 6/12/22 4:11 PM, kdevel wrote:
On Tuesday, 7 June 2022 at 18:37:13 UTC, Steven Schveighoffer wrote:
[...]
My very common use of `scope(failure)` for my DB code:
```d
conn.exec("START TRANSACTION");
scope(success) conn.exec("COMMIT");
scope(failure) conn.exec("ROLLBACK");
```
Are there multiple (successful) returns in your code or why are you
executing the COMMIT under a scope exit clause?
No, usually just one (the implicit final return).
If there is only one
successful return wouldn't it enhance code readability when an explicit
COMMIT precedes the return?
It just introduces another place where I can mess up the transaction. I
can write these 3 lines, and be done, never having to worry about the
transaction code again.
This is one of the major features of scope guards.
Is the ROLLBACK really necessary?
Yes. If you don't execute the rollback and start executing more DB
calls, they all get included in the transaction (and might be expected
to be).
Isn't the transaction not committed
(i.e. rolled back) when the db handle is closed?
Possibly, but I don't close the handle. It goes back to a pool to get
reused.
Why not have a
Transaction class (struct)?
A transaction class/struct cannot hook normal return and throwing
differently (the destructor has no way of knowing whether an exception
is in flight).
-Steve
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 23:57:19 UTC, Steven Schveighoffer
wrote:
On 6/5/22 6:09 PM, kdevel wrote:
On Sunday, 5 June 2022 at 20:53:32 UTC, Steven Schveighoffer
wrote:
[...]
For this purpose nobody needs a separate subclass named
`Error`. That works with `Exception`s.
You can use Exceptions instead. But the difference is they
are part of the program instead of considered a check on the
program itself.
I have no clue what that means.
An assert (or thrown error) is not ever supposed to happen, so
it's not part of the program. It's a check to ensure that the
program is sound and valid. You can feel free to insert as many
asserts as you want, and it should not be considered part of
the program.
It basically says "If this condition is false, this entire
program is invalid, and I don't know how to continue from here."
Who is the narrator here? It is the author of the function which
contains the assert. But this is not necessarily the author of
the code which calls this function. Why should the author of the
function be allowed *to terminate the whole the program* (be it
via assert or exit)?
[...]
My code does not throw `Error`s. It always throws `Exception`s.
Then I guess you should just ignore Errors and let the runtime
handle them? I'm not sure why this discussion is happening.
Because phobos/runtime throw Errors?
changing all range functions to use the `checkedPopFront` and
`checkedFront`, just to find the error. Instead of letting
the asserts do their job.
It is not clear to me, what you now complain about. You first
criticized that in
```
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
there is triple checking that r is not empty, namely in
!r.empty, in r.front and in r.popFront. One check, !r.emtpy,
will suffice. Hence one can safely use
```
for(auto r = range; !r.empty; r.popFront_unchecked) {
auto elem = r.front_unchecked;
}
```
You can do this, and then if it fails, you have to guess that
maybe you did something wrong, and start using the checked
versions.
What shall go wrong in this example? Of course the compiler might
generate wrong code. But it is not the aim of putting enforce and
asserts into program code to guard against the bugs in the
compiler.
Re: Comparing Exceptions and Errors
On Tuesday, 7 June 2022 at 18:37:13 UTC, Steven Schveighoffer
wrote:
[...]
My very common use of `scope(failure)` for my DB code:
```d
conn.exec("START TRANSACTION");
scope(success) conn.exec("COMMIT");
scope(failure) conn.exec("ROLLBACK");
```
Are there multiple (successful) returns in your code or why are
you executing the COMMIT under a scope exit clause? If there is
only one successful return wouldn't it enhance code readability
when an explicit COMMIT precedes the return?
Is the ROLLBACK really necessary? Isn't the transaction not
committed (i.e. rolled back) when the db handle is closed? Why
not have a Transaction class (struct)?
Re: Comparing Exceptions and Errors
On 6/7/22 3:58 PM, frame wrote:
On Tuesday, 7 June 2022 at 18:37:13 UTC, Steven Schveighoffer wrote:
My very common use of `scope(failure)` for my DB code:
```d
conn.exec("START TRANSACTION");
scope(success) conn.exec("COMMIT");
scope(failure) conn.exec("ROLLBACK");
```
This is hard to encapsulate into a type, as dtors only hook
`scope(exit)` essentially.
-Steve
That's fine as the Throwable is still thrown but since `scope(failure)`
acts as like catch-block, people may use it as replacement if there is
something more to do.
I personally like the clean, short syntax when I don't care about the
actual exception or if I know the called function has already handled
the exception and just rethrows it:
```d
bool fun() {
scope(failure) {
// do something
return false;
}
badCode();
return true;
}
```
Wait, what?
I'm looking at the spec and it says:
A scope(exit) or scope(success) statement may not exit with a
throw, goto, break, continue, or return; nor may it be entered with a goto.
Which specifically does not include scope(failure).
I would never have expected that to be a "feature"...
I know this is bad as I'm capturing a possible error here - but this is
what an unexperienced coder would do, because it works. The better
approach would be to use `scope(exception)` (not typed, just as
keyword) that allows to act only on Exceptions so one cannot break
Exception vs Error paradigma and this also clarifies that usage of
`scope(failure)` is potentially dangerous if you return from it.
Anyway, I just put that here - maybe you mention it in your blog.
Ali's linked bug report suggests that it's happening for Errors too,
which is going to cause major problems if something didn't get cleaned
up properly.
I will update the blog post. My recommendation is going to be, don't
circumvent the propagation of the throwable for now. Instead use a `try`
+ `catch(Exception)`. I think we need a compiler change to not allow
return if it's catching an Error.
There may be other "weird" cases that are not covered by the spec. Is it
legal to goto a label inside scope(failure)?
Thanks for the heads up!
-Steve
Re: Comparing Exceptions and Errors
On 6/7/22 12:58, frame wrote:
> ```d
> bool fun() {
> scope(failure) {
> // do something
> return false;
> }
>
> badCode();
> return true;
> }
> ```
>
> I know this is bad as I'm capturing a possible error here - but this is
> what an unexperienced coder would do, because it works.
WAT! I think that's a bug. Simply having a 'return' statement suppresses
rethrowing the exception? Too subtle! The following bug is related:
https://issues.dlang.org/show_bug.cgi?id=21443
And the spec does not mention 'return' in scope(failure) having such a
huge effect. Uncommenting the 'return' line below causes wildly
different program behavior:
import std.stdio;
void main() {
writeln("main is calling zar");
zar();
}
void zar() {
scope (failure) {
writeln("zar is failing");
}
writeln("zar is calling bar");
bar();
}
void bar() {
scope (failure) {
writeln("bar is failing");
// return;
}
writeln("bar is calling foo");
foo();
}
void foo() {
writeln("foo is throwing");
throw new Exception("Oops!");
}
BUG! :)
Ali
Re: Comparing Exceptions and Errors
On Tuesday, 7 June 2022 at 18:37:13 UTC, Steven Schveighoffer
wrote:
My very common use of `scope(failure)` for my DB code:
```d
conn.exec("START TRANSACTION");
scope(success) conn.exec("COMMIT");
scope(failure) conn.exec("ROLLBACK");
```
This is hard to encapsulate into a type, as dtors only hook
`scope(exit)` essentially.
-Steve
That's fine as the Throwable is still thrown but since
`scope(failure)` acts as like catch-block, people may use it as
replacement if there is something more to do.
I personally like the clean, short syntax when I don't care about
the actual exception or if I know the called function has already
handled the exception and just rethrows it:
```d
bool fun() {
scope(failure) {
// do something
return false;
}
badCode();
return true;
}
```
I know this is bad as I'm capturing a possible error here - but
this is what an unexperienced coder would do, because it works.
The better approach would be to use `scope(exception)` (not
typed, just as keyword) that allows to act only on Exceptions so
one cannot break Exception vs Error paradigma and this also
clarifies that usage of `scope(failure)` is potentially dangerous
if you return from it.
Anyway, I just put that here - maybe you mention it in your blog.
Re: Comparing Exceptions and Errors
On 6/7/22 12:28 PM, frame wrote:
On Friday, 3 June 2022 at 23:40:50 UTC, Steven Schveighoffer wrote:
During the last beerconf, I wrote a short blog post about how `Error`
and `Exception` are different, and why you should never continue after
catching `Error`s.
I know the thematics but I still wonder why we only have
`scope(failure)` then? Anywhere where you will use this shiny thing with
a return statement will also catch any error that have occurred.
`scope(exit)` doesn't allow return statements, thus the only properly
clean design would be an additional `scope(exception)` guard.
My very common use of `scope(failure)` for my DB code:
```d
conn.exec("START TRANSACTION");
scope(success) conn.exec("COMMIT");
scope(failure) conn.exec("ROLLBACK");
```
This is hard to encapsulate into a type, as dtors only hook
`scope(exit)` essentially.
-Steve
Re: Comparing Exceptions and Errors
On Friday, 3 June 2022 at 23:40:50 UTC, Steven Schveighoffer wrote: During the last beerconf, I wrote a short blog post about how `Error` and `Exception` are different, and why you should never continue after catching `Error`s. I know the thematics but I still wonder why we only have `scope(failure)` then? Anywhere where you will use this shiny thing with a return statement will also catch any error that have occurred. `scope(exit)` doesn't allow return statements, thus the only properly clean design would be an additional `scope(exception)` guard.
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 18:08:17 UTC, Ola Fosheim Grøstad wrote: There is no reason for D to undercut users of @safe code. (Wrong usage of the term «undercut», but you get the idea…)
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 17:52:12 UTC, Steven Schveighoffer wrote: Then that's part of the algorithm. You can use an Exception, and then handle the exception by calling the real sort. If in the future, you decide that it can properly sort with that improvement, you remove the Exception. That is different from e.g. using a proven algorithm, like quicksort, but failing to implement it properly. No? Why do you find it so? Adding a buggy optimization is exactly failing to implement it properly. There is a reference, the optimization should work exactly like the reference, but didn't. Using asserts in @safe code should be no different than using asserts in Python code. Python code <=> safe D code. Python library implemented in C <=> trusted D code. There is no reason for D to undercut users of @safe code. If anything D should try to use @safe to provide benefits that C++ users don't get.
Re: Comparing Exceptions and Errors
On 6/6/22 12:15 PM, Ola Fosheim Grøstad wrote: On Monday, 6 June 2022 at 15:54:16 UTC, Steven Schveighoffer wrote: If it's an expected part of the sorting algorithm that it *may fail to sort*, then that's not an Error, that's an Exception. No, it is not expected. Let me rewrite my answer to Sebastiaan to fit with the sort scenario: For instance, you may have a formally verified sort function, but it is too slow. So you optimize one selected bottle neck, but that cannot be verified, because verification is hard. That specific unverified softspot is guarded by an assert. The compiler may remove it or not. Your shipped product fails, because the hard to read optimization wasn't perfect. So you trap the thrown assert and call the reference implementation instead. The cool thing with actors/tasks is that you can make them as small and targeted and revert to fallbacks if they fail. (Assuming 100% @safe code.) Then that's part of the algorithm. You can use an Exception, and then handle the exception by calling the real sort. If in the future, you decide that it can properly sort with that improvement, you remove the Exception. That is different from e.g. using a proven algorithm, like quicksort, but failing to implement it properly. -Steve
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 16:15:19 UTC, Ola Fosheim Grøstad wrote: On Monday, 6 June 2022 at 15:54:16 UTC, Steven Schveighoffer wrote: If it's an expected part of the sorting algorithm that it *may fail to sort*, then that's not an Error, that's an Exception. No, it is not expected. Let me rewrite my answer to Sebastiaan to fit with the sort scenario: Let me sketch up another scenario. Let's say I am making an online game and I need early feedback from beta-testers. So I run my beta-service with lots of asserts and logging, when actors fail I discard them and relaunch them. If the server went down on the first assert I wouldn't be able to test my server at all, because there would be no users willing to participate in a betatest where the server goes down every 20 seconds! That is a very bad high risk-factor, that totally dominates this use scenario. An engineer has to fill words such as «reliability», «utility», «probability» and «risk» with meaning that match the use scenario and make deliberate choices (cost-benefit-risk considerations). That includes choosing an actor model, and each actor has to prevent failure from affecting other actors. (by definition of «actor»).
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 15:54:16 UTC, Steven Schveighoffer wrote: If it's an expected part of the sorting algorithm that it *may fail to sort*, then that's not an Error, that's an Exception. No, it is not expected. Let me rewrite my answer to Sebastiaan to fit with the sort scenario: For instance, you may have a formally verified sort function, but it is too slow. So you optimize one selected bottle neck, but that cannot be verified, because verification is hard. That specific unverified softspot is guarded by an assert. The compiler may remove it or not. Your shipped product fails, because the hard to read optimization wasn't perfect. So you trap the thrown assert and call the reference implementation instead. The cool thing with actors/tasks is that you can make them as small and targeted and revert to fallbacks if they fail. (Assuming 100% @safe code.) It says that the programmer cannot attribute exactly where this went wrong because otherwise, he would have accounted for it, or thrown an Exception instead (or some other mitigation). He can make a judgement. If this happened in a safe pure function then it would most likely be the result of what is what meant to do: check that the assumptions of the algorithm holds. Anything from memory corruption, to faulty hardware, to bugs in the code, could be the cause. That is not what asserts check! They will be removed if the static analyzer is powerful enough. All the information to remove the assert should be in the source code. You are asserting that *given all the constraints of the type system* then the assert should hold. Memory corruption could make an assert succeed when it should not, because then anything can happen! It cannot catch memory corruption reliably because it is not excluded from optimization. You need something else for that, something that turns off optimization for all asserts. Exactly. Use Exceptions if it's recoverable, Errors if it's not. This is what is not true, asserts says only something about the algorithm it is embedded in, it says that the algorithm makes a wrong assumption, and that is all. It says nothing about the calling environment. A failed assert could be because of undefined behavior. It doesn't *imply* it, but it cannot be ruled out. And a successful assert could happen because of undefined behaviour or optimization! If you want these types of guards then you need to propose a type of asserts that would be excluded from optimization. (which might be a good idea!) In the case of UB anything can happen. It is up to the programmer to make that judgment based on the use scenario. It is a matter of probabilisitic calculations in relation to the use scenario of the application. As I pointed out elsewhere: «reliability» has to be defined in terms of the use scenario by a skilled human being, not in terms of some kind of abstract thinking about compiler design.
Re: Comparing Exceptions and Errors
On 6/6/22 12:59 AM, Ola Fosheim Grøstad wrote: On Sunday, 5 June 2022 at 23:57:19 UTC, Steven Schveighoffer wrote: It basically says "If this condition is false, this entire program is invalid, and I don't know how to continue from here." No, it says: this function failed to uphold this invariant. You can perfectly well recover if you know what that function touches. For instance if a sort function fails, then you can call a slower sort function. If it's an expected part of the sorting algorithm that it *may fail to sort*, then that's not an Error, that's an Exception. Or in terms of actors/tasks: if one actor-solver fails numerically, then you can recover and use a different actor-solver. If the condition is recoverable => Exception. If it is not recoverable => Error. An assert says nothing about the whole program. An assert says that something is wrong, and this was not expected or foreseen. It says that the programmer cannot attribute exactly where this went wrong because otherwise, he would have accounted for it, or thrown an Exception instead (or some other mitigation). Anything from memory corruption, to faulty hardware, to bugs in the code, could be the cause. An assert only says that the logic of that particular function is not meeting the SPEC. That's one possible reason. But if you are *planning* on possibly not meeting the spec (such as your sort example), that is a different story. Only the programmer knows if recovery is possible, not the compiler. Exactly. Use Exceptions if it's recoverable, Errors if it's not. A failed assert is not implying undefined behaviour in @safe code. A failed assert could be because of undefined behavior. It doesn't *imply* it, but it cannot be ruled out. -Steve
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 06:56:46 UTC, Ola Fosheim Grøstad wrote: On Monday, 6 June 2022 at 06:14:59 UTC, Sebastiaan Koppe wrote: Those are not places where you would put an assert. The only place to put an assert is when *you* know there is no recovery. No, asserts are orthogonal to recovery. They just specify the assumed constraints in the implementation of the algorithm. You can view them as comments that can be read by a computer and checked for that specific function. I guess an informal way to express this is: *Asserts are comments that you would need to make when explaining why the algorithm works to another person (or to convince yourself that it works).* As far as unnecessary asserts, it would be nice to have something more powerful than static assert, something that could reason about runtime issues that are simple and issue errors if it could not establish it. E.g.: ``` int i = 0; …later… i++; …much later… compiletime_assert(i>0); ```
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 06:14:59 UTC, Sebastiaan Koppe wrote: Those are not places where you would put an assert. The only place to put an assert is when *you* know there is no recovery. No, asserts are orthogonal to recovery. They just specify the assumed constraints in the implementation of the algorithm. You can view them as comments that can be read by a computer and checked for that specific function. For instance you can have a formally proven reference implementation full of asserts, then one optimized version where you keep critical asserts or just the post condition. If the optimized version fails, then you can revert to the reference (with no or few asserts, because it is already formally verified). There is nothing wrong with having many asserts or asserts you «know» to hold. They are helpful when you modify code and datastructures. Maybe one could have more selective ways to leave out asserts (e.g. based on revision) so that you remove most asserts in actors that has not changed since version 1.0 and retain more asserts in new actors. Also, if you fully check the full post condition (in @safe code) then you can remove all asserts in release as they are inconsequential. So the picture is more nuanced and it should be up to the programmer to decide, but maybe a more expressive and selective regime is useful.
Re: Comparing Exceptions and Errors
On Monday, 6 June 2022 at 04:59:05 UTC, Ola Fosheim Grøstad wrote: For instance if a sort function fails, then you can call a slower sort function. Or in terms of actors/tasks: if one actor-solver fails numerically, then you can recover and use a different actor-solver. Those are not places where you would put an assert. The only place to put an assert is when *you* know there is no recovery. Like you have been arguing, there aren't many places like that. So don't use it. --- 9 out of 10 times when I see an assert in code review I ask the author to reconsider. Often it only requires a little tweak. I guess you could say I have found asserts to be misused.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 23:57:19 UTC, Steven Schveighoffer wrote: It basically says "If this condition is false, this entire program is invalid, and I don't know how to continue from here." No, it says: this function failed to uphold this invariant. You can perfectly well recover if you know what that function touches. For instance if a sort function fails, then you can call a slower sort function. Or in terms of actors/tasks: if one actor-solver fails numerically, then you can recover and use a different actor-solver. An assert says nothing about the whole program. An assert only says that the logic of that particular function is not meeting the SPEC. That’s all. If you use asserts for something else then you don’t follow the semantic purpose of asserts. Only the programmer knows if recovery is possible, not the compiler. A failed assert is not implying undefined behaviour in @safe code.
Re: Comparing Exceptions and Errors
On 6/5/22 6:09 PM, kdevel wrote:
On Sunday, 5 June 2022 at 20:53:32 UTC, Steven Schveighoffer wrote:
[...]
For this purpose nobody needs a separate subclass named `Error`. That
works with `Exception`s.
You can use Exceptions instead. But the difference is they are part of
the program instead of considered a check on the program itself.
I have no clue what that means.
An assert (or thrown error) is not ever supposed to happen, so it's not
part of the program. It's a check to ensure that the program is sound
and valid. You can feel free to insert as many asserts as you want, and
it should not be considered part of the program.
It basically says "If this condition is false, this entire program is
invalid, and I don't know how to continue from here."
[...]
If the code threw an `Exception` instead of an `Error` everything
would be fine.
I think the point of Errors is that you can remove them for efficiency.
elephant/room.
Why? If you have a correct program, you shouldn't ever have errors
thrown. If you do have errors thrown that's something you need to
address ASAP.
My code does not throw `Error`s. It always throws `Exception`s.
Then I guess you should just ignore Errors and let the runtime handle
them? I'm not sure why this discussion is happening.
[...]
Consider the normal flow of a range in a foreach loop, it's:
```d
// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
If both `popFront` and `front` also always call `empty` you are
calling `empty` 3 times per loop, with an identical value for the
2nd and 3rd calls.
Solution: Implement explicitly unchecked popFront() and front()
versions.
That's terrible. Now I have to instrument my code whenever I have a
weird unknown error,
Well, no. Just use the default. It will throw an exception if something
goes wrong. Of course, I must admid, if you want to compete with those
I-have-the-fastest-serving-http-server-guys (N.B.: HTTP not HTTPS! Did
anybody notice that?) reporting 50 Gigarequests per second (or so) then
you probably have to *measure* *the* *performance* of the
range/container implementation. But not earlier.
It becomes a tedious job of me to check "did I already check this? Oh I
did, so I can call the unchecked version", and hope that a future me
doesn't remove the original check.
Just always check, and then if you care about performance turn the
checks off. If they are on, and it crashes the program, you need to fix
it before turning them off. It's never a solution to just turn the
safety checks off when they are failing to allow the program to "work".
changing all range functions to use the `checkedPopFront` and
`checkedFront`, just to find the error. Instead of letting the asserts
do their job.
It is not clear to me, what you now complain about. You first criticized
that in
```
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
there is triple checking that r is not empty, namely in !r.empty, in
r.front and in r.popFront. One check, !r.emtpy, will suffice. Hence one
can safely use
```
for(auto r = range; !r.empty; r.popFront_unchecked) {
auto elem = r.front_unchecked;
}
```
You can do this, and then if it fails, you have to guess that maybe you
did something wrong, and start using the checked versions.
I have encountered numerous times when I trigger these checks, and it's
always a problem with how I wrote my code. If instead I just got a
segfault, now I have to start digging, instrumenting, etc.
If you don't trust whomever you can simply keep the checked versions.
The test will then be performed thrice regardless if a test failure will
result in throwing an `Exception` or throwing an `Error`. AFAICS.
As the author of the range, I put the asserts in for *my* benefit,
without affecting the user (if they so desire). I never trust the user.
If you write perfect code, turn asserts off and you won't have a
problem, right?
If you don't write perfect code, leave them on, and you can diagnose the
problem when it occurs instead of having to redo everything.
The thing is, I don't want there to be a battle between performance and
correctness. This solves the issue by allowing correctness to be
checked, without hindering the possibility of compiling with performance.
Yet, I do think D compilers don't make it easy to turn off checks in
selected module. And as previously stated, I think some things that are
Errors should actually be Exceptions.
-Steve
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 21:08:11 UTC, Steven Schveighoffer wrote: [...] Just FYI, this is a *different discussion* from whether Errors should be recoverable. The wording of this "question" bothers me really. What does "Errors" mean here? If you mean thrown object having a (sub)type of `Error` the relevant question is: ARE `Error`s recoverable? If they ARE recoverable then every program can written in a way it handles even `Error`s gracefully. Whether specific conditions are considered an Error or an Exception is something on which I have differing opinions than the language. However, I do NOT have a problem with having an assert/invariant mechanism to help prove my program is correct. I may be not a top-notch expert on this topic but IMO there is no facility which can perform such a proof.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 20:53:32 UTC, Steven Schveighoffer
wrote:
[...]
For this purpose nobody needs a separate subclass named
`Error`. That works with `Exception`s.
You can use Exceptions instead. But the difference is they are
part of the program instead of considered a check on the
program itself.
I have no clue what that means.
[...]
If the code threw an `Exception` instead of an `Error`
everything would be fine.
I think the point of Errors is that you can remove them for
efficiency.
elephant/room.
Why? If you have a correct program, you shouldn't ever have
errors thrown. If you do have errors thrown that's something
you need to address ASAP.
My code does not throw `Error`s. It always throws `Exception`s.
In other words, just like asserts or bounds checks, they are
not expected to be part of the normal working program.
"removing [Errors, asserts, bounds checks] is a bit like
wearing a life-jacket to practice in the harbour, but then
leaving the life-jackets behind when your ship leaves for open
ocean" [1]
It's more like leaving the floaties behind when you are doing a
swim meet.
Nice new question for job interviews.
[...]
Consider the normal flow of a range in a foreach loop, it's:
```d
// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
If both `popFront` and `front` also always call `empty` you
are calling `empty` 3 times per loop, with an identical value
for the 2nd and 3rd calls.
Solution: Implement explicitly unchecked popFront() and
front() versions.
That's terrible. Now I have to instrument my code whenever I
have a weird unknown error,
Well, no. Just use the default. It will throw an exception if
something goes wrong. Of course, I must admid, if you want to
compete with those I-have-the-fastest-serving-http-server-guys
(N.B.: HTTP not HTTPS! Did anybody notice that?) reporting 50
Gigarequests per second (or so) then you probably have to
*measure* *the* *performance* of the range/container
implementation. But not earlier.
changing all range functions to use the `checkedPopFront` and
`checkedFront`, just to find the error. Instead of letting the
asserts do their job.
It is not clear to me, what you now complain about. You first
criticized that in
```
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
there is triple checking that r is not empty, namely in !r.empty,
in r.front and in r.popFront. One check, !r.emtpy, will suffice.
Hence one can safely use
```
for(auto r = range; !r.empty; r.popFront_unchecked) {
auto elem = r.front_unchecked;
}
```
If you don't trust whomever you can simply keep the checked
versions. The test will then be performed thrice regardless if a
test failure will result in throwing an `Exception` or throwing
an `Error`. AFAICS.
[...]
Re: Comparing Exceptions and Errors
On 6/5/22 11:03, kdevel wrote: > On Sunday, 5 June 2022 at 17:04:49 UTC, Ali Çehreli wrote: >> On 6/5/22 08:07, kdevel wrote: > [...] >> Like many other programmers who include me, Sean Parent may be right.[1] >> >> Other than being a trivial example to make a point, the code I've >> shown may be taking advantage of the "structure of array" optimization. > > "Premature optimization is ..." You pulled important phrases like "Sean Parent" and "incidental data structure" and I countered with "structure of array" but "premature optimization" finally beat my argument. Wait... No! Proving my example ineffective does not disprove my argument. So, I take this exchange as a fun break. > It's not "the" assert but another one. Nonetheless I take up the challenge: I hereby withdraw my example. Ali
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 21:08:11 UTC, Steven Schveighoffer wrote: Just FYI, this is a *different discussion* from whether Errors should be recoverable. Ok, but do you a difference between being recoverable anywhere and being recoverable at the exit-point of an execution unit like an Actor/Task? Whether specific conditions are considered an Error or an Exception is something on which I have differing opinions than the language. Ok, like null-dereferencing and division-by-zero perhaps. However, I do NOT have a problem with having an assert/invariant mechanism to help prove my program is correct. Or rather the opposite, prove that a specific function is incorrect for a specific input configuration. The question is, if a single function is incorrect for some specific input, why would you do anything more than disabling that function?
Re: Comparing Exceptions and Errors
On 6/5/22 12:27 PM, Ola Fosheim Grøstad wrote: Ok, so I am a bit confused about what is Error and what is not… According to core.exception there is wide array of runtime Errors: ``` RangeError ArrayIndexError ArraySliceError AssertError FinalizeError OutOfMemoryError InvalidMemoryOperationError ForkError SwitchError ``` I am not sure that any of these should terminate anything outside the offending actor, but I could be wrong as it is hard to tell exactly when some of those are thrown. Just FYI, this is a *different discussion* from whether Errors should be recoverable. Whether specific conditions are considered an Error or an Exception is something on which I have differing opinions than the language. However, I do NOT have a problem with having an assert/invariant mechanism to help prove my program is correct. -Steve
Re: Comparing Exceptions and Errors
On 6/5/22 8:45 AM, kdevel wrote:
On Sunday, 5 June 2022 at 01:43:06 UTC, Steven Schveighoffer wrote:
[...]
But you aren't perfect, and so maybe you make a mistake, and trigger
an Error. The compiler handles this unexpected condition by unwinding
the stack back to the main function, printing the error and exiting,
so you can go fix whatever mistake you made.
For this purpose nobody needs a separate subclass named `Error`. That
works with `Exception`s.
You can use Exceptions instead. But the difference is they are part of
the program instead of considered a check on the program itself.
There's a need for both.
[...]
If the code threw an `Exception` instead of an `Error` everything
would be fine.
I think the point of Errors is that you can remove them for efficiency.
elephant/room.
Why? If you have a correct program, you shouldn't ever have errors
thrown. If you do have errors thrown that's something you need to
address ASAP.
In other words, just like asserts or bounds checks, they are not
expected to be part of the normal working program.
"removing [Errors, asserts, bounds checks] is a bit like wearing a
life-jacket to practice in the harbour, but then leaving the
life-jackets behind when your ship leaves for open ocean" [1]
It's more like leaving the floaties behind when you are doing a swim meet.
[...]
Consider the normal flow of a range in a foreach loop, it's:
```d
// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
If both `popFront` and `front` also always call `empty` you are
calling `empty` 3 times per loop, with an identical value for the 2nd
and 3rd calls.
Solution: Implement explicitly unchecked popFront() and front() versions.
That's terrible. Now I have to instrument my code whenever I have a
weird unknown error, changing all range functions to use the
`checkedPopFront` and `checkedFront`, just to find the error. Instead of
letting the asserts do their job.
Having the assert allows diagnosing invalid programs without crashing
your program,
That depends on the understanding of "crashing a program". If library
code throws an `Error` instead of an `Exception` I have to isolate that
code in a subprocess in order to make my program gracefully handle the
error condition.
You don't gracefully handle the error condition. It's like saying
gracefully handling running into the guardrail on a road. You just
crash, and hope you don't die. You don't just graze into it and keep
going thinking "well, the guardrail did it's job, glad it's there, I
plan on using it every time I go around that corner."
Think of CGI processes which provide output direct to a customer. If
there is an assert the customer will see the famous Internal Server
Error message (in case of apache httpd).
An assert triggering means, your code did something invalid. It should
crash/exit.
Now we can have *totally separate* debates on what should be an Error
and what should be an Exception. And not to belittle your point, I
understand that there can be a philosophy that you *only* want
recoverable throwables for certain code domains (I myself also have that
feeling for e.g. out of bounds errors). It's just not what D picked as
an error handling scheme. We have both recoverable exceptions, and non
recoverable errors.
-Steve
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 17:04:49 UTC, Ali Çehreli wrote:
On 6/5/22 08:07, kdevel wrote:
[...]
Like many other programmers who include me, Sean Parent may be
right.[1]
Other than being a trivial example to make a point, the code
I've shown may be taking advantage of the "structure of array"
optimization.
"Premature optimization is ..."
[...]
> struct T {
> int a;
> int b;
> }
>
> struct S {
> T [] t;
>
>void add(int i) {
> t ~= T (i, i * 10);
>}
>
>void foo() {
> // Look Ma no assert!
The assert may have been moved to another place:
It's not "the" assert but another one. Nonetheless I take up the
challenge:
struct T {
int a;
int b;
invariant() {
// Look Pa it's still here!
assert(b == a * 10);
}
}
struct T {
int i;
this (int i)
{
this.i = i;
import std.checkedint;
Checked!(int, Throw) (i) * 10;
}
int a () const { return i; }
int b () const { return i * 10; }
}
struct S {
const (T) [] t;
void add(int i) {// <-- Both arrays always same size
t ~= T (i);
}
void foo() {
// ...
}
}
void main() {
auto s = S();
s.add(42);
s.foo();
// s.a ~= 1; // does not compile
// s.t[0].b = 3; // no compile either
// s.t[0].i = 7; // cannot change i, does not compile
s.add (2^^27); // fine
// s.add (2^^28); // throws Overflow on binary operator, fine
s.foo();
}
Ali
[...]
I'll try to not bring up his name again ;-)
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:
void add(int i) {// <-- Both arrays always same size
a ~= i;
b ~= i * 10;
}
void foo() {
assert(a.length == b.length); // <-- Invariant check
// ...
}
Maybe it would help if we can agree that this assert ought to be
statically proven to hold and the assert would therefore never be
evaluated in running code. Emitting asserts is just a sign of
failed statical analysis (which is common, but that is the most
sensible interpretation from a verification viewpoint).
The purpose of asserts is not to test the environment. The
purpose is to "prove" that the specified invariant of the
function holds for all legal input.
It follows that the goal of an assert is not to test if the
program is in a legal state!
I understand why you say this, but if this was the case then we
could not remove any asserts by statical analysis. :-/
Re: Comparing Exceptions and Errors
On 6/5/22 08:07, kdevel wrote:
> On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:
> [...]
>> struct S {
>> int[] a;
>> int[] b;
>>
>> void add(int i) {// <-- Both arrays always same size
>> a ~= i;
>> b ~= i * 10;
>> }
>>
>> void foo() {
>> assert(a.length == b.length); // <-- Invariant check
>> // ...
>> }
>> }
>>
>> void main() {
>> auto s = S();
>> s.add(42);
>> s.foo();
>> }
>>
>> The code is written in a way that both arrays will *always* have equal
>> number of elements.
>
> I think this is what Sean Parent called "incidental data structure" [1].
Like many other programmers who include me, Sean Parent may be right.[1]
Other than being a trivial example to make a point, the code I've shown
may be taking advantage of the "structure of array" optimization. I am
sure Sean Parent knows that as well.
> I would refactor the code:
Most likely me too.
> struct T {
> int a;
> int b;
> }
>
> struct S {
> T [] t;
>
>void add(int i) {
> t ~= T (i, i * 10);
>}
>
>void foo() {
> // Look Ma no assert!
The assert may have been moved to another place:
struct T {
int a;
int b;
invariant() {
// Look Pa it's still here!
assert(b == a * 10);
}
}
Ali
[1] I stopped following Sean Parent when he dismissed D and me by waving
his hand behind and walking away: "A language with reference types? No,
thanks." That happened at the end of one of his presentations here at
the Silicon Valley C++, which I happened to co-host. I am sure he is
observant enough to one day realize that C++ has reference types by
convention. (I recently posted links to C++ core guidelines proving that
point of mine, one of which is something to the effect of "never pass
polymorphic types by-value".)
Re: Comparing Exceptions and Errors
Ok, so I am a bit confused about what is Error and what is not… According to core.exception there is wide array of runtime Errors: ``` RangeError ArrayIndexError ArraySliceError AssertError FinalizeError OutOfMemoryError InvalidMemoryOperationError ForkError SwitchError ``` I am not sure that any of these should terminate anything outside the offending actor, but I could be wrong as it is hard to tell exactly when some of those are thrown. InvalidMemoryOperationError sound bad, of course, but the docs says «An invalid memory operation error occurs in circumstances when the garbage collector has detected an operation it cannot reliably handle. The default D GC is not re-entrant, so this can happen due to allocations done from within finalizers called during a garbage collection cycle.» This sounds more like an issue that needs fixing… On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote: The code is written in a way that both arrays will *always* have equal number of elements. And there is a "silly" check for that invariant. So far so good. The issue is what to do *when* that assert fails. Are you sure that it was a silly programmer mistake? I am not sure at all. Ok, so this is a question of the layers: ``` top layer -- D | E | - middle layer -- B | C | | bottom layer --- A If the failed assert is happening in a lower layer A then code on the outer layer should fault (or roll back a transaction). Whether it is reasonable to capture that error and suppress it depends on how independent you want those layers to be in your architecture. It also depends on the nature of layer A. If the failed assert happens in middle layer section B, then the D would be affected, but not A, C or E. The key issue is that the nature of layers is informal in the language (in fact, in most languages, a weakness), so only the programmer can tell what is reasonable or not. In fact, when we think about it; most aspects about what is expected from a program is informal… so it is very difficult to make judgment at the compiler level. Is the only other culprit the runtime kernel? I really don't know who else may be involved. I mean the application's «custom actor kernel», a hardened piece of code that is not modified frequently and heavily tested. The goal is to keep uncertainty local to an actor so that you can toss out misbehaving actors and keep the rest of the system working smoothly (99.5% of the time, 50 minutes downtime per week). Actors are expected to contain bugs because the game system is continuously modified (to balance the game play, to get new content, more excitement, whatever…). This is why we want 100% @safe code as a feature. There are also bugs in unrelated actor code writing over each others' memory. But that cannot happen if I decide to make actors 100% @safe and only let them interact with each other through my «custom actor kernel»? You are free to choose to catch Errors and continue under the assumption that it is safe to do so. Now I am confused!! Steven wrote «I've thought in the past that throwing an error really should not throw, but log the error (including the call stack), and then exit without even attempting to unwind the stack.» Surely, the perspective being promoted is to make sure that Errors cannot be stopped from propagating? That is why this becomes a discussion? If an Error can propagate through "nothrow" then the compiler should emit handler code for it and issue a warning. If you don't want that then the programmer should safe guard against it, meaning: manually catch and abort or do manual checks in all locations above it where Errors can arise. The compiler knows where. Not really sure why D has "nothrow", it doesn't really fit with the rest of the language? To interface with C++ perhaps?? If that is the case, just adopt C++ "noexcept" semantics, use assert() for debugging only, in "nothrow" code. And discourage the use of "nothrow". Heck, throw in a compiler switch to turn off "nothrow" if that is safe. The advice in the article still holds for me. I think the main difference is in the assumptions we make about an Errors: Is it a logic error in actor code or some crazy state that will cause weirder results if we continue. We can't know for sure. And this is no different from other languages with a runtime. You cannot be sure, but it probably isn't a runtime issue, and even if it was… players will be more upset by not being able to play than to have some weird effects happening. Same for chat service. Same for being able to read Wikipedia-caches (worse to have no access than to have 1% of pages garbled on display until the server is updated). Different settings need different solutions. So, maybe interfacing with C++ requires "nothrow", but if that is the only reason… why let everybody pay a
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 15:07:13 UTC, kdevel wrote: ... I would refactor the code: I really liked this one. The way it solves and at same time restrict the "external access" with that struct of (a,b) makes the code easier to maintain too. Glad I keep lurking around this forum. Matheus.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 14:24:39 UTC, Ali Çehreli wrote:
[...]
struct S {
int[] a;
int[] b;
void add(int i) {// <-- Both arrays always same size
a ~= i;
b ~= i * 10;
}
void foo() {
assert(a.length == b.length); // <-- Invariant check
// ...
}
}
void main() {
auto s = S();
s.add(42);
s.foo();
}
The code is written in a way that both arrays will *always*
have equal number of elements.
I think this is what Sean Parent called "incidental data
structure" [1]. I would refactor the code:
struct T {
int a;
int b;
}
struct S {
T [] t;
void add(int i) {
t ~= T (i, i * 10);
}
void foo() {
// Look Ma no assert!
// ...
}
}
void main() {
auto s = S();
s.add(42);
s.foo();
s.a ~= 1; // does not compile
s.foo();
}
```
[1]
Re: Comparing Exceptions and Errors
On 6/5/22 04:43, kdevel wrote:
> On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote:
> [...]
>> Errors are thrown when the program is discovered to be in an invalid
>> state.
>
> The following program throws an `Error` in popFront:
>
> import std.range;
>
> void main ()
> {
>int [1] a;
>auto q = a[1..$]; // line 6
>q.popFront; // line 7, throws core.exception.AssertError
> }
>
> When the program counter (PC) is at line 6 the program is in a valid
state.
>
> At no time the program is in an invalid state and it would not pass into
> an invalid state if popFront threw an `Exception` instead of an `Error`.
That looks like an argument for changing the behavior of D runtime when
an out-of-bounds access occurs. Of course, it is too late to change that
at this time but perhaps there can be a compiler switch like
-boundstype=[error|exception].
The programmer has many options to prove that nothing crazy is
happening. A common example suitable for many cases:
enforce(!q.empty); // (Throws Exception)
q.popFront;
Ali
Re: Comparing Exceptions and Errors
On 6/4/22 23:31, Ola Fosheim Grøstad wrote:
> On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote:
>> Errors are thrown when the program is discovered to be in an invalid
>> state. We don't know what happened and when. For example, we don't
>> know whether the memory has been overwritten by some rogue code.
>
> That is not very probable in 100% @safe code. You are basically saying
> that D cannot compete with Go and other «safe» languages.
I did not mean that. I think we have a misunderstanding at a fundamental
level.
> Dereferencing
> a null pointer usually means that some code failed to create an instance
> and check for it.
Dereferencing a null pointer does not throw Error but fine. I agree.
>> What happened? What can we assume. We don't know and we cannot assume
>> any state.
>
> So D will never be able to provide actors and provide fault tolerance.
Let me show an example. Here is a piece of code that could be running in
an actor:
struct S {
int[] a;
int[] b;
void add(int i) {// <-- Both arrays always same size
a ~= i;
b ~= i * 10;
}
void foo() {
assert(a.length == b.length); // <-- Invariant check
// ...
}
}
void main() {
auto s = S();
s.add(42);
s.foo();
}
The code is written in a way that both arrays will *always* have equal
number of elements. And there is a "silly" check for that invariant. So
far so good. The issue is what to do *when* that assert fails.
Are you sure that it was a silly programmer mistake? I am not sure at all.
>> Is the service in a usable state?
>
> Yes, the actor code failed. The actor code change frequently, not the
> runtime kernel.
Is the only other culprit the runtime kernel? I really don't know who
else may be involved.
>> Possibly. Not shutting down might produce incorrect results. Do we
>> prefer up but incorrect or dead?
>
> I prefer that service keeps running: chat service, game service, data
> delivered with hashed «checksum». Not all software are database engines
> where you have to pessimize about bugs in the runtime kernel.
There are also bugs in unrelated actor code writing over each others'
memory.
It is possible that the service will do unexpected or very wrong things.
But you answer my question: Your game server can do weird things.
Hopefully all acceptable by paying customers.
> If the data delivered is good enough for the client and better than
> nothing then the service should keep running!!!
Yes! How can you be sure data is good when the silly assertion above
failed. How could that happen? Is there any logical way to describe it
was actor code's mistake? I don't think so. Let's assume the
commented-out parts do not touch the arrays at all.
You are free to choose to catch Errors and continue under the assumption
that it is safe to do so. The advice in the article still holds for me.
I think the main difference is in the assumptions we make about an
Errors: Is it a logic error in actor code or some crazy state that will
cause weirder results if we continue. We can't know for sure.
>> I hope there is a way of aborting the program when there are invariant
>
> Invariants are USUALLY local. I dont write global spaghetti code. As a
> programmer you should be able to distinguish between local and global
> failure.
>
> You are assuming that the programmer is incapable of making judgements.
> That is assuming way too much.
I resent causing that misunderstanding. I apologize.
The only assumption I make about the programmer is that they do not mix
Error and Exception so that in the end an Error points at a situation
that warrants aborting the mission. Hm... Thinking more about it,
assuming that an Error is due to a local programmer error would be a
judgment.
Ali
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 07:21:18 UTC, Ola Fosheim Grøstad wrote: [...] The reality is that software is layered. Faults at different layers should have different consequences at the discretion of a capable programmer. +1
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 01:43:06 UTC, Steven Schveighoffer
wrote:
[...]
But you aren't perfect, and so maybe you make a mistake, and
trigger an Error. The compiler handles this unexpected
condition by unwinding the stack back to the main function,
printing the error and exiting, so you can go fix whatever
mistake you made.
For this purpose nobody needs a separate subclass named `Error`.
That works with `Exception`s.
[...]
If the code threw an `Exception` instead of an `Error`
everything would be fine.
I think the point of Errors is that you can remove them for
efficiency.
elephant/room.
In other words, just like asserts or bounds checks, they are
not expected to be part of the normal working program.
"removing [Errors, asserts, bounds checks] is a bit like wearing
a life-jacket to practice in the harbour, but then leaving the
life-jackets behind when your ship leaves for open ocean" [1]
[...]
Consider the normal flow of a range in a foreach loop, it's:
```d
// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
If both `popFront` and `front` also always call `empty` you are
calling `empty` 3 times per loop, with an identical value for
the 2nd and 3rd calls.
Solution: Implement explicitly unchecked popFront() and front()
versions.
Having the assert allows diagnosing invalid programs without
crashing your program,
That depends on the understanding of "crashing a program". If
library code throws an `Error` instead of an `Exception` I have
to isolate that code in a subprocess in order to make my program
gracefully handle the error condition.
Think of CGI processes which provide output direct to a customer.
If there is an assert the customer will see the famous Internal
Server Error message (in case of apache httpd).
[...]
[1] http://wiki.c2.com/?AssertionsAsDefensiveProgramming
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote:
[...]
Errors are thrown when the program is discovered to be in an
invalid state.
The following program throws an `Error` in popFront:
import std.range;
void main ()
{
int [1] a;
auto q = a[1..$]; // line 6
q.popFront; // line 7, throws
core.exception.AssertError
}
When the program counter (PC) is at line 6 the program is in a
valid state.
At no time the program is in an invalid state and it would not
pass into an invalid state if popFront threw an `Exception`
instead of an `Error`.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 11:13:48 UTC, Adam D Ruppe wrote: On Sunday, 5 June 2022 at 10:38:44 UTC, Ola Fosheim Grøstad wrote: That is a workaround that makes other languages more attractive. It is what a lot of real world things do since it provides additional layers of protection while still being pretty easy to use. Yes, it is not a bad solution in many cases. It is a classic solution for web servers, but web servers typically don't retain a variety of mutable state of many different types (a webserver can do well with just a shared memcache). My code did and still does simply catch Error and proceed. Most Errors are completely harmless; RangeError, for example, is thrown *before* the out-of-bounds write, meaning it prevented the damage, not just notified you of it. It was fully recoverable in practice before that silly Dec '17 dmd change, and tbh still is after it in a lot of code. Yes, this is pretty much how I write validators of input in Python web services. I don't care if the validator failed or if the input failed, in either case the input has to be stopped, but the service can continue. If there is a suspected logic failure, log and/or send notification to the developer, but for the end user it is good enough that they "for now" send some other input (e.g. avoiding some unicode letter or whatever). Or perhaps redefine RangeError into RangeException, OutOfMemoryError as OutOfMemoryException, and such for the other preventative cases and carry on with joy, productivity, and correctness. For a system level language such decisions ought to be in the hand of the developer so that he can write his own runtime. Maybe some kind of transformers so that libraries can produce Errors, but have them transformed to something else at boundaries. If I want to write an actor-based runtime and do all my application code as 100% @safe actors that are fully «reliable», then that should be possible in a system level language. The programmer's definition and evaluation of «reliability» in the context of a «casual game server» should carry more weight than some out-of-context-reasoning about «computer science» (it isn't actually).
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 10:38:44 UTC, Ola Fosheim Grøstad wrote: That is a workaround that makes other languages more attractive. It is what a lot of real world things do since it provides additional layers of protection while still being pretty easy to use. *Correctness **is** probabilistic.* Even in the case of 100% verified code, as there is a possibility that the spec is wrong. I'm of the opinion that the nothrow implementation right now is misguided. It is a relatively recent change to dmd (merged December 2017). My code did and still does simply catch Error and proceed. Most Errors are completely harmless; RangeError, for example, is thrown *before* the out-of-bounds write, meaning it prevented the damage, not just notified you of it. It was fully recoverable in practice before that silly Dec '17 dmd change, and tbh still is after it in a lot of code. If it was up to me, that patch would be reverted and the spec modified to codify the old status quo. Or perhaps redefine RangeError into RangeException, OutOfMemoryError as OutOfMemoryException, and such for the other preventative cases and carry on with joy, productivity, and correctness.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 00:18:43 UTC, Adam Ruppe wrote: Run it in a separate process with minimum shared memory. That is a workaround that makes other languages more attractive. It does not work when I want to have 1000+ actors in my game server (which at this point most likely will be written in Go, sadly). So a separate process is basically a non-solution. At this point Go seems to be the best technology of all the bad options! A pity, as it is not an enjoyable language IMO, but the goals are more important than the means… The problem here is that people are running an argument as if most D software is control-software for chemical processes or database kernels. Then people quote writings on safety measures that has been evolved in the context/mindset of control-software in the 80s and 90s. And that makes no sense, when only Funkwerk (and possibly 1 or 2 others) write such software in D. The reality is, most languages call C-libraries and have C-code in their runtime, under the assumption that those C-libaries and runtimes have been hardened and proven to be reliable with low probability of failure. *Correctness **is** probabilistic.* Even in the case of 100% verified code, as there is a possibility that the spec is wrong. *Reliability measures are dependent on the used context*. What «reliable» means depends on skilled judgment utilized to evaluate the software in the use context. «reliable» is not a context independent absolute.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 07:28:52 UTC, Sebastiaan Koppe wrote: Go has panic. Other languages have similar constructs. And recover. So D will never be able to provide actors and provide fault tolerance. I guess it depends on the programmer. But it isn’t if you cannot prevent Error from propagating. Is the service in a usable state? Yes, the actor code failed. The actor code change frequently, not the runtime kernel. The actor code is free to call anything, including @system @trusted code. How is this different from FFI in other languages? As a programmer you make a judgment. The D argument is to prevent the programmer from making a judgment? How would the actor framework know when an error is due to a silly bug in an isolated How can I know that a failure in Python code isn’t caused by C There is no difference in the situation. I make a judgment as a programmer.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 07:21:18 UTC, Ola Fosheim Grøstad wrote: You can make the same argument for an interpreter: if an assert fails in the intrrpreter code then that could be a fault in the interpreter therefore you should shut down all programs being run by that interpreter. Typo: if an assert fails in the interpreted code, then that could be a sign that the interpreter itself is flawed. Should you then stop all programs run by that interpreter? The point: in the real world you need more options than the nuclear option. Pessimizing everywhere is not giving higher satisfaction for the end user. (Iphone keyboard)
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 06:31:42 UTC, Ola Fosheim Grøstad wrote: On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote: That is not very probable in 100% @safe code. You are basically saying that D cannot compete with Go and other «safe» languages. Go has panic. Other languages have similar constructs. So D will never be able to provide actors and provide fault tolerance. I guess it depends on the programmer. Is the service in a usable state? Yes, the actor code failed. The actor code change frequently, not the runtime kernel. The actor code is free to call anything, including @system code. How would the actor framework know when an error is due to a silly bug in an isolated function or if it is something more serious?
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 03:43:16 UTC, Paul Backus wrote: See here: https://bloomberg.github.io/bde-resources/pdfs/Contracts_Undefined_Behavior_and_Defensive_Programming.pdf Not all software is banking applications. If an assert fails that means that the program logic is wrong, not that the program is in an invalid state. (Invalid state is a stochastic consequence and detection can happen mich later). So that means that you should just stop the program. It means that you should shut down all running instances of that program on all computers across the globe. That is the logical consequence of this perspective, and it makes sense for a banking database. It does not make sense for the constructor of Ants in a computer game service. It is better to have an enjoyable game delivered with fewer ants than a black screen all weekend. You can make the same argument for an interpreter: if an assert fails in the intrrpreter code then that could be a fault in the interpreter therefore you should shut down all programs being run by that interpreter. The reality is that software is layered. Faults at different layers should have different consequences at the discretion of a capable programmer.
Re: Comparing Exceptions and Errors
On Sunday, 5 June 2022 at 00:40:26 UTC, Ali Çehreli wrote: Errors are thrown when the program is discovered to be in an invalid state. We don't know what happened and when. For example, we don't know whether the memory has been overwritten by some rogue code. That is not very probable in 100% @safe code. You are basically saying that D cannot compete with Go and other «safe» languages. Dereferencing a null pointer usually means that some code failed to create an instance and check for it. My code can detect that the failure is local under the assumption thay the runtime isnt a piece of trash. What happened? What can we assume. We don't know and we cannot assume any state. So D will never be able to provide actors and provide fault tolerance. Is the service in a usable state? Yes, the actor code failed. The actor code change frequently, not the runtime kernel. Possibly. Not shutting down might produce incorrect results. Do we prefer up but incorrect or dead? I prefer that service keeps running: chat service, game service, data delivered with hashed «checksum». Not all software are database engines where you have to pessimize about bugs in the runtime kernel. If the data delivered is good enough for the client and better than nothing then the service should keep running!!! I hope there is a way of aborting the program when there are invariant Invariants are USUALLY local. I dont write global spaghetti code. As a programmer you should be able to distinguish between local and global failure. You are assuming that the programmer is incapable of making judgements. That is assuming way too much.
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 22:03:08 UTC, kdevel wrote: On Saturday, 4 June 2022 at 14:05:14 UTC, Paul Backus wrote: This is entirely a question of API design. If it should be the caller's responsibility to check for some condition before calling the function, then you can throw an `Error` when that condition does not hold (or more likely, use an `assert` or an `in` contract to check for it). Provided one does not catch `Error`s this means one has to isolate such an API design by using a subprocess. This is what one usually tries to avoid. See here: http://joeduffyblog.com/2016/02/07/the-error-model/#bugs-arent-recoverable-errors And also the following section: http://joeduffyblog.com/2016/02/07/the-error-model/#reliability-fault-tolerance-and-isolation If it should be the callee's responsibility to check, you should throw an `Exception` (or use `enforce`). If the library always throws exceptions it can be used in both API "designs". In the case that the implementor of the caller expects `Error`s instead of `Exceptions` she could use a small wrapper which catches the Exceptions and rethrows them as `Error`s. Likewise for error codes. Using contracts and invariants impedes this approach. See here: https://bloomberg.github.io/bde-resources/pdfs/Contracts_Undefined_Behavior_and_Defensive_Programming.pdf
Re: Comparing Exceptions and Errors
On 6/4/22 9:43 PM, Steven Schveighoffer wrote: But I think you are still not supposed to continue execution. I'm not sure what a compiler might assume at this point, and I unfortunately can't find in the language specification where it states this. It might not be in there at all, the spec is sometimes lacking compared to the implementation. BTW, I think this is the main reason why it keeps coming up for D learners, and why I wrote the article in the first place. It would be good (if it's not already in the spec) to have something mentioned about the pitfalls of catching Errors. -Steve
Re: Comparing Exceptions and Errors
On 6/4/22 6:56 PM, kdevel wrote:
On Saturday, 4 June 2022 at 16:55:31 UTC, Steven Schveighoffer wrote:
[...]
The point of an `Error` is that your code can assume it cannot happen.
If it does happen, the code is invalid.
According to my favorite dictionary "assume" means "take for granted"
[1]. If `Error`s may happen how can code (or its author) "assume" that
`Error`s cannot happen?
You don't assume it, you guarantee it. You are expected to provide a
guarantee to the compiler that your code won't throw these errors.
But you aren't perfect, and so maybe you make a mistake, and trigger an
Error. The compiler handles this unexpected condition by unwinding the
stack back to the main function, printing the error and exiting, so you
can go fix whatever mistake you made.
It's kind of like a segfault. There's no valid reason to read memory you
don't own (yes, I know you can use segfaults to trigger loading of
memory, I'm not talking about that kind of segfault). So what do you do
when an unexpected segfault happens? You crash the program, and exit. In
this case, the language is giving you by default a hint about where it
occurred, and if you desire, you can get more information by catching
the error where you want and doing more checks, etc.
This is reflected in the fact that the compiler will omit cleanup code
if an `Error` is thrown (it can assume that it will never happen).
But instead the compiler should *emit* the cleanup code and we would not
have to discuss here carefully avoiding to name the root cause of all
this entanglements.
A compiler *could* do this, and in fact, the compiler used to do this.
But I think you are still not supposed to continue execution. I'm not
sure what a compiler might assume at this point, and I unfortunately
can't find in the language specification where it states this. It might
not be in there at all, the spec is sometimes lacking compared to the
implementation.
However, I did ask Walter about this last beerconf, and he said to treat
a throw/catch of an error like a goto, anything can happen.
The point of using `Error` is for a last resort check for program
correctness (because you failed to validate the input before getting
to that point).
If the code threw an `Exception` instead of an `Error` everything would
be fine.
I think the point of Errors is that you can remove them for efficiency.
In other words, just like asserts or bounds checks, they are not
expected to be part of the normal working program. Exceptions are part
of the program, and provide a different mechanism of handling error
conditions.
[...]
A great example are range functions. Often times you see at the
beginning of any `popFront` method the statement `assert(!empty);`.
This is universally accepted, as you shouldn't be calling `popFront`
if you haven't checked for `empty`.
Yep.
```
core.exception.AssertError@[...]linux/bin64/../../src/phobos/std/range/primitives.d(2280):
Attempting to popFront() past the end of an array of int
```
I see no difference to the potentially invalid array index case. It
would ease the use of the range if it threw an `Exception`.
But it's possible to turn off asserts and make the code run faster. I
personally never turn them off on certain programs (web server) because
the penalty is not noticeable enough. But if these were Exceptions, they
*could not be turned off*.
Consider the normal flow of a range in a foreach loop, it's:
```d
// foreach(elem; range)
for(auto r = range; !r.empty; r.popFront) {
auto elem = r.front;
}
```
If both `popFront` and `front` also always call `empty` you are calling
`empty` 3 times per loop, with an identical value for the 2nd and 3rd
calls. Having the assert allows diagnosing invalid programs without
crashing your program, but also allowing full performance when you want it.
Phobos' `RedBlackTree` has an `invariant` which walks the entire RBT and
validates the red-black property holds *before and after every method
call*. This is not what you would want for performant code as it
completely destroys the complexity guarantees. Yet it's there to help
diagnose problems with RBT if you are working on modifying it. These
kinds of checks are to help the developer prove their code is correct
without having to continually prove it's correct for normal use.
-Steve
Re: Comparing Exceptions and Errors
On 6/4/22 10:17, Ola Fosheim Grøstad wrote: > Why can't Error unwind the stack properly? Errors are thrown when the program is discovered to be in an invalid state. We don't know what happened and when. For example, we don't know whether the memory has been overwritten by some rogue code. Or perhaps a bit got flipped in memory. When the state of the program is discovered to be outside of what we think is normal, we cannot execute further code. It would be madness to wish that some cleanup code would do the right thing when e.g. we were sure that an array would never be empty but we found it to be empty. What happened? What can we assume. We don't know and we cannot assume any state. (As has been stated many times in this thread and elsewhere, Exceptions are different. They don't have anything to do with invariants.) > In a not-miniscule service you can be pretty certain that some ±1 bugs > will be there, especially in a service that is receiving new features on > a regular basis. So, if you get an index/key error/null-dereferencing > that wasn't checked for, unwinding that actor/task/handler makes sense, > shutting down the service doesn't make sense. Is the service in a usable state? > If you allow the whole service to go down then you have opened a > Denial-of-Service vector Possibly. Not shutting down might produce incorrect results. Do we prefer up but incorrect or dead? > I am not a fan of Go, but it is difficult to find a more balanced > solution, and Go 1.18 has generics, so it is becoming more competitive! I hope there is a way of aborting the program when there are invariant violations discovered. Ali
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 22:31:38 UTC, Ola Fosheim Grøstad wrote: So what do you have to do to avoid having Errors thrown? How do you make your task/handler fault tolerant in 100% @safe code? Run it in a separate process with minimum shared memory.
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 16:55:31 UTC, Steven Schveighoffer wrote: [...] The point of an `Error` is that your code can assume it cannot happen. If it does happen, the code is invalid. According to my favorite dictionary "assume" means "take for granted" [1]. If `Error`s may happen how can code (or its author) "assume" that `Error`s cannot happen? That makes absolutely no sense to me. This is reflected in the fact that the compiler will omit cleanup code if an `Error` is thrown (it can assume that it will never happen). But instead the compiler should *emit* the cleanup code and we would not have to discuss here carefully avoiding to name the root cause of all this entanglements. The point of using `Error` is for a last resort check for program correctness (because you failed to validate the input before getting to that point). If the code threw an `Exception` instead of an `Error` everything would be fine. [...] I actually replaced some arrays with an `Exception` throwing wrapper because I didn't want to crash the whole server for certain cases, and I didn't want to continuously validate array indexes. +1 [...] A great example are range functions. Often times you see at the beginning of any `popFront` method the statement `assert(!empty);`. This is universally accepted, as you shouldn't be calling `popFront` if you haven't checked for `empty`. Yep. ``` core.exception.AssertError@[...]linux/bin64/../../src/phobos/std/range/primitives.d(2280): Attempting to popFront() past the end of an array of int ``` I see no difference to the potentially invalid array index case. It would ease the use of the range if it threw an `Exception`. [1] https://www.thefreedictionary.com/assume
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 22:01:57 UTC, Steven Schveighoffer wrote: You shouldn't retry on Error, and you shouldn't actually have any Errors thrown. So what do you have to do to avoid having Errors thrown? How do you make your task/handler fault tolerant in 100% @safe code?
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 14:05:14 UTC, Paul Backus wrote: [...] What does that mean? Am I `Error` blind? Generally you do not need to subclass `Error` yourself. The most common way of throwing an `Error` in user code is to use `assert`, which (with default compiler flags) throws an `AssertError` on failure. Function contracts and struct/class invariants work the same way. `git grep -Enw 'assert|unittest'` reveals that my code contains assert statements only in unittests. Someone (was it Walter?) once pointed out that asserts are ignored in relase mode (like C assert) and that for my purposes `enforce` (i.e. throw an Exception) is best suited. 3. Can you provide some piece of code which *must* throw `Error` and cannot throw an appropriate Exception? This is entirely a question of API design. If it should be the caller's responsibility to check for some condition before calling the function, then you can throw an `Error` when that condition does not hold (or more likely, use an `assert` or an `in` contract to check for it). Provided one does not catch `Error`s this means one has to isolate such an API design by using a subprocess. This is what one usually tries to avoid. If it should be the callee's responsibility to check, you should throw an `Exception` (or use `enforce`). If the library always throws exceptions it can be used in both API "designs". In the case that the implementor of the caller expects `Error`s instead of `Exceptions` she could use a small wrapper which catches the Exceptions and rethrows them as `Error`s. Likewise for error codes. Using contracts and invariants impedes this approach.
Re: Comparing Exceptions and Errors
On 6/4/22 2:46 PM, Ola Fosheim Grøstad wrote: On Saturday, 4 June 2022 at 18:32:48 UTC, Sebastiaan Koppe wrote: Most wont throw a Error though. And typical services have canary releases and rollback. So you just fix it, which you have to do anyway. I take it you mean manual rollback, but the key issue is that you want to retry on failure. Not infrequently the source for the failure will be in the environment, the code just didn't handle the failure correctly. You shouldn't retry on Error, and you shouldn't actually have any Errors thrown. I'll draw a line in the sand here -- OutOfMemoryError shouldn't be an Error, but an Exception. Because there's no way you can check if an allocation will succeed before doing it, and arguably, there are ways to deal with out of memory problems without shutting down the process. On a service with SLA of 99.999% the probable "failure time" would be 6 seconds per week, so if you can retry you may still run fine even if you failed to check correctly for an error on that specific subsystem. That makes the system more resilient/robust. Exceptions are perfectly fine to catch and retry. Anticipating the failing condition, and throwing an exception instead is a viable solution. -Steve
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 18:32:48 UTC, Sebastiaan Koppe wrote: Most wont throw a Error though. And typical services have canary releases and rollback. So you just fix it, which you have to do anyway. I take it you mean manual rollback, but the key issue is that you want to retry on failure. Not infrequently the source for the failure will be in the environment, the code just didn't handle the failure correctly. On a service with SLA of 99.999% the probable "failure time" would be 6 seconds per week, so if you can retry you may still run fine even if you failed to check correctly for an error on that specific subsystem. That makes the system more resilient/robust.
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 17:17:13 UTC, Ola Fosheim Grøstad wrote: Why can't Error unwind the stack properly? It does normally, but it doesn't destruct objects when those are in `nothrow` functions. Nothrow functions don't throw, so have no cleanup. You could argue it is strange that assert throws... In a not-miniscule service you can be pretty certain that some ±1 bugs will be there, especially in a service that is receiving new features on a regular basis. Most wont throw a Error though. And typical services have canary releases and rollback. So you just fix it, which you have to do anyway. Not saying its perfect, but if you only use asserts when you have to, and handle other things using the type system, it doesn't actually happen all that often.
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 16:55:50 UTC, Sebastiaan Koppe wrote: The reasoning is simple: Error + nothrow will sidestep any RAII you may have. Since you cannot know what potentially wasn't destructed, the only safe course of action is to abandon ship. Why can't Error unwind the stack properly? Yes, in plenty of cases that is completely overkill. Then again, programs should be written to not assert in the first place. In a not-miniscule service you can be pretty certain that some ±1 bugs will be there, especially in a service that is receiving new features on a regular basis. So, if you get an index/key error/null-dereferencing that wasn't checked for, unwinding that actor/task/handler makes sense, shutting down the service doesn't make sense. If you allow the whole service to go down then you have opened a Denial-of-Service vector, which is a problem if the service is attracting attention from teens/immature adults. (E.g. games, social apps, political sites, educational sites etc). Considering most asserts I have seen are either due to a bad api or just laziness - and shouldn't have to exist in the first place - maybe it's not that bad. Well, problem is if a usually reliable subsystem is intermittently flaky, and you get this behaviour, then that isn't something you can assume will be caught in tests (you cannot test for all scenarios, only the likely ones). I am not a fan of Go, but it is difficult to find a more balanced solution, and Go 1.18 has generics, so it is becoming more competitive! At the end of the day you don't have to love a language to choose it… and for a service, runtime behaviour is more important than other issues.
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 01:17:28 UTC, Steven Schveighoffer wrote: If a thread does not catch an error and end the program, that's a defect in druntime I think. If it tries to rethrow the exception in the main thread (oh, man I have to check... Yeah, this is what it does), then it's entirely possible that the main thread will never even get to the `Error`! Yes, for that reason, and others, people should not use that api. If we are paranoid and want to do as little as possible, at least we should attempt to copy a string literal to stderr. Something like "Thread exiting with Error." And exit(1) right after that. Yes, that is what it should do. Probably yes.
Re: Comparing Exceptions and Errors
On 6/4/22 7:57 AM, kdevel wrote: On Friday, 3 June 2022 at 23:40:50 UTC, Steven Schveighoffer wrote: During the last beerconf, I wrote a short blog post about how `Error` and `Exception` are different, and why you should never continue after catching `Error`s. Feedback welcome, I didn't announce here when I wrote it because it's kind of small/insignificant, but maybe it can help newcomers to the language: https://www.schveiguy.com/blog/2022/05/comparing-exceptions-and-errors-in-d/ Here my feedback: 1. What if div is called with x = -2147483648 and y = -1? Isn't code which allows a divisor == 0 to propagate to the CPU an error? Must the code thus not throw an object instantiated from a subclass of `Error`? Well, that is just a toy example to show code that might use an Error. It's not intended to be a fully-fleshed-out function. I recommend simply using the divide operator in real code. The point of an `Error` is that your code can assume it cannot happen. If it does happen, the code is invalid. This is reflected in the fact that the compiler will omit cleanup code if an `Error` is thrown (it can assume that it will never happen). The point of using `Error` is for a last resort check for program correctness (because you failed to validate the input before getting to that point). What if I have that function div used in code which is called from say controller code of a CGI binary. Or likewise from a vibe.d-thread servicing a web request? How do I isolate that fault? Do I have to spawn a subprocess as Walter suggested in the case of memory corruption [1]? [This is of course all rhetorical!] vibe should exit the process if an `Error` is thrown. There is a version you can specify to have it catch `Error`, but it would only be for debugging. https://vibed.org/docs#compile-time-configuration (see `VibeDebugCatchAll`) One thing that always bugs me in my vibe code is out of bounds errors for arrays. I actually replaced some arrays with an `Exception` throwing wrapper because I didn't want to crash the whole server for certain cases, and I didn't want to continuously validate array indexes. 2. Since 2017 or so I have written some 10 KLOC of D, maybe about two dozen classes deriving from Exception. But I did not annotate any of my methods or function with "nothrow" nor did I author any class deriving from `Error`. What does that mean? Am I `Error` blind? As long as you aren't catching `Throwable` or `Error`, you should be fine. Simply not marking things `nothrow` doesn't mean that they won't be inferred `nothrow`. `auto` and template functions are inferred. In practice, there are probably very very few places where this can bite you. Which also means, if it does bite, it's going to be really really hard to track down. 3. Can you provide some piece of code which *must* throw `Error` and cannot throw an appropriate Exception? As Paul said, this is up to your API. If you specify that you assume the inputs to the function are cleansed, then you can correctly throw an Error if they are out of spec. A great example are range functions. Often times you see at the beginning of any `popFront` method the statement `assert(!empty);`. This is universally accepted, as you shouldn't be calling `popFront` if you haven't checked for `empty`. -Steve
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 14:19:22 UTC, Ola Fosheim Grøstad wrote: Also, what it is the purpose of @safe if you have to kill all threads? Such rigidity will just make Go look all the more attractive for service providers! I agree with this, but given the current semantics there is nothing else to do but teardown everything upon first sight of Error. The reasoning is simple: Error + nothrow will sidestep any RAII you may have. Since you cannot know what potentially wasn't destructed, the only safe course of action is to abandon ship. Yes, in plenty of cases that is completely overkill. Then again, programs should be written to not assert in the first place. Considering most asserts I have seen are either due to a bad api or just laziness - and shouldn't have to exist in the first place - maybe it's not that bad.
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 01:17:28 UTC, Steven Schveighoffer wrote: I've thought in the past that throwing an error really should not throw, but log the error (including the call stack), and then exit without even attempting to unwind the stack. But code at least expects an attempt to throw the Error up the stack, so code that is expecting to catch it would break. This is too harsh for a service that is read-only, meaning a service that only read from a database and never writes to it. All running threads have to be given a chance to exit gracefully, at the very minimum. What is the source for these errors anyway? A filesystem not responding? A crashed device driver? A race condition? A deadlock? Starvation? Many sources for errors can be recovered from by rescheduling in a different order at a different time. What I'd like to see is a fault tolerant 100% @safe actor pattern with local per-actor GC. By fault tolerant I mean that the actor is killed and then a new actor is rescheduled (could be an alternative "reference" implementation or the same after a time delay). Also, what it is the purpose of @safe if you have to kill all threads? Such rigidity will just make Go look all the more attractive for service providers!
Re: Comparing Exceptions and Errors
On Saturday, 4 June 2022 at 11:57:32 UTC, kdevel wrote: 2. Since 2017 or so I have written some 10 KLOC of D, maybe about two dozen classes deriving from Exception. But I did not annotate any of my methods or function with "nothrow" nor did I author any class deriving from `Error`. What does that mean? Am I `Error` blind? Generally you do not need to subclass `Error` yourself. The most common way of throwing an `Error` in user code is to use `assert`, which (with default compiler flags) throws an `AssertError` on failure. Function contracts and struct/class invariants work the same way. 3. Can you provide some piece of code which *must* throw `Error` and cannot throw an appropriate Exception? This is entirely a question of API design. If it should be the caller's responsibility to check for some condition before calling the function, then you can throw an `Error` when that condition does not hold (or more likely, use an `assert` or an `in` contract to check for it). If it should be the callee's responsibility to check, you should throw an `Exception` (or use `enforce`).
Re: Comparing Exceptions and Errors
On Friday, 3 June 2022 at 23:40:50 UTC, Steven Schveighoffer wrote: During the last beerconf, I wrote a short blog post about how `Error` and `Exception` are different, and why you should never continue after catching `Error`s. Feedback welcome, I didn't announce here when I wrote it because it's kind of small/insignificant, but maybe it can help newcomers to the language: https://www.schveiguy.com/blog/2022/05/comparing-exceptions-and-errors-in-d/ -Steve Here my feedback: 1. What if div is called with x = -2147483648 and y = -1? Isn't code which allows a divisor == 0 to propagate to the CPU an error? Must the code thus not throw an object instantiated from a subclass of `Error`? What if I have that function div used in code which is called from say controller code of a CGI binary. Or likewise from a vibe.d-thread servicing a web request? How do I isolate that fault? Do I have to spawn a subprocess as Walter suggested in the case of memory corruption [1]? [This is of course all rhetorical!] 2. Since 2017 or so I have written some 10 KLOC of D, maybe about two dozen classes deriving from Exception. But I did not annotate any of my methods or function with "nothrow" nor did I author any class deriving from `Error`. What does that mean? Am I `Error` blind? 3. Can you provide some piece of code which *must* throw `Error` and cannot throw an appropriate Exception? [1] http://forum.dlang.org/post/t6ef8c$1cu5$1...@digitalmars.com Re: Why is D unpopular?
Re: Comparing Exceptions and Errors
On 6/3/22 18:17, Steven Schveighoffer wrote:
> If a thread does not catch an error and end the program
No, the thread dies and the main thread doesn't know anything about it.
Unless if std.concurrency is being used and the main thread looks for
and receives a LinkTerminated message. (If spawnLinked was used to start
the thread.)
But still, there is no trace of the Error. So I do the following in my
thread functions:
void myThread() {
try {
} catch (Exception exc) {
// ...
} catch (Error exc) {
// Report on stderr
exit(1); // Yes, takes down the main thread.
// Could choose to do otherwise...
}
}
Ali
Re: Comparing Exceptions and Errors
On 6/3/22 8:44 PM, Ali Çehreli wrote: One feedback I have is about non-main threads. Although everybody agrees that Errors should not be caught, the main thread does so and prints the useful output that you show in the article. Yes, the one exception to the no-catch error rule is to print/log diagnostic information. But you should never *continue executing* after that. If the main thread is allowed to do that, we should do as much (or less) in non-main threads as well. Otherwise, threads disappear without a trace. If a thread does not catch an error and end the program, that's a defect in druntime I think. If it tries to rethrow the exception in the main thread (oh, man I have to check... Yeah, this is what it does), then it's entirely possible that the main thread will never even get to the `Error`! If we are paranoid and want to do as little as possible, at least we should attempt to copy a string literal to stderr. Something like "Thread exiting with Error." And exit(1) right after that. Yes, that is what it should do. But all of that is wishful programming because if the program is in an illegal state, should we attempt to report it? If we should not, shouldn't the main thread should not either? (Sorry, non-native Inglish speaker here. :p) It's useful to do *something*, so it's not just an empty log file with no idea where the error occurred. I've thought in the past that throwing an error really should not throw, but log the error (including the call stack), and then exit without even attempting to unwind the stack. But code at least expects an attempt to throw the Error up the stack, so code that is expecting to catch it would break. -Steve
Re: Comparing Exceptions and Errors
On 6/3/22 16:40, Steven Schveighoffer wrote: https://www.schveiguy.com/blog/2022/05/comparing-exceptions-and-errors-in-d/ Cool! I didn't know catching Errors behaves specially in unittest blocks and contracts. One feedback I have is about non-main threads. Although everybody agrees that Errors should not be caught, the main thread does so and prints the useful output that you show in the article. If the main thread is allowed to do that, we should do as much (or less) in non-main threads as well. Otherwise, threads disappear without a trace. If we are paranoid and want to do as little as possible, at least we should attempt to copy a string literal to stderr. Something like "Thread exiting with Error." And exit(1) right after that. But all of that is wishful programming because if the program is in an illegal state, should we attempt to report it? If we should not, shouldn't the main thread should not either? (Sorry, non-native Inglish speaker here. :p) Ali
