Re: Exhaustiveness in switch

[Remi Forax]

- Mail original -
> De: "Brian Goetz" <brian.go...@oracle.com>
> À: "amber-spec-experts" <amber-spec-experts@openjdk.java.net>
> Envoyé: Jeudi 10 Mai 2018 22:12:37
> Objet: Exhaustiveness in switch

> In the long (and indirect) thread on "Expression Switch Exception
> Naming", we eventually meandered around to the question of when should
> the compiler deem an expression switch to be exhaustive, and therefore
> emit a catch-all throwing default. Let's step back from this for a bit
> and remind ourselves why we care about this.
> 
> Superficially, having to write a throwing default for a condition we
> believe to be impossible is annoying:
> 
>     switch (trafficLight) {
>     case RED -> stop();
>     case GREEN -> driveFast();
>     case YELLOW -> driveFaster();
>     default -> throw new ExasperationException("No, we haven't
> added any new traffic light colors since the invention of the
> automobile, so I have no idea what " + trafficLight + " is");
>     }
> 
> The annoyance here, though, is twofold:
>  - I have to write code for something which I think can't happen;
>  - That code is annoying to write.
> 
> In the above, we "knew" another traffic light color was impossible, and
> we listed them all -- and the compiler knew it. This is particularly
> irritating.  However, we often also see cases like this:
> 
>     void processVowel(letter) {
>         switch (letter) {
>             case A: ...
>     case E: ...
>         case I: ...
>             case O: ...
>     case U: ...
>         default: throw new IllegalStateException("Not a vowel: " +
> letter);
>     }
> 
> Here, the annoyance is slightly different, in that I could not
> reasonably expect the compiler to know I'd covered all the vowels.  In
> fact, I think the explicit exception in this case is useful, in that it
> documents an invariant known to the programmer but not captured in the
> type system.  But it is still annoying that I have to construct a format
> string, construct an exception, and throw it; if there were easier ways
> to do that, I might be less annoyed.  Without diving into the bikeshed,
> maybe this looks something like:
> 
>     default: throw IllegalStateException.format("Not a vowel: %s",
> vowel);
> 
> The details aren't relevant, but the point is: maybe a small-ish library
> tweak would reduce the annoyance of writing such clauses. (This one
> isn't so bad, but Dan excavated a bunch that were way worse.)  But,
> let's set this aside for a moment, and return back to the point of why
> we want the compiler to provide a throwing default.
> 
> I think most of the discussion has centered on the problem of a novel
> value showing up at runtime.  This is surely an issue, and must be dealt
> with, but the central issue is: a default is never able to distinguish
> between a runtime-novel value and a value we just forgot to include at
> compile time.  It doesn't matter whether this default throws (as the
> implicit default in an expression switch) or does nothing (as the
> implicit default in statement switches today does).
> 
> We agreed that we should not require the user to provide a default when
> they provide case clauses that cover the target type as of compile time
> (true+false for boolean, all the members of a sealed type, etc.)  This
> is because the default you'd be forced to put in otherwise (for
> expression switches) is actually harmful; if the type were later
> modified to have more values, an explicit default would swallow them,
> rather than yielding an error at recompilation time.  So it is not only
> annoying, but actually could cover up errors.
> 
> We then went off on the wrong tangent, though, where we wondered whether
> it was OK to implicitly assume enums were sealed, since some enums are
> clearly intended to acquire new values.  But the mistake was focusing on
> the wrong aspect of sealed-ness (the statement of intent to not add more
> values), rather than the compiler's ability to reason credibly about
> known possible values.
> 
> So, backing up, I think we should always treat a "complete" enum
> expression switch specially -- don't require a default, and implicitly
> add a throwing one, if all the cases are specified. This way, if the
> assumption that you've covered all the cases is later broken via
> separate compilation, on recompilation, you'll discover this early,
> rather than at runtime.  (You'll still get runtime protection either
> way.)  Regardless of whether we think the enum will be extended in the
> future or not.  There's no need for enums to de

Re: Exhaustiveness in switch

[Remi Forax]

> De: "Kevin Bourrillion" <kev...@google.com>
> À: "Brian Goetz" <brian.go...@oracle.com>
> Cc: "amber-spec-experts" <amber-spec-experts@openjdk.java.net>
> Envoyé: Jeudi 10 Mai 2018 22:44:33
> Objet: Re: Exhaustiveness in switch

> * I think that for most occurrences of `default: throw...`, by far, the user
> really doesn't benefit from being able to choose the exception type or the
> message at all. A standardized choice of exception, and an autogenerated
> message (that automatically includes the switch target, which users usually
> don't bother to do themselves!), may be strictly better.

+1, 
as one of my student put it, it's important to always go banana predictably. 

Rémi 

> On Thu, May 10, 2018 at 1:12 PM, Brian Goetz < [ 
> mailto:brian.go...@oracle.com |
> brian.go...@oracle.com ] > wrote:

>> In the long (and indirect) thread on "Expression Switch Exception Naming", we
>> eventually meandered around to the question of when should the compiler deem 
>> an
>> expression switch to be exhaustive, and therefore emit a catch-all throwing
>> default. Let's step back from this for a bit and remind ourselves why we care
>> about this.

>> Superficially, having to write a throwing default for a condition we believe 
>> to
>> be impossible is annoying:

>> switch (trafficLight) {
>> case RED -> stop();
>> case GREEN -> driveFast();
>> case YELLOW -> driveFaster();
>> default -> throw new ExasperationException("No, we haven't added any new 
>> traffic
>> light colors since the invention of the automobile, so I have no idea what " 
>> +
>> trafficLight + " is");
>> }

>> The annoyance here, though, is twofold:
>> - I have to write code for something which I think can't happen;
>> - That code is annoying to write.

>> In the above, we "knew" another traffic light color was impossible, and we
>> listed them all -- and the compiler knew it. This is particularly irritating.
>> However, we often also see cases like this:

>> void processVowel(letter) {
>> switch (letter) {
>> case A: ...
>> case E: ...
>> case I: ...
>> case O: ...
>> case U: ...
>> default: throw new IllegalStateException("Not a vowel: " + letter);
>> }

>> Here, the annoyance is slightly different, in that I could not reasonably 
>> expect
>> the compiler to know I'd covered all the vowels. In fact, I think the 
>> explicit
>> exception in this case is useful, in that it documents an invariant known to
>> the programmer but not captured in the type system. But it is still annoying
>> that I have to construct a format string, construct an exception, and throw 
>> it;
>> if there were easier ways to do that, I might be less annoyed. Without diving
>> into the bikeshed, maybe this looks something like:

>> default: throw IllegalStateException.format("Not a vowel: %s", vowel);

>> The details aren't relevant, but the point is: maybe a small-ish library 
>> tweak
>> would reduce the annoyance of writing such clauses. (This one isn't so bad, 
>> but
>> Dan excavated a bunch that were way worse.) But, let's set this aside for a
>> moment, and return back to the point of why we want the compiler to provide a
>> throwing default.

>> I think most of the discussion has centered on the problem of a novel value
>> showing up at runtime. This is surely an issue, and must be dealt with, but 
>> the
>> central issue is: a default is never able to distinguish between a
>> runtime-novel value and a value we just forgot to include at compile time. It
>> doesn't matter whether this default throws (as the implicit default in an
>> expression switch) or does nothing (as the implicit default in statement
>> switches today does).

>> We agreed that we should not require the user to provide a default when they
>> provide case clauses that cover the target type as of compile time 
>> (true+false
>> for boolean, all the members of a sealed type, etc.) This is because the
>> default you'd be forced to put in otherwise (for expression switches) is
>> actually harmful; if the type were later modified to have more values, an
>> explicit default would swallow them, rather than yielding an error at
>> recompilation time. So it is not only annoying, but actually could cover up
>> errors.

>> We then went off on the wrong tangent, though, where we wondered whether it 
>> was
>> OK to implicitly assume enums were sealed, since some enums are clearly
>> intended to acquire new value

Re: Exhaustiveness in switch

[Brian Goetz]

OK, but consider these cases.

Case 1:
    enum E { A, B }
    ...

    switch (e) {
    case A -> ...
    case B -> ...
    default: throw ...;
    }

Now, we add "C" to E and recompile.  No compilation error, and it throws 
on input C.


Case 2:

    enum E { A, B }
    ...

    switch (e) {
    case A -> ...
    case B -> ...
    // implicit throwing default
    }

Again, we add C and recompile.  Now we get a compilation error, because 
the switch isn't exhaustive.


Case 3: compiler doesn't try to reason about exhaustiveness of enums.

    enum E { A, B }
    ...

    switch (e) {
    case A -> ...
    case B -> ...
    }

And we get a compilation error because there's no default.

I was under the impression you were a big fan of (2), because then the 
compiler notifies you when your switch becomes non-exhaustive.  With an 
explicit default (1), or no implicit throwing default (3), you lose that.


Can you clarify?

On 5/10/2018 5:05 PM, Kevin Bourrillion wrote:
The only distinction I am discussing now is whether it is implicit or 
explicit.  It was never my intention to argue that the benefits came 
from the implicitness - I just want there to be a way to choose 
between the two behaviors.

Re: Exhaustiveness in switch

[Dan Smith]

> On May 10, 2018, at 2:46 PM, Guy Steele  wrote:
> 
>> On May 10, 2018, at 4:54 PM, Brian Goetz > > wrote:
>> 
>>> * It would feel strange to even bother applying this exhaustiveness goo to 
>>> byte switches. If we ever had ranges of course then, any type of switch 
>>> could join the party. (I don't know whether ranges are a thing we're 
>>> considering or not and I'm not pushing that we do.)
>> 
>> Yeah, its on the edge.  Its a no-brainer for `boolean`, its nuts for `int` 
>> (without ranges), but its vaguely defensible for `byte`.  Though I can't 
>> really get too excited about it.
> 
> The choice is not just among four sizes of integer.  One could imagine 
> recognizing certain idioms such as
> 
>   switch (myInt & 7) {
> case 2, 3, 5, 7 -> “prime”;
> case 0, 1, 4 -> “square”;
> case 6 -> “perfect”;
>   }
> 
> and understanding that they are exhaustive.  Dunno if the compiler guys want 
> to go there.

We can teach the compiler a few properties of primitive numbers, but we can't 
effectively teach it about a random WidgetStateMachine. (Like, maybe if one 
field is null, then the WidgetState will be one of 3 possible enum values, out 
of 33 total. This is totally obvious to people who know about 
WidgetStateMachines, why does the compiler make me write extra code?!?)

But an idea I've raised is that while the compiler's ability to prove 
exhaustiveness will always be limited, a user might appreciate _asserting_ 
exhaustiveness in some extremely lightweight way. That would opt out of any 
compile-time checking ("looks like you missed a case"), and instead, like a 
cast, would give you runtime checks for unexpected inputs ("you told me I 
wouldn't see this value, but I did").

Brian's response to this, conveyed in the OP, is that he sees forcing an 
explicit "default -> unexpected input" case as a feature, not a bug. Which, 
yeah, I can see a reader sometimes wanting an explanation in an error message. 
But sometimes I think you just want to implicitly assert something you expect 
to be true ("this Object is a String", "this value can be dereferenced", "this 
number is a valid array index").

The bottom line is that it's not clear whether that extra modifier on switch 
(or whatever) which would opt in to runtime checking is worth the extra 
complexity. But maybe there's something there, in conjunction with a modifier 
(or whatever) which would opt in to compile-time checking.

Re: Exhaustiveness in switch

[Brian Goetz]

* Users of Guava's Preconditions love the support for 
"%s"-formatting.  But on the other hand, in code where you /are 
definitely/ about to throw something, why wouldn't I just use string 
concatenation? That is generally easier to read because you don't have 
to mentally correlate specifiers to faraway arguments. (Okay: one 
reason might be if your message is long enough that you'd like to use 
a multi-line raw string literal for it, where concatenation is 
BadBadBad, but I can't say how common this is or isn't at the moment.)


I think this is YMMV.  Some users (this one included) always feel dirty 
using string concatenation.  I would prefer to use formatting almost all 
the time.  That exceptions don't incorporate formatting is mostly an 
artifact of the fact that the last big overhaul in exception APIs was in 
1.4 (when we added chaining) and format was added in 1.5.  (Because of 
the way it was done in 1.4, where the constructor parameter of the 
general form is (String, Throwable), we can't overload with a (String, 
Object...) constructor that would treat the string as a format string, 
which is surely what we would have done had we had format() in 1.0.)


* I think that for most occurrences of `default: throw...`, by far, 
the user really doesn't benefit from being able to choose the 
exception type or the message at all. A standardized choice of 
exception, and an autogenerated message (that automatically includes 
the switch target, which users usually don't bother to do 
themselves!), may be strictly better. Can we consider providing them a 
shortcut (strawman: just `default: throw;`!)?


We did actually discuss this very notion.  While its attractive, it also 
feels like an attractor for an infinite stream of "can you let me 
customize x/y/z" requests.  So we wanted to explore whether there are 
library moves we can make that get us to this benefit first. For 
example, if you could say


    default: throw switchException();

where switchExpression() was a static-import of 
Intrinsics.switchExpression() (which could still capture 
Object.toString() of the target, as well as a description of the switch 
context), that's a smaller hammer with more flexibility.


* For exhaustive switches (so far: enums), the ideal we have been 
trying to shoot for is that the user makes an informed, intelligent 
choice about whether they want the compile error or the runtime error 
when a new one is added. So far, with what we've implemented at 
Google, it seems that we're falling fall short of that ideal. We have 
been strongly considering the idea that you /shouldn't/ be allowed to 
simply omit a default at all, because what that /means/ is far too 
opaque. It would be better if the user had to do something explicit in 
either case. Could it be that the idea in the previous bullet actually 
provides this? We would go back to always requiring an explicit 
`default` for an e-switch, but you still get to make an explicit 
choice which behavior you're asking for.


That sounds like a turnaround.  Previously, you'd argued that specifying 
all the enum cases and omitting a default was ideal because then you 
have a valuable tripwire against new values being added without them 
getting swept under the rug.


* It would feel strange to even bother applying this exhaustiveness 
goo to /byte/ switches. If we ever had ranges of course then, any 
type of switch could join the party. (I don't know whether ranges are 
a thing we're considering or not and I'm not pushing that we do.)


Yeah, its on the edge.  Its a no-brainer for `boolean`, its nuts for 
`int` (without ranges), but its vaguely defensible for `byte`. Though I 
can't really get too excited about it.

Re: Exhaustiveness in switch

[Kevin Bourrillion]

This is just a quick initial response and that it will veer off-topic:

* Users of Guava's Preconditions love the support for "%s"-formatting.  But
on the other hand, in code where you *are definitely* about to throw
something, why wouldn't I just use string concatenation? That is generally
easier to read because you don't have to mentally correlate specifiers to
faraway arguments. (Okay: one reason might be if your message is long
enough that you'd like to use a multi-line raw string literal for it, where
concatenation is BadBadBad, but I can't say how common this is or isn't at
the moment.)

* If you do add those methods, though, you don't want to just call
String.format() inside -- or if you do, you at least want to catch
InvalidFormatException and produce a fallback message in that case; don't
let that except supplant the one the user is really trying to create
(especially if that is unchecked).

* I think that for most occurrences of `default: throw...`, by far, the
user really doesn't benefit from being able to choose the exception type or
the message at all. A standardized choice of exception, and an
autogenerated message (that automatically includes the switch target, which
users usually don't bother to do themselves!), may be strictly better. Can
we consider providing them a shortcut (strawman: just `default: throw;`!)?

* For exhaustive switches (so far: enums), the ideal we have been trying to
shoot for is that the user makes an informed, intelligent choice about
whether they want the compile error or the runtime error when a new one is
added. So far, with what we've implemented at Google, it seems that we're
falling fall short of that ideal. We have been strongly considering the
idea that you *shouldn't* be allowed to simply omit a default at all,
because what that *means* is far too opaque. It would be better if the user
had to do something explicit in either case. Could it be that the idea in
the previous bullet actually provides this? We would go back to always
requiring an explicit `default` for an e-switch, but you still get to make
an explicit choice which behavior you're asking for.

* It would feel strange to even bother applying this exhaustiveness goo to
*byte* switches. If we ever had ranges of course then, any type of
switch could join the party. (I don't know whether ranges are a thing we're
considering or not and I'm not pushing that we do.)





On Thu, May 10, 2018 at 1:12 PM, Brian Goetz  wrote:

> In the long (and indirect) thread on "Expression Switch Exception Naming",
> we eventually meandered around to the question of when should the compiler
> deem an expression switch to be exhaustive, and therefore emit a catch-all
> throwing default. Let's step back from this for a bit and remind ourselves
> why we care about this.
>
> Superficially, having to write a throwing default for a condition we
> believe to be impossible is annoying:
>
> switch (trafficLight) {
> case RED -> stop();
> case GREEN -> driveFast();
> case YELLOW -> driveFaster();
> default -> throw new ExasperationException("No, we haven't added
> any new traffic light colors since the invention of the automobile, so I
> have no idea what " + trafficLight + " is");
> }
>
> The annoyance here, though, is twofold:
>  - I have to write code for something which I think can't happen;
>  - That code is annoying to write.
>
> In the above, we "knew" another traffic light color was impossible, and we
> listed them all -- and the compiler knew it. This is particularly
> irritating.  However, we often also see cases like this:
>
> void processVowel(letter) {
> switch (letter) {
> case A: ...
> case E: ...
> case I: ...
> case O: ...
> case U: ...
> default: throw new IllegalStateException("Not a vowel: " +
> letter);
> }
>
> Here, the annoyance is slightly different, in that I could not reasonably
> expect the compiler to know I'd covered all the vowels.  In fact, I think
> the explicit exception in this case is useful, in that it documents an
> invariant known to the programmer but not captured in the type system.  But
> it is still annoying that I have to construct a format string, construct an
> exception, and throw it; if there were easier ways to do that, I might be
> less annoyed.  Without diving into the bikeshed, maybe this looks something
> like:
>
> default: throw IllegalStateException.format("Not a vowel: %s",
> vowel);
>
> The details aren't relevant, but the point is: maybe a small-ish library
> tweak would reduce the annoyance of writing such clauses. (This one isn't
> so bad, but Dan excavated a bunch that were way worse.)  But, let's set
> this aside for a moment, and return back to the point of why we want the
> compiler to provide a throwing default.
>
> I think most of the discussion has centered on the problem of a novel
> value showing 

Exhaustiveness in switch

[Brian Goetz]

In the long (and indirect) thread on "Expression Switch Exception 
Naming", we eventually meandered around to the question of when should 
the compiler deem an expression switch to be exhaustive, and therefore 
emit a catch-all throwing default. Let's step back from this for a bit 
and remind ourselves why we care about this.


Superficially, having to write a throwing default for a condition we 
believe to be impossible is annoying:


    switch (trafficLight) {
    case RED -> stop();
    case GREEN -> driveFast();
    case YELLOW -> driveFaster();
    default -> throw new ExasperationException("No, we haven't 
added any new traffic light colors since the invention of the 
automobile, so I have no idea what " + trafficLight + " is");

    }

The annoyance here, though, is twofold:
 - I have to write code for something which I think can't happen;
 - That code is annoying to write.

In the above, we "knew" another traffic light color was impossible, and 
we listed them all -- and the compiler knew it. This is particularly 
irritating.  However, we often also see cases like this:


    void processVowel(letter) {
        switch (letter) {
            case A: ...
    case E: ...
        case I: ...
            case O: ...
    case U: ...
        default: throw new IllegalStateException("Not a vowel: " + 
letter);

    }

Here, the annoyance is slightly different, in that I could not 
reasonably expect the compiler to know I'd covered all the vowels.  In 
fact, I think the explicit exception in this case is useful, in that it 
documents an invariant known to the programmer but not captured in the 
type system.  But it is still annoying that I have to construct a format 
string, construct an exception, and throw it; if there were easier ways 
to do that, I might be less annoyed.  Without diving into the bikeshed, 
maybe this looks something like:


    default: throw IllegalStateException.format("Not a vowel: %s", 
vowel);


The details aren't relevant, but the point is: maybe a small-ish library 
tweak would reduce the annoyance of writing such clauses. (This one 
isn't so bad, but Dan excavated a bunch that were way worse.)  But, 
let's set this aside for a moment, and return back to the point of why 
we want the compiler to provide a throwing default.


I think most of the discussion has centered on the problem of a novel 
value showing up at runtime.  This is surely an issue, and must be dealt 
with, but the central issue is: a default is never able to distinguish 
between a runtime-novel value and a value we just forgot to include at 
compile time.  It doesn't matter whether this default throws (as the 
implicit default in an expression switch) or does nothing (as the 
implicit default in statement switches today does).


We agreed that we should not require the user to provide a default when 
they provide case clauses that cover the target type as of compile time 
(true+false for boolean, all the members of a sealed type, etc.)  This 
is because the default you'd be forced to put in otherwise (for 
expression switches) is actually harmful; if the type were later 
modified to have more values, an explicit default would swallow them, 
rather than yielding an error at recompilation time.  So it is not only 
annoying, but actually could cover up errors.


We then went off on the wrong tangent, though, where we wondered whether 
it was OK to implicitly assume enums were sealed, since some enums are 
clearly intended to acquire new values.  But the mistake was focusing on 
the wrong aspect of sealed-ness (the statement of intent to not add more 
values), rather than the compiler's ability to reason credibly about 
known possible values.


So, backing up, I think we should always treat a "complete" enum 
expression switch specially -- don't require a default, and implicitly 
add a throwing one, if all the cases are specified. This way, if the 
assumption that you've covered all the cases is later broken via 
separate compilation, on recompilation, you'll discover this early, 
rather than at runtime.  (You'll still get runtime protection either 
way.)  Regardless of whether we think the enum will be extended in the 
future or not.  There's no need for enums to declare themselves "sealed" 
or "non-sealed" (and such a declaration would likely be incorrect 
anyway, as it asks users to predict the future, which is error-prone.)


Given this, I'm willing to use ICCE as a base type for the implicit 
exception (though there should be more specific subtypes.)



Now, statement switches.  It seems sad that we can't get the same kind 
of compile-time assistance over statement switches than we do over 
expression switches.  We're somewhat locked in by compatibility here; 
statement switches today get an implicit "default: nothing" clause if 
they have no default, and we cannot (and don't want to) break this.  So 
the next best thing is if the user could say "I want to get