> On Jun 9, 2016, at 11:42 AM, Dave Abrahams <[email protected]> wrote: > > > on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com > <http://matthew-at-anandabits.com/>> wrote: > >>> On Jun 9, 2016, at 9:55 AM, Dave Abrahams <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> >>> on Wed Jun 08 2016, Matthew Johnson <matthew-AT-anandabits.com >>> <http://matthew-at-anandabits.com/> <http://matthew-at-anandabits.com/ >>> <http://matthew-at-anandabits.com/>>> wrote: >>> >> >>>>> On Jun 8, 2016, at 1:33 PM, Dave Abrahams <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> >>>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com >>>>> <http://matthew-at-anandabits.com/>> wrote: >>>>> >>>> >>>>>>> On Jun 7, 2016, at 9:15 PM, Dave Abrahams <[email protected] >>>>>>> <mailto:[email protected]>> wrote: >>>>>>> >>>>>>> >>>>>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com >>>>>>> <http://matthew-at-anandabits.com/> <http://matthew-at-anandabits.com/ >>>>>>> <http://matthew-at-anandabits.com/>>> wrote: >>>>>>> >>>>>> >>>>>>>>> On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution >>>>>>>>> <[email protected] <mailto:[email protected]>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> on Tue Jun 07 2016, Matthew Johnson <[email protected] >>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>> >>>>>>>> >>>>>>>>>>> , but haven't realized >>>>>>>>>>> that if you step around the type relationships encoded in Self >>>>>>>>>>> requirements and associated types you end up with types that appear >>>>>>>>>>> to >>>>>>>>>>> interoperate but in fact trap at runtime unless used in exactly the >>>>>>>>>>> right way. >>>>>>>>>> >>>>>>>>>> Trap at runtime? How so? Generalized existentials should still be >>>>>>>>>> type-safe. >>>>>>>>> >>>>>>>>> There are two choices when you erase static type relationships: >>>>>>>>> >>>>>>>>> 1. Acheive type-safety by trapping at runtime >>>>>>>>> >>>>>>>>> FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap >>>>>>>>> >>>>>>>>> 2. Don't expose protocol requirements that involve these >>>>>>>>> relationships, >>>>>>>>> which would prevent the code above from compiling and prevent >>>>>>>>> FloatingPoint from conforming to itself. >>>>>>>>> >>>>>>>>>> Or are you talking about the hypothetical types / behaviors people >>>>>>>>>> think they want when they don’t fully understand what is happening... >>>>>>>>> >>>>>>>>> I don't know what you mean here. I think generalized existentials >>>>>>>>> will >>>>>>>>> be nice to have, but I think most people will want them to do >>>>>>>>> something >>>>>>>>> they can't possibly do. >>>>>>>> >>>>>>>> Exactly. What I meant is that people think they want that expression >>>>>>>> to compile because they don’t understand that the only thing it can do >>>>>>>> is trap. I said “hypothetical” because producing a compile time error >>>>>>>> rather than a runtime trap is the only sane thing to do. Your comment >>>>>>>> surprised me because I can’t imagine we would move forward in Swift >>>>>>>> with the approach of trapping. >>>>>>> >>>>>>> I would very much like to be able to create instances of “Collection >>>>>>> where Element == Int” so we can throw away the wrappers in the stdlib. >>>>>>> That will require some type mismatches to be caught at runtime via >>>>>>> trapping. >>>>>> >>>>>> For invalid index because the existential accepts a type erased index? >>>>> >>>>> Exactly. >>>>> >>>>>> How do you decide where to draw the line here? It feels like a very >>>>>> slippery slope for a language where safety is a stated priority to >>>>>> start adopting a strategy of runtime trapping for something as >>>>>> fundamental as how you expose members on an existential. >>>>> >>>>> If you don't do this, the alternative is that “Collection where Element >>>>> == Int” does not conform to Collection. >>>> >>>> This isn’t directly related to having self or associated type >>>> requirements. It is true of all existentials. >>> >>> That is just an implementation limitation today, IIUC. What I'm talking >>> about here would make it impossible for some to do that. >> >> If it is just an implementation limitation I am happy to hear that. >> >>> >>>> If that changes for simple existentials and generalized existentials >>>> expose all members (as in the latest draft of the proposal) maybe it >>>> will be possible for all existentials to conform to their protocol. >>> >>> Not without introducing runtime traps. See my “subscript function” >>> example. >> >>> >>>> >>>>> That's weird and not very >>>>> useful. You could expose all the methods that were on protocol >>>>> extensions of Collection on this existential, unless they used >>>>> associated types other than the element type. But you couldn't pass the >>>>> existential to a generic function like >>>>> >>>>> func scrambled<C: Collection>(_ c: C) -> [C.Element] >>>>> >>>>>> IMO you should *have* to introduce unsafe behavior like that manually. >>>>> >>>>> Collection where Element == Int & Index == * >>>>> >>>>> ? >>>> >>>> I didn’t mean directly through the type of the existential. >>> >>> My question is, why not? That is still explicit. >> >> It’s not explicit in the sense that nobody wrote `fatalError` or >> similar in their code. It’s too easy to write something like that >> without realizing that it introduces the possibility of a crash. If >> we adopt syntax like that to introduce an existential that introduces >> traps we should at least require members that can be trap to be >> invoked using a `!` suffix or something like that to make it clear to >> users that a trap will happen if they are not extremely careful when >> using that member. >> >> More generally though, I don’t want the rules of the language to be >> written in a way that causes the compiler to synthesize traps in such >> a general way. >> >> The existential should not introduce a precondition that isn’t already >> present in the semantics of the protocol itself. If the semantics of >> the protocol do not place preconditions on arguments beyond their type >> (such as “must be a valid index into this specific instance”) the >> compiler should not allow the existential to conform if a trap is >> required in some circumstances. That is a new precondition and >> therefore the existential does not actually fulfill the requirements >> of the protocol. >> >> I could *maybe* live with a solution where protocol requirements are >> marked as trapping, etc depending on the specific argument received at >> runtime. This is a total straw man syntax, but maybe `IndexableBase` >> would declare the subscript `@trapping` (probably something different >> but I hope this communicates the idea). This alerts users to the fact >> that they need to be extra careful - not any value of `Self.Index` is >> valid and you can get a crash if you’re not careful. >> >> Having this semantic explicit in the definition of the protocol opens >> the door to maybe considering an existential synthesized by the >> compiler that traps because it doesn’t introduce a new precondition >> that wasn’t already present in the protocol. >> >> I would want to give consideration to specific details of a proposal >> along these lines before deciding how I feel about it, but I have a >> more open mind to this approach than introducing traps not present in >> the preconditions of the protocol. >> >> /// You can subscript a collection with any valid index other than the >> /// collection's end index. The end index refers to the position one past >> /// the last element of a collection, so it doesn't correspond with an >> /// element. >> /// >> /// - Parameter position: The position of the element to access. `position` >> /// must be a valid index of the collection that is not equal to the >> /// `endIndex` property. >> @trapping public subscript(position: Self.Index) -> Self._Element { get } >> >>> >>>> One obvious mechanism for introducing unsafe behavior is to write >>>> manual type erasure wrappers like we do today. >>>> >>>> Another possibility would be to allow extending the existential type >>>> (not the protocol). This would allow you to write overloads on the >>>> Collection existential that takes some kind of type erased index if >>>> that is what you want and either trap if you receive an invalid index >>>> or better (IMO) return an `Element?`. I’m not sure how extensions on >>>> existentials might be implemented, but this is an example of the kind >>>> of operation you might want available on it that you wouldn’t want >>>> available on all Collection types. >>>> >>>>> >>>>>> Collection indices are already something that isn’t fully statically >>>>>> safe so I understand why you might want to allow this. >>>>> >>>>> By the same measure, so are Ints :-) >>>>> >>>>> The fact that a type's methods have preconditions does *not* make it >>>>> “statically unsafe.” >>>> >>>> That depends on what you mean by safe. Sure, those methods aren’t >>>> going corrupt memory, but they *are* going to explicitly and >>>> intentionally crash for some inputs. That doesn’t qualify as “fully >>>> safe” IMO. >>> >>> Please pick a term other than “unsafe” here; it's not unsafe in the >>> sense we mean the word in Swift. It's safe in exactly the same way that >>> array indexes and integers are. When you violate a precondition, it >>> traps. >> >> I am happy to use any word you like here. >> >> Can you clarify what you mean by the word safe in Swift? It doesn’t >> appear to be limited to memory safety in the public about page >> https://swift.org/about/ <https://swift.org/about/> >> <https://swift.org/about/ <https://swift.org/about/>>: > > I mean memory- and type-safe. > >> Safe. The most obvious way to write code should also behave in a safe >> manner. Undefined behavior is the enemy of safety, and developer >> mistakes should be caught before software is in production. Opting for >> safety sometimes means Swift will feel strict, but we believe that >> clarity saves time in the long run. >> >> Safety >> >> Swift was designed from the outset to be safer than C-based languages, >> and eliminates entire classes of unsafe code. Variables are always >> initialized before use, arrays and integers are checked for overflow, >> and memory is managed automatically. Syntax is tuned to make it easy >> to define your intent — for example, simple three-character keywords >> define a variable (var) or constant (let). >> >> Another safety feature is that by default Swift objects can never be >> nil, and trying to make or use a nil object will results in a >> compile-time error. This makes writing code much cleaner and safer, >> and prevents a common cause of runtime crashes. However, there are >> cases where nil is appropriate, and for these situations Swift has an >> innovative feature known as optionals. An optional may contain nil, >> but Swift syntax forces you to safely deal with it using ? to indicate >> to the compiler you understand the behavior and will handle it safely. >> >> This positioning statement makes it appear as if preventing common >> causes of crashes falls within the meaning of safe that Swift is >> using. Having existentials introduce new preconditions and traps when >> they are not met does not seem aligned with that goal IMO. > > Static typing “increases safety,” in the casual sense. That doesn't > mean that an operation that traps on a failed precondition check is > “unsafe.” > >>> The user doesn't do anything “manual” to introduce that trapping >>> behavior for integers. Preconditions are a natural part of most types. >> >> The user doesn’t, but isn’t the overflow trap implemented in the >> standard library? > > Whether it is or is not is an implementation detail. > >> Regardless, this is a specific case that has been given explicit >> design attention by humans. The precondition is designed, not >> introduced by compiler rules that haven’t considered the specific case >> in question. >> >>> >>>>>> But I don’t think having the language's existentials do this >>>>>> automatically is the right approach. Maybe there is another >>>>>> approach that could be used in targeted use cases where the less >>>>>> safe behavior makes sense and is carefully designed. >>>>> >>>>> Whether it makes sense or not really depends on the use-cases. There's >>>>> little point in generalizing existentials if the result isn't very useful. >>>> >>>> Usefulness depends on your perspective. >>> >>> Of course. As I've said, let's look at the use cases. >> >> Agree. We can consider those in depth when the time comes to ramp up >> discussion of Austin’s proposal. >> >>> >>>> I have run into several scenarios where they would be very useful >>>> without needing to be prone to crashes when used incorrectly. One >>>> obvious basic use case is storing things in a heterogenous collection >>>> where you bind . >>> >>> bind what? >> >> Sorry, I must have gotten distracted and not finished that paragraph. >> I meant to say bind the associated types that are necessary for your >> use case. Sometimes you bind *all* of the associated types to >> concrete types and the protocol has no `Self` requirements. In that >> case there is no trouble at all in conforming the type-erased >> “existential" to the protocol itself. Austin’s proposal would >> eliminate the need to manually write these “existentials” manually. >> >>>> >>>>> The way to find out is to take a look at the examples we currently have >>>>> of protocols with associated types or Self requirements and consider >>>>> what you'd be able to do with their existentials if type relationships >>>>> couldn't be erased. >>>>> >>>>> We have known use-cases, currently emulated in the standard library, for >>>>> existentials with erased type relationships. *If* these represent the >>>>> predominant use cases for something like generalized existentials, it >>>>> seems to me that the language feature should support that. Note: I have >>>>> not seen anyone build an emulation of the other kind of generalized >>>>> existential. My theory: there's a good reason for that :-). >>>> >>>> AFAIK (and I could be wrong) the only rules in the language that >>>> require the compiler to synthesize a trap except using a nil IUO, `!` >>>> on a nil Optional, and an invalid `as` cast . These are all >>>> syntactically explicit unsafe / dangerous operations. All other traps >>>> are in the standard library (array index, overflow, etc). Most >>>> important about all of these cases is that they have received direct >>>> human consideration. >>> >>> There is no distinction in the user model between what might be >>> synthesized by the language and what appears on standard library types. >> >> Maybe I shouldn’t have made that distinction. >> >> The point I am trying to emphasize is that each of these are special >> cases that have received direct human consideration. The potential >> for a trap is not introduced by language rules that apply to >> user-defined constructs in without consideration of the specific >> details of that construct. >> >>> >>>> Introducing a language (not library) mechanism that exposes members on >>>> generalized existentials in a way that relies on runtime traps for >>>> type safety feels to me like a pretty dramatic turn agains the stated >>>> priority of safety. It will mean you must understand exactly what is >>>> going on and be extremely careful to use generalized existentials >>>> without causing crashes. This will either make Swift code much more >>>> crashy or will scare people away from using generalized existentials >>>> (and maybe both). >>> >>> I don't accept either of those statements without seeing some analysis >>> of the use-cases. For example, I don't believe that AnyCollection et al >>> are particularly crash-prone. The likelihood that you'll use the wrong >>> index type with a collection is very, very low. I'm less certain of >>> what happens with Self requirements in real cases. >> >> But again, I believe this is an exceptional case as the precondition >> is explicitly stated in the semantics of the protocol. > > IIUC, it has been cited by Doug as the exemplar of the > predominantly-requested case by a 10:1 ratio!
In terms of forming the existential, storing it in variables, accepting arguments of that type, etc yes. I don’t know how many of those requests expect it to conform to the protocol and expect to be able to use it in generic code constrained to the protocol. > >> IMO the burden of proof should be on the side that proposes a >> mechanism to introduce traps, not the side that proposes avoiding >> them. > > If you really want to make this about sides and burdens, the burden of > proof always rests with the side proposing to extend the language. We > shouldn't be making changes without understanding how they will play out > in real use-cases. I agree with this. But if we are discussing two different options for extending the language I think the option that doesn’t introduce crashes should be preferred without pretty compelling reasons to choose the option that can introduce crashes. > >>>> Neither of those outcomes is good. >>>> >>>> Collection indices are a somewhat special case as there is already a >>>> strong precondition that people are familiar with because it would be >>>> too costly to performance and arguably too annoying to deal with an >>>> Optional result in every array lookup. IMO that is why the library is >>>> able to get away with it in the current type erased AnyCollection. >>>> But this is not a good model for exposing any members on an >>>> existential that do not already have a strong precondition that causes >>>> a trap when violated. >>>> >>>> I think a big reason why you maybe haven’t seen a lot of examples of >>>> people writing type erased “existentials" is because it is a huge pain >>>> in the neck to write this stuff manually today. People may be >>>> designing around the need for them. I haven’t seen a huge sampling of >>>> type erased “existentials" other people are writing but I haven’t >>>> written any that introduce a trap like this. The only traps are in >>>> the “abstract" base class whose methods will never be called (and >>>> wouldn’t even be implemented if they could be marked abstract). >>>> >>>> What specific things do you think we need to be able to do that rely >>>> on the compiler synthesizing a trap in the way it exposes the members >>>> of the existential? >>> >>> I don't know. I'm saying, I don't think we understand the use-cases >>> well enough to make a determination. >> >> That’s fair. I agree that use cases should be carefully considered. >> >>> >>>> Here are a few examples from Austin’s proposal that safely use >>>> existential collections. I don’t understand why you think this >>>> approach is insufficient. Maybe you could supply a concrete example >>>> of a use case that can’t be written with the mechanism in Austin’s >>>> proposal. >>>> >>>> https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure >>>> >>>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure> >>>> >>>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure >>>> >>>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure>> >>>> >>>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure >>>> >>>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure><https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure >>>> >>>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure>>> >>>> >>>> let a : Any<Collection> >>>> >>>> // A variable whose type is the Index associated type of the underlying >>>> // concrete type of 'a'. >>>> let theIndex : a.Index = ... >>>> >>>> // A variable whose type is the Element associated type of the underlying >>>> // concrete type of 'a'. >>>> let theElement : a.Element = ... >>>> >>>> // Given a mutable collection, swap its first and last items. >>>> // Not a generic function. >>>> func swapFirstAndLast(inout collection: >>>> Any<BidirectionalMutableCollection>) { >>>> // firstIndex and lastIndex both have type "collection.Index" >>>> guard let firstIndex = collection.startIndex, >>>> lastIndex = collection.endIndex?.predecessor(collection) where >>>> lastIndex != firstIndex else { >>>> print("Nothing to do") >>>> return >>>> } >>>> >>>> // oldFirstItem has type "collection.Element" >>>> let oldFirstItem = collection[firstIndex] >>>> >>>> collection[firstIndex] = collection[lastIndex] >>>> collection[lastIndex] = oldFirstItem >>>> } >>>> >>>> var a : Any<BidirectionalMutableCollection where .Element == String> = ... >>>> >>>> let input = "West Meoley" >>>> >>>> // Not actually necessary, since the compiler knows "a.Element" is String. >>>> // A fully constrained anonymous associated type is synonymous with the >>>> concrete >>>> // type it's forced to take on, and the two are interchangeable. >>>> // However, 'as' casting is still available if desired. >>>> let anonymousInput = input as a.Element >>>> >>>> a[a.startIndex] = anonymousInput >>>> >>>> // as mentioned, this also works: >>>> a[a.startIndex] = input >>>> >>>> // If the collection allows it, set the first element in the collection to >>>> a given string. >>>> func setFirstElementIn(inout collection: Any<Collection> toString string: >>>> String) { >>>> if let element = string as? collection.Element { >>>> // At this point, 'element' is of type "collection.Element" >>>> collection[collection.startIndex] = element >>>> } >>>> } >>> >>> Neither of these look like they actually make *use* of the fact that >>> there's type erasure involved (and therefore should probably be written >>> as generics?). The interesting cases with Any<Collection...>, for the >>> purposes of this discussion, arise when you have multiple instances of >>> the same existential type that wrap different concrete types. >> >> One use case I have found is to work around the lack of higher-kinder >> types. > > Really, now: a use-case for feature A that is a workaround for the lack > of feature B hardly justifies adding feature A! We do want to add > higher-kinded types eventually. Good to know. I thought higher-kinder types were on the “maybe if someone shows a compelling enough use case” list. AFAIK this is the first time a member of the core team has stated the intent to add them. If that is the case I agree that this use case isn’t relevant. The workaround isn’t great because it loses type information that is critical to the optimizer (but it’s all we have available today). > >> If you have a protocol where specific implementations will return >> different types, but all conform to a second protocol you can define >> the protocol in terms of a generic type-erased wrapper which conforms >> to the second protocol and accepts type arguments that match the >> associated types (thus binding the associated types to concrete >> types). I have found this to be a useful technique (granted it is a >> workaround and I’m not sure how useful it would continue to be if >> Swift eventually gets higher-kinder types). >> >>> >>> Another problem I see: in this new world, what is the model for choosing >>> whether to write a function as a protocol extension/generic, or as a >>> regular function taking existential parameters? Given that either of >>> the above could have been written either way, we need to be able to >>> answer that question. When existentials don't conform to their >>> protocols, it seems to me that the most general thing to do is use >>> existentials whenever you can, and only resort to using generics when >>> forced by the type system. This does not seem like a particularly good >>> programming model to me, but I might be convinced otherwise. >> >> That doesn’t seem like a particularly good programming model to me either. >> >> The rule of thumb I am operating with for protocols with Self or >> associated type requirements is to prefer generics and use type >> erasure / existentials when that isn’t possible. For example, when >> heterogeneity is required or when you can’t form the necessary type in >> a protocol requirement (as in the preceding example). >> >> This heuristic has been working out pretty well for me thus far. > > I do worry a bit that people will choose the opposite heuristic. > > It would be somewhat reassuring to me if we could prove to ourselves > that, using your heuristic, one is never forced to copy/paste a generic > function implementation into a corresponding function that uses > existentials. > >> The primary impact of introducing a language mechanism for generalized >> existentials in my code would be to eliminate a lot of manual type >> erasing boilerplate. > > If your code has many manual type erasing wrappers corresponding to > protocols with associated types and/or Self requirements that also never > have to trap type mismatches, that would certainly be instructive > empirical data. Would you care to share the protocols and wrappers you > are talking about? I put together a sample implementation of a Cocoa-like responder chain in Swift a while ago when the “Swift dynamism” debate was raging. It isn't intended to be a Swifty design. It is intended to be similar to Cocoa and show techniques that can be used to do things similar to Cocoa’s responder chain and targer-action in Swift. It uses a type erased wrapper for actions that binds `Sender` while hiding the concrete `Action` type and also the `Handler` associated type. It cannot and should not conform to the protocol it is derived from and could be replaced with the generalized existentials in Austin’s proposal. https://gist.github.com/anandabits/ec26f67f682093cf18b170c21bcf433e <https://gist.github.com/anandabits/ec26f67f682093cf18b170c21bcf433e> This is a good example to start with because it is related to a topic that has been hotly debated and is clearly something a lot of people want to be able to do. > >>> Anyway, my overall point is that this all seems like something we *can* >>> do and that nicely fills gaps in the type system, but not necessarily >>> something we *should* do until we better understand what it's actually >>> *for* and how it affects the programming model. >> >> That’s a very fair position to take. :) >> >>> >>> -- >>> Dave >> > > -- > Dave
_______________________________________________ swift-evolution mailing list [email protected] https://lists.swift.org/mailman/listinfo/swift-evolution
