Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[John McCall via swift-evolution]

> On Nov 10, 2016, at 5:16 PM, Dave Abrahams  wrote:
> on Thu Nov 10 2016, John McCall  wrote:
> 
>>> On Nov 10, 2016, at 9:31 AM, Joe Groff  wrote:
 On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
 
> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>  wrote:
>> On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
>> 
>> wrote:
>> 
>> 
>> 
>> on Mon Nov 07 2016, John McCall  wrote:
>> 
 On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
  wrote:
 
 
 Given that we're headed for ABI (and thus stdlib API) stability, I've
 been giving lots of thought to the bottom layer of our collection
>>> 
 abstraction and how it may limit our potential for efficiency.  In
 particular, I want to keep the door open for optimizations that work on
 contiguous memory regions.  Every cache-friendly data structure, even 
 if
 it is not an array, contains contiguous memory regions over which
 operations can often be vectorized, that should define boundaries for
 parallelism, etc.  Throughout Cocoa you can find patterns designed to
 exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
 out in readv/writev, and MPI datatypes essentially boil down to
 identifying the contiguous parts of data structures.  My point is that
 this is an important class of optimization, with numerous real-world
 examples.
 
 If you think about what it means to build APIs for contiguous memory
 into abstractions like Sequence or Collection, at least without
 penalizing the lowest-level code, it means exposing 
 UnsafeBufferPointers
 as a first-class part of the protocols, which is really
 unappealing... unless you consider that *borrowed* UnsafeBufferPointers
 can be made safe.  
 
 [Well, it's slightly more complicated than that because
 UnsafeBufferPointer is designed to bypass bounds checking in release
 builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
 something—that checks bounds unconditionally... but] the point remains
 that
 
 A thing that is unsafe when it's arbitrarily copied can become safe if
 you ensure that it's only borrowed (in accordance with well-understood
 lifetime rules).
>>> 
>>> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
>>> doesn't prevent you from making your own copy, which could then escape
>>> the scope that was guaranteeing safety.
>>> 
>>> This is fixable, of course, but it's a more significant change to the
>>> type and how it would be used.
>> 
>> It sounds like you're saying that, to get static safety benefits from
>> ownership, we'll need a whole parallel universe of safe move-only
>> types. Seems a cryin' shame.
> 
> We've discussed the possibility of types being able to control
> their "borrowed" representation. Even if this isn't something we
> generalize, arrays and contiguous buffers might be important
> enough to the language that your safe BufferPointer could be
> called 'borrowed ArraySlice', with the owner backreference
> optimized out of the borrowed representation. Perhaps Array's own
> borrowed representation would benefit from acting like a slice
> rather than a whole-buffer borrow too.
 
 The disadvantage of doing this is that it much more heavily
 penalizes the case where we actually do a copy from a borrowed
 reference — it becomes an actual array copy, not just a reference
 bump.
>>> 
>>> Fair point, though the ArraySlice/Array dichotomy strikes me as
>>> already kind of encouraging this—you might pass ArraySlices down
>>> into your algorithm, but we encourage people to use Array at storage
>>> and API boundaries, forcing copies.
>> 
>> Fair point.  In practice, though, I think most algorithms won't need
>> to "escape" that array slice.
> 
> I disagree. I'm working on some generic matching algorithms (to lay the
> foundation for String search and regexes).  There's going to be a broad
> category of functions in this area that work on Strings and return
> SubStrings, or work on Collections and return slices thereof.  Often
> they'll be called from a context where the resultant slices don't
> outlive the collection, but they still do need to be returned.

Ok.  You're right, I was thinking about arrays more than I was thinking about 
strings.

Anyway, if you're talking about returning the value back, you're talking about
something we can't support as just a borrowed value without some sort of
lifetime-qualification system.

Re: [swift-evolution] [Out of scope] Discussion on general Darwin/GlibC module

[Drew Crawford via swift-evolution]

grep -R "import Glibc" ~/Code --include "*.swift" | wc -l
297

As someone who might be characterized as suffering from the problem this
proposal purports to solve, I am not convinced.

The primary problem here is that "libc" is a misnomer.  Did you mean
musl, dietlibc, or glibc?  Did you mean "whatever libc my distro likes?"
Swift in practice only supports one per platform, but that is a bug not
a feature, and that bug should not be standardized.  We could try to
invent some syntax to specify one but now we are back with the current
system again.

The other problem is that in all my usages, "import Glibc" is not a real
problem I face.  The real problems are that "the libcs *plural*" are
*just different*.  Darwin has timeval64, glibc does not, and you'd
better check your arch and pick the right one, only on one platform.
SO_REUSEADDR has one type in Brand X and another type in Brand Y.  Don't
even get me *started* on poll, EREs, or half a dozen other behavioral
variations.

Taking two different libraries and pretending they are the same is not
the solution, it's the disease.  The way out of this swamp for most
developers is to use a real Swift library, the same damn Swift library,
on all platforms (sadly, Foundation today does not meet this
requirement).  The way out of this swamp for crazy people like me who
must write to the metal is to actually write to the metal, to the
particular libc being targeted, not to a hypothetical platonic ideal
libc which does not exist.

I realize that four lines at the top of my files is a *visible*
annoyance, but fixing it just promotes it to an invisible one.

Drew

--
  Drew Crawford
  d...@sealedabstract.com



On Wed, Nov 9, 2016, at 12:58 PM, Alex Blewitt via swift-evolution wrote:
> Although out of scope for phase 1, something that keeps cropping up in
> a variety of Linux/Darwin Swift scripts is the conditional inclusion
> of Darwin or GlibC per platform. The last point was an observation
> that creating a 'nice' wrapper for LibC or a cleaned up POSIX API is a
> non-goal:
>
> https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html
>
>> I think it makes sense to have a cross platform “libc” which is an
>> alias for darwin, glibc, or whatever, and just leave it at that.
>> Other proposals for a “POSIX” module have gotten bogged down because
>> inevitably the idea comes up to make the resultant API nicer in
>> various ways: rename creat, handle errno more nicely, make use of
>> multiple return values, … etc.  The problem with this approach is
>> that we don’t *want* people using these layer of APIs, we want higher
>> level Foundation-like APIs to be used.  ...*
* I think we should formally decide that a “nice” wrapper for libc is a
  non-goal.  There is too much that doesn’t make sense to wrap at this
  level - the only Swift code that should be using this is the
  implementation of higher level API, and such extremely narrow cases
  that we can live with them having to handle the problems of dealing
  with the raw APIs directly.  -Chris
>
> I have created a draft for a proposal to create such a module.
> Comments are welcome.
>
> Alex
>
> ---
>
> # Libc module for Swift
>
> * Proposal: [SE-](-filename.md)
> * Authors: [Alex Blewitt](https://github.com/alblue)
> * Review Manager: TBD
> * Status: **Under discussion**
>
> ## Introduction
>
> When running on Darwin, the base module is called `Darwin`.
> When running
> on Linux or other operating systems, it's called `GlibC`.
>
> This repeatedly leads to code such as:
>
> 
> #if os(Linux)
>   import Glibc
> #else
>   import Darwin
> #endif
> ```
>
> As the set of operating systems evolve, one of these
> conditional imports
> needs to be updated. Instead of repeating this, make it
> available via a
> standard `Libc` module in the base Swift library.
>
> Swift-evolution thread: [Discussion thread topic for that proposal]
> (https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html)
>
> ## Motivation
>
> The [set of platforms]
> (https://github.com/apple/swift/blob/fdf6ee20e4ca1fd32482f4b7b88a97ebdda52cd2/lib/Basic/LangOptions.cpp#L26-L36)
> that Swift currently runs on can be divided into two; Darwin and XNU
> based systems
> (macOS, iOS, watchOS, tvOS), Windows, and Unix based systems
> (Linux, FreeBSD, Android, PS4).
>
> The base module on Darwin is called `Darwin`, while on Linux and
> other Unix systems the base module is called `Glibc`. The base
> module is typically conditionally included when working at a
> lower layer
> than Foundation (which has the same detail involved in importing the
> base module).
>
> As a result, conditionally importing the right version typically uses
> a conditional test based on the operating system, and the same code is
> seen in a number of different modules, both internal to Swift and
> external:
>
> * [Test for mmap in stdlib]
>   
> 

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Dave Abrahams via swift-evolution]

on Thu Nov 10 2016, Joe Groff  wrote:

>> On Nov 10, 2016, at 1:02 PM, Dave Abrahams  wrote:
>> 
>> 
>> on Thu Nov 10 2016, Stephen Canon  wrote:
>> 
>
 On Nov 10, 2016, at 1:30 PM, Dave Abrahams via swift-evolution 
  wrote:
 
 
 on Thu Nov 10 2016, Joe Groff  wrote:
 
>>> 
>> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
>> 
>>> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>>>  wrote:
 On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
  wrote:
 
> 
 
 on Mon Nov 07 2016, John McCall  wrote:
 
>> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>> 
>> Given that we're headed for ABI (and thus stdlib API) stability, I've
>> been giving lots of thought to the bottom layer of our collection
> 
>> abstraction and how it may limit our potential for efficiency.  In
>> particular, I want to keep the door open for optimizations that work 
>> on
>> contiguous memory regions.  Every cache-friendly data structure, 
>> even if
>> it is not an array, contains contiguous memory regions over which
>> operations can often be vectorized, that should define boundaries for
>> parallelism, etc.  Throughout Cocoa you can find patterns designed to
>> exploit this fact when possible (NSFastEnumeration).  Posix I/O 
>> bottoms
>> out in readv/writev, and MPI datatypes essentially boil down to
>> identifying the contiguous parts of data structures.  My point is 
>> that
>> this is an important class of optimization, with numerous real-world
>> examples.
>> 
>> If you think about what it means to build APIs for contiguous memory
>> into abstractions like Sequence or Collection, at least without
>> penalizing the lowest-level code, it means exposing 
>> UnsafeBufferPointers
>> as a first-class part of the protocols, which is really
>> unappealing... unless you consider that *borrowed* 
>> UnsafeBufferPointers
>> can be made safe.  
>> 
>> [Well, it's slightly more complicated than that because
>> UnsafeBufferPointer is designed to bypass bounds checking in release
>> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
>> something—that checks bounds unconditionally... but] the point 
>> remains
>> that
>> 
>> A thing that is unsafe when it's arbitrarily copied can become safe 
>> if
>> you ensure that it's only borrowed (in accordance with 
>> well-understood
>> lifetime rules).
> 
> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
> doesn't prevent you from making your own copy, which could then escape
> the scope that was guaranteeing safety.
> 
> This is fixable, of course, but it's a more significant change to the
> type and how it would be used.
 
 It sounds like you're saying that, to get static safety benefits from
 ownership, we'll need a whole parallel universe of safe move-only
 types. Seems a cryin' shame.
>>> 
>>> We've discussed the possibility of types being able to control
>>> their "borrowed" representation. Even if this isn't something we
>>> generalize, arrays and contiguous buffers might be important enough
>>> to the language that your safe BufferPointer could be called
>>> 'borrowed ArraySlice', with the owner backreference optimized
>>> out of the borrowed representation. Perhaps Array's own borrowed
>>> representation would benefit from acting like a slice rather than a
>>> whole-buffer borrow too.
>> 
>> The disadvantage of doing this is that it much more heavily
>> penalizes the case where we actually do a copy from a borrowed
>> reference — it becomes an actual array copy, not just a reference
>> bump.
> 
> Fair point, though the ArraySlice/Array dichotomy strikes me as
> already kind of encouraging this—you might pass ArraySlices down into
> your algorithm, but we encourage people to use Array at storage and
> API boundaries, forcing copies.
> 
> From a philosophical perspective of making systems Swift feel like
> "the same language" as Swift today, it feels better to me to try to
> express this as making our high-level safe abstractions efficient
> rather than making our low-level unsafe abstractions safe. 
 
 +1, or maybe 10
 
 What worries me is that if systems programmers are trying to get static
 

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Dave Abrahams via swift-evolution]

on Thu Nov 10 2016, John McCall  wrote:

>> On Nov 10, 2016, at 9:31 AM, Joe Groff  wrote:
>>> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
>>> 
 On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
  wrote:
> On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
> 
> wrote:
>
> 
> 
> on Mon Nov 07 2016, John McCall  wrote:
> 
>>> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>>>  wrote:
>>> 
>>> 
>>> Given that we're headed for ABI (and thus stdlib API) stability, I've
>>> been giving lots of thought to the bottom layer of our collection
>> 
>>> abstraction and how it may limit our potential for efficiency.  In
>>> particular, I want to keep the door open for optimizations that work on
>>> contiguous memory regions.  Every cache-friendly data structure, even if
>>> it is not an array, contains contiguous memory regions over which
>>> operations can often be vectorized, that should define boundaries for
>>> parallelism, etc.  Throughout Cocoa you can find patterns designed to
>>> exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
>>> out in readv/writev, and MPI datatypes essentially boil down to
>>> identifying the contiguous parts of data structures.  My point is that
>>> this is an important class of optimization, with numerous real-world
>>> examples.
>>> 
>>> If you think about what it means to build APIs for contiguous memory
>>> into abstractions like Sequence or Collection, at least without
>>> penalizing the lowest-level code, it means exposing UnsafeBufferPointers
>>> as a first-class part of the protocols, which is really
>>> unappealing... unless you consider that *borrowed* UnsafeBufferPointers
>>> can be made safe.  
>>> 
>>> [Well, it's slightly more complicated than that because
>>> UnsafeBufferPointer is designed to bypass bounds checking in release
>>> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
>>> something—that checks bounds unconditionally... but] the point remains
>>> that
>>> 
>>> A thing that is unsafe when it's arbitrarily copied can become safe if
>>> you ensure that it's only borrowed (in accordance with well-understood
>>> lifetime rules).
>> 
>> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
>> doesn't prevent you from making your own copy, which could then escape
>> the scope that was guaranteeing safety.
>> 
>> This is fixable, of course, but it's a more significant change to the
>> type and how it would be used.
> 
> It sounds like you're saying that, to get static safety benefits from
> ownership, we'll need a whole parallel universe of safe move-only
> types. Seems a cryin' shame.
 
 We've discussed the possibility of types being able to control
 their "borrowed" representation. Even if this isn't something we
 generalize, arrays and contiguous buffers might be important
 enough to the language that your safe BufferPointer could be
 called 'borrowed ArraySlice', with the owner backreference
 optimized out of the borrowed representation. Perhaps Array's own
 borrowed representation would benefit from acting like a slice
 rather than a whole-buffer borrow too.
>>> 
>>> The disadvantage of doing this is that it much more heavily
>>> penalizes the case where we actually do a copy from a borrowed
>>> reference — it becomes an actual array copy, not just a reference
>>> bump.
>> 
>> Fair point, though the ArraySlice/Array dichotomy strikes me as
>> already kind of encouraging this—you might pass ArraySlices down
>> into your algorithm, but we encourage people to use Array at storage
>> and API boundaries, forcing copies.
>
> Fair point.  In practice, though, I think most algorithms won't need
> to "escape" that array slice.

I disagree. I'm working on some generic matching algorithms (to lay the
foundation for String search and regexes).  There's going to be a broad
category of functions in this area that work on Strings and return
SubStrings, or work on Collections and return slices thereof.  Often
they'll be called from a context where the resultant slices don't
outlive the collection, but they still do need to be returned.

>> From a philosophical perspective of making systems Swift feel like
>> "the same language" as Swift today, it feels better to me to try to
>> express this as making our high-level safe abstractions efficient
>> rather than making our low-level unsafe abstractions safe. Given our
>> short-term goals for the borrow model as I understand them, I don't
>> think we can really make a BufferPointer-like type safe in the way
>> Dave is suggesting, since the pointer 

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[John McCall via swift-evolution]

> On Nov 10, 2016, at 9:31 AM, Joe Groff  wrote:
>> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
>> 
>>> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>>>  wrote:
 On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
  wrote:
 
 
 on Mon Nov 07 2016, John McCall  wrote:
 
>> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>> 
>> Given that we're headed for ABI (and thus stdlib API) stability, I've
>> been giving lots of thought to the bottom layer of our collection
> 
>> abstraction and how it may limit our potential for efficiency.  In
>> particular, I want to keep the door open for optimizations that work on
>> contiguous memory regions.  Every cache-friendly data structure, even if
>> it is not an array, contains contiguous memory regions over which
>> operations can often be vectorized, that should define boundaries for
>> parallelism, etc.  Throughout Cocoa you can find patterns designed to
>> exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
>> out in readv/writev, and MPI datatypes essentially boil down to
>> identifying the contiguous parts of data structures.  My point is that
>> this is an important class of optimization, with numerous real-world
>> examples.
>> 
>> If you think about what it means to build APIs for contiguous memory
>> into abstractions like Sequence or Collection, at least without
>> penalizing the lowest-level code, it means exposing UnsafeBufferPointers
>> as a first-class part of the protocols, which is really
>> unappealing... unless you consider that *borrowed* UnsafeBufferPointers
>> can be made safe.  
>> 
>> [Well, it's slightly more complicated than that because
>> UnsafeBufferPointer is designed to bypass bounds checking in release
>> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
>> something—that checks bounds unconditionally... but] the point remains
>> that
>> 
>> A thing that is unsafe when it's arbitrarily copied can become safe if
>> you ensure that it's only borrowed (in accordance with well-understood
>> lifetime rules).
> 
> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
> doesn't prevent you from making your own copy, which could then escape
> the scope that was guaranteeing safety.
> 
> This is fixable, of course, but it's a more significant change to the
> type and how it would be used.
 
 It sounds like you're saying that, to get static safety benefits from
 ownership, we'll need a whole parallel universe of safe move-only
 types. Seems a cryin' shame.
>>> 
>>> We've discussed the possibility of types being able to control their 
>>> "borrowed" representation. Even if this isn't something we generalize, 
>>> arrays and contiguous buffers might be important enough to the language 
>>> that your safe BufferPointer could be called 'borrowed ArraySlice', with 
>>> the owner backreference optimized out of the borrowed representation. 
>>> Perhaps Array's own borrowed representation would benefit from acting like 
>>> a slice rather than a whole-buffer borrow too.
>> 
>> The disadvantage of doing this is that it much more heavily penalizes the 
>> case where we actually do a copy from a borrowed reference — it becomes an 
>> actual array copy, not just a reference bump.
> 
> Fair point, though the ArraySlice/Array dichotomy strikes me as already kind 
> of encouraging this—you might pass ArraySlices down into your algorithm, but 
> we encourage people to use Array at storage and API boundaries, forcing 
> copies.

Fair point.  In practice, though, I think most algorithms won't need to 
"escape" that array slice.

> From a philosophical perspective of making systems Swift feel like "the same 
> language" as Swift today, it feels better to me to try to express this as 
> making our high-level safe abstractions efficient rather than making our 
> low-level unsafe abstractions safe. Given our short-term goals for the borrow 
> model as I understand them, I don't think we can really make a 
> BufferPointer-like type safe in the way Dave is suggesting, since the pointer 
> fields *inside* the struct need to be first class lifetime-qualified rather 
> than the value of the struct itself. Since Array and ArraySlice already 
> communicate an ownership stake in the memory they reference, a borrowed Array 
> or ArraySlice value *would* safely and efficiently provide access to 
> contiguous memory with only support for first-order borrowed/consumed 
> property declarations and not full first class lifetime support.

I agree.

John.
___
swift-evolution mailing 

Re: [swift-evolution] [swift-dev] Pure Cocoa NSNumbers and AnyHashable

[Matthew Johnson via swift-evolution]

> On Nov 10, 2016, at 2:41 PM, Joe Groff  wrote:
> 
>> 
>> On Nov 10, 2016, at 11:51 AM, Matthew Johnson > > wrote:
>> 
>>> 
>>> On Nov 10, 2016, at 1:44 PM, Joe Groff >> > wrote:
>>> 
 
 On Nov 10, 2016, at 11:42 AM, Matthew Johnson > wrote:
 
 
> On Nov 10, 2016, at 1:34 PM, Joe Groff via swift-dev  > wrote:
> 
> 
>> On Nov 10, 2016, at 10:30 AM, Philippe Hausler > > wrote:
>> 
>> So I think there are a few rough edges here not just the hashing or 
>> equality. I think the issue comes down to the subclass of NSNumber that 
>> is being used - it is defeating not only hashing but also performance 
>> and allocation optimizations in Foundation.
>> 
>> So what would we have to do to get rid of the “type preserving” NSNumber 
>> subclass?
> 
> The type-preserving subclasses remember the exact Swift type that a value 
> was bridged from, to preserve type specificity of casts so that e.g. `0 
> as Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> 
> AnyObject round-trip involves ObjC, thereby providing somewhat more 
> consistent behavior between Darwin and Corelibs-based Swift. If we were 
> willing to give that up, and say that NSNumbers just flat-out lose type 
> info and can cast back to any Swift number type, then it seems to me we 
> could use the pure Cocoa subclasses.
 
 Would these only be value-preserving casts and return nil if information 
 loss would occur?  I think that might be preferable anyway.  Maybe I’m 
 just not thinking hard enough, but when would the original type 
 information be important as long as information isn’t lost?  When I 
 interact with these casts and NSNumber (JSON parsing, etc) I generally *do 
 not* want an attempted cast that would be value-preserving to ever fail.
>>> 
>>> I'm inclined to agree that the cast should be value-preserving rather than 
>>> type-preserving. There was concern about the behavior being different on 
>>> Darwin and Linux, which is why we try to be type-preserving so that pure 
>>> Swift code that uses number values with Any or other polymorphic interfaces 
>>> behaves consistently with Cocoa Foundation code that has to traffic in 
>>> NSNumber for the same effect.
>> 
>> Are you saying that Swift on Darwin can’t have value-preserving behavior?  
>> It seems like I’m missing something here.  If value-preserving is the 
>> desirable behavior can you elaborate on the specific challenges getting in 
>> the way of having that behavior everywhere?
> 
> It would require a change to the type relationships between the number value 
> types on Swift. They are currently plain, unrelated struct types, so you 
> can't do something like:
> 
>   let x: Int = 1
>   x as? Float // produces nil
> 
> We could conceivably special-case the number types in Swift so that they do 
> value-preserving casts, and maybe that's even a good idea, but we don't today.

What bothers me about the current behavior is that when you have a numeric 
value of type `Any` its casting behavior depends on how it was constructed.  
This makes it easy to write code that works with some numeric values of type 
Any and be badly broken for others.  One can argue this is ok because the 
values have different types underlying types, but I think it turns out to be 
pretty confusing and problematic for numeric types in practice.  

This problem is compounded by the fact that *most* of the time when we work 
with opaque numeric values we’re working with actually working with values that 
do cast to all of the standard library numeric types, but aren’t necessarily 
value-preserving.  They can truncate or overflow.

To clarify what bothers me about the current behavior I’ll give an example:

let json = "{ \"one\": 1, \"onePointZero\": 1.0, \"onePointOne\": 1.1, 
\"onePointNine\": 1.1, \"largerThanInt8Max\": 270 }"
let data = json.data(using: .utf8)!

// casts always succeed, but might truncate or overflow
//let dict = try! JSONSerialization.jsonObject(with: data, options: []) as! 
[String: Any]

// casts always succeed, but might truncate or overflow
//let dict: [String: Any] = ["one": 1 as NSNumber, "onePointZero": 1.0 as 
NSNumber, "onePointOne": 1.1 as NSNumber, "onePointNine": 1.9 as NSNumber, 
"largerThanInt8Max": 270 as NSNumber]

// casts always fail unless the target type matches the original type
let dict: [String: Any] = ["one": 1, "onePointZero": 1.0, "onePointOne": 1.1, 
"onePointNine": 1.9, "largerThanInt8Max": 270]

let oneAsDouble = dict["one"] as? Double// 1 or nil
let onePointZeroAsInt = dict["onePointZero"] as? Int   // 1 or 

Re: [swift-evolution] [Proposal] Type Narrowing

[Jay Abbott via swift-evolution]

Consider this code:

struct Pet {
let name: String
weak var owner: Person?

init(name: String, owner: Person?) {
self.name = name
self.owner = owner
owner?.pet = self
}

mutating func transferOwnership(to newOwner: Person) {
let previousOwner = owner
owner = newOwner
newOwner.pet = self
if(previousOwner != nil) {
previousOwner!.pet = nil
}
}

func feed() {
}
}
class Person {
let name: String
var pet: Pet?

init(name: String) {
self.name = name
}

func givePetAway(to someone: Person) {
if pet != nil {
pet!.transferOwnership(to: someone)
//pet!.feed()
}
}
}
let bert = Person(name: "Bert")let ernie = Person(name: "Ernie")var
elmo = Pet(name: "Elmo", owner: nil)

elmo.transferOwnership(to: bert)print("Bert's pet is \(bert.pet) -
Ernie's pet is \(ernie.pet)")

bert.givePetAway(to: ernie)print("Bert's pet is \(bert.pet) - Ernie's
pet is \(ernie.pet)")

This works as expected, but if you uncomment pet!.feed() in givePetAway(to:)
it will crash, because the mutating function modifies the two-way
relationship between pet and owner.

In the code I use if pet != nil to demonstrate, in your proposal for unwrap,
if I used it to unwrap pet (a value-type, but accessed through self so it
can be modified after unwrapping because it’s not nil at the moment) the
compiler would assume I could use it and pet.feed() would crash, just as
pet!.feed() does now. In your proposal for type narrowing, it would be the
same problem. This is like your foo.value example from the proposal.

I don’t think you can get around the fact that the compiler can’t guarantee
a type-narrowed or even unwrapped mutable value, which is why if let works
as it does with an immutable snapshot.

However, type narrowing for immutable values would still be good.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Joe Groff via swift-evolution]

> On Nov 10, 2016, at 1:02 PM, Dave Abrahams  wrote:
> 
> 
> on Thu Nov 10 2016, Stephen Canon  wrote:
> 
>>> On Nov 10, 2016, at 1:30 PM, Dave Abrahams via swift-evolution 
>>>  wrote:
>>> 
>>> 
>>> on Thu Nov 10 2016, Joe Groff  wrote:
>>> 
>> 
> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
> 
>> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>>  wrote:
>>> On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
>>>  wrote:
>>> 
 
>>> 
>>> on Mon Nov 07 2016, John McCall  wrote:
>>> 
> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> Given that we're headed for ABI (and thus stdlib API) stability, I've
> been giving lots of thought to the bottom layer of our collection
 
> abstraction and how it may limit our potential for efficiency.  In
> particular, I want to keep the door open for optimizations that work 
> on
> contiguous memory regions.  Every cache-friendly data structure, even 
> if
> it is not an array, contains contiguous memory regions over which
> operations can often be vectorized, that should define boundaries for
> parallelism, etc.  Throughout Cocoa you can find patterns designed to
> exploit this fact when possible (NSFastEnumeration).  Posix I/O 
> bottoms
> out in readv/writev, and MPI datatypes essentially boil down to
> identifying the contiguous parts of data structures.  My point is that
> this is an important class of optimization, with numerous real-world
> examples.
> 
> If you think about what it means to build APIs for contiguous memory
> into abstractions like Sequence or Collection, at least without
> penalizing the lowest-level code, it means exposing 
> UnsafeBufferPointers
> as a first-class part of the protocols, which is really
> unappealing... unless you consider that *borrowed* 
> UnsafeBufferPointers
> can be made safe.  
> 
> [Well, it's slightly more complicated than that because
> UnsafeBufferPointer is designed to bypass bounds checking in release
> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
> something—that checks bounds unconditionally... but] the point remains
> that
> 
> A thing that is unsafe when it's arbitrarily copied can become safe if
> you ensure that it's only borrowed (in accordance with well-understood
> lifetime rules).
 
 UnsafeBufferPointer today is a copyable type.  Having a borrowed value
 doesn't prevent you from making your own copy, which could then escape
 the scope that was guaranteeing safety.
 
 This is fixable, of course, but it's a more significant change to the
 type and how it would be used.
>>> 
>>> It sounds like you're saying that, to get static safety benefits from
>>> ownership, we'll need a whole parallel universe of safe move-only
>>> types. Seems a cryin' shame.
>> 
>> We've discussed the possibility of types being able to control
>> their "borrowed" representation. Even if this isn't something we
>> generalize, arrays and contiguous buffers might be important enough
>> to the language that your safe BufferPointer could be called
>> 'borrowed ArraySlice', with the owner backreference optimized
>> out of the borrowed representation. Perhaps Array's own borrowed
>> representation would benefit from acting like a slice rather than a
>> whole-buffer borrow too.
> 
> The disadvantage of doing this is that it much more heavily
> penalizes the case where we actually do a copy from a borrowed
> reference — it becomes an actual array copy, not just a reference
> bump.
 
 Fair point, though the ArraySlice/Array dichotomy strikes me as
 already kind of encouraging this—you might pass ArraySlices down into
 your algorithm, but we encourage people to use Array at storage and
 API boundaries, forcing copies.
 
 From a philosophical perspective of making systems Swift feel like
 "the same language" as Swift today, it feels better to me to try to
 express this as making our high-level safe abstractions efficient
 rather than making our low-level unsafe abstractions safe. 
>>> 
>>> +1, or maybe 10
>>> 
>>> What worries me is that if systems programmers are trying to get static
>>> guarantees that there's no ARC traffic, they won't be willing to handle
>>> a copyable thing that carries ownership.
>> 
>> FWIW, we (frequently) only need a static guarantee of no ARC traffic

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Dave Abrahams via swift-evolution]

on Thu Nov 10 2016, Stephen Canon  wrote:

>> On Nov 10, 2016, at 1:30 PM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>> 
>> on Thu Nov 10 2016, Joe Groff  wrote:
>> 
>
 On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
 
> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>  wrote:
>> On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>>> 
>> 
>> on Mon Nov 07 2016, John McCall  wrote:
>> 
 On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
  wrote:
 
 
 Given that we're headed for ABI (and thus stdlib API) stability, I've
 been giving lots of thought to the bottom layer of our collection
>>> 
 abstraction and how it may limit our potential for efficiency.  In
 particular, I want to keep the door open for optimizations that work on
 contiguous memory regions.  Every cache-friendly data structure, even 
 if
 it is not an array, contains contiguous memory regions over which
 operations can often be vectorized, that should define boundaries for
 parallelism, etc.  Throughout Cocoa you can find patterns designed to
 exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
 out in readv/writev, and MPI datatypes essentially boil down to
 identifying the contiguous parts of data structures.  My point is that
 this is an important class of optimization, with numerous real-world
 examples.
 
 If you think about what it means to build APIs for contiguous memory
 into abstractions like Sequence or Collection, at least without
 penalizing the lowest-level code, it means exposing 
 UnsafeBufferPointers
 as a first-class part of the protocols, which is really
 unappealing... unless you consider that *borrowed* UnsafeBufferPointers
 can be made safe.  
 
 [Well, it's slightly more complicated than that because
 UnsafeBufferPointer is designed to bypass bounds checking in release
 builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
 something—that checks bounds unconditionally... but] the point remains
 that
 
 A thing that is unsafe when it's arbitrarily copied can become safe if
 you ensure that it's only borrowed (in accordance with well-understood
 lifetime rules).
>>> 
>>> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
>>> doesn't prevent you from making your own copy, which could then escape
>>> the scope that was guaranteeing safety.
>>> 
>>> This is fixable, of course, but it's a more significant change to the
>>> type and how it would be used.
>> 
>> It sounds like you're saying that, to get static safety benefits from
>> ownership, we'll need a whole parallel universe of safe move-only
>> types. Seems a cryin' shame.
> 
> We've discussed the possibility of types being able to control
> their "borrowed" representation. Even if this isn't something we
> generalize, arrays and contiguous buffers might be important enough
> to the language that your safe BufferPointer could be called
> 'borrowed ArraySlice', with the owner backreference optimized
> out of the borrowed representation. Perhaps Array's own borrowed
> representation would benefit from acting like a slice rather than a
> whole-buffer borrow too.
 
 The disadvantage of doing this is that it much more heavily
 penalizes the case where we actually do a copy from a borrowed
 reference — it becomes an actual array copy, not just a reference
 bump.
>>> 
>>> Fair point, though the ArraySlice/Array dichotomy strikes me as
>>> already kind of encouraging this—you might pass ArraySlices down into
>>> your algorithm, but we encourage people to use Array at storage and
>>> API boundaries, forcing copies.
>>> 
>>> From a philosophical perspective of making systems Swift feel like
>>> "the same language" as Swift today, it feels better to me to try to
>>> express this as making our high-level safe abstractions efficient
>>> rather than making our low-level unsafe abstractions safe. 
>> 
>> +1, or maybe 10
>> 
>> What worries me is that if systems programmers are trying to get static
>> guarantees that there's no ARC traffic, they won't be willing to handle
>> a copyable thing that carries ownership.
>
> FWIW, we (frequently) only need a static guarantee of no ARC traffic
> *within a critical section*. If we can guarantee that whatever ARC
> operations need to be done happen in a precisely-controlled manner at
> a known interface boundary, that’s often good enough.

I don't think you 

Re: [swift-evolution] [swift-dev] Pure Cocoa NSNumbers and AnyHashable

[Joe Groff via swift-evolution]

> On Nov 10, 2016, at 11:51 AM, Matthew Johnson  wrote:
> 
>> 
>> On Nov 10, 2016, at 1:44 PM, Joe Groff > > wrote:
>> 
>>> 
>>> On Nov 10, 2016, at 11:42 AM, Matthew Johnson >> > wrote:
>>> 
>>> 
 On Nov 10, 2016, at 1:34 PM, Joe Groff via swift-dev > wrote:
 
 
> On Nov 10, 2016, at 10:30 AM, Philippe Hausler  > wrote:
> 
> So I think there are a few rough edges here not just the hashing or 
> equality. I think the issue comes down to the subclass of NSNumber that 
> is being used - it is defeating not only hashing but also performance and 
> allocation optimizations in Foundation.
> 
> So what would we have to do to get rid of the “type preserving” NSNumber 
> subclass?
 
 The type-preserving subclasses remember the exact Swift type that a value 
 was bridged from, to preserve type specificity of casts so that e.g. `0 as 
 Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> 
 AnyObject round-trip involves ObjC, thereby providing somewhat more 
 consistent behavior between Darwin and Corelibs-based Swift. If we were 
 willing to give that up, and say that NSNumbers just flat-out lose type 
 info and can cast back to any Swift number type, then it seems to me we 
 could use the pure Cocoa subclasses.
>>> 
>>> Would these only be value-preserving casts and return nil if information 
>>> loss would occur?  I think that might be preferable anyway.  Maybe I’m just 
>>> not thinking hard enough, but when would the original type information be 
>>> important as long as information isn’t lost?  When I interact with these 
>>> casts and NSNumber (JSON parsing, etc) I generally *do not* want an 
>>> attempted cast that would be value-preserving to ever fail.
>> 
>> I'm inclined to agree that the cast should be value-preserving rather than 
>> type-preserving. There was concern about the behavior being different on 
>> Darwin and Linux, which is why we try to be type-preserving so that pure 
>> Swift code that uses number values with Any or other polymorphic interfaces 
>> behaves consistently with Cocoa Foundation code that has to traffic in 
>> NSNumber for the same effect.
> 
> Are you saying that Swift on Darwin can’t have value-preserving behavior?  It 
> seems like I’m missing something here.  If value-preserving is the desirable 
> behavior can you elaborate on the specific challenges getting in the way of 
> having that behavior everywhere?

It would require a change to the type relationships between the number value 
types on Swift. They are currently plain, unrelated struct types, so you can't 
do something like:

let x: Int = 1
x as? Float // produces nil

We could conceivably special-case the number types in Swift so that they do 
value-preserving casts, and maybe that's even a good idea, but we don't today.

-Joe___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review extended] SE-0143: Conditional Conformances

[T.J. Usiyan via swift-evolution]

+1 on the revised version.

I agree with the conservative approach of disallowing overlapping/multiple
conformance with plans to consider expanding behavior later.

On Wed, Nov 9, 2016 at 7:03 PM, Joe Groff via swift-evolution <
swift-evolution@swift.org> wrote:

> During the first round of review for SE-0143, there was concern about the
> clarity of the proposal as written, since the discussion fixated on a
> number of ancillary issues not intended to be core to the proposal. Doug
> has revised the proposal for another round of feedback, so the review
> period for this proposal has been extended through *November 15*.
>
> -Joe
>
> On Sep 28, 2016, at 3:15 PM, Joe Groff via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> Hello Swift community,
>
> The review of “Conditional Conformances” begins now and runs through
> October 7. The proposal is available here:
> https://github.com/apple/swift-evolution/blob/master/
> proposals/0143-conditional-conformances.md
>
> Reviews are an important part of the Swift evolution process. All
> reviews should be sent to the swift-evolution mailing list at
>
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
> or, if you would like to keep your feedback private, directly to
> the review manager. When replying, please try to keep the proposal link
> at the top of the message:
>
> Proposal link:
>
> https://github.com/apple/swift-evolution/blob/master/
> proposals/0143-conditional-conformances.md
>
> Reply text
>
> Other replies
>
> What goes into a review?
>
> The goal of the review process is to improve the proposal under
> review through constructive criticism and, eventually, determine the
> direction of Swift. When writing your review, here are some questions you
> might want to answer in your review:
>
>- What is your evaluation of the proposal?
>- Is the problem being addressed significant enough to warrant
>a change to Swift?
>- Does this proposal fit well with the feel and direction of Swift?
>- If you have used other languages or libraries with a
>similar feature, how do you feel that this proposal compares to those?
>- How much effort did you put into your review? A glance, a
>quick reading, or an in-depth study?
>
> More information about the Swift evolution process is available at
>
> https://github.com/apple/swift-evolution/blob/master/process.md
>
> Thank you,
>
> -Joe
>
> Review Manager
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-dev] Pure Cocoa NSNumbers and AnyHashable

[Matthew Johnson via swift-evolution]

> On Nov 10, 2016, at 1:44 PM, Joe Groff  wrote:
> 
>> 
>> On Nov 10, 2016, at 11:42 AM, Matthew Johnson  wrote:
>> 
>> 
>>> On Nov 10, 2016, at 1:34 PM, Joe Groff via swift-dev  
>>> wrote:
>>> 
>>> 
 On Nov 10, 2016, at 10:30 AM, Philippe Hausler  wrote:
 
 So I think there are a few rough edges here not just the hashing or 
 equality. I think the issue comes down to the subclass of NSNumber that is 
 being used - it is defeating not only hashing but also performance and 
 allocation optimizations in Foundation.
 
 So what would we have to do to get rid of the “type preserving” NSNumber 
 subclass?
>>> 
>>> The type-preserving subclasses remember the exact Swift type that a value 
>>> was bridged from, to preserve type specificity of casts so that e.g. `0 as 
>>> Any as AnyObject as Any as? Float` doesn't succeed even if the Any <-> 
>>> AnyObject round-trip involves ObjC, thereby providing somewhat more 
>>> consistent behavior between Darwin and Corelibs-based Swift. If we were 
>>> willing to give that up, and say that NSNumbers just flat-out lose type 
>>> info and can cast back to any Swift number type, then it seems to me we 
>>> could use the pure Cocoa subclasses.
>> 
>> Would these only be value-preserving casts and return nil if information 
>> loss would occur?  I think that might be preferable anyway.  Maybe I’m just 
>> not thinking hard enough, but when would the original type information be 
>> important as long as information isn’t lost?  When I interact with these 
>> casts and NSNumber (JSON parsing, etc) I generally *do not* want an 
>> attempted cast that would be value-preserving to ever fail.
> 
> I'm inclined to agree that the cast should be value-preserving rather than 
> type-preserving. There was concern about the behavior being different on 
> Darwin and Linux, which is why we try to be type-preserving so that pure 
> Swift code that uses number values with Any or other polymorphic interfaces 
> behaves consistently with Cocoa Foundation code that has to traffic in 
> NSNumber for the same effect.

Are you saying that Swift on Darwin can’t have value-preserving behavior?  It 
seems like I’m missing something here.  If value-preserving is the desirable 
behavior can you elaborate on the specific challenges getting in the way of 
having that behavior everywhere?

> 
> -Joe

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0145: Package Manager Version Pinning

[Anders Bertelrud via swift-evolution]

Thank you, Daniel.  I have marked the SE-0145 as such in the swift-evolution 
repository.

Anders

> On 2016-11-10, at 09.41, Daniel Dunbar  wrote:
> 
> Thanks to everyone who participated in this review!
> 
> Based on the pretty universal negative feedback, we are going to reject this 
> proposal as is, and take it back for another round of revisions.
> 
> Our revised plan is:
> 1. To introduce an "autopin" behavior to cover the problem Paul outlined 
> where `pin --all` effectively needs to be "sticky" for any new dependencies 
> which come into play.
> 2. To make auto pinning on by default, with an explicit mechanism for 
> projects to opt out.
> 
> I hope to have this written up for review next week.
> 
> Thanks!
>  - Daniel
> 
>> On Nov 4, 2016, at 9:06 AM, Paul Cantrell via swift-evolution 
>> > wrote:
>> 
>> What is your evaluation of the proposal?
>> 
>> General +1, with reservations. Novel elements of the proposed behavior will 
>> need careful evaluation and refinement as we see how they plays out in 
>> practice, with open-mindedness from both users and the core team.
>> 
>> Breaking it down:
>> 
>> +1 on having this feature in some form. It’s essential.
>> 
>> +1 on making user choice about .gitignore the thing that controls whether 
>> and how pinning is shared within a team. That’s simple, clear, accommodates 
>> a wide range of needs, and is consistent with other package managers.
>> 
>> +1 that pinfiles have no effect whatsoever on dependent projects. That’s the 
>> only sensible way for it to work, but since there was some debate about 
>> that, I’ll just reiterate support.
>> 
>> -1 on making dependencies unpinned by default. Trying to induce unexpected 
>> behavior to encourage testing can be a good technique — in contexts where 
>> testing is the goal. My gut tells me that doing this when building is the 
>> goal will cause a lot of confusion and kvetching. I follow the proposal’s 
>> argument that unexpected breakages are a nice way to make strict semver a 
>> community norm … and I just do not buy it.
>> 
>> However, given that we hashed this out at great length and the core team is 
>> still enamored of the idea, I’m willing to give it a try! I’d love to be 
>> proved wrong.
>> 
>> +1 on the proposed command structure given that I’ve lost the aforementioned 
>> “always pin” argument. Living in a sometimes-pinned-sometimes-not world is 
>> going to be confusing, but the proposed commands help as best they can.
>> 
>> ¿-1? This is a big one. If I do:
>> 
>> swift package pin --all
>> 
>> …and then add a new dependency, is the new dependency also pinned? It should 
>> be. To pin or not to pin is typically a project- and team-wide policy 
>> decision. I do see the use case for pinning just one ill-behaved dependency, 
>> but more typically pinning is something that is built in to a team’s testing 
>> process and their assumptions about a whole build’s behavior.
>> 
>> The proposal is vague on this point, but could be interpreted to mean that 
>> --all does not pin new dependencies: “Dependencies are never automatically 
>> pinned, pinning is only ever taken as a result of an explicit user action.” 
>> (Aside: there should be a semicolon instead of a comma in that sentence.) I 
>> assume — hope — that this is not the case! If --all does not affect new 
>> dependencies, then I’m -1 on the proposal.
>> 
>> ¿-1? The proposal mentions that SwiftPM already effectively performs local 
>> pinning, but is ambiguous about whether this behavior remains separate from 
>> the new pinfile. I’m dubious about having two separate pinning mechanisms, 
>> one visible and one invisible.
>> 
>> +1 to the proposal’s repeated mentions of clear output and helpful 
>> diagnostics. Since this proposal introduces behavior that’s somewhat off the 
>> beaten path for package managers, this will be essential.
>> 
>> On the pin/lock controversy
>> 
>> I don’t care. Computer science is full of heavily overloaded terms where a 
>> loose underlying concept takes on radically different meanings (bridge, 
>> channel, dispatch, edge, graph, header, key, model, module, node, open, 
>> parameter, port, process, protocol, query, return, row, source, union). We 
>> do just fine disambiguating all these in context, thank you very much. 
>> Renaming “lock” to “pin” solves a problem that doesn’t exist.
>> 
>> However, we programmers are _also_ used to dealing with synonyms or 
>> partially overlapping near-synonyms (nil / null; closure / lambda / block; 
>> field / instance variable; tagged union / associated type enum; etc) and we 
>> also do just fine with those too. I’m sure we’ll learn to deal with lock / 
>> pin, and nobody will care after 6 months.
>> 
>> In short, “pin” is an unobjectionable solution to a non-problem. Core team 
>> is excited about “pin?” Grand. It’s a fine term. Do it and move on.
>> 
>> 
>> Is the problem being 

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Stephen Canon via swift-evolution]

> On Nov 10, 2016, at 1:30 PM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Thu Nov 10 2016, Joe Groff  wrote:
> 
>>> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
>>> 
 On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
  wrote:
> On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
>  wrote:
> 
>> 
> 
> on Mon Nov 07 2016, John McCall  wrote:
> 
>>> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>>>  wrote:
>>> 
>>> 
>>> Given that we're headed for ABI (and thus stdlib API) stability, I've
>>> been giving lots of thought to the bottom layer of our collection
>> 
>>> abstraction and how it may limit our potential for efficiency.  In
>>> particular, I want to keep the door open for optimizations that work on
>>> contiguous memory regions.  Every cache-friendly data structure, even if
>>> it is not an array, contains contiguous memory regions over which
>>> operations can often be vectorized, that should define boundaries for
>>> parallelism, etc.  Throughout Cocoa you can find patterns designed to
>>> exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
>>> out in readv/writev, and MPI datatypes essentially boil down to
>>> identifying the contiguous parts of data structures.  My point is that
>>> this is an important class of optimization, with numerous real-world
>>> examples.
>>> 
>>> If you think about what it means to build APIs for contiguous memory
>>> into abstractions like Sequence or Collection, at least without
>>> penalizing the lowest-level code, it means exposing UnsafeBufferPointers
>>> as a first-class part of the protocols, which is really
>>> unappealing... unless you consider that *borrowed* UnsafeBufferPointers
>>> can be made safe.  
>>> 
>>> [Well, it's slightly more complicated than that because
>>> UnsafeBufferPointer is designed to bypass bounds checking in release
>>> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
>>> something—that checks bounds unconditionally... but] the point remains
>>> that
>>> 
>>> A thing that is unsafe when it's arbitrarily copied can become safe if
>>> you ensure that it's only borrowed (in accordance with well-understood
>>> lifetime rules).
>> 
>> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
>> doesn't prevent you from making your own copy, which could then escape
>> the scope that was guaranteeing safety.
>> 
>> This is fixable, of course, but it's a more significant change to the
>> type and how it would be used.
> 
> It sounds like you're saying that, to get static safety benefits from
> ownership, we'll need a whole parallel universe of safe move-only
> types. Seems a cryin' shame.
 
 We've discussed the possibility of types being able to control
 their "borrowed" representation. Even if this isn't something we
 generalize, arrays and contiguous buffers might be important enough
 to the language that your safe BufferPointer could be called
 'borrowed ArraySlice', with the owner backreference optimized
 out of the borrowed representation. Perhaps Array's own borrowed
 representation would benefit from acting like a slice rather than a
 whole-buffer borrow too.
>>> 
>>> The disadvantage of doing this is that it much more heavily
>>> penalizes the case where we actually do a copy from a borrowed
>>> reference — it becomes an actual array copy, not just a reference
>>> bump.
>> 
>> Fair point, though the ArraySlice/Array dichotomy strikes me as
>> already kind of encouraging this—you might pass ArraySlices down into
>> your algorithm, but we encourage people to use Array at storage and
>> API boundaries, forcing copies.
>> 
>> From a philosophical perspective of making systems Swift feel like
>> "the same language" as Swift today, it feels better to me to try to
>> express this as making our high-level safe abstractions efficient
>> rather than making our low-level unsafe abstractions safe. 
> 
> +1, or maybe 10
> 
> What worries me is that if systems programmers are trying to get static
> guarantees that there's no ARC traffic, they won't be willing to handle
> a copyable thing that carries ownership.

FWIW, we (frequently) only need a static guarantee of no ARC traffic *within a 
critical section*. If we can guarantee that whatever ARC operations need to be 
done happen in a precisely-controlled manner at a known interface boundary, 
that’s often good enough.

– Steve

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Dave Abrahams via swift-evolution]

on Thu Nov 10 2016, Joe Groff  wrote:

>> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
>> 
>>> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>>>  wrote:
 On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
  wrote:
 
>
 
 on Mon Nov 07 2016, John McCall  wrote:
 
>> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>> 
>> Given that we're headed for ABI (and thus stdlib API) stability, I've
>> been giving lots of thought to the bottom layer of our collection
> 
>> abstraction and how it may limit our potential for efficiency.  In
>> particular, I want to keep the door open for optimizations that work on
>> contiguous memory regions.  Every cache-friendly data structure, even if
>> it is not an array, contains contiguous memory regions over which
>> operations can often be vectorized, that should define boundaries for
>> parallelism, etc.  Throughout Cocoa you can find patterns designed to
>> exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
>> out in readv/writev, and MPI datatypes essentially boil down to
>> identifying the contiguous parts of data structures.  My point is that
>> this is an important class of optimization, with numerous real-world
>> examples.
>> 
>> If you think about what it means to build APIs for contiguous memory
>> into abstractions like Sequence or Collection, at least without
>> penalizing the lowest-level code, it means exposing UnsafeBufferPointers
>> as a first-class part of the protocols, which is really
>> unappealing... unless you consider that *borrowed* UnsafeBufferPointers
>> can be made safe.  
>> 
>> [Well, it's slightly more complicated than that because
>> UnsafeBufferPointer is designed to bypass bounds checking in release
>> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
>> something—that checks bounds unconditionally... but] the point remains
>> that
>> 
>> A thing that is unsafe when it's arbitrarily copied can become safe if
>> you ensure that it's only borrowed (in accordance with well-understood
>> lifetime rules).
> 
> UnsafeBufferPointer today is a copyable type.  Having a borrowed value
> doesn't prevent you from making your own copy, which could then escape
> the scope that was guaranteeing safety.
> 
> This is fixable, of course, but it's a more significant change to the
> type and how it would be used.
 
 It sounds like you're saying that, to get static safety benefits from
 ownership, we'll need a whole parallel universe of safe move-only
 types. Seems a cryin' shame.
>>> 
>>> We've discussed the possibility of types being able to control
>>> their "borrowed" representation. Even if this isn't something we
>>> generalize, arrays and contiguous buffers might be important enough
>>> to the language that your safe BufferPointer could be called
>>> 'borrowed ArraySlice', with the owner backreference optimized
>>> out of the borrowed representation. Perhaps Array's own borrowed
>>> representation would benefit from acting like a slice rather than a
>>> whole-buffer borrow too.
>> 
>> The disadvantage of doing this is that it much more heavily
>> penalizes the case where we actually do a copy from a borrowed
>> reference — it becomes an actual array copy, not just a reference
>> bump.
>
> Fair point, though the ArraySlice/Array dichotomy strikes me as
> already kind of encouraging this—you might pass ArraySlices down into
> your algorithm, but we encourage people to use Array at storage and
> API boundaries, forcing copies.
>
> From a philosophical perspective of making systems Swift feel like
> "the same language" as Swift today, it feels better to me to try to
> express this as making our high-level safe abstractions efficient
> rather than making our low-level unsafe abstractions safe. 

+1, or maybe 10

What worries me is that if systems programmers are trying to get static
guarantees that there's no ARC traffic, they won't be willing to handle
a copyable thing that carries ownership.

> Given our short-term goals for the borrow model as I understand them,
> I don't think we can really make a BufferPointer-like type safe in the
> way Dave is suggesting, since the pointer fields *inside* the struct
> need to be first class lifetime-qualified rather than the value of the
> struct itself. 

Well now, those fields don't have to be public API, and it would be fine
for the implementation of this thing to use unsafe constructs.

> Since Array and ArraySlice already communicate an ownership stake in
> the memory they reference, a borrowed Array or ArraySlice value
> *would* safely and efficiently provide access to contiguous memory
> with 

Re: [swift-evolution] Some concerns on custom operators

[Joe Groff via swift-evolution]

> On Nov 10, 2016, at 7:23 AM, Jay Abbott via swift-evolution 
>  wrote:
> 
> Would it make sense to allow some kind of operator aliasing on import, so 
> that developers can at least work-around library conflicts?

Definitely. I think something like the import improvements Robert Widmann was 
proposing a while back are sorely needed, not only for operators but for 
managing conflicts in general.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0145: Package Manager Version Pinning

[Daniel Dunbar via swift-evolution]

Thanks to everyone who participated in this review!

Based on the pretty universal negative feedback, we are going to reject this 
proposal as is, and take it back for another round of revisions.

Our revised plan is:
1. To introduce an "autopin" behavior to cover the problem Paul outlined where 
`pin --all` effectively needs to be "sticky" for any new dependencies which 
come into play.
2. To make auto pinning on by default, with an explicit mechanism for projects 
to opt out.

I hope to have this written up for review next week.

Thanks!
 - Daniel

> On Nov 4, 2016, at 9:06 AM, Paul Cantrell via swift-evolution 
>  wrote:
> 
> What is your evaluation of the proposal?
> 
> General +1, with reservations. Novel elements of the proposed behavior will 
> need careful evaluation and refinement as we see how they plays out in 
> practice, with open-mindedness from both users and the core team.
> 
> Breaking it down:
> 
> +1 on having this feature in some form. It’s essential.
> 
> +1 on making user choice about .gitignore the thing that controls whether and 
> how pinning is shared within a team. That’s simple, clear, accommodates a 
> wide range of needs, and is consistent with other package managers.
> 
> +1 that pinfiles have no effect whatsoever on dependent projects. That’s the 
> only sensible way for it to work, but since there was some debate about that, 
> I’ll just reiterate support.
> 
> -1 on making dependencies unpinned by default. Trying to induce unexpected 
> behavior to encourage testing can be a good technique — in contexts where 
> testing is the goal. My gut tells me that doing this when building is the 
> goal will cause a lot of confusion and kvetching. I follow the proposal’s 
> argument that unexpected breakages are a nice way to make strict semver a 
> community norm … and I just do not buy it.
> 
> However, given that we hashed this out at great length and the core team is 
> still enamored of the idea, I’m willing to give it a try! I’d love to be 
> proved wrong.
> 
> +1 on the proposed command structure given that I’ve lost the aforementioned 
> “always pin” argument. Living in a sometimes-pinned-sometimes-not world is 
> going to be confusing, but the proposed commands help as best they can.
> 
> ¿-1? This is a big one. If I do:
> 
> swift package pin --all
> 
> …and then add a new dependency, is the new dependency also pinned? It should 
> be. To pin or not to pin is typically a project- and team-wide policy 
> decision. I do see the use case for pinning just one ill-behaved dependency, 
> but more typically pinning is something that is built in to a team’s testing 
> process and their assumptions about a whole build’s behavior.
> 
> The proposal is vague on this point, but could be interpreted to mean that 
> --all does not pin new dependencies: “Dependencies are never automatically 
> pinned, pinning is only ever taken as a result of an explicit user action.” 
> (Aside: there should be a semicolon instead of a comma in that sentence.) I 
> assume — hope — that this is not the case! If --all does not affect new 
> dependencies, then I’m -1 on the proposal.
> 
> ¿-1? The proposal mentions that SwiftPM already effectively performs local 
> pinning, but is ambiguous about whether this behavior remains separate from 
> the new pinfile. I’m dubious about having two separate pinning mechanisms, 
> one visible and one invisible.
> 
> +1 to the proposal’s repeated mentions of clear output and helpful 
> diagnostics. Since this proposal introduces behavior that’s somewhat off the 
> beaten path for package managers, this will be essential.
> 
> On the pin/lock controversy
> 
> I don’t care. Computer science is full of heavily overloaded terms where a 
> loose underlying concept takes on radically different meanings (bridge, 
> channel, dispatch, edge, graph, header, key, model, module, node, open, 
> parameter, port, process, protocol, query, return, row, source, union). We do 
> just fine disambiguating all these in context, thank you very much. Renaming 
> “lock” to “pin” solves a problem that doesn’t exist.
> 
> However, we programmers are _also_ used to dealing with synonyms or partially 
> overlapping near-synonyms (nil / null; closure / lambda / block; field / 
> instance variable; tagged union / associated type enum; etc) and we also do 
> just fine with those too. I’m sure we’ll learn to deal with lock / pin, and 
> nobody will care after 6 months.
> 
> In short, “pin” is an unobjectionable solution to a non-problem. Core team is 
> excited about “pin?” Grand. It’s a fine term. Do it and move on.
> 
> 
> Is the problem being addressed significant enough to warrant a change to 
> Swift?
> 
> It’s essential. SwiftPM will be impractical in many real-world situations 
> until this is sorted out.
> 
> 
> Does this proposal fit well with the feel and direction of Swift?
> 
> Yes, it’s consistent with the general approach of SwiftPM.
> 
> 
> If you have 

Re: [swift-evolution] Contiguous Memory and the Effect of Borrowing on Safety

[Joe Groff via swift-evolution]

> On Nov 8, 2016, at 9:29 AM, John McCall  wrote:
> 
>> On Nov 8, 2016, at 7:44 AM, Joe Groff via swift-evolution 
>>  wrote:
>>> On Nov 7, 2016, at 3:55 PM, Dave Abrahams via swift-evolution 
>>>  wrote:
>>> 
>>> 
>>> on Mon Nov 07 2016, John McCall  wrote:
>>> 
> On Nov 6, 2016, at 1:20 PM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> Given that we're headed for ABI (and thus stdlib API) stability, I've
> been giving lots of thought to the bottom layer of our collection
 
> abstraction and how it may limit our potential for efficiency.  In
> particular, I want to keep the door open for optimizations that work on
> contiguous memory regions.  Every cache-friendly data structure, even if
> it is not an array, contains contiguous memory regions over which
> operations can often be vectorized, that should define boundaries for
> parallelism, etc.  Throughout Cocoa you can find patterns designed to
> exploit this fact when possible (NSFastEnumeration).  Posix I/O bottoms
> out in readv/writev, and MPI datatypes essentially boil down to
> identifying the contiguous parts of data structures.  My point is that
> this is an important class of optimization, with numerous real-world
> examples.
> 
> If you think about what it means to build APIs for contiguous memory
> into abstractions like Sequence or Collection, at least without
> penalizing the lowest-level code, it means exposing UnsafeBufferPointers
> as a first-class part of the protocols, which is really
> unappealing... unless you consider that *borrowed* UnsafeBufferPointers
> can be made safe.  
> 
> [Well, it's slightly more complicated than that because
> UnsafeBufferPointer is designed to bypass bounds checking in release
> builds, and to ensure safety you'd need a BoundsCheckedBuffer—or
> something—that checks bounds unconditionally... but] the point remains
> that
> 
> A thing that is unsafe when it's arbitrarily copied can become safe if
> you ensure that it's only borrowed (in accordance with well-understood
> lifetime rules).
 
 UnsafeBufferPointer today is a copyable type.  Having a borrowed value
 doesn't prevent you from making your own copy, which could then escape
 the scope that was guaranteeing safety.
 
 This is fixable, of course, but it's a more significant change to the
 type and how it would be used.
>>> 
>>> It sounds like you're saying that, to get static safety benefits from
>>> ownership, we'll need a whole parallel universe of safe move-only
>>> types. Seems a cryin' shame.
>> 
>> We've discussed the possibility of types being able to control their 
>> "borrowed" representation. Even if this isn't something we generalize, 
>> arrays and contiguous buffers might be important enough to the language that 
>> your safe BufferPointer could be called 'borrowed ArraySlice', with the 
>> owner backreference optimized out of the borrowed representation. Perhaps 
>> Array's own borrowed representation would benefit from acting like a slice 
>> rather than a whole-buffer borrow too.
> 
> The disadvantage of doing this is that it much more heavily penalizes the 
> case where we actually do a copy from a borrowed reference — it becomes an 
> actual array copy, not just a reference bump.

Fair point, though the ArraySlice/Array dichotomy strikes me as already kind of 
encouraging this—you might pass ArraySlices down into your algorithm, but we 
encourage people to use Array at storage and API boundaries, forcing copies.

From a philosophical perspective of making systems Swift feel like "the same 
language" as Swift today, it feels better to me to try to express this as 
making our high-level safe abstractions efficient rather than making our 
low-level unsafe abstractions safe. Given our short-term goals for the borrow 
model as I understand them, I don't think we can really make a 
BufferPointer-like type safe in the way Dave is suggesting, since the pointer 
fields *inside* the struct need to be first class lifetime-qualified rather 
than the value of the struct itself. Since Array and ArraySlice already 
communicate an ownership stake in the memory they reference, a borrowed Array 
or ArraySlice value *would* safely and efficiently provide access to contiguous 
memory with only support for first-order borrowed/consumed property 
declarations and not full first class lifetime support.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Type Narrowing

[Haravikk via swift-evolution]

> On 10 Nov 2016, at 16:53, Jay Abbott  wrote:
> 
> Haravikk,
> 
> I think you missed ilya’s point with the owner/pet example:
> 
> // This is inside the Owner class...
> func freeMyPetIfIHaveOne {
>   if pet != nil {
> pet.setFree() // this sets pet = nil
> // self.pet is now nil - not due to concurrency, it was set to nil on 
> this thread in the function above.
> // However, the compiler considers it a non-optional at this point
> pet.doStuff() // Compiler allows, bad things happen!
>   }
> }
> As Dennis mentioned, narrowing only works for immutable values, and since 
> optionals are always mutable it defeats the whole justification for it. I 
> like the concept, but maybe it should be for immutables only?
> 
If pet is of type Optional then a .setFree() method on T cannot set self = 
nil, as self is of type T only in that scope.

The only way you can do something like that is to declare the setFree() method 
on Optional where Element:T, but that won't work on the updated proposal which 
uses a keyword to explicitly unwrap the variable (to avoid precisely that kind 
of Optional vs T method conflict), you can view the updated proposals here:

https://github.com/Haravikk/swift-evolution/blob/master/proposals/-optional-unwrapping.md
 

https://github.com/Haravikk/swift-evolution/blob/master/proposals/-type-narrowing.md
 


I've also checked and a similar case like this for polymorphic types shouldn't 
be an issue either, as you can't assign self to a wider (or even orthogonal) 
type.

In other words, the only way that .setFree() can make changes that would break 
narrowing would be to have a reference to the instance you're working with, 
which is what the proposals now guard against. But for value types this should 
not be an issue at all.

This is all of course unless I'm missing something else, but I tried in a 
playground and I can't assign anything to self that would break 
narrowing/unwrapping that I can see, except through a reference type.___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Type Narrowing

[Jay Abbott via swift-evolution]

Haravikk,

I think you missed ilya’s point with the owner/pet example:

// This is inside the Owner class...func freeMyPetIfIHaveOne {
  if pet != nil {
pet.setFree() // this sets pet = nil
// self.pet is now nil - not due to concurrency, it was set to nil
on this thread in the function above.
// However, the compiler considers it a non-optional at this point
pet.doStuff() // Compiler allows, bad things happen!
  }
}

As Dennis mentioned, narrowing only works for immutable values, and since
optionals are always mutable it defeats the whole justification for it. I
like the concept, but maybe it should be for immutables only?
​

On Thu, 10 Nov 2016 at 11:27 Haravikk via swift-evolution <
swift-evolution@swift.org> wrote:

> On 10 Nov 2016, at 10:32, Dennis Lysenko 
> wrote:
>
> So a lot of concerns here especially ilya's are ones that wouldn't be
> brought up if people looked at existing successful implementations like
> Kotlin where they are clearly solved. (fyi, answer is only narrowing with
> immutable values.)
>
> Personally I think type narrowing with explicit opt-in has no value. All
> or nothing, the whole meat of the proposal is in it being implicit.
>
> Right you are, I think an explicit keyword is only required for optionals;
> stripping them out into their own proposal simplifies things considerably.
> I've tweaked the type-narrowing specific proposal to return implicit
> narrowing (or explicit via the is keyword and assignment, depending upon
> how you want to look at it).
>
> I do think there is still value in handling reference types as well, but
> I'm proposing that this is done with a new force is! keyword which forces
> the narrowing (but causes a runtime concurrent modification error if the
> type no longer matches what the type-checker expects), as well as a new
> @concurrency(safe) attribute that can be used to indicate variables that
> posses a safe reference to an instance, e.g- for types that use a storage
> class for copy-on-write functionality, or where the value is local to a
> method etc. (though this isn't enforced).
>
> if it makes compilation times even slower I'm probably against it - xcode
> in general has been driving me up a wall lately with a matter of minutes
> for compiling and signing our (not huge) project, so any compiler speed
> improvements take on increased precedence for me.
>
> While I agree that the current performance can leave a lot to be desired,
> I don't think that this should actually slow it down by any meaningful
> amount; most variables won't narrow so will just be a single type as
> normal, and with optionals removed from the narrowing process the
> type-checker should only ever need to compare against the narrowest type on
> the stack, i.e- the only time wider types are considered is when you assign
> a wider type to a narrowed variable, and that's just popping types off the
> stack to get the new current type.
>
> But yeah, if it were have a big impact on performance I'd recommend
> delaying it until major optimisation work has been done, but I don't see
> that it should make much of a difference, and it really shouldn't be any
> more of a burden than shadowing is today.
>
> On Wed, Nov 9, 2016, 13:52 Haravikk via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> So I'm trying to re-write the proposal with the use of a keyword for
> unwrapping in mind, to keep it simpler for myself I've done this as two
> separate proposals for the time being, one for simpler unwrapping of
> optionals, and one for type-narrowing of polymorphic types:
>
>
> https://github.com/Haravikk/swift-evolution/blob/master/proposals/-optional-unwrapping.md
>
> https://github.com/Haravikk/swift-evolution/blob/master/proposals/-type-narrowing.md
>
> In addition to feedback on each proposal, I'm interested to know whether
> people think it is better to keep these separate? They're still very
> similar features, but the differences make it pretty awkward to keep them
> in one big proposal.
>
> I've also given up on integrating enums generically into it; as I don't
> think it's possible to do it in a similar enough way, and some extension to
> pattern matching would probably be better anyway.
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] stored properties in extensions (was: associated objects)

[Paul Cantrell via swift-evolution]

This line of thought suggests that allowing stored properties for same-module 
extensions is not only much more feasible to implement, but also makes more 
sense at the user level.

> On Nov 10, 2016, at 4:36 AM, Jay Abbott  wrote:
> 
> Perhaps some types don’t lend themselves to being extended?
> 
> Intuitively I would think any extensions should not affect the core behaviour 
> at all. So if I extended a type by adding a property x, two instances with 
> everything else the same and different values of x should still be considered 
> equal by a type-specific equality check. For example you would agree that 1 + 
> 1 = 2 but what if the numbers were coloured red, blue, and yellow 
> respectively, like fridge-magnets, should 1(red) + 1(blue) = 2(yellow)? I 
> think yes. The colour is an extension, it doesn’t change the fundamental 
> concept or behaviour of an integer number.
> 
> I see extensions as a way to add functionality (and potentially data), but 
> without affecting the core behaviour. If you wanted to change behaviour then 
> you should use inheritance or composition to create something new. You can’t 
> then use your own type for instances created by a library, unless it gives 
> you a way to do that, the library would expect its own types to behave in a 
> predictable way, similarly they should behave the same way when extended.
> 
> 
> On Thu, 3 Nov 2016 at 15:14 Thorsten Seitz via swift-evolution 
> > wrote:
> Has anybody thought about the semantic issues of out-of-module extensions 
> with stored properties apart from the implementation issues?
> 
> Such properties could potentially wreak havoc with the semantics of the type 
> being extended. How would these properties play nice with an existing 
> definition of equality, for example? How can it be guaranteed that their 
> value is consistent with the remaining state? And kept that way in case of 
> mutability?
> 
> -Thorsten
> 
> > Am 15.10.2016 um 03:01 schrieb Paul Cantrell via swift-evolution 
> > >:
> >
> >
> >> On Oct 9, 2016, at 3:43 PM, Charles Srstka via swift-evolution 
> >> > wrote:
> >>
> >> Let extensions introduce stored properties, but only in the same module as 
> >> the type’s definition. Then, the compiler can just take any extensions 
> >> into consideration when it’s determining the size of the type, just as if 
> >> the properties had been declared in the type. Declaring stored properties 
> >> on an extension outside of the type’s module results in a compiler error, 
> >> exactly as today. This would, without any performance drawbacks, solve one 
> >> of the big problems that people are hoping to solve via stored properties 
> >> in extensions—the ability to organize members by protocol conformance.
> >
> > Yes please! A big strong +1 to this from me. I can think of several 
> > specific chunks of problem code that this would clean up immensely.
> >
> > Contra Karl in another message, it’s _in-module_ stored property extensions 
> > that I want most frequently. By far.
> >
> > It seems to me that Charles’s idea could be introduced as its own proposal. 
> > If out-of-module stored property extensions do eventually become feasible, 
> > then Charles’s proposal is a good stepping stone. If they never do, then 
> > his proposal has done no harm.
> >
> > I realize this probably falls into the post-ABI stability bucket, but I’d 
> > love to help write/support the proposal when its time comes.
> >
> > Cheers,
> >
> > Paul
> >
> > ___
> > swift-evolution mailing list
> > swift-evolution@swift.org 
> > https://lists.swift.org/mailman/listinfo/swift-evolution 
> > 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-evolution 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Some concerns on custom operators

[Jay Abbott via swift-evolution]

Would it make sense to allow some kind of operator aliasing on import, so
that developers can at least work-around library conflicts?

On Wed, 9 Nov 2016 at 21:59 Dave Abrahams via swift-evolution <
swift-evolution@swift.org> wrote:

>
> on Wed Nov 09 2016, John McCall  wrote:
>
> >> On Nov 9, 2016, at 1:24 PM, Dave Abrahams via swift-evolution <
> swift-evolution@swift.org> wrote:
> >> on Wed Nov 09 2016, John McCall >
> > wrote:
> >>
>  On Nov 9, 2016, at 9:25 AM, Anton Zhilin via swift-evolution <
> swift-evolution@swift.org> wrote:
>  • Upon implementation of SE-0077 in Swift 3, some libraries started
> to drop operators entirely:
> >
> >>> link #1, link #2.
>  • Declarations of the same custom operator with different precedence
> groups create a conflict.
>  • The conflict can be resolved manually, but the resolution has to be
> made in every file that uses
> >>> the operator, which defeats the reason for using operators in the
> first place.
>  • This is a part of a larger problem of conflict resolution, for
> which we don’t currently have a
> >>> systematic approach.
> >>>
> >>> It makes sense to me to provide a more module-wide conflict resolution
> >>> mechanism.  Maybe we can have some sort of "internal export" mechanism
> >>> where a file can introduce imports into other files within a project.
> >>>
> • Many libraries dealing with custom operators choose
>  to import Runes, which is basically a stockpile of operator
>  declarations. But it conflicts with Result, Swiftx and Operadics.
> >>>
> >>> Won't this just shake itself out pretty soon, assuming these projects
> >>> have any interest in interoperating?
> >>
> >> This is a well-known library interoperability dynamic, and IMO we can't
> >> expect the solution for conflicting libraries to be that you have to get
> >> the library authors to communicate with one another.  That effectively
> >> fixes nothing for the poor app developer who integrates these libraries.
> >
> > I agree that we need to solve that problem, which is why I suggested an
> approach
> > for solving that problem in the previous paragraph.
>
> Sorry if I didn't read carefully enough.
>
> > But it's still reasonable for us as "wardens of the ecosystem" to ask
> > library authors to consider how their libraries interoperate with
> > their peers.
>
> Sure; that's part of the job of writing a library.
>
> > We can also make a stronger effort to ignore spurious conflicts in the
> > language, of course, e.g. by only complaining if conflicting
> > precedencegroup declarations would yield different parsing results;
> > but that logic would get unworkably complex pretty quick.
> >
> > John.
>
> --
> -Dave
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Type Narrowing

[Haravikk via swift-evolution]

> On 10 Nov 2016, at 10:32, Dennis Lysenko  wrote:
> So a lot of concerns here especially ilya's are ones that wouldn't be brought 
> up if people looked at existing successful implementations like Kotlin where 
> they are clearly solved. (fyi, answer is only narrowing with immutable 
> values.)
> 
> Personally I think type narrowing with explicit opt-in has no value. All or 
> nothing, the whole meat of the proposal is in it being implicit.
> 
Right you are, I think an explicit keyword is only required for optionals; 
stripping them out into their own proposal simplifies things considerably. I've 
tweaked the type-narrowing specific proposal to return implicit narrowing (or 
explicit via the is keyword and assignment, depending upon how you want to look 
at it).

I do think there is still value in handling reference types as well, but I'm 
proposing that this is done with a new force is! keyword which forces the 
narrowing (but causes a runtime concurrent modification error if the type no 
longer matches what the type-checker expects), as well as a new 
@concurrency(safe) attribute that can be used to indicate variables that posses 
a safe reference to an instance, e.g- for types that use a storage class for 
copy-on-write functionality, or where the value is local to a method etc. 
(though this isn't enforced).
> if it makes compilation times even slower I'm probably against it - xcode in 
> general has been driving me up a wall lately with a matter of minutes for 
> compiling and signing our (not huge) project, so any compiler speed 
> improvements take on increased precedence for me.
> 
While I agree that the current performance can leave a lot to be desired, I 
don't think that this should actually slow it down by any meaningful amount; 
most variables won't narrow so will just be a single type as normal, and with 
optionals removed from the narrowing process the type-checker should only ever 
need to compare against the narrowest type on the stack, i.e- the only time 
wider types are considered is when you assign a wider type to a narrowed 
variable, and that's just popping types off the stack to get the new current 
type.

But yeah, if it were have a big impact on performance I'd recommend delaying it 
until major optimisation work has been done, but I don't see that it should 
make much of a difference, and it really shouldn't be any more of a burden than 
shadowing is today.

>> On Wed, Nov 9, 2016, 13:52 Haravikk via swift-evolution 
>> > wrote:
>> So I'm trying to re-write the proposal with the use of a keyword for 
>> unwrapping in mind, to keep it simpler for myself I've done this as two 
>> separate proposals for the time being, one for simpler unwrapping of 
>> optionals, and one for type-narrowing of polymorphic types:
>> 
>> https://github.com/Haravikk/swift-evolution/blob/master/proposals/-optional-unwrapping.md
>>  
>> 
>> https://github.com/Haravikk/swift-evolution/blob/master/proposals/-type-narrowing.md
>>  
>> 
>> 
>> In addition to feedback on each proposal, I'm interested to know whether 
>> people think it is better to keep these separate? They're still very similar 
>> features, but the differences make it pretty awkward to keep them in one big 
>> proposal.
>> 
>> I've also given up on integrating enums generically into it; as I don't 
>> think it's possible to do it in a similar enough way, and some extension to 
>> pattern matching would probably be better anyway.
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] stored properties in extensions (was: associated objects)

[Jay Abbott via swift-evolution]

Perhaps some types don’t lend themselves to being extended?

Intuitively I would think any extensions should not affect the core
behaviour at all. So if I extended a type by adding a property x, two
instances with everything else the same and different values of x should
still be considered equal by a type-specific equality check. For example
you would agree that 1 + 1 = 2 but what if the numbers were coloured red,
blue, and yellow respectively, like fridge-magnets, should 1(red) + 1(blue)
= 2(yellow)? I think yes. The colour is an extension, it doesn’t change the
fundamental concept or behaviour of an integer number.

I see extensions as a way to add functionality (and potentially data), but
without affecting the core behaviour. If you wanted to change behaviour
then you should use inheritance or composition to create something new. You
can’t then use your own type for instances created by a library, unless it
gives you a way to do that, the library would expect its own types to
behave in a predictable way, similarly they should behave the same way when
extended.
​

On Thu, 3 Nov 2016 at 15:14 Thorsten Seitz via swift-evolution <
swift-evolution@swift.org> wrote:

> Has anybody thought about the semantic issues of out-of-module extensions
> with stored properties apart from the implementation issues?
>
> Such properties could potentially wreak havoc with the semantics of the
> type being extended. How would these properties play nice with an existing
> definition of equality, for example? How can it be guaranteed that their
> value is consistent with the remaining state? And kept that way in case of
> mutability?
>
> -Thorsten
>
> > Am 15.10.2016 um 03:01 schrieb Paul Cantrell via swift-evolution <
> swift-evolution@swift.org>:
> >
> >
> >> On Oct 9, 2016, at 3:43 PM, Charles Srstka via swift-evolution <
> swift-evolution@swift.org> wrote:
> >>
> >> Let extensions introduce stored properties, but only in the same module
> as the type’s definition. Then, the compiler can just take any extensions
> into consideration when it’s determining the size of the type, just as if
> the properties had been declared in the type. Declaring stored properties
> on an extension outside of the type’s module results in a compiler error,
> exactly as today. This would, without any performance drawbacks, solve one
> of the big problems that people are hoping to solve via stored properties
> in extensions—the ability to organize members by protocol conformance.
> >
> > Yes please! A big strong +1 to this from me. I can think of several
> specific chunks of problem code that this would clean up immensely.
> >
> > Contra Karl in another message, it’s _in-module_ stored property
> extensions that I want most frequently. By far.
> >
> > It seems to me that Charles’s idea could be introduced as its own
> proposal. If out-of-module stored property extensions do eventually become
> feasible, then Charles’s proposal is a good stepping stone. If they never
> do, then his proposal has done no harm.
> >
> > I realize this probably falls into the post-ABI stability bucket, but
> I’d love to help write/support the proposal when its time comes.
> >
> > Cheers,
> >
> > Paul
> >
> > ___
> > swift-evolution mailing list
> > swift-evolution@swift.org
> > https://lists.swift.org/mailman/listinfo/swift-evolution
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Proposal] Type Narrowing

[Dennis Lysenko via swift-evolution]

So a lot of concerns here especially ilya's are ones that wouldn't be
brought up if people looked at existing successful implementations like
Kotlin where they are clearly solved. (fyi, answer is only narrowing with
immutable values.)

Personally I think type narrowing with explicit opt-in has no value. All or
nothing, the whole meat of the proposal is in it being implicit.

I see too many people predisposed to considering this as if it's "compiler
magic" to the point where I don't feel the cost of arguing is worth what it
would bring to the language. Sure, it's a nice piece of syntax sugar but
it's not going to revolutionise it, and if it makes compilation times even
slower I'm probably against it - xcode in general has been driving me up a
wall lately with a matter of minutes for compiling and signing our (not
huge) project, so any compiler speed improvements take on increased
precedence for me.

Just my 2c.

Dennis

On Wed, Nov 9, 2016, 13:52 Haravikk via swift-evolution <
swift-evolution@swift.org> wrote:

> So I'm trying to re-write the proposal with the use of a keyword for
> unwrapping in mind, to keep it simpler for myself I've done this as two
> separate proposals for the time being, one for simpler unwrapping of
> optionals, and one for type-narrowing of polymorphic types:
>
>
> https://github.com/Haravikk/swift-evolution/blob/master/proposals/-optional-unwrapping.md
>
> https://github.com/Haravikk/swift-evolution/blob/master/proposals/-type-narrowing.md
>
> In addition to feedback on each proposal, I'm interested to know whether
> people think it is better to keep these separate? They're still very
> similar features, but the differences make it pretty awkward to keep them
> in one big proposal.
>
> I've also given up on integrating enums generically into it; as I don't
> think it's possible to do it in a similar enough way, and some extension to
> pattern matching would probably be better anyway.
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution