Re: [swift-evolution] Pitch: Cross-module inlining and specialization

[Andrew Trick via swift-evolution]

> On Oct 2, 2017, at 11:20 PM, Slava Pestov via swift-evolution 
>  wrote:
> 
> 
>> On Oct 2, 2017, at 11:11 PM, Slava Pestov via swift-evolution 
>> > wrote:
>> 
>>> This semantic doesn’t make sense to me, and I think we need to change it.  
>>> I think we are better served with the semantics of “the body may be 
>>> inlined, but doesn’t have to.”
>> 
>> That is the effect it has today. The decision to inline or not is made by 
>> the optimizer, and @inlinable doesn’t change anything here; it makes the 
>> body available if the optimizer chooses to do so.
> 
> Also remember we have the @inline(never) attribute. It’s not underscored so 
> I’m assuming it’s an “official” part of the language. And "@inline(never) 
> @inlinable" is a perfectly valid combination — it serializes the SIL for the 
> function body, and while inlining it is prohibited, it is still subject to 
> specialization, function signature optimizations, etc.
> 
> Slava

FWIW, the @inlinable name has always confused me. Methods not marked @inlinable 
are still internally inlinable. "Inlining" is already a term of art with 
specific semantics in other languages, and even in Swift is it's own thing to 
be controlled independently from resilience. The real issue I have with the 
name is that it says nothing about resilience. I’ll never forget that fragility 
is the opposite of resilience. I can't see how a @fragile attribute would ever 
be misconstrued.

As for the various shades of fragility of data types, I don't see why that 
can't be handled as qualifiers or additional optional attributes for expert 
developers. It’s just a matter of picking a reasonable default.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 30, 2017, at 10:45 AM, Taylor Swift  wrote:
> 
> this function initializeMemory(as:from:) says it will be 
> removed in Swift 4.0. It is now Swift 4.0. can I remove it?

It looks safe to remove. However, the doc comments in the same file should be 
updated to refer to `initializeMemory(as:from:count:)`.

+cc Nate Cook

-Andy

> On Sat, Sep 30, 2017 at 2:15 AM, Taylor Swift  > wrote:
> 
> 
> On Thu, Sep 28, 2017 at 7:59 PM, Andrew Trick  > wrote:
> 
>> On Sep 6, 2017, at 10:15 PM, Taylor Swift > > wrote:
>> 
>> okay so I think so far what’s been agreed on is 
>> 
>> 1. rename “Bytes” to “Memory” in the raw pointer API. Note: this brings the 
>> `copyBytes(from:)` collection method into the scope of this proposal
>> 
>> 2. change raw offsets to be in terms of bytes, not typed strides. This 
>> argument will be called `atByteOffset:` and will only appear in 
>> UnsafeMutableRawBufferPointer. “at:” arguments are no longer needed in 
>> UnsafeMutableRawPointer, since we can just use pointer arithmetic now.
>> 
>> 
>> 3. move UnsafeMutableBufferPointer’s `at:` arguments to the front of the 
>> parameter list. mostly cause any pointer arithmetic happens in the front so 
>> structurally we want to mirror that.
>> 
>> 4. add dual (to:) single element initializers and assigners to 
>> UnsafeMutablePointer, but not UnsafeMutableRawPointer because it’s 
>> apparently not useful there. 
>> `UnsafeMutableRawPointer.initializeMemory(as:repeating:count:)` still 
>> loses its default count to prevent confusion with its buffer variant.
>> 
>> 5. memory deallocation on buffer pointers is clearly documented to only be 
>> defined behavior when the buffer matches a whole heap block. 
> 
> 
> Kelvin,
> 
> Attempting to limit the scope of this proposal backfired. I was hoping to 
> avoid discussing changes to the slice API, instead providing basic 
> functionality within the buffer API itself. However, Dave Abrahams has argued 
> that once the slice API is extended, the positional arguments are extraneous 
> and less clear.
> 
> Instead of
> 
>   buf.intialize(at: i, from: source)
> 
> We want to force a more obvious idiom:
> 
>   buf[i.. 
> I think this is a reasonable argument and convinced myself that it's possible 
> to extend the slice API. I'm also convinced now that we don't need overloads 
> to handle an UnsafeBufferPointer source, instead we can provide a single 
> generic entry point on both UnsafeMutableBufferPointer and its slice, 
> optimized within the implementation:
> 
>  `initialize(from: S) -> (S.Iterator, Index)
> 
> We can always drop down to the UnsafePointer API to get back to a direct 
> unsafe implementation as a temporary workaround for performance issues.
> 
> Let's set aside for now whether we support full or partial 
> initialization/assignment, how to handle moveInitialize, and whether we need 
> to return the Iterator/Index. This is going to require another iteration on 
> swift-evolution, which we should discuss in a separate thread. 
> 
> At this point, I suggest removing the controversial aspects of SE-0184 so 
> that we can put the other changes behind us and focus future discussion 
> around a smaller follow-up proposal.
> 
> Here I've summarized the changes that I think could be accepted as-is:
> https://gist.github.com/atrick/c1ed7afb598e5cc943bdac7683914e3e 
> 
> 
> If you amend SE-0184 accordingly and file a new PR, I think it can be quickly 
> approved.
> 
> -Andy
> 
> 
> Part one of SE-0184 is here as SE-0184 A 
> 
>  
> I’ll implement it tomorrow and then make the PR
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 29, 2017, at 4:03 PM, Taylor Swift  wrote:
> 
> 
> 
> On Sep 29, 2017, at 5:56 PM, Dave Abrahams  > wrote:
> 
>> 
>> 
>>> On Sep 29, 2017, at 3:48 PM, Taylor Swift >> > wrote:
>>> 
>>> 
>>> 
>>> On Fri, Sep 29, 2017 at 4:13 PM, Andrew Trick >> > wrote:
>>> 
>>> 
 On Sep 29, 2017, at 1:23 PM, Taylor Swift > wrote:
 
 Instead of
 
   buf.intialize(at: i, from: source)
 
 We want to force a more obvious idiom:
 
   buf[i..>> 
>>> Sorry, a better analogy would have been:
>>> 
>>>  buf[i...].intialize(from: source)
>>> 
>>> Whether you specify the slice’s end point depends on whether you want to 
>>> completely initialize that slice or whether you’re just filling up as much 
>>> of the buffer as you can. It also depends on whether `source` is also a 
>>> buffer (of known size) or some arbitrary Sequence.
>>> 
>>> Otherwise, point  taken.
>>> 
>>> -Andy
>>> 
>>> After thinking about this more, one-sided ranges might provide just the 
>>> expressivity we need. What if:
>>> 
>>> buf[offset...].initialize(from: source) // initializes source.count 
>>> elements from source starting from offset
>>> 
>>> buf[offset ..< endIndex].initialize(from: source) // initializes up to 
>>> source.count elements from source starting from offset
>>> 
>>> 
>>> The one sided one does not give a full initialization guarantee. The two 
>>> sided one guarantees the entire segment is initialized.
>> 
>> In every other context, x[i...] is equivalent to x[i..> 
>> I don't think breaking that precedent is a good idea.
>> 
>>> For move operations, the one sided one will fully deinitialize the source 
>>> buffer while the two sided one will only deinitialize endIndex - offset 
>>> elements. 
>> 
>> —
>> -Dave
> 
> well since people want to use subscript notation so much we need some way of 
> expressing case 1. writing both bounds in the subscript seems to imply a full 
> initialization (and thus partial movement) guarantee.

Presumably, in your use case, you’re working directly with buffers on both 
sides (please point us to the real code). With the slicing approach, that would 
be done as follows:

bufA[i ..< i + bufB.count].initialize(from: bufB)

That will enforce full initialization of the slice: precondition(self.count == 
source.count).

Your point stands that it is redundant. That point will need to be weighed 
against other points, which I think should be discussed in another thread.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 29, 2017, at 1:23 PM, Taylor Swift  wrote:
> 
> I think this is a reasonable argument and convinced myself that it's possible 
> to extend the slice API. I'm also convinced now that we don't need overloads 
> to handle an UnsafeBufferPointer source, instead we can provide a single 
> generic entry point on both UnsafeMutableBufferPointer and its slice, 
> optimized within the implementation:
> 
>  `initialize(from: S) -> (S.Iterator, Index)
> 
> We can always drop down to the UnsafePointer API to get back to a direct 
> unsafe implementation as a temporary workaround for performance issues. 
> 
> Using Sequences throws out a whole host of assumptions we’ve been taking 
> advantage of. We can’t check for source.countanymore since that requires 
> traversing the entire Sequence. And if the performance is so bad relative to 
> the UnsafePointer API, we have to ask what the purpose of such a buffer API 
> would even be. The whole purpose of the buffer API was to make it easier to 
> do things with pointers without having to keep track of buffer lengths 
> externally. It should be built around the UnsafePointer API, not the much 
> higher level Sequence API.

Those are good points. I’m also somewhat concerned about -Onone performance. 
Can you point to some code in your library that becomes excessively redundant 
by specifying the source.count at the call site?

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 10:15 PM, Taylor Swift  wrote:
> 
> okay so I think so far what’s been agreed on is 
> 
> 1. rename “Bytes” to “Memory” in the raw pointer API. Note: this brings the 
> `copyBytes(from:)` collection method into the scope of this proposal
> 
> 2. change raw offsets to be in terms of bytes, not typed strides. This 
> argument will be called `atByteOffset:` and will only appear in 
> UnsafeMutableRawBufferPointer. “at:” arguments are no longer needed in 
> UnsafeMutableRawPointer, since we can just use pointer arithmetic now.
> 
> 
> 3. move UnsafeMutableBufferPointer’s `at:` arguments to the front of the 
> parameter list. mostly cause any pointer arithmetic happens in the front so 
> structurally we want to mirror that.
> 
> 4. add dual (to:) single element initializers and assigners to 
> UnsafeMutablePointer, but not UnsafeMutableRawPointer because it’s apparently 
> not useful there. 
> `UnsafeMutableRawPointer.initializeMemory(as:repeating:count:)` still 
> loses its default count to prevent confusion with its buffer variant.
> 
> 5. memory deallocation on buffer pointers is clearly documented to only be 
> defined behavior when the buffer matches a whole heap block. 


Kelvin,

Attempting to limit the scope of this proposal backfired. I was hoping to avoid 
discussing changes to the slice API, instead providing basic functionality 
within the buffer API itself. However, Dave Abrahams has argued that once the 
slice API is extended, the positional arguments are extraneous and less clear.

Instead of

  buf.intialize(at: i, from: source)

We want to force a more obvious idiom:

  buf[i.. (S.Iterator, Index)

We can always drop down to the UnsafePointer API to get back to a direct unsafe 
implementation as a temporary workaround for performance issues.

Let's set aside for now whether we support full or partial 
initialization/assignment, how to handle moveInitialize, and whether we need to 
return the Iterator/Index. This is going to require another iteration on 
swift-evolution, which we should discuss in a separate thread. 

At this point, I suggest removing the controversial aspects of SE-0184 so that 
we can put the other changes behind us and focus future discussion around a 
smaller follow-up proposal.

Here I've summarized the changes that I think could be accepted as-is:
https://gist.github.com/atrick/c1ed7afb598e5cc943bdac7683914e3e

If you amend SE-0184 accordingly and file a new PR, I think it can be quickly 
approved.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 9, 2017, at 8:37 AM, Andrew Trick <atr...@apple.com> wrote:
> 
> 
>> On Sep 9, 2017, at 3:15 AM, Jean-Daniel <mail...@xenonium.com> wrote:
>> 
>> 
>>> Le 8 sept. 2017 à 03:03, Andrew Trick via swift-evolution 
>>> <swift-evolution@swift.org> a écrit :
>>> 
>>> 
>>>> On Sep 7, 2017, at 5:37 PM, Joe Groff <jgr...@apple.com> wrote:
>>>>> 
>>>>> The important thing is that the UnsafeBufferPointer API is clearly 
>>>>> documented. We do not want users to think it’s ok to deallocate a smaller 
>>>>> buffer than they allocated.
>>>>> 
>>>>> Unfortunately, there’s actually no way to assert this in the runtime 
>>>>> because malloc_size could be larger than the allocated capacity. 
>>>>> Incorrect code could happen to work and we can live with that.
>>>> 
>>>> Would it be sufficient to assert that malloc_good_size(passedCapacity) == 
>>>> malloc_size(base) ? It wouldn't be perfect but could still catch a lot of 
>>>> misuses.
>>> 
>>> That theory does hold up for a million random values, but I don’t know if 
>>> we can rely on malloc_size never being larger than roundUp(sz, 16). Greg?
>> 
>> You can’t. This may be true while alloc size if less than a page, but a 
>> quick test show that:
>> 
>> malloc_size(malloc(4097)) = 4608
> 
> Thanks, I was being a bit silly...
> We also have malloc_good_size(4097) = 4608.
> 
> What I was getting at is, can malloc_good_size be “dumb” for any legal 
> implementation of malloc zones?
> 
> Or can we assert malloc_good_size(x) == malloc_size(malloc(x)?
> 
> -Andy

Answer:
- this assumption is obviously dependent on a particular implementation of libc.
- users implement their malloc zone however they like (although Swift doesn't 
strictly *need* to be compatible with them).
- even current implementations of libc have various operating modes that could 
violate the assertion, you would need to guard the check
  with a runtime condition.

-Andy
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 9, 2017, at 3:15 AM, Jean-Daniel <mail...@xenonium.com> wrote:
> 
> 
>> Le 8 sept. 2017 à 03:03, Andrew Trick via swift-evolution 
>> <swift-evolution@swift.org> a écrit :
>> 
>> 
>>> On Sep 7, 2017, at 5:37 PM, Joe Groff <jgr...@apple.com> wrote:
>>>> 
>>>> The important thing is that the UnsafeBufferPointer API is clearly 
>>>> documented. We do not want users to think it’s ok to deallocate a smaller 
>>>> buffer than they allocated.
>>>> 
>>>> Unfortunately, there’s actually no way to assert this in the runtime 
>>>> because malloc_size could be larger than the allocated capacity. Incorrect 
>>>> code could happen to work and we can live with that.
>>> 
>>> Would it be sufficient to assert that malloc_good_size(passedCapacity) == 
>>> malloc_size(base) ? It wouldn't be perfect but could still catch a lot of 
>>> misuses.
>> 
>> That theory does hold up for a million random values, but I don’t know if we 
>> can rely on malloc_size never being larger than roundUp(sz, 16). Greg?
> 
> You can’t. This may be true while alloc size if less than a page, but a quick 
> test show that:
> 
> malloc_size(malloc(4097)) = 4608

Thanks, I was being a bit silly...
We also have malloc_good_size(4097) = 4608.

What I was getting at is, can malloc_good_size be “dumb” for any legal 
implementation of malloc zones?

Or can we assert malloc_good_size(x) == malloc_size(malloc(x)?

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 5:56 PM, Jordan Rose  wrote:
> 
> 
> 
>> On Sep 7, 2017, at 17:46, Andrew Trick > > wrote:
>> 
>> 
>>> On Sep 7, 2017, at 5:40 PM, Joe Groff >> > wrote:
>>> 
>>> 
 
> But then given that, I don't understand why the 'capacity' parameter is 
> necessary. Under what circumstances would it actually be faster than 
> "just" calling malloc_size?
 
 The runtime may need to hash the address or traverse a lookup table to 
 find out which allocation pool the block resides in. Now, that’s only if 
 some platform ditches full malloc compatibility for user allocations, so 
 I’m not sure how realistic it is.
>>> 
>>> It seems to me that you could still provide malloc/free compatibility with 
>>> a zone that had to do a relatively expensive traversal on free() to recover 
>>> the pool the memory came from; malloc/free just wouldn't be the ideal 
>>> interface in that situation.
>>> 
>>> -Joe
>> 
>> Joe is right, and I just learned how amazing malloc zones are.
> 
> As long as you support multiple allocators (or hide everything behind 
> malloc/free), there's already a cost of malloc_zone_from_ptr or equivalent. 
> Without seeing a concrete use case, I wouldn't want to stay with the 
> harder-to-use API in UnsafePointer itself. It might be a feature of a 
> particular allocator that you need to keep the capacity around, but it isn't 
> something generally true about Swift's memory model, and probably never will 
> be.
> 
> (Interesting reference points: malloc/malloc.h and the implementation of 
> malloc on macOS 
> 
>  - search for "free(void *ptr)".)
> 
> Jordan

I’m primarily arguing from the point of view that UnsafeBufferPointer should 
pass it’s deallocation capacity and should be implementable in terms of 
UnsafePointer. But I’m fine hiding the capacity argument from the public API 
for now. We know what the proposal author wants to do, so unless Joe still 
feels strongly, we could accept the proposal as-is, put the API decision to 
rest and focus on better documentation and and assertions.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 5:37 PM, Joe Groff  wrote:
>> 
>> The important thing is that the UnsafeBufferPointer API is clearly 
>> documented. We do not want users to think it’s ok to deallocate a smaller 
>> buffer than they allocated.
>> 
>> Unfortunately, there’s actually no way to assert this in the runtime because 
>> malloc_size could be larger than the allocated capacity. Incorrect code 
>> could happen to work and we can live with that.
> 
> Would it be sufficient to assert that malloc_good_size(passedCapacity) == 
> malloc_size(base) ? It wouldn't be perfect but could still catch a lot of 
> misuses.


That theory does hold up for a million random values, but I don’t know if we 
can rely on malloc_size never being larger than roundUp(sz, 16). Greg?

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 5:40 PM, Joe Groff  wrote:
> 
> 
>> 
>>> But then given that, I don't understand why the 'capacity' parameter is 
>>> necessary. Under what circumstances would it actually be faster than "just" 
>>> calling malloc_size?
>> 
>> The runtime may need to hash the address or traverse a lookup table to find 
>> out which allocation pool the block resides in. Now, that’s only if some 
>> platform ditches full malloc compatibility for user allocations, so I’m not 
>> sure how realistic it is.
> 
> It seems to me that you could still provide malloc/free compatibility with a 
> zone that had to do a relatively expensive traversal on free() to recover the 
> pool the memory came from; malloc/free just wouldn't be the ideal interface 
> in that situation.
> 
> -Joe

Joe is right, and I just learned how amazing malloc zones are.
-Andy
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 5:17 PM, Jordan Rose  wrote:
> 
> 
> 
>> On Sep 7, 2017, at 17:09, Andrew Trick > > wrote:
>> 
>> 
>>> On Sep 7, 2017, at 2:29 PM, Jordan Rose >> > wrote:
>>> 
 
 We do want the Swift memory model to be consistent with the reality that 
 on most platforms, we need the runtime to track block size.
>>> 
>>> I don't know where this comes from. If you don't need to be 
>>> malloc-compatible, you don't strictly "need" this information. (It would 
>>> break tools like 'leaks', though.)
>> 
>> There are two distinct but related issues (1) malloc compatibility (2) 
>> malloc/free like functionality. I know developers sometimes expect or want 
>> #2. Realistically, we will always want the runtime to provide malloc_size, 
>> even if it’s not super fast, so we’re not giving up anything long term by 
>> providing #2. The fact the #1 is also a likely goal on major platforms just 
>> reinforces that position.
> 
> I don't understand why "realistically, we will always want the runtime to 
> provide malloc_size". Could you explain why?

The standard library already makes good use of malloc_size. More to the point, 
I think “realistically" it needs to be possible to write tools for memory 
analysis and debugging. I meant “always” in the temporal sense. I don’t think 
all memory needs this, only memory that is directly under the user’s control.

> But then given that, I don't understand why the 'capacity' parameter is 
> necessary. Under what circumstances would it actually be faster than "just" 
> calling malloc_size?

The runtime may need to hash the address or traverse a lookup table to find out 
which allocation pool the block resides in. Now, that’s only if some platform 
ditches full malloc compatibility for user allocations, so I’m not sure how 
realistic it is.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 2:29 PM, Jordan Rose  wrote:
> 
>> 
>> We do want the Swift memory model to be consistent with the reality that on 
>> most platforms, we need the runtime to track block size.
> 
> I don't know where this comes from. If you don't need to be 
> malloc-compatible, you don't strictly "need" this information. (It would 
> break tools like 'leaks', though.)

There are two distinct but related issues (1) malloc compatibility (2) 
malloc/free like functionality. I know developers sometimes expect or want #2. 
Realistically, we will always want the runtime to provide malloc_size, even if 
it’s not super fast, so we’re not giving up anything long term by providing #2. 
The fact the #1 is also a likely goal on major platforms just reinforces that 
position.

>> If our memory model states that the runtime tracks capacity of manually 
>> allocated blocks, then the deallocation capacity should be optional to 
>> reflect that.
>> 
>> Still waiting to hear any arguments that something about that memory model 
>> is bad.
> 
> I don't see an advantage to having the parameter if we're going to promise 
> that it's always present. It is additional complexity for very little win in 
> a generic interface. If someone wants to write a special allocation entry 
> point that actually makes use of this, they can do so, but it shouldn't live 
> on UnsafePointer.
> 
> (We also have nothing that prevents you from doing `(ptr+1).deallocate()`, 
> but, well, "unsafe".)

I also don’t see any usability advantage in providing the extra argument for 
anyone directly using UnsafePointer, which is why I initially objected to Joe 
Groff’s request. Then I realized I care less about usability and potential 
confusion than I care that UnsafePointer API can be viewed is a specification 
of the Swift's basic memory management functionality. We want to communicate 
that data structures doing manual allocation/deallocation should provide the 
allocated capacity during deallocation if it is available. The runtime could 
make good use of that. In particular, I want UnsafeBufferPointer’s deallocate() 
to be able to call UnsafePointer.deallocate(allocatedCapacity: buffer.count), 
rather than implementing it in terms of Builtins.

Incidentally, FWIW, I think compiled code should continue to be required to 
pass the capacity to the runtime during deallocation. That way, any changes in 
the runtime implementation of deallocation, particularly extra address checks, 
are isolated to the standard library.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 11:39 AM, Jordan Rose  wrote:
> 
>> This discussion needs to be grounded by reiterating role of the API. 
>> UnsafePointer specifies the memory model without extraneous functionality or 
>> convenience.
>> 
>> The UnsafePointer.deallocate() API *is not*:
>> 
>> - a common, expected, or encouraged way to deallocate
>> 
>> - the simplest, safest, or most convenient way to deallocate
>> 
>> - necessarilly the most optimal path for deallocation
> 
> I don't think this is correct. UnsafePointer.deallocate is the API you must 
> use to deallocate memory allocated with UnsafePointer.allocate.

No, all of Swift’s APIs for manual allocation/deallocation need to be 
compatible. UnsafeBufferPointer is highly preferable to UnsafePointer today. In 
the future we will have a safe API for manual allocation, still based on the 
underlying model specified by UnsafePointer.

UnsafePointer.allocate() is *not*

- a common, expected, or encouraged way to allocate

- the simplest, safest, or most convenient way to allocate

- necessarily the most optimal path for allocation

Even though high-level APIs are specified in terms of this model, they can be 
implemented via their own fast-paths.

>  Myquestion is whether it's acceptable to break all the people who didn't 
> know this and are using it to deallocate memory allocated with malloc or new 
> on Apple platforms. It sounds like the answer to that is "no, we want to be 
> malloc-compatible", and therefore the 'capacity' parameter isn't currently 
> serving a purpose today. We will always need to check if the memory is 
> actually in the Swift pool before even believing the 'capacity' parameter.

We don’t need to claim that manually allocated Swift memory is malloc 
compatible on every platform that happens to have malloc. We do want the Swift 
memory model to be consistent with the reality that on most platforms, we need 
the runtime to track block size.

> (It is definitely true that the intent was for this to be the allocation 
> capacity, and I'm surprised you interpreted it as supporting partial 
> deallocation. But we probably can't fix that at this point.)

I never misinterpreted the meaning of the API, but apparently multiple people 
on the evolution list did. Regardless, that is not valid reason for changing 
the API. It’s only a valid reason for improving documentation and encouraging 
the use of safer APIs.

If our memory model states that the runtime tracks capacity of manually 
allocated blocks, then the deallocation capacity should be optional to reflect 
that.

Still waiting to hear any arguments that something about that memory model is 
bad.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 7, 2017, at 8:06 AM, Taylor Swift  wrote:
> 
> I don’t see any source for this claim in the documentation 
> ,
>  or the source code 
> .
>  As far as I can tell the expected behavior is that partial deallocation 
> “should” work.
> 
> On Thu, Sep 7, 2017 at 8:59 AM, Joe Groff  > wrote:
> 
> The segfaulting example is an incorrect usage. The only valid parameters to 
> deallocate(capacity:) are the base address of an allocation, and the original 
> capacity passed into allocate(); it has never been intended to support 
> partial deallocation of allocated blocks. It seems to me like this proposal 
> is based on a misunderstanding of how the API works. The documentation and/or 
> name should be clarified.
> 
> -Joe
> 
> > “fixing” this bug will cause programs that once operated on previously 
> > valid assumptions of “free()” semantics to behave differently, without any 
> > warnings ever being generated. Conversely incorrect code will suddenly 
> > become “correct” though this is less of a problem.
> >
> >> A sized implementation may fail more obviously when you violate the 
> >> contract in the future. Not having sized deallocation is a known 
> >> deficiency of the C model we've been fairly diligent about avoiding in 
> >> Swift's allocation interfaces, and it would be extremely unfortunate for 
> >> us to backpedal from it.
> >>
> >> -Joe

This discussion needs to be grounded by reiterating role of the API. 
UnsafePointer specifies the memory model without extraneous functionality or 
convenience.

The UnsafePointer.deallocate() API *is not*:

- a common, expected, or encouraged way to deallocate

- the simplest, safest, or most convenient way to deallocate

- necessarilly the most optimal path for deallocation

There is only one decision that needs to be made here. Does the Swift runtime 
track allocation size for manually allocated blocks? I think the answer should 
be "yes", or at least haven't heard a strong argument against it. 
UnsafePointer.deallocate() needs to direcly reflect that model by making 
`allocatedCapacity` an *optional* argument.

Discussion about whether this API is unsafe, misleading, suboptimal or 
incorrectly implemented are secondary. Those are all deficiencies in the 
current documentation, current implementation, and availability of higher-level 
APIs.

Note that yesterday I argued that an optional argument wasn't worth the 
potential for confusion. That's true from a practical perspective, but I had 
lost sight of need to clearly specify the memory model. We want the Swift 
runtime to both have the functionality for tracking block size and also allow 
user code to track it more efficiently. Both those intentions need to be 
reflected in this API.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 10:15 PM, Taylor Swift  wrote:
> 
> okay so I think so far what’s been agreed on is 
> 
> 1. rename “Bytes” to “Memory” in the raw pointer API. Note: this brings the 
> `copyBytes(from:)` collection method into the scope of this proposal
> 
> 2. change raw offsets to be in terms of bytes, not typed strides. This 
> argument will be called `atByteOffset:` and will only appear in 
> UnsafeMutableRawBufferPointer. “at:” arguments are no longer needed in 
> UnsafeMutableRawPointer, since we can just use pointer arithmetic now.
> 
> 
> 3. move UnsafeMutableBufferPointer’s `at:` arguments to the front of the 
> parameter list. mostly cause any pointer arithmetic happens in the front so 
> structurally we want to mirror that.
> 
> 4. add dual (to:) single element initializers and assigners to 
> UnsafeMutablePointer, but not UnsafeMutableRawPointer because it’s apparently 
> not useful there. 
> `UnsafeMutableRawPointer.initializeMemory(as:repeating:count:)` still 
> loses its default count to prevent confusion with its buffer variant.
> 
> 5. memory deallocation on buffer pointers is clearly documented to only be 
> defined behavior when the buffer matches a whole heap block. 

Yay for all 5.
-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 5:17 PM, Taylor Swift  wrote:
> 
> 
> 
> On Sep 6, 2017, at 7:01 PM, Andrew Trick  > wrote:
> 
>> 
>>> On Sep 6, 2017, at 4:54 PM, Taylor Swift >> > wrote:
>>> 
 The semantics of buffer.deallocate() needs to be: free `buffer.count` 
 bytes of memory at `buffer.baseAddress`. So, that will always be the fast 
 path!
 Kelvin, do you agree with that?
>>> 
>>> this could be problematic if you have multiple contiguous buffers carved 
>>> out of the same heap block. i agree that this is the best semantics for 
>>> buffer pointers but we need the sized backend in Swift before this is 
>>> possible else we will end up in the same boat we’re in right now with 
>>> `deallocate(capacity:)` where we would have to make buffer deallocate heap 
>>> block-based for now and then pull the rug out from underneath users later 
>>> in order to switch to the improved semantics
>> 
>> If I understand your proposal, it’s only valid to deallocate a buffer that 
>> was allocated with the same capacity. Anything else should assert.
>> -Andy
> 
> the proposal isn’t specific enough there and that’s my fault but this seems 
> like a good solution. in the future if we get a sized backend we can loosen 
> the assertions and make the partial heap block buffer case defined behavior.

The important thing is that the UnsafeBufferPointer API is clearly documented. 
We do not want users to think it’s ok to deallocate a smaller buffer than they 
allocated.

Unfortunately, there’s actually no way to assert this in the runtime because 
malloc_size could be larger than the allocated capacity. Incorrect code could 
happen to work and we can live with that.

This is really the same issue that we punted on earlier… there’s no way to 
indicate that a buffer owns its memory. So we need to rely on clear 
documentation and discourage buffer “rebasing”.

In the future, a safe buffer will always own its memory and buffer slices will 
refer back to it. 

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 4:54 PM, Taylor Swift  wrote:
> 
>> The semantics of buffer.deallocate() needs to be: free `buffer.count` bytes 
>> of memory at `buffer.baseAddress`. So, that will always be the fast path!
>> Kelvin, do you agree with that?
> 
> this could be problematic if you have multiple contiguous buffers carved out 
> of the same heap block. i agree that this is the best semantics for buffer 
> pointers but we need the sized backend in Swift before this is possible else 
> we will end up in the same boat we’re in right now with 
> `deallocate(capacity:)` where we would have to make buffer deallocate heap 
> block-based for now and then pull the rug out from underneath users later in 
> order to switch to the improved semantics

If I understand your proposal, it’s only valid to deallocate a buffer that was 
allocated with the same capacity. Anything else should assert.
-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 4:46 PM, Taylor Swift  wrote:
> 
>> A sized implementation may fail more obviously when you violate the contract 
>> in the future. Not having sized deallocation is a known deficiency of the C 
>> model we've been fairly diligent about avoiding in Swift's allocation 
>> interfaces, and it would be extremely unfortunate for us to backpedal from 
>> it.
>> 
>> -Joe
> 
> Which is worse, an active gaping hole in Swift’s memory system, or 
> potentially discouraging users from using a hypothetical future allocation 
> API? 
> 
> Making sure the existing allocation API is working properly is a prerequisite 
> to introducing a future more advanced allocation API.

I think we agree that the runtime should assert when it is passed an invalid 
deallocation size. The problem you’re describing could still occur with the 
UnsafeBufferPointer.deallocate() method if the user happened to slice and 
rebase the buffer.

-Andy ___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 3:42 PM, Joe Groff  wrote:
> 
> 
>> On Sep 6, 2017, at 3:26 PM, Andrew Trick  wrote:
>> 
>> 
>>> On Sep 6, 2017, at 2:31 PM, Joe Groff  wrote:
>>> 
 
 The fact that we’re using malloc and free is already part of the ABI 
 because old libraries need to be able to deallocate memory allocated by 
 newer libraries.
>>> 
>>> The compiler doesn't ever generate calls directly to malloc and free, and 
>>> the runtime entry points we do use already take size and alignment on both 
>>> allocation and deallocation.
>> 
>> True, we’ve never said that UnsafePointer deallocation is compatible with C, 
>> and I we don't currently expose malloc_size functionality in any API AFAIK.
>> 
 Within the standard library we could make use of some new deallocation 
 fast path in the future without worrying about backward deployment.
 
 Outside of the standard library, clients will get the benefits of whatever 
 allocator is available on their deployed platform because we now encourage 
 them to use UnsafeBufferPointer.deallocate(). We can change the 
 implementation inside UnsafeBufferPointer all we want, as long as it’s 
 still malloc-compatible.
 
 I’m sure we’ll want to provide a better allocation/deallocation API in the 
 future for systems programmers based on move-only types. That will already 
 be deployment-limited.
 
 Absolute worst case, we introduce a sized UnsafePointer.deallocate in the 
 future. Any new code outside the stdlib that wants the performance 
 advantage would either need to
 - trivially wrap deallocation using UnsafeBufferPointer
 - create a trivial UnsafePointer.deallocate thunk under an availability 
 flag
>>> 
>>> Since we already have sized deallocate, why would we take it away? If the 
>>> name is misleading, we can change the name.
>> 
>> I don't think leaving it as an optional argument is worthwhile, as I 
>> explained above. Deallocation capacity is either required or we drop it 
>> completely. If we keep it, then `allocatedCapacity` is the right name.
>> 
>> The reason for taking it away, beside being misleading, is that it exposes 
>> another level of unsafety.
>> 
>> My thinking has been that this is not the allocation fast path of the 
>> future, and the stdlib itself could track the size of unsafely-allocated 
>> blocks if it ever used a different underlying allocator.
>> 
>> Now I realize this isn't really about fast/slow deallocation paths. Removing 
>> `capacity` or even making it optional forces all future Swift 
>> implementations to provide malloc_size functionality for any piece of memory 
>> that is compatible with the Unsafe API.
>> 
>> I'm actually ok with that, because I think it's generally desirable for 
>> application memory to reside in either in malloc-compatible blocks or fixed 
>> size pools. i.e. I think malloc_size is something the platform needs. 
>> However, you seem to think this would be too restrictive in the future. How 
>> is this a known problem for C and what's your confidence level it will be a 
>> problem for Swift?
> 
> No, I agree that being malloc-compatible is a reasonable goal; on Apple 
> platforms, being able to register any custom allocator we come up with as a 
> malloc zone would mean that the platform's existing memory profiling and 
> debugging tools can still work. Even if we have a scheme where the allocator 
> directly reaches into per-thread fixed-sized pools, it seems to me like it'd 
> be hard to make malloc_size impossible to implement, though it might be slow, 
> asking each pool for each thread whether an address is inside it. Strongly 
> encouraging, if not requiring, user code to provide deallocation sizes seems 
> to me like a prerequisite to making that sort of design a net win over plain 
> malloc/free. 
> 
> -Joe

Ok good. For growable buffers, we also want the OS to give us malloc_size which 
may be larger than requested capacity.

The semantics of buffer.deallocate() needs to be: free `buffer.count` bytes of 
memory at `buffer.baseAddress`. So, that will always be the fast path!
Kelvin, do you agree with that?

Any future safe API for manual buffer allocation/deallocation will also track 
the buffer size, so that will also be the fast path.

In the future, pointer.deallocate() without capacity might become a slower 
path. But I think that the UnsafePointer allocation/deallocation paths are 
Swift's equivalent of malloc/free. If the client code knows the buffer size, it 
shouldn't be using those. In fact, we don't want to force the client to track 
capacity if deallocation is *not* performance critical.

I suppose an optional `allocatedCapacity` argument would be ok, since we should 
be asserting in the runtime anyway. It just adds complexity to the API and I 
don't buy the backward deployment argument.

-Andy

___

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 2:31 PM, Joe Groff  wrote:
> 
>> 
>> The fact that we’re using malloc and free is already part of the ABI because 
>> old libraries need to be able to deallocate memory allocated by newer 
>> libraries.
> 
> The compiler doesn't ever generate calls directly to malloc and free, and the 
> runtime entry points we do use already take size and alignment on both 
> allocation and deallocation.

True, we’ve never said that UnsafePointer deallocation is compatible with C, 
and I we don't currently expose malloc_size functionality in any API AFAIK.

>> Within the standard library we could make use of some new deallocation fast 
>> path in the future without worrying about backward deployment.
>> 
>> Outside of the standard library, clients will get the benefits of whatever 
>> allocator is available on their deployed platform because we now encourage 
>> them to use UnsafeBufferPointer.deallocate(). We can change the 
>> implementation inside UnsafeBufferPointer all we want, as long as it’s still 
>> malloc-compatible.
>> 
>> I’m sure we’ll want to provide a better allocation/deallocation API in the 
>> future for systems programmers based on move-only types. That will already 
>> be deployment-limited.
>> 
>> Absolute worst case, we introduce a sized UnsafePointer.deallocate in the 
>> future. Any new code outside the stdlib that wants the performance advantage 
>> would either need to
>> - trivially wrap deallocation using UnsafeBufferPointer
>> - create a trivial UnsafePointer.deallocate thunk under an availability flag
> 
> Since we already have sized deallocate, why would we take it away? If the 
> name is misleading, we can change the name.


I don't think leaving it as an optional argument is worthwhile, as I explained 
above. Deallocation capacity is either required or we drop it completely. If we 
keep it, then `allocatedCapacity` is the right name.

The reason for taking it away, beside being misleading, is that it exposes 
another level of unsafety.

My thinking has been that this is not the allocation fast path of the future, 
and the stdlib itself could track the size of unsafely-allocated blocks if it 
ever used a different underlying allocator.

Now I realize this isn't really about fast/slow deallocation paths. Removing 
`capacity` or even making it optional forces all future Swift implementations 
to provide malloc_size functionality for any piece of memory that is compatible 
with the Unsafe API.

I'm actually ok with that, because I think it's generally desirable for 
application memory to reside in either in malloc-compatible blocks or fixed 
size pools. i.e. I think malloc_size is something the platform needs. However, 
you seem to think this would be too restrictive in the future. How is this a 
known problem for C and what's your confidence level it will be a problem for 
Swift?

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 6, 2017, at 1:12 PM, Joe Groff via swift-evolution 
>  wrote:
> 
>> 
>> On Sep 6, 2017, at 1:06 PM, Taylor Swift  wrote:
>> 
>> 
>> 
>> On Wed, Sep 6, 2017 at 12:41 PM, Joe Groff via swift-evolution 
>>  wrote:
>>> Currently, memory is deallocated by an instance method on 
>>> UnsafeMutablePointer, deallocate(count:). Like much of the Swift pointer 
>>> API, performing this operation on a buffer pointer requires extracting 
>>> baseAddress! and count. It is very common for the allocation code above to 
>>> be immediately followed by:
>>> 
>>> defer
>>> 
>>> {
>>>buffer.
>>> baseAddress?.deallocate(capacity: buffer.count
>>> )
>>> }
>>> 
>>> This method is extremely problematic because nearly all users, on first 
>>> seeing the signature of deallocate(capacity:), will naturally conclude from 
>>> the capacity label that deallocate(capacity:) is equivalent to some kind of 
>>> realloc()that can only shrink the buffer. However this is not the actual 
>>> behavior — deallocate(capacity:) actually ignores the capacity argument and 
>>> just calls free() on self. The current API is not only awkward and 
>>> suboptimal, it is misleading. You can write perfectly legal Swift code that 
>>> shouldn’t segfault, but still can, for example
>>> 
>>> var ptr = UnsafeMutablePointer.allocate(capacity: 100
>>> )
>>> ptr.
>>> initialize(to: 13, count: 100
>>> )
>>> ptr.
>>> deallocate(capacity: 50) // deallocate the second half of the memory 
>>> block
>>> ptr[0] // segmentation fault
>>> where the first 50 addresses should still be valid if the documentation 
>>> is to be read literally.
>> 
>> The fact that the Swift runtime currently uses malloc/free is an 
>> implementation detail. Tracking deallocation size is a very useful 
>> optimization for better allocator backends, and C++ underwent an ABI break 
>> to make it so that sized delete can be supported. Maybe we can change the 
>> name to `deallocate(allocatedCapacity:)` to make it clear that it isn't 
>> resizing the memory, and/or make the capacity argument optional so that you 
>> can pay for the overhead of the allocator deriving the size if it's 
>> inconvenient for the calling code to carry the size around, but we shouldn't 
>> remove the functionality altogether.
>> 
>> -Joe
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
>> The idea is to get the house in order by removing all parameters from 
>> deallocate(), since that’s what it really does right now. Then, in the 
>> future, if Swift gets a more sophisticated allocator backend, a new method 
>> like deallocate(capacity:) or reallocate(toCapacity:) could be added without 
>> conflicting with the currently named deallocate(capacity:). However, using 
>> the function signature to pretend that it does something it can’t actually 
>> do right now is extremely dangerous.
> 
> I don't think that's a good idea in this case, because it's not unlikely we 
> would explore an optimized allocator soon after ABI stability, and 
> retrofitting these interfaces in a future version of Swift would put a 
> deployment target limit on when they can be used, and mean that a lot of user 
> code would need to be retrofitted to carry allocated capacities where needed 
> to see any benefit.
> 
> -Joe


The fact that we’re using malloc and free is already part of the ABI because 
old libraries need to be able to deallocate memory allocated by newer libraries.

Within the standard library we could make use of some new deallocation fast 
path in the future without worrying about backward deployment.

Outside of the standard library, clients will get the benefits of whatever 
allocator is available on their deployed platform because we now encourage them 
to use UnsafeBufferPointer.deallocate(). We can change the implementation 
inside UnsafeBufferPointer all we want, as long as it’s still malloc-compatible.

I’m sure we’ll want to provide a better allocation/deallocation API in the 
future for systems programmers based on move-only types. That will already be 
deployment-limited.

Absolute worst case, we introduce a sized UnsafePointer.deallocate in the 
future. Any new code outside the stdlib that wants the performance advantage 
would either need to
- trivially wrap deallocation using UnsafeBufferPointer
- create a trivial UnsafePointer.deallocate thunk under an availability flag

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 5, 2017, at 6:28 PM, Taylor Swift  wrote:
> 
>> 
>> UMRP (raw pointer):
>> --- func initializeMemory(as:at:(=0)count:(1)to:)
>> +++ func initializeMemory(as:repeating:count:) // remove default count 
>> 
>> still extremely suspicious of this but i’m willing to compromise. also there 
>> should be an `initializeMemory(at:to:)` to match the typed methods
>> 
>> Do you mean initializeMemory(as:to:)?
> 
> I don’t think it’s necessary since raw buffers are not normally used to hold 
> a single typed element. We don’t need symmetry between raw and typed pointers 
> and I wouldn’t want to add an API that will never be used. So, I would be ok 
> with it only if there’s some use case that I’m overlooking.
> 
> The only case i can think of is initializing a buffer header but then again i 
> rarely use raw pointers so i’m not sure how much this feature would be used. 
> However, this function is already 
> 
>  in the API (with a default count of 1 and offset of 0) so there’s that.

Yeah, but you’re deprecating the old API entry point anyway. The `at` argument 
was never used AFAICT. I doubt code is using the default `count` and it seems 
like a fixit could handle it.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 5, 2017, at 9:53 AM, Taylor Swift <kelvin1...@gmail.com> wrote:
> 
> we agreed to deprecate the strided at:? Note that 
> UnsafeMutableRawBufferPointer would still need a byteOffset: argument. I’m 
> also still not comfortable with duplicating functionality for the sake of 
> having two names

I’ll summarize what I think happened in this thread...

I don't think the suggested copyBytes rename was controversial:

UMRP (raw pointer):
--- copyBytes(from:count)
+++ copyMemory(from:bytes:)


UMRBP (raw buffer):
--- copyBytes(from:)
+++ copyMemory(atByteOffset:from:)

--- copyBytes(from:)
+++ copyMemory(from:)


In the new raw initializeMemory methods, Xiaodi and I agreed to make
it more clear that the offset is in terms of `self` rather than
`from`, and to further reduce ambiguity by forcing manual stride
computation and using an explicit "offset" label. The call site will
be just as explicit as dropping down to `baseAddress` but without any
pointer conversion boilerplate.

UMRBP (raw buffer):
+++ func initializeMemory(atByteOffset:as:from:)
+++ func moveInitializeMemory(atByteOffset:as:from:)


Than, in the one existing initializeMemory method, just drop the strided index.
It's never used, we want to be consistent in using a byte offset
for the raw index, and that can be expressed just as easily as pointer
arithmetic:

UMRP (raw pointer):
--- func initializeMemory(as:T.Type, at:Int = 0, count:Int = 1, to:T)
+++ func initializeMemory(as:T.Type, repeating:T, count:Int = 1)

Then you can simply leave these methods as-is:
> func initializeMemory(as:T.Type, from:UnsafePointer, count:Int)
> func moveInitializeMemory(as:T.Type, from:UnsafeMutablePointer, 
> count:Int) 


We don't have a consensus, but I think the suggestion to distinguish
between single value vs. multiple semantics was good. Otherwise,
adding the default count could be very misleading. Normally, we try to
minimize surface area, but adding two methods for the single-value case
avoids ambiguity between the buffer and pointer semantics:

UMP (pointer)
--- func initialize(to:count:(=1))
+++ func initialize(to:)
+++ func initialize(repeating:count:) // no default count
+++ func assign(to:)
+++ func assign(repeating:count:) // no default count

UMRP (raw pointer):
--- func initializeMemory(as:at:(=0)count:(1)to:)
+++ func initializeMemory(as:repeating:count:) // remove default count

-Andy

> On Tue, Sep 5, 2017 at 11:31 AM, Andrew Trick <atr...@apple.com 
> <mailto:atr...@apple.com>> wrote:
> I think we’ve agreed to a few minor updates to this proposal. Since there 
> hasn’t been any other feedback on the thread it may be worth posting an 
> amended proposal so we all know what we’ve agreed on.
> 
> -Andy
> 
>> On Sep 3, 2017, at 8:23 PM, Andrew Trick via swift-evolution 
>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>> 
>> 
>>> On Sep 3, 2017, at 8:05 PM, Xiaodi Wu <xiaodi...@gmail.com 
>>> <mailto:xiaodi...@gmail.com>> wrote:
>>> 
>>> If we use byte offset, then the at parameter in UnsafeMutableRawPointer 
>>> should be removed, since pointer arithmetic can be used instead (just like 
>>> with UnsafeMutablePointer).
>>> 
>>> I agree that it seems quite sensible to remove the ‘at’ parameter 
>>> altogether from the UMRP method.
>> 
>> No code in tree or on github is using the `at` argument. I think it can be 
>> removed. A fixit should still be possible.
>> 
>>> Not convinced moving the at: argument to come before the as: argument is 
>>> worth it in terms of source breakage.
>>> 
>>> Since much of this proposal involves shuffling and relabeling arguments, 
>>> I’d argue it’s better to break slight more source in one go for the optimal 
>>> API than to break slightly less for a slightly less optimal API, no? (This 
>>> is assuming there is agreement that ‘at:as:’ is less prone to 
>>> misinterpretation than ‘as:at:’.)
>> 
>> 
>> To be clear, we’re just talking about 
>> UnsafeMutableRawBufferPointer.initializeMemory now, so this is purely 
>> additive.
>> I think the label needs to be `atByteOffset`, and placing it before `as` 
>> makes a lot of sense because it no longer depends on the type’s stride. 
>> 
>> -Andy
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org <mailto:swift-evolution@swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>> <https://lists.swift.org/mailman/listinfo/swift-evolution>
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

I think we’ve agreed to a few minor updates to this proposal. Since there 
hasn’t been any other feedback on the thread it may be worth posting an amended 
proposal so we all know what we’ve agreed on.

-Andy

> On Sep 3, 2017, at 8:23 PM, Andrew Trick via swift-evolution 
> <swift-evolution@swift.org> wrote:
> 
> 
>> On Sep 3, 2017, at 8:05 PM, Xiaodi Wu <xiaodi...@gmail.com 
>> <mailto:xiaodi...@gmail.com>> wrote:
>> 
>> If we use byte offset, then the at parameter in UnsafeMutableRawPointer 
>> should be removed, since pointer arithmetic can be used instead (just like 
>> with UnsafeMutablePointer).
>> 
>> I agree that it seems quite sensible to remove the ‘at’ parameter altogether 
>> from the UMRP method.
> 
> No code in tree or on github is using the `at` argument. I think it can be 
> removed. A fixit should still be possible.
> 
>> Not convinced moving the at: argument to come before the as: argument is 
>> worth it in terms of source breakage.
>> 
>> Since much of this proposal involves shuffling and relabeling arguments, I’d 
>> argue it’s better to break slight more source in one go for the optimal API 
>> than to break slightly less for a slightly less optimal API, no? (This is 
>> assuming there is agreement that ‘at:as:’ is less prone to misinterpretation 
>> than ‘as:at:’.)
> 
> 
> To be clear, we’re just talking about 
> UnsafeMutableRawBufferPointer.initializeMemory now, so this is purely 
> additive.
> I think the label needs to be `atByteOffset`, and placing it before `as` 
> makes a lot of sense because it no longer depends on the type’s stride. 
> 
> -Andy
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 3, 2017, at 8:38 PM, Taylor Swift  wrote:
> 
> what was the reasoning for making raw at: offset in strides and not bytes?

So, you’re talking about UnsafeMutableRawPointer(as:at:count:to:)…

The thinking was that it was typical for users to continue filling in the 
memory using the same type, and that manually performing address arithmetic is 
error prone.

Now that we're considering consistency with the raw buffer pointer API, which 
didn’t exist at the time, the thinking is that `at` can be ambiguous and that 
manual address arithmetic is actually less error prone and increases clarity at 
the call site.

Does that adequately sum of the last few messages in this thread?

-Andy


> On Sep 3, 2017, at 10:22 PM, Andrew Trick  > wrote:
> 
>> 
>>> On Sep 3, 2017, at 8:05 PM, Xiaodi Wu >> > wrote:
>>> 
>>> If we use byte offset, then the at parameter in UnsafeMutableRawPointer 
>>> should be removed, since pointer arithmetic can be used instead (just like 
>>> with UnsafeMutablePointer).
>>> 
>>> I agree that it seems quite sensible to remove the ‘at’ parameter 
>>> altogether from the UMRP method.
>> 
>> No code in tree or on github is using the `at` argument. I think it can be 
>> removed. A fixit should still be possible.
>> 
>>> Not convinced moving the at: argument to come before the as: argument is 
>>> worth it in terms of source breakage.
>>> 
>>> Since much of this proposal involves shuffling and relabeling arguments, 
>>> I’d argue it’s better to break slight more source in one go for the optimal 
>>> API than to break slightly less for a slightly less optimal API, no? (This 
>>> is assuming there is agreement that ‘at:as:’ is less prone to 
>>> misinterpretation than ‘as:at:’.)
>> 
>> 
>> To be clear, we’re just talking about 
>> UnsafeMutableRawBufferPointer.initializeMemory now, so this is purely 
>> additive.
>> I think the label needs to be `atByteOffset`, and placing it before `as` 
>> makes a lot of sense because it no longer depends on the type’s stride. 
>> 
>> -Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 3, 2017, at 8:05 PM, Xiaodi Wu  wrote:
> 
> If we use byte offset, then the at parameter in UnsafeMutableRawPointer 
> should be removed, since pointer arithmetic can be used instead (just like 
> with UnsafeMutablePointer).
> 
> I agree that it seems quite sensible to remove the ‘at’ parameter altogether 
> from the UMRP method.

No code in tree or on github is using the `at` argument. I think it can be 
removed. A fixit should still be possible.

> Not convinced moving the at: argument to come before the as: argument is 
> worth it in terms of source breakage.
> 
> Since much of this proposal involves shuffling and relabeling arguments, I’d 
> argue it’s better to break slight more source in one go for the optimal API 
> than to break slightly less for a slightly less optimal API, no? (This is 
> assuming there is agreement that ‘at:as:’ is less prone to misinterpretation 
> than ‘as:at:’.)


To be clear, we’re just talking about 
UnsafeMutableRawBufferPointer.initializeMemory now, so this is purely additive.
I think the label needs to be `atByteOffset`, and placing it before `as` makes 
a lot of sense because it no longer depends on the type’s stride. 

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 2, 2017, at 5:34 PM, Xiaodi Wu  wrote:
> 
> On Sat, Sep 2, 2017 at 4:41 PM, Andrew Trick  > wrote:
> 
>> On Sep 2, 2017, at 2:06 PM, Taylor Swift via swift-evolution 
>> > wrote:
>> 
>> the subscript doesn’t know about the stride that you want to use. If you 
>> want to get rid of the `at:` ambiguity, you have to get rid of it 
>> everywhere, or users will just wind up having to remember two ways (one 
>> ambiguous and one less ambiguous) of doing the same thing, instead of one 
>> (ambiguous) way of doing it.
>> 
>> Certainly, that's a good point. On rethink and another re-reading of the 
>> proposal, it's unclear to me that the addition of `at` arguments to 
>> UnsafeMutablePointer is sufficiently justified by the proposal text. Is it 
>> merely that it's shorter than writing `foo + MemoryLayout.stride * 
>> offset`? With the ambiguity of `at`, it seems that the current way of 
>> writing it, though longer, is certainly less ambiguous. 
>> 
>> Please reread it; UnsafeMutablePointer’s methods do not use `at:`.
> 
> Regarding the typed buffer pointers, I think it is clear by convention, and 
> will be well documented, that the `at` label refers to a position in `self`.
> 
> The raw buffer pointer API isn’t so obvious. Since the `at` refers to `self` 
> it might more logically be a byte offset. Note that `at` as a label name 
> always refers to a strided index.
> 
> This would be a bit more explicit:
> UnsafeRawBufferPointer.initializeMemory(as:T.self, atByteOffset: position * 
> MemoryLayout.stride, from: bufferOfT)
> 
> But possibly less convenient… Since that `at` label currently on 
> UnsafeRawPointer.initializeMemory is almost never used, I don’t think we need 
> to worry too much about convenience.
> 
> That existing `at` label on UnsafeRawPointer.initializeMemory, would also 
> need to be renamed, which is fine.
> 
> If I may suggest one more incremental improvement in clarity, it would be to 
> move `at[ByteOffset]` to be the first argument; this eliminates the possible 
> reading that we are offsetting the source buffer:
> 
> ```
> UnsafeRawBufferPointer.initializeMemory(atByteOffset: position * 
> MemoryLayout.stride, as:T.self, from: bufferOfT)
> ```

Sure, that probably makes sense if we decide to go with a byte offset vs. 
stride index.
-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

> On Sep 2, 2017, at 2:06 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> the subscript doesn’t know about the stride that you want to use. If you want 
> to get rid of the `at:` ambiguity, you have to get rid of it everywhere, or 
> users will just wind up having to remember two ways (one ambiguous and one 
> less ambiguous) of doing the same thing, instead of one (ambiguous) way of 
> doing it.
> 
> Certainly, that's a good point. On rethink and another re-reading of the 
> proposal, it's unclear to me that the addition of `at` arguments to 
> UnsafeMutablePointer is sufficiently justified by the proposal text. Is it 
> merely that it's shorter than writing `foo + MemoryLayout.stride * 
> offset`? With the ambiguity of `at`, it seems that the current way of writing 
> it, though longer, is certainly less ambiguous. 
> 
> Please reread it; UnsafeMutablePointer’s methods do not use `at:`.

Regarding the typed buffer pointers, I think it is clear by convention, and 
will be well documented, that the `at` label refers to a position in `self`.

The raw buffer pointer API isn’t so obvious. Since the `at` refers to `self` it 
might more logically be a byte offset. Note that `at` as a label name always 
refers to a strided index.

This would be a bit more explicit:
UnsafeRawBufferPointer.initializeMemory(as:T.self, atByteOffset: position * 
MemoryLayout.stride, from: bufferOfT)

But possibly less convenient… Since that `at` label currently on 
UnsafeRawPointer.initializeMemory is almost never used, I don’t think we need 
to worry too much about convenience.

That existing `at` label on UnsafeRawPointer.initializeMemory, would also need 
to be renamed, which is fine.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0184: Unsafe[Mutable][Raw][Buffer]Pointer: add missing methods, adjust existing labels for clarity, and remove deallocation size

[Andrew Trick via swift-evolution]

Thanks for the review as always…

> On Sep 2, 2017, at 12:13 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> 
> 
> On Sat, Sep 2, 2017 at 10:03 AM, Xiaodi Wu via swift-evolution 
> > wrote:
> On Sat, Sep 2, 2017 at 12:27 AM, Douglas Gregor via swift-evolution 
> > wrote:
> Hello Swift community,
> 
> The review of SE-0184 "Unsafe[Mutable][Raw][Buffer]Pointer: add missing 
> methods, adjust existing labels for clarity, and remove deallocation size" 
> begins now and runs through September 7, 2017. The proposal is available here:
> 
> https://github.com/apple/swift-evolution/blob/master/proposals/0184-unsafe-pointers-add-missing.md
>  
> 
> ## UnsafeMutableBufferPointer
> 
> Please clarify: why are you proposing that the `at:` arguments in 
> `UnsafeMutableBufferPointer` and `UnsafeMutableRawBufferPointer` _should not_ 
> receive default values, but the `at:` arguments in `UnsafeMutableRawPointer` 
> _should_ receive a default value of `0`?
> 
> 
>  The extant API for `UnsafeMutableRawPointer` already included these default 
> arguments which seem to be widely used in the stdlib,, the proposal tries to 
> avoid these kinds of source breakages wherever possible. We avoid providing 
> the default arguments on buffer pointers because we want the fact that it 
> takes a start–length segment pair to be obvious at the call site.

That’s right, the buffer pointer API is just one level above pointers. It 
primarily offers the safety of tracking its capacity and bounds checking in 
debug mode. It does not help you safely manage fully initialized buffer slices 
(we *do* want to provide that convenience later as a higher-level non-pointer 
type). Instead, it exposes segments within the buffer for initialization, 
assignment, deinitialization. It needs to be obvious in the client code, at 
every call site, that the caller is responsible for tracking those segments. 
Earlier iterations of this proposal attempted to hide this as a convenience, 
but that led to dangerous scenarios.

Taking at Kelvin’s example:

var image = UnsafeMutableBufferPointer.allocate(capacity : maxPixels)

var filled:Int = 0
for scanline: UnsafeMutableBufferPointer in scanlines {
image.moveInitialize(at: filled, from: scanline)
filled += scanline.count
}
…
// We don’t want to allow any defaults here.
// It’s important for the client code to explicitly correlate the range
// that it initialized with the range that it deinitialized.
image.deinitialize(at: 0, count: filled) 

> Concerns:
> 
> ## UnsafeMutablePointer
> 
> It's alarming that omitting `count` in `initialize(repeating:count:)` (and 
> assign, etc.) means initialize _one_ element, but elsewhere (such as 
> `UnsafeMutableBufferPointer` means initialize _all_ elements. The behavior of 
> the proposed API also contradicts its own spelling on its face: 
> `initialize(repeating: foo)` means *do not repeat* `foo`.
> 
> Yes, I understand the argument that `*BufferPointer` types have an intrinsic 
> count, etc., but in the context of code where types are inferred, `let foo = 
> T.allocate(capacity: 100); foo.initialize(repeating: bar)` should not mean 
> one thing for `*BufferPointer` types and a totally different thing for plain 
> `*Pointer` types--particularly when both can be allocated with a certain 
> capacity greater than one.
> 
> Either `count` should always be required, or for convenience there should be 
> a separate overload `initialize(pointee:)` that does not require `count`.
> 
> 
> I understand the naming is not optimal, but reams of discussion on this list 
> have concluded that it’s the least bad alternative available. We can’t just 
> get rid of the default value for `count:` because usage in real code bases 
> shows that this default argument is actually extremely useful. I believe well 
> over 90% of the calls to these methods in the standard library currently rely 
> on the default argument value. Renaming the `repeating:` argument to `to:` 
> would make the API inconsistent and hide the fact that plain pointers are 
> still capable of operating on many elements in sequence — “`to:count:`” makes 
> no grammatical sense to read — “to” is a singular preposition.

The *only* thing I really don’t like about the proposed API is the potential 
confusion between UnsafeMutablePointer calls with a default count and 
UnsafeMutableBufferPointer calls using the implied buffer count. But this is 
where we ended up because the default count turns out to be useful. I’m willing 
to live with it since I don’t see a great alternative.

> 
> ## UnsafeMutableRawBufferPointer
> 
> In `copyBytes`, the use of `Bytes` to emphasize that it's the memory that's 
> being copied is thoughtful, but it is inconsistent with the other method 
> names 

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

Kelvin,

Please resubmit a clean swift-evolution PR now. I personally think this is 
ready for formal review given that all feedback was positive and all issues 
brought up during review have been addressed.

-Andy

> On Aug 22, 2017, at 12:59 PM, Michael Ilseman  wrote:
> 
> This is an excellent, thoroughly thought out, and well written proposal! I’m 
> eager to see these improvements land.
> 
> 
>> On Aug 22, 2017, at 11:33 AM, Taylor Swift > > wrote:
>> 
>> 
>> 
>> On Tue, Aug 22, 2017 at 2:35 AM, Andrew Trick > > wrote:
>> 
>>> On Aug 21, 2017, at 10:59 PM, Taylor Swift >> > wrote:
>>> 
>>> Sorry to bring this up again, but I was not able to defend the addition of 
>>> `UnsafeMutableBufferPointer.de 
>>> initialize()`. It is incorrect for 
>>> the typical use case and doesn't appear to solve any important use case. 
>>> The *only* fully initializing method is `initialize(repeating:)`, but that 
>>> will usually be used for trivial values, which should not be deinitialized. 
>>> It's preferable for the user to explicitly deinitialize just the segments 
>>> that they know were initialized, which can be done on the base pointer. The 
>>> only benefit in having a `deinitialize` on the buffer is to communicate to 
>>> users who see the `initialize` API for the first time that it is their 
>>> responsibility to deinitialize if the type requires it. To that end, we 
>>> could add a `deinitialize(at:count:)` method, communicating the symmetry 
>>> with `initialize(at:from:). Naturally `index + count <= self.count`.
>>> 
>>> -Andy
>>> 
>>> I don’t agree with this. If `deinitialize()` is a problem because it 
>>> deinitializes the entire buffer, so are `moveAssign` and `moveInitialize`. 
>>> They all assume the released buffer operand is fully initialized. 
>>> `deinitialize()` has just as much use as the other full-buffer releasing 
>>> methods. Just take the image buffer example there 
>> 
>> `moveAssign` and `moveInitialize` assume that the sub-buffer being moved 
>> from is fully initialized. That’s already obvious because the user is asking 
>> to move source.count elements. I don’t see any use cases where it would pose 
>> a problem. If the user is moving out of a partially initialized buffer, they 
>> have already to sliced (and unfortunately rebased) the buffer. OTOH 
>> `deinitialize` is incorrect for normal use cases. I don’t see any practical 
>> analogy between those APIs.
>>> let pixels:Int = scanlines.map{ $0.count }.reduce(0, +)
>>> var image = UnsafeMutableBufferPointer.allocate(capacity: pixels)
>>> 
>>> var filled:Int = 0
>>> for scanline:UnsafeMutableBufferPointer in scanlines 
>>> {
>>> image.moveInitialize(at: filled, from: scanline)
>>> filled += scanline.count
>>> }
>>> 
>>> image.deinitialize()
>> 
>> We don’t want developers to do this. Instead we want to see an explicitly 
>> named association between the number of items initialized and deinitialized:
>> 
>> image.deinitialize(at: 0, count: filled)
>> 
>> Flipping this around, it could be even more common to be writing into a 
>> larger than necessary buffer (pixels > filled). If we’re providing 
>> auto-slicing initializers, then deinitialization should follow the same 
>> approach, rather than:
>> 
>> UnsafeMutableBufferPointer(rebasing: image[0, filled]).deinitialize()
>>> image.deallocate()
>>> 
>>> and replace `Pixel` with a class type like `UIButton`.
>>> And `deinitialize(at:count:)` is bad because you’re asking for a count on a 
>>> buffer method. `moveAssign` and `moveInitialize` can take range parameters 
>>> because they each have a second operand that supplies the count number. 
>>> `deinitialize` doesn’t. That means calls could end up looking like 
>>> 
>>> buffer.deinitialize(at: 0, count: buffer.count)
>>> 
>>> which is exactly what we were trying to avoid in the first place.
>> 
>> But there is no value in avoiding the `count` argument here. That’s not a 
>> valid motivation for introducing `deinitialize` on a buffer, and we’d be 
>> better off not introducing it at all.
>> 
>> The only valid motivation I can come up with for introducing `deinitialize` 
>> on buffer is to remind developers who are only looking at the buffer API 
>> (and not the plain pointer API) that it’s their responsibility to manually 
>> deinitialize (it doesn’t automatically happen on deallocation or 
>> destruction).
>> 
>> -Andy
>> 
>> 
>> I replaced UnsafeMutableBufferPointer.deinitialize() with 
>> UnsafeMutableBufferPointer.deinitialize(at:count:)
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 21, 2017, at 10:59 PM, Taylor Swift  wrote:
> 
> Sorry to bring this up again, but I was not able to defend the addition of 
> `UnsafeMutableBufferPointer.deinitialize()`. It is incorrect for the typical 
> use case and doesn't appear to solve any important use case. The *only* fully 
> initializing method is `initialize(repeating:)`, but that will usually be 
> used for trivial values, which should not be deinitialized. It's preferable 
> for the user to explicitly deinitialize just the segments that they know were 
> initialized, which can be done on the base pointer. The only benefit in 
> having a `deinitialize` on the buffer is to communicate to users who see the 
> `initialize` API for the first time that it is their responsibility to 
> deinitialize if the type requires it. To that end, we could add a 
> `deinitialize(at:count:)` method, communicating the symmetry with 
> `initialize(at:from:). Naturally `index + count <= self.count`.
> 
> -Andy
> 
> I don’t agree with this. If `deinitialize()` is a problem because it 
> deinitializes the entire buffer, so are `moveAssign` and `moveInitialize`. 
> They all assume the released buffer operand is fully initialized. 
> `deinitialize()` has just as much use as the other full-buffer releasing 
> methods. Just take the image buffer example there 

`moveAssign` and `moveInitialize` assume that the sub-buffer being moved from 
is fully initialized. That’s already obvious because the user is asking to move 
source.count elements. I don’t see any use cases where it would pose a problem. 
If the user is moving out of a partially initialized buffer, they have already 
to sliced (and unfortunately rebased) the buffer. OTOH `deinitialize` is 
incorrect for normal use cases. I don’t see any practical analogy between those 
APIs.
> let pixels:Int = scanlines.map{ $0.count }.reduce(0, +)
> var image = UnsafeMutableBufferPointer.allocate(capacity: pixels)
> 
> var filled:Int = 0
> for scanline:UnsafeMutableBufferPointer in scanlines 
> {
> image.moveInitialize(at: filled, from: scanline)
> filled += scanline.count
> }
> 
> image.deinitialize()
We don’t want developers to do this. Instead we want to see an explicitly named 
association between the number of items initialized and deinitialized:

image.deinitialize(at: 0, count: filled)

Flipping this around, it could be even more common to be writing into a larger 
than necessary buffer (pixels > filled). If we’re providing auto-slicing 
initializers, then deinitialization should follow the same approach, rather 
than:

UnsafeMutableBufferPointer(rebasing: image[0, filled]).deinitialize()
> image.deallocate()
> 
> and replace `Pixel` with a class type like `UIButton`.
> And `deinitialize(at:count:)` is bad because you’re asking for a count on a 
> buffer method. `moveAssign` and `moveInitialize` can take range parameters 
> because they each have a second operand that supplies the count number. 
> `deinitialize` doesn’t. That means calls could end up looking like 
> 
> buffer.deinitialize(at: 0, count: buffer.count)
> 
> which is exactly what we were trying to avoid in the first place.

But there is no value in avoiding the `count` argument here. That’s not a valid 
motivation for introducing `deinitialize` on a buffer, and we’d be better off 
not introducing it at all.

The only valid motivation I can come up with for introducing `deinitialize` on 
buffer is to remind developers who are only looking at the buffer API (and not 
the plain pointer API) that it’s their responsibility to manually deinitialize 
(it doesn’t automatically happen on deallocation or destruction).

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 20, 2017, at 6:03 PM, Taylor Swift  wrote:
> 
> New draft of the proposal is up here: 
>   
> >
> 
> Important changes start here 
> .

I have more feedback for you after discussing with Ben and Michael.

Under the section "Background". The retaining/releasing counts are confusing 
because we normally talk about "+0/+1" with respect to a single value. Here 
you're summing the reference count effect across different source and 
destination values. I think you can drop the -1/+0/+1 from the cell labels and 
just label the headers as "Source-Copy(+1) vs. Source-Move(+0)" and 
"Dest-Initializing(+0)" vs. "Dest-Destroying(-1)".

Ben noticed that we still don't have `UnsafeMutableBufferPointer.deallocate()`. 
Please add that as well.

I think you should also add the default `alignedTo:Int = 
MemoryLayout.alignment` to `UnsafeRawMutablePointer`. It is effectively 
part of Swift's memory model... [Background: We're providing a language level 
default guarantee of word-aligned storage. We don't want to expose the 
platform's default alignment. There is no such thing as a "maximal" alignment 
for all imported primitive types. We expect allocators to typically provide 
16-byte alignment, but when developers are relying on that for correctness, we 
want it to be explicit. Developers usually rely on word alignment so there's no 
value in making that explicit.]

The source breaking changes can be marked deprecated for now, but can only be 
marked unavailable in Swift 5 mode or later.

We anticipate adding a ContiguouslyStored protocol "soon", which could be used 
to reduce the amount of overloading on the buffer `initialize`/`assign` APIs. 
But we need a lot more time to iterate on that design and it doesn't appear 
that migrating to it would be source breaking w.r.t. your proposal.

I think we would be receptive to a "non-Optional baseAddress" proposal now if 
you or anyone else wants to pitch that. I know Dave Abrahams had a working 
implementation at some point. That would weaken some of the incentive to 
continue adding more convenience to buffer slices.

Sorry to bring this up again, but I was not able to defend the addition of 
`UnsafeMutableBufferPointer.deinitialize()`. It is incorrect for the typical 
use case and doesn't appear to solve any important use case. The *only* fully 
initializing method is `initialize(repeating:)`, but that will usually be used 
for trivial values, which should not be deinitialized. It's preferable for the 
user to explicitly deinitialize just the segments that they know were 
initialized, which can be done on the base pointer. The only benefit in having 
a `deinitialize` on the buffer is to communicate to users who see the 
`initialize` API for the first time that it is their responsibility to 
deinitialize if the type requires it. To that end, we could add a 
`deinitialize(at:count:)` method, communicating the symmetry with 
`initialize(at:from:). Naturally `index + count <= self.count`.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 21, 2017, at 3:11 PM, Michael Ilseman  wrote:
> 
>> 
>> I can’t defend breaking existing source without having seen real code that 
>> was actually written incorrectly. I don’t see the downside of using the same 
>> deprecation strategy as the other changes. I expect code that was already 
>> written to be correct and future code to not call the deprecated API.
>> 
> 
> It would have to be deprecated in Swift 4 mode. For beyond-4 mode, are you 
> arguing it should remain deprecated or can it become obsoleted?

We can defer that decision. We could conceivably obsolete 
`deallocate(capacity:)` in beyond-4, but keep the rest of the changes as 
deprecated until beyond-beyond-swift-4.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 20, 2017, at 6:03 PM, Taylor Swift  wrote:
> 
> New draft of the proposal is up here: 
>   
> >
> 
> Important changes start here 
> .

This should be brought to the attention of swift-evolution:

> The old `deallocate(capacity:)` method should be marked as `unavailable` 
> since it currently encourages dangerously incorrect code. This avoids 
> misleading future users, forces current users to address this potentially 
> catastrophic memory bug, and leaves the possibility open for us to add a 
> `deallocate(capacity:)` method in the future, or perhaps even a 
> `reallocate(toCapacity:)` method.

I can’t defend breaking existing source without having seen real code that was 
actually written incorrectly. I don’t see the downside of using the same 
deprecation strategy as the other changes. I expect code that was already 
written to be correct and future code to not call the deprecated API.

-Andy

> On Sun, Aug 20, 2017 at 1:40 PM, Kelvin Ma  > wrote:
> actually never mind that, UnsafeMutablePointer should be the only type to not 
> support at: arguments since offsetting them is easy with +.
> 
> On Aug 20, 2017, at 12:12 AM, Taylor Swift via swift-evolution 
> > wrote:
> 
>> 
>> 
>> On Sat, Aug 19, 2017 at 10:28 PM, Andrew Trick > > wrote:
>> 
>>> On Aug 19, 2017, at 6:42 PM, Taylor Swift >> > wrote:
>>> 
>>> 
>>> 
>>> On Sat, Aug 19, 2017 at 9:31 PM, Andrew Trick >> > wrote:
>>> 
 On Aug 19, 2017, at 6:16 PM, Taylor Swift > wrote:
 
 What you’re describing is basically an earlier version of the proposal 
 which had a slightly weaker precondition (source >= destination) than 
 yours (source == destination). That one basically ignored the Sequence 
 methods at the expense of greater API surface area.
>>> 
>>> The Sequence methods don’t provide the simpler, more convenient form of 
>>> initialization/deinitialization that I thought you wanted. I see two 
>>> reasonable options.
>>> 
>>> 1. Don’t provide any new buffer initialization/deinitialization 
>>> convenience. i.e. drop UsafeMutableBufferPointer moveInitialize, 
>>> moveAssign, and deinitialize from your proposal.
>>> 
>>> 2. Provide the full set of convenience methods: initialize, assign, 
>>> moveInitialize, and moveAssign assuming self.count==source.count. And 
>>> provide deinitialize() to be used only in conjunction with those new 
>>> initializers.
>>> 
>>> The question is really whether those new methods are going to significantly 
>>> simplify your code. If not, #1 is the conservative choice. Don't provide 
>>> convenience which could be misused. Put off solving that problem until we 
>>> can design a new move-only buffer type that tracks partially initialized 
>>> state.
>>> 
>>> -Andy 
>>> 
>>> 
>>> I’m not sure the answer is to just omit methods from 
>>> UnsafeMutableBufferPointer since most of the original complaints circulated 
>>> around having to un-nil baseAddress to do anything with them.
>> 
>> I know un-nil’ing baseAddress is horrible, but I don’t think working around 
>> that is an important goal yet. Eventually there will be a much safer, more 
>> convenient mechanism for manual allocation that doesn’t involve “pointers". 
>> I also considered adding API surface to UnsafeMutableBufferPointer.Slice, 
>> but that’s beyond what we should do now and may also become irrelevant when 
>> we have a more sophisticated buffer type. 
>> 
>>> What if only unary methods should be added to UnsafeMutableBufferPointer 
>>> without count:, meaning:
>>> 
>>> initialize(repeating:)
>> 
>> I actually have no problem with this one... except that it could be confused 
>> with UnsafeMutablePointer.initialize(repeating:), but I’ll ignore that since 
>> we already discussed it.
>> 
>>> assign(repeating:)
>>> deinitialize()
>> 
>> These are fine only if we have use cases that warrant them AND those use 
>> cases are expected to fully initialize the buffer, either via 
>> initialize(repeating:) or initialize(from: buffer) with 
>> precondition(source.count==self.count). They don’t really make sense for the 
>> use case that I’m familiar with. Without clear motivating code patterns, 
>> they aren’t worth the risk. “API Completeness” doesn’t have intrinsic value.
>> 
>> An example use for assign(repeating:) would be to zero out an image buffer.
>>  
>> 
>>> and the other methods should take both an 

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 20, 2017, at 7:25 PM, Taylor Swift  wrote:
> 
> a method like `buffer.segment(0 ..< 5)` or something which would avoid 
> constructing the slice type (and save typing that long 
> `UnsafeMutableBufferPointer(rebasing:)` name) like that would be nice . but 
> that’s just sugar and not very important if you ask me

Yes, I agree on all points.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 19, 2017, at 9:12 PM, Taylor Swift  wrote:
> 
>> and the other methods should take both an offset parameter instead of a 
>> count parameter:
>> 
>> initialize(from:at:)
>> assign(from:at:)
>> moveInitialize(from:at:)
>> moveAssign(from:at:)
>> 
>> which provides maximum explicitness. This requires improvements to buffer 
>> pointer slicing though. But I’m not a fan of the mission creep that’s 
>> working into this proposal (i only originally wrote the thing to get 
>> allocate(capacity:) and deallocate() into UnsafeMutableBufferPointer!)
> 
> I’m open to that, with source.count <= self.count + index. They are 
> potentially ambiguous (the `at` could refer to a source index) but consistent 
> with the idea that this API is for copying an entire source buffer into a 
> slice of the destination buffer. Again, we need to find real code that 
> benefits from this, but I expect the stdlib could use these.

In case that typo wasn’t obvious, we actually want the precondition: `offset + 
source.count <= self.count` (your latest proposal draft is correct).

> The more I think the more I believe using from:at: is the right approach. The 
> only problem is that it would have to be written as a generic on Collectionor 
> Sequence to avoid having to provide up to 4 overloads for each operation, 
> since we would want these to work well with buffer slices as well as buffers 
> themselves. That puts them uncomfortably close to the turf of the existing 
> buffer pointer Sequence API though.

It would have to be a generic method taking a RandomAccessCollection. Calling 
that would rely on tricky type inference. For now, I prefer the explicitly 
typed API in your current proposal.

> Or we could make UnsafeMutableBufferPointer its own slice type. Right now 
> MutableRandomAccessSlice takes up 4 
> words of storage when it really only needs two.


A slice needs to be a separate type because of the indexing semantics:
https://github.com/apple/swift-evolution/commit/c8165b41b188c3d095425a0b4636fcf299ee9036
 


Note that it's actually a feature that Subsequence refers back to its original 
buffer. If we cared enough, we could define a buffer slice type that exposes 
most of the buffer API. I just don't think we need to worry about that level of 
convenience yet, especially with you adding the `at:` labels. I'm fine if 
developers resort to `init(rebasing:)` in rare cases. Again, once we have a 
safe move-only or reference counted buffer type, that will be the better choice 
for Swift APIs. For now, this is still largely driven by C interop.

I like this proposal as it stands!
https://github.com/kelvin13/swift-evolution/blob/9d305ced7b82b9cf5854b55b3f0d08952853d046/proposals/0184-improved-pointers.md

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 19, 2017, at 6:42 PM, Taylor Swift  wrote:
> 
> 
> 
> On Sat, Aug 19, 2017 at 9:31 PM, Andrew Trick  > wrote:
> 
>> On Aug 19, 2017, at 6:16 PM, Taylor Swift > > wrote:
>> 
>> What you’re describing is basically an earlier version of the proposal which 
>> had a slightly weaker precondition (source >= destination) than yours 
>> (source == destination). That one basically ignored the Sequence methods at 
>> the expense of greater API surface area.
> 
> The Sequence methods don’t provide the simpler, more convenient form of 
> initialization/deinitialization that I thought you wanted. I see two 
> reasonable options.
> 
> 1. Don’t provide any new buffer initialization/deinitialization convenience. 
> i.e. drop UsafeMutableBufferPointer moveInitialize, moveAssign, and 
> deinitialize from your proposal.
> 
> 2. Provide the full set of convenience methods: initialize, assign, 
> moveInitialize, and moveAssign assuming self.count==source.count. And provide 
> deinitialize() to be used only in conjunction with those new initializers.
> 
> The question is really whether those new methods are going to significantly 
> simplify your code. If not, #1 is the conservative choice. Don't provide 
> convenience which could be misused. Put off solving that problem until we can 
> design a new move-only buffer type that tracks partially initialized state.
> 
> -Andy 
> 
> 
> I’m not sure the answer is to just omit methods from 
> UnsafeMutableBufferPointer since most of the original complaints circulated 
> around having to un-nil baseAddress to do anything with them.

I know un-nil’ing baseAddress is horrible, but I don’t think working around 
that is an important goal yet. Eventually there will be a much safer, more 
convenient mechanism for manual allocation that doesn’t involve “pointers". I 
also considered adding API surface to UnsafeMutableBufferPointer.Slice, but 
that’s beyond what we should do now and may also become irrelevant when we have 
a more sophisticated buffer type. 

> What if only unary methods should be added to UnsafeMutableBufferPointer 
> without count:, meaning:
> 
> initialize(repeating:)

I actually have no problem with this one... except that it could be confused 
with UnsafeMutablePointer.initialize(repeating:), but I’ll ignore that since we 
already discussed it.

> assign(repeating:)
> deinitialize()

These are fine only if we have use cases that warrant them AND those use cases 
are expected to fully initialize the buffer, either via initialize(repeating:) 
or initialize(from: buffer) with precondition(source.count==self.count). They 
don’t really make sense for the use case that I’m familiar with. Without clear 
motivating code patterns, they aren’t worth the risk. “API Completeness” 
doesn’t have intrinsic value.

> and the other methods should take both an offset parameter instead of a count 
> parameter:
> 
> initialize(from:at:)
> assign(from:at:)
> moveInitialize(from:at:)
> moveAssign(from:at:)
> 
> which provides maximum explicitness. This requires improvements to buffer 
> pointer slicing though. But I’m not a fan of the mission creep that’s working 
> into this proposal (i only originally wrote the thing to get 
> allocate(capacity:) and deallocate() into UnsafeMutableBufferPointer!)

I’m open to that, with source.count <= self.count + index. They are potentially 
ambiguous (the `at` could refer to a source index) but consistent with the idea 
that this API is for copying an entire source buffer into a slice of the 
destination buffer. Again, we need to find real code that benefits from this, 
but I expect the stdlib could use these.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 19, 2017, at 6:16 PM, Taylor Swift  wrote:
> 
> What you’re describing is basically an earlier version of the proposal which 
> had a slightly weaker precondition (source >= destination) than yours (source 
> == destination). That one basically ignored the Sequence methods at the 
> expense of greater API surface area.

The Sequence methods don’t provide the simpler, more convenient form of 
initialization/deinitialization that I thought you wanted. I see two reasonable 
options.

1. Don’t provide any new buffer initialization/deinitialization convenience. 
i.e. drop UsafeMutableBufferPointer moveInitialize, moveAssign, and 
deinitialize from your proposal.

2. Provide the full set of convenience methods: initialize, assign, 
moveInitialize, and moveAssign assuming self.count==source.count. And provide 
deinitialize() to be used only in conjunction with those new initializers.

The question is really whether those new methods are going to significantly 
simplify your code. If not, #1 is the conservative choice. Don't provide 
convenience which could be misused. Put off solving that problem until we can 
design a new move-only buffer type that tracks partially initialized state.

-Andy 

> On Sat, Aug 19, 2017 at 9:08 PM, Andrew Trick  > wrote:
> 
>> On Aug 19, 2017, at 6:03 PM, Taylor Swift > > wrote:
>> 
>> 
>> 
>> On Sat, Aug 19, 2017 at 8:52 PM, Andrew Trick > > wrote:
 
 The problem is I would expect to be able to safely call deinitialize() and 
 friends after calling initialize(from:). If Element is a class type and 
 initialize doesn’t fill the entire buffer range, calling deinitialize() 
 will crash. That being said, since copy(from:bytes:) and copyBytes(from:) 
 don’t do any initialization and have no direct counterparts in 
 UnsafeMutableBufferPointer, it’s okay if they have different behavior than 
 the other methods.
>>> 
>>> You astutely pointed out that the UnsafeMutableBufferPointer.deinitialize() 
>>> method is dangerous, and I asked you to add a warning to its comments. 
>>> However, given the danger, I think we need to justify adding the method to 
>>> begin with. Are there real use cases that greatly benefit from it?
>>> 
>>> I agree that’s a problem, which is why i was iffy on supporting partial 
>>> initialization to begin with. The use case is for things like growing 
>>> collections where you have to periodically move to larger storage. However, 
>>> deinitialize is no more dangerous than moveInitialize, 
>>> assign(repeating:count:), or moveAssign; they all deinitialize at least one 
>>> entire buffer. If deinitialize is to be omitted, so must a majority of the 
>>> unsafe pointer API.
>> 
>> Here's an alternative. Impose the precondition(source.count == self.count) 
>> to the following UnsafeMutableBufferPointer convenience methods that you 
>> propose adding:
>> 
>> +++ func assign(from:UnsafeBufferPointer)
>> +++ func assign(from:UnsafeMutableBufferPointer)
>> +++ func moveAssign(from:UnsafeMutableBufferPointer)
>> +++ func moveInitialize(from:UnsafeMutableBufferPointer)
>> +++ func initialize(from:UnsafeBufferPointer)
>> +++ func initialize(from:UnsafeMutableBufferPointer)
>> 
>> I don't that introduces any behavior that is inconsistent with other 
>> methods. `copyBytes` is a totally different thing that only works on trivial 
>> types. The currently dominant use case for UnsafeBufferPointer, partially 
>> initialized backing store, does not need to use your new convenience 
>> methods. It can continue dropping down to pointer+count style 
>> initialization/deinitialization.
>> 
>> -Andy
>>  
>> the latest draft does not have 
>> assign(from:UnsafeMutableBufferPointer) or  
>> initialize(from:UnsafeMutableBufferPointer), it uses the generic 
>> Sequence methods that are already there that do not require that 
>> precondition.
> 
> Sorry, I was pasting from your original proposal. Here are the relevant 
> methods from the latest draft:
> 
> https://github.com/kelvin13/swift-evolution/blob/1b7738513c00388b8de3b09769eab773539be386/proposals/0184-improved-pointers.md
>  
> 
> 
> +++ func moveInitialize(from:UnsafeMutableBufferPointer)
> +++ func moveAssign(from:UnsafeMutableBufferPointer)
> 
> But with the precondition, the `assign` method could be reasonably added 
> back, right?
> +++ func assign(from:UnsafeMutableBufferPointer)
> 
> Likewise, I don’t have a problem with initialize(from: UnsafeBufferPointer) 
> where self.count==source.count. The Sequence initializer is different. It’s 
> designed for the Array use case and forces the caller to deal with partial 
> initialization. 
> 
> UnsafeMutableRawBufferPointer.moveInitializeMemory on the other 

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 19, 2017, at 5:33 PM, Taylor Swift  wrote:
> 
> I agree it’s probably a bad idea to add the default arg to those two 
> functions. However, the default argument in initialize(repeating:count:) is 
> there for backwards compatibility since it already had it before and there’s 
> like a hundred places in the stdlib that use this default value.

Alright, I could agree to that if no one else wants to weigh in. As long as you 
remove the default from the memory binding API.

-Andy

> On Sat, Aug 19, 2017 at 6:02 PM, Andrew Trick  > wrote:
> 
>> On Aug 15, 2017, at 9:47 PM, Taylor Swift via swift-evolution 
>> > wrote:
>> 
>> Implementation is here: https://github.com/apple/swift/pull/11464 
>> 
>> 
>> On Sat, Aug 12, 2017 at 8:23 PM, Taylor Swift > > wrote:
>> I’ve revised the proposal based on what I learned from trying to implement 
>> these changes. I think it’s worth tacking the existing methods that take 
>> Sequences at the same time as this actually makes the design a bit simpler.
>> > >
>> 
>> The previous version 
>>  of this 
>> document ignored the generic initialization methods on 
>> UnsafeMutableBufferPointer and UnsafeMutableRawBufferPointer, leaving them 
>> to be overhauled at a later date, in a separate proposal. Instead, this 
>> version of the proposal leverages those existing methods to inform a more 
>> compact API design which has less surface area, and is more future-proof 
>> since it obviates the need to design and add another (redundant) set of 
>> protocol-oriented pointer APIs later.
>> 
>> On Tue, Aug 8, 2017 at 12:52 PM, Taylor Swift > > wrote:
>> Since Swift 5 just got opened up for proposals, SE-184 Improved Pointers is 
>> ready for community review, and I encourage everyone to look it over and 
>> provide feedback. Thank you!
>> >  
>> >
> 
> 
> Thanks for continuing to improve this proposal. It’s in great shape now.
> 
> Upon rereading it today I have to say I strongly object to the `count = 1` 
> default in the following two cases:
> 
> + UnsafeMutablePointer.withMemoryRebound(to: count: Int = 1)
> + UnsafeMutableRawPointer.bindMemory(to:T.Type, count:Int = 1)
>   -> UnsafeMutablePointer
> 
> To aid understanding, it needs to be clear at the call-site that binding 
> memory only applies to the specified number of elements. It's a common 
> mistake for users to think they can obtain a pointer to a different type, 
> then use that pointer as a base to access other elements. These APIs are 
> dangerous expert interfaces. We certainly don't want to make their usage more 
> concise at the expense of clarity.
> 
> In general, I think there's very little value in the `count=1` default, and 
> it creates potential confusion on the caller side between the `BufferPointer` 
> API and the `Pointer` API. For example:
> 
> + initialize(repeating:Pointee, count:Int = 1)
> 
> Seeing `p.initialize(repeating: x)`, the user may think `p` refers to the 
> buffer instead of a pointer into the buffer and misunderstand the behavior.
> 
> + UnsafeMutablePointer.deinitialize(count: Int = 1)
> 
> Again, `p.deinitialize()` looks to me like it might be deinitializing an 
> entire buffer.
> 
> If the `count` label is always explicit, then there's a clear distinction 
> between the low-level `pointer` APIs and the `buffer` APIs.
> 
> The pointer-to-single-element case never seemed interesting enough to me to 
> worry about making convenient. If I'm wrong about that, is there some 
> real-world code you can point to where the count=1 default significantly 
> improves clarity?
> 
> -Andy
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 19, 2017, at 6:03 PM, Taylor Swift  wrote:
> 
> 
> 
> On Sat, Aug 19, 2017 at 8:52 PM, Andrew Trick  > wrote:
>>> 
>>> The problem is I would expect to be able to safely call deinitialize() and 
>>> friends after calling initialize(from:). If Element is a class type and 
>>> initialize doesn’t fill the entire buffer range, calling deinitialize() 
>>> will crash. That being said, since copy(from:bytes:) and copyBytes(from:) 
>>> don’t do any initialization and have no direct counterparts in 
>>> UnsafeMutableBufferPointer, it’s okay if they have different behavior than 
>>> the other methods.
>> 
>> You astutely pointed out that the UnsafeMutableBufferPointer.deinitialize() 
>> method is dangerous, and I asked you to add a warning to its comments. 
>> However, given the danger, I think we need to justify adding the method to 
>> begin with. Are there real use cases that greatly benefit from it?
>> 
>> I agree that’s a problem, which is why i was iffy on supporting partial 
>> initialization to begin with. The use case is for things like growing 
>> collections where you have to periodically move to larger storage. However, 
>> deinitialize is no more dangerous than moveInitialize, 
>> assign(repeating:count:), or moveAssign; they all deinitialize at least one 
>> entire buffer. If deinitialize is to be omitted, so must a majority of the 
>> unsafe pointer API.
> 
> Here's an alternative. Impose the precondition(source.count == self.count) to 
> the following UnsafeMutableBufferPointer convenience methods that you propose 
> adding:
> 
> +++ func assign(from:UnsafeBufferPointer)
> +++ func assign(from:UnsafeMutableBufferPointer)
> +++ func moveAssign(from:UnsafeMutableBufferPointer)
> +++ func moveInitialize(from:UnsafeMutableBufferPointer)
> +++ func initialize(from:UnsafeBufferPointer)
> +++ func initialize(from:UnsafeMutableBufferPointer)
> 
> I don't that introduces any behavior that is inconsistent with other methods. 
> `copyBytes` is a totally different thing that only works on trivial types. 
> The currently dominant use case for UnsafeBufferPointer, partially 
> initialized backing store, does not need to use your new convenience methods. 
> It can continue dropping down to pointer+count style 
> initialization/deinitialization.
> 
> -Andy
>  
> the latest draft does not have 
> assign(from:UnsafeMutableBufferPointer) or  
> initialize(from:UnsafeMutableBufferPointer), it uses the generic 
> Sequence methods that are already there that do not require that precondition.

Sorry, I was pasting from your original proposal. Here are the relevant methods 
from the latest draft:

https://github.com/kelvin13/swift-evolution/blob/1b7738513c00388b8de3b09769eab773539be386/proposals/0184-improved-pointers.md

+++ func moveInitialize(from:UnsafeMutableBufferPointer)
+++ func moveAssign(from:UnsafeMutableBufferPointer)

But with the precondition, the `assign` method could be reasonably added back, 
right?
+++ func assign(from:UnsafeMutableBufferPointer)

Likewise, I don’t have a problem with initialize(from: UnsafeBufferPointer) 
where self.count==source.count. The Sequence initializer is different. It’s 
designed for the Array use case and forces the caller to deal with partial 
initialization. 

UnsafeMutableRawBufferPointer.moveInitializeMemory on the other hand probably 
doesn't need that precondition since there's no way to deinitialize. It just 
needs clear comments.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

>> 
>> The problem is I would expect to be able to safely call deinitialize() and 
>> friends after calling initialize(from:). If Element is a class type and 
>> initialize doesn’t fill the entire buffer range, calling deinitialize() will 
>> crash. That being said, since copy(from:bytes:) and copyBytes(from:) don’t 
>> do any initialization and have no direct counterparts in 
>> UnsafeMutableBufferPointer, it’s okay if they have different behavior than 
>> the other methods.
> 
> You astutely pointed out that the UnsafeMutableBufferPointer.deinitialize() 
> method is dangerous, and I asked you to add a warning to its comments. 
> However, given the danger, I think we need to justify adding the method to 
> begin with. Are there real use cases that greatly benefit from it?
> 
> I agree that’s a problem, which is why i was iffy on supporting partial 
> initialization to begin with. The use case is for things like growing 
> collections where you have to periodically move to larger storage. However, 
> deinitialize is no more dangerous than moveInitialize, 
> assign(repeating:count:), or moveAssign; they all deinitialize at least one 
> entire buffer. If deinitialize is to be omitted, so must a majority of the 
> unsafe pointer API.

Here's an alternative. Impose the precondition(source.count == self.count) to 
the following UnsafeMutableBufferPointer convenience methods that you propose 
adding:

+++ func assign(from:UnsafeBufferPointer)
+++ func assign(from:UnsafeMutableBufferPointer)
+++ func moveAssign(from:UnsafeMutableBufferPointer)
+++ func moveInitialize(from:UnsafeMutableBufferPointer)
+++ func initialize(from:UnsafeBufferPointer)
+++ func initialize(from:UnsafeMutableBufferPointer)

I don't that introduces any behavior that is inconsistent with other methods. 
`copyBytes` is a totally different thing that only works on trivial types. The 
currently dominant use case for UnsafeBufferPointer, partially initialized 
backing store, does not need to use your new convenience methods. It can 
continue dropping down to pointer+count style initialization/deinitialization.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 9, 2017, at 8:51 AM, Taylor Swift  wrote:
> 
> 
> 
> On Wed, Aug 9, 2017 at 2:34 AM, Andrew Trick  > wrote:
> 
>> On Aug 8, 2017, at 11:10 PM, Taylor Swift > > wrote:
>> 
>> 
>> On Wed, Aug 9, 2017 at 1:51 AM, Andrew Trick > > wrote:
>> 
>>> On Aug 8, 2017, at 8:44 PM, Taylor Swift >> > wrote:
>>> 
>>> cool,, as for UnsafeMutableRawBufferPointer.copy(from:bytes:), I cannot 
>>> find such a function anywhere in the API. There is copyBytes(from:) 
>>> ,
>>>  but the documentation is messed up and mentions a nonexistent count: 
>>> argument over and over again. The documentation also doesn’t mention what 
>>> happens if there is a length mismatch, so users are effectively relying on 
>>> an implementation detail. I don’t know how to best resolve this.
>> 
>> We currently have `UnsafeMutableRawBufferPointer.copyBytes(from:)`. I don’t 
>> think your proposal changes that. The current docs refer to the `source` 
>> parameter, which is correct. Docs refer to the parameter name, not the label 
>> name. So `source.count` is the size of the input. I was pointing out that it 
>> has the semantics: `debugAssert(source.count <= self.count)`.
>> 
>> Your proposal changes `UnsafeRawPointer.copyBytes(from:count:)` to 
>> `UnsafeRawPointer.copy(from:bytes:)`. Originally we wanted to those API 
>> names to match, but I’m fine with your change. What is more important is 
>> that the semantics are the same as `copyBytes(from:)`. Furthermore, any new 
>> methods that you add that copy into a raw buffer (e.g. 
>> initializeMemory(as:from:count:)) should have similar behavior.
>> 
>>  
>> I’m fine with switching to taking the count from the source, though I think 
>> taking the count from the destination is slightly better because 1) the use 
>> cases I mentioned in the other email, and 2) all the other memorystate 
>> functions use self.count instead of source.count, if they take a source 
>> argument. But being consistent with the raw pointer version is more 
>> important.
> 
> If it’s copying from a buffer it should not take a count, if it’s copying 
> from a pointer it obviously needs to take a count. What I mean by the two 
> versions being named consistently is simply that they’re both named 
> `copyBytes`. That really isn’t important though. The overflow/underflow 
> semantics being consistent are important.
> 
> (Incidentally, the reason “bytes” needs to be in the somewhere name is 
> because this method isn’t capable of copying nontrivial values)
> 
>> Should the methods that don’t deal with raw buffers also be modified to use 
>> the source argument (i.e. UnsafeMutableBufferPointer.initialize(from:))?
> 
> I’m not sure what you mean by this. It also allows the destination to be 
> larger than the source. Initializing from a sequence does not trap on 
> overflow because we can’t guarantee the size of the sequence ahead of time. 
> When I talk about consistent overflow/underflow semantics, I’m only talking 
> about initializing one unsafe buffer/pointer from another unsafe 
> buffer/pointer.
> 
>> Also, was there a reason why UnsafeMutableRawBufferPointer.copyBytes(from:) 
>> uses the source’s count instead of its own? Right now this behavior is 
>> “technically” undocumented behavior (as the public docs haven’t been 
>> updated) so if there was ever a time to change it, now would be it.
> 
> Mainly because partial initialization is more expected than dropping data on 
> the floor. Ultimately, this should be whatever typical developers would 
> expect the behavior to be. I would be very hesitant to change the behavior 
> now though.
> 
> -Andy
> 
> The problem is I would expect to be able to safely call deinitialize() and 
> friends after calling initialize(from:). If Element is a class type and 
> initialize doesn’t fill the entire buffer range, calling deinitialize() will 
> crash. That being said, since copy(from:bytes:) and copyBytes(from:) don’t do 
> any initialization and have no direct counterparts in 
> UnsafeMutableBufferPointer, it’s okay if they have different behavior than 
> the other methods.

You astutely pointed out that the UnsafeMutableBufferPointer.deinitialize() 
method is dangerous, and I asked you to add a warning to its comments. However, 
given the danger, I think we need to justify adding the method to begin with. 
Are there real use cases that greatly benefit from it?

We have to accept that UnsafeBufferPointer is simply not a safe API for 
managing initialization and deinitialization. Adding a convenience method only 
makes it less safe.

The standard library *should* vend a safe API for initializing and 
deinitializing manually allocated memory. However, that will require 

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 15, 2017, at 9:47 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> Implementation is here: https://github.com/apple/swift/pull/11464 
> 
> 
> On Sat, Aug 12, 2017 at 8:23 PM, Taylor Swift  > wrote:
> I’ve revised the proposal based on what I learned from trying to implement 
> these changes. I think it’s worth tacking the existing methods that take 
> Sequences at the same time as this actually makes the design a bit simpler.
>  >
> 
> The previous version 
>  of this 
> document ignored the generic initialization methods on 
> UnsafeMutableBufferPointer and UnsafeMutableRawBufferPointer, leaving them to 
> be overhauled at a later date, in a separate proposal. Instead, this version 
> of the proposal leverages those existing methods to inform a more compact API 
> design which has less surface area, and is more future-proof since it 
> obviates the need to design and add another (redundant) set of 
> protocol-oriented pointer APIs later.
> 
> On Tue, Aug 8, 2017 at 12:52 PM, Taylor Swift  > wrote:
> Since Swift 5 just got opened up for proposals, SE-184 Improved Pointers is 
> ready for community review, and I encourage everyone to look it over and 
> provide feedback. Thank you!
>   
> >


Thanks for continuing to improve this proposal. It’s in great shape now.

Upon rereading it today I have to say I strongly object to the `count = 1` 
default in the following two cases:

+ UnsafeMutablePointer.withMemoryRebound(to: count: Int = 1)
+ UnsafeMutableRawPointer.bindMemory(to:T.Type, count:Int = 1)
  -> UnsafeMutablePointer

To aid understanding, it needs to be clear at the call-site that binding memory 
only applies to the specified number of elements. It's a common mistake for 
users to think they can obtain a pointer to a different type, then use that 
pointer as a base to access other elements. These APIs are dangerous expert 
interfaces. We certainly don't want to make their usage more concise at the 
expense of clarity.

In general, I think there's very little value in the `count=1` default, and it 
creates potential confusion on the caller side between the `BufferPointer` API 
and the `Pointer` API. For example:

+ initialize(repeating:Pointee, count:Int = 1)

Seeing `p.initialize(repeating: x)`, the user may think `p` refers to the 
buffer instead of a pointer into the buffer and misunderstand the behavior.

+ UnsafeMutablePointer.deinitialize(count: Int = 1)

Again, `p.deinitialize()` looks to me like it might be deinitializing an entire 
buffer.

If the `count` label is always explicit, then there's a clear distinction 
between the low-level `pointer` APIs and the `buffer` APIs.

The pointer-to-single-element case never seemed interesting enough to me to 
worry about making convenient. If I'm wrong about that, is there some 
real-world code you can point to where the count=1 default significantly 
improves clarity?

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 18, 2017, at 5:36 PM, Taylor Swift  wrote:
> 
> Should the immutable buffer pointer types also get deallocate()?

Both UnsafePointer and UnsafeBufferPointer should get deallocate. The Raw API 
already has those methods.

-Andy

> On Fri, Aug 18, 2017 at 7:55 PM, Andrew Trick  > wrote:
> 
>> On Aug 15, 2017, at 9:47 PM, Taylor Swift via swift-evolution 
>> > wrote:
>> 
>> Implementation is here: https://github.com/apple/swift/pull/11464 
>> 
>> 
>> On Sat, Aug 12, 2017 at 8:23 PM, Taylor Swift > > wrote:
>> I’ve revised the proposal based on what I learned from trying to implement 
>> these changes. I think it’s worth tacking the existing methods that take 
>> Sequences at the same time as this actually makes the design a bit simpler.
>> > >
>> 
>> The previous version 
>>  of this 
>> document ignored the generic initialization methods on 
>> UnsafeMutableBufferPointer and UnsafeMutableRawBufferPointer, leaving them 
>> to be overhauled at a later date, in a separate proposal. Instead, this 
>> version of the proposal leverages those existing methods to inform a more 
>> compact API design which has less surface area, and is more future-proof 
>> since it obviates the need to design and add another (redundant) set of 
>> protocol-oriented pointer APIs later.
>> 
>> On Tue, Aug 8, 2017 at 12:52 PM, Taylor Swift > > wrote:
>> Since Swift 5 just got opened up for proposals, SE-184 Improved Pointers is 
>> ready for community review, and I encourage everyone to look it over and 
>> provide feedback. Thank you!
>> >  
>> >
>> 
> 
> 
> Would you mind adding a deallocate method to (nonmutable) 
> UnsafePointer/UnsafeBufferPointer to take care of
> [SR-3309](https://bugs.swift.org/browse/SR-3309 
> )?
> 
> There’s simply nothing in the memory model that requires mutable memory for 
> deallocation.
> 
> It fits right in with this proposal and hardly seems worth a separate one.
> 
> -Andy
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 15, 2017, at 9:47 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> Implementation is here: https://github.com/apple/swift/pull/11464 
> 
> 
> On Sat, Aug 12, 2017 at 8:23 PM, Taylor Swift  > wrote:
> I’ve revised the proposal based on what I learned from trying to implement 
> these changes. I think it’s worth tacking the existing methods that take 
> Sequences at the same time as this actually makes the design a bit simpler.
>  >
> 
> The previous version 
>  of this 
> document ignored the generic initialization methods on 
> UnsafeMutableBufferPointer and UnsafeMutableRawBufferPointer, leaving them to 
> be overhauled at a later date, in a separate proposal. Instead, this version 
> of the proposal leverages those existing methods to inform a more compact API 
> design which has less surface area, and is more future-proof since it 
> obviates the need to design and add another (redundant) set of 
> protocol-oriented pointer APIs later.
> 
> On Tue, Aug 8, 2017 at 12:52 PM, Taylor Swift  > wrote:
> Since Swift 5 just got opened up for proposals, SE-184 Improved Pointers is 
> ready for community review, and I encourage everyone to look it over and 
> provide feedback. Thank you!
>   
> >
> 


Would you mind adding a deallocate method to (nonmutable) 
UnsafePointer/UnsafeBufferPointer to take care of
[SR-3309](https://bugs.swift.org/browse/SR-3309 
)?

There’s simply nothing in the memory model that requires mutable memory for 
deallocation.

It fits right in with this proposal and hardly seems worth a separate one.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] pure functions

[Andrew Trick via swift-evolution]

> On Aug 18, 2017, at 2:45 PM, David Sweeris via swift-evolution 
>  wrote:
> 
> If we define our semantics carefully enough, could we realistically make it 
> so that "@pure" or "@impure" (or whatever) would simply be a note to the 
> compiler, letting it skip that bit of analysis for the function in question?
> 
> 
> 
>> The secondary concern is that you need to build out the model enough that 
>> you don’t prevent abstractions.  A pure function should be able to create an 
>> instance of a struct, mutate it (i.e. calling non-pure functions) etc.  This 
>> requires a non-trivial design, and as the design complexity creeps, you run 
>> the risk of it getting out of control.
> 
> Did we ever get around to figuring which definition(s) of "pure" that we 
> wanted to support, or did the discussion always get out of scope before we 
> got that far? I know the issue's been brought up, but I don't recall if it 
> was discussed enough to reach any conclusions.

It became clear that we weren’t going to change Swift’s default semantics in 
the version 2/3 timeframe, and we made the effort to do sufficient whole-module 
analysis to make up for it. I see this primarily being a resilience issue in 
the future. Eventually I think we’ll want public APIs to be able to declare 
some level of statically enforced purity. We might even get some level of 
default guarantee as a way of “deprecating global var”. But that whole 
discussion has rightly been punted.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 8, 2017, at 11:10 PM, Taylor Swift  wrote:
> 
> 
> On Wed, Aug 9, 2017 at 1:51 AM, Andrew Trick  > wrote:
> 
>> On Aug 8, 2017, at 8:44 PM, Taylor Swift > > wrote:
>> 
>> cool,, as for UnsafeMutableRawBufferPointer.copy(from:bytes:), I cannot find 
>> such a function anywhere in the API. There is copyBytes(from:) 
>> ,
>>  but the documentation is messed up and mentions a nonexistent count: 
>> argument over and over again. The documentation also doesn’t mention what 
>> happens if there is a length mismatch, so users are effectively relying on 
>> an implementation detail. I don’t know how to best resolve this.
> 
> We currently have `UnsafeMutableRawBufferPointer.copyBytes(from:)`. I don’t 
> think your proposal changes that. The current docs refer to the `source` 
> parameter, which is correct. Docs refer to the parameter name, not the label 
> name. So `source.count` is the size of the input. I was pointing out that it 
> has the semantics: `debugAssert(source.count <= self.count)`.
> 
> Your proposal changes `UnsafeRawPointer.copyBytes(from:count:)` to 
> `UnsafeRawPointer.copy(from:bytes:)`. Originally we wanted to those API names 
> to match, but I’m fine with your change. What is more important is that the 
> semantics are the same as `copyBytes(from:)`. Furthermore, any new methods 
> that you add that copy into a raw buffer (e.g. 
> initializeMemory(as:from:count:)) should have similar behavior.
> 
>  
> I’m fine with switching to taking the count from the source, though I think 
> taking the count from the destination is slightly better because 1) the use 
> cases I mentioned in the other email, and 2) all the other memorystate 
> functions use self.count instead of source.count, if they take a source 
> argument. But being consistent with the raw pointer version is more important.

If it’s copying from a buffer it should not take a count, if it’s copying from 
a pointer it obviously needs to take a count. What I mean by the two versions 
being named consistently is simply that they’re both named `copyBytes`. That 
really isn’t important though. The overflow/underflow semantics being 
consistent are important.

(Incidentally, the reason “bytes” needs to be in the somewhere name is because 
this method isn’t capable of copying nontrivial values)

> Should the methods that don’t deal with raw buffers also be modified to use 
> the source argument (i.e. UnsafeMutableBufferPointer.initialize(from:))?

I’m not sure what you mean by this. It also allows the destination to be larger 
than the source. Initializing from a sequence does not trap on overflow because 
we can’t guarantee the size of the sequence ahead of time. When I talk about 
consistent overflow/underflow semantics, I’m only talking about initializing 
one unsafe buffer/pointer from another unsafe buffer/pointer.

> Also, was there a reason why UnsafeMutableRawBufferPointer.copyBytes(from:) 
> uses the source’s count instead of its own? Right now this behavior is 
> “technically” undocumented behavior (as the public docs haven’t been updated) 
> so if there was ever a time to change it, now would be it.

Mainly because partial initialization is more expected than dropping data on 
the floor. Ultimately, this should be whatever typical developers would expect 
the behavior to be. I would be very hesitant to change the behavior now though.

-Andy

> —
> 
> Another thing. The initialization methods that you’re adding to 
> `UnsafeRawPointer` and `UnsafeRawBufferPointer` should return typed 
> `UnsafePointer` and `UnsafeBufferPointer` respectively.
> 
> I’ll fix that once the current pending edit 
>  gets merged.
>  
> 
> Thanks,
> 
> -Andy
> 
>> On Tue, Aug 8, 2017 at 11:33 PM, Andrew Trick > > wrote:
>> 
>>> On Aug 8, 2017, at 8:29 PM, Taylor Swift >> > wrote:
>>> 
>>> 
>>> 
>>> On Tue, Aug 8, 2017 at 11:24 PM, Andrew Trick >> > wrote:
>>> 
 On Aug 8, 2017, at 6:51 PM, Taylor Swift > wrote:
 
 
 
 On Tue, Aug 8, 2017 at 9:38 PM, Andrew Trick > wrote:
 
> > UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)
> 
> Well, I think it's somewhat ridiculous for users to write this every time 
> they allocate a buffer:
> 
> `UnsafeMutableRawBufferPointer.allocate(bytes: size, alignedTo: 
> MemoryLayout.alignment)`
> 
> If anyone reading the code is unsure about the Swift API's alignment
> guarantee, it's trivial to check the API docs.
> 
> You could 

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 8, 2017, at 8:44 PM, Taylor Swift  wrote:
> 
> cool,, as for UnsafeMutableRawBufferPointer.copy(from:bytes:), I cannot find 
> such a function anywhere in the API. There is copyBytes(from:) 
> ,
>  but the documentation is messed up and mentions a nonexistent count: 
> argument over and over again. The documentation also doesn’t mention what 
> happens if there is a length mismatch, so users are effectively relying on an 
> implementation detail. I don’t know how to best resolve this.

We currently have `UnsafeMutableRawBufferPointer.copyBytes(from:)`. I don’t 
think your proposal changes that. The current docs refer to the `source` 
parameter, which is correct. Docs refer to the parameter name, not the label 
name. So `source.count` is the size of the input. I was pointing out that it 
has the semantics: `debugAssert(source.count <= self.count)`.

Your proposal changes `UnsafeRawPointer.copyBytes(from:count:)` to 
`UnsafeRawPointer.copy(from:bytes:)`. Originally we wanted to those API names 
to match, but I’m fine with your change. What is more important is that the 
semantics are the same as `copyBytes(from:)`. Furthermore, any new methods that 
you add that copy into a raw buffer (e.g. initializeMemory(as:from:count:)) 
should have similar behavior.

—

Another thing. The initialization methods that you’re adding to 
`UnsafeRawPointer` and `UnsafeRawBufferPointer` should return typed 
`UnsafePointer` and `UnsafeBufferPointer` respectively.

Thanks,

-Andy

> On Tue, Aug 8, 2017 at 11:33 PM, Andrew Trick  > wrote:
> 
>> On Aug 8, 2017, at 8:29 PM, Taylor Swift > > wrote:
>> 
>> 
>> 
>> On Tue, Aug 8, 2017 at 11:24 PM, Andrew Trick > > wrote:
>> 
>>> On Aug 8, 2017, at 6:51 PM, Taylor Swift >> > wrote:
>>> 
>>> 
>>> 
>>> On Tue, Aug 8, 2017 at 9:38 PM, Andrew Trick >> > wrote:
>>> 
 > UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)
 
 Well, I think it's somewhat ridiculous for users to write this every time 
 they allocate a buffer:
 
 `UnsafeMutableRawBufferPointer.allocate(bytes: size, alignedTo: 
 MemoryLayout.alignment)`
 
 If anyone reading the code is unsure about the Swift API's alignment
 guarantee, it's trivial to check the API docs.
 
 You could introduce a clearly documented default `alignedTo`
 argument. The reason I didn't do that is that the runtime won't
 respect it anyway. But I think it would be fair to go ahead with the
 API and file a bug against the runtime.
 
 Default argument of MemoryLayout.alignment is the way to go but as 
 you said i don’t know if that is actually allowed/works. An alternative is 
 to have two allocate methods each, one that takes an alignment argument 
 and one that doesn’t (and aligns to pointer alignment) but that feels 
 inelegant. Default arguments would be better.
>>> 
>>> Default argument makes sense to me too. Then the raw buffer pointer and 
>>> regular raw pointer APIs can be consistent with each other.
>>> 
>>> Runtime bug: https://bugs.swift.org/browse/SR-5664 
>>> 
>>> 
>>> yikes i was not aware of this. I don’t think it’s bad enough to warrant 
>>> dropping the argument like with deallocate(capacity:) but I can imagine bad 
>>> things happening to code that crams extra inhabitants into pointers.
>> 
>> If we ever need to do pointer adjustment during deallocation to accommodate 
>> alignment, then I think the Swift runtime can track that. I see no reason to 
>> muddy the UnsafeRawPointer API with it. So, I agree with your proposed 
>> change to drop `alignedTo` there.
>> 
>> -Andy
>> 
>> oh lol I was talking about assuming the pointer returned by 
>> allocate(bytes:alignedTo:) is a multiple of alignedTo. Some code might be 
>> relying on the last few bits of the pointer being zero; i.e. sticking bit 
>> flags there like how some implementations store the red/black color 
>> information in a red-black tree node.
> 
> Oh, sure. But I think it will be easy to fix the runtime. We could probably 
> do it before the proposal is accepted if necessary.
> -Andy
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 8, 2017, at 8:29 PM, Taylor Swift  wrote:
> 
> 
> 
> On Tue, Aug 8, 2017 at 11:24 PM, Andrew Trick  > wrote:
> 
>> On Aug 8, 2017, at 6:51 PM, Taylor Swift > > wrote:
>> 
>> 
>> 
>> On Tue, Aug 8, 2017 at 9:38 PM, Andrew Trick > > wrote:
>> 
>>> > UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)
>>> 
>>> Well, I think it's somewhat ridiculous for users to write this every time 
>>> they allocate a buffer:
>>> 
>>> `UnsafeMutableRawBufferPointer.allocate(bytes: size, alignedTo: 
>>> MemoryLayout.alignment)`
>>> 
>>> If anyone reading the code is unsure about the Swift API's alignment
>>> guarantee, it's trivial to check the API docs.
>>> 
>>> You could introduce a clearly documented default `alignedTo`
>>> argument. The reason I didn't do that is that the runtime won't
>>> respect it anyway. But I think it would be fair to go ahead with the
>>> API and file a bug against the runtime.
>>> 
>>> Default argument of MemoryLayout.alignment is the way to go but as you 
>>> said i don’t know if that is actually allowed/works. An alternative is to 
>>> have two allocate methods each, one that takes an alignment argument and 
>>> one that doesn’t (and aligns to pointer alignment) but that feels 
>>> inelegant. Default arguments would be better.
>> 
>> Default argument makes sense to me too. Then the raw buffer pointer and 
>> regular raw pointer APIs can be consistent with each other.
>> 
>> Runtime bug: https://bugs.swift.org/browse/SR-5664 
>> 
>> 
>> yikes i was not aware of this. I don’t think it’s bad enough to warrant 
>> dropping the argument like with deallocate(capacity:) but I can imagine bad 
>> things happening to code that crams extra inhabitants into pointers.
> 
> If we ever need to do pointer adjustment during deallocation to accommodate 
> alignment, then I think the Swift runtime can track that. I see no reason to 
> muddy the UnsafeRawPointer API with it. So, I agree with your proposed change 
> to drop `alignedTo` there.
> 
> -Andy
> 
> oh lol I was talking about assuming the pointer returned by 
> allocate(bytes:alignedTo:) is a multiple of alignedTo. Some code might be 
> relying on the last few bits of the pointer being zero; i.e. sticking bit 
> flags there like how some implementations store the red/black color 
> information in a red-black tree node.

Oh, sure. But I think it will be easy to fix the runtime. We could probably do 
it before the proposal is accepted if necessary.
-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 8, 2017, at 6:51 PM, Taylor Swift  wrote:
> 
> 
> 
> On Tue, Aug 8, 2017 at 9:38 PM, Andrew Trick  > wrote:
> 
>> > UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)
>> 
>> Well, I think it's somewhat ridiculous for users to write this every time 
>> they allocate a buffer:
>> 
>> `UnsafeMutableRawBufferPointer.allocate(bytes: size, alignedTo: 
>> MemoryLayout.alignment)`
>> 
>> If anyone reading the code is unsure about the Swift API's alignment
>> guarantee, it's trivial to check the API docs.
>> 
>> You could introduce a clearly documented default `alignedTo`
>> argument. The reason I didn't do that is that the runtime won't
>> respect it anyway. But I think it would be fair to go ahead with the
>> API and file a bug against the runtime.
>> 
>> Default argument of MemoryLayout.alignment is the way to go but as you 
>> said i don’t know if that is actually allowed/works. An alternative is to 
>> have two allocate methods each, one that takes an alignment argument and one 
>> that doesn’t (and aligns to pointer alignment) but that feels inelegant. 
>> Default arguments would be better.
> 
> Default argument makes sense to me too. Then the raw buffer pointer and 
> regular raw pointer APIs can be consistent with each other.
> 
> Runtime bug: https://bugs.swift.org/browse/SR-5664 
> 
> 
> yikes i was not aware of this. I don’t think it’s bad enough to warrant 
> dropping the argument like with deallocate(capacity:) but I can imagine bad 
> things happening to code that crams extra inhabitants into pointers.

If we ever need to do pointer adjustment during deallocation to accommodate 
alignment, then I think the Swift runtime can track that. I see no reason to 
muddy the UnsafeRawPointer API with it. So, I agree with your proposed change 
to drop `alignedTo` there.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 8, 2017, at 5:49 PM, Taylor Swift  wrote:
> 
> 
> 
> On Tue, Aug 8, 2017 at 7:53 PM, Andrew Trick  > wrote:
> 
>> On Aug 8, 2017, at 9:52 AM, Taylor Swift via swift-evolution 
>> > wrote:
>> 
>> Since Swift 5 just got opened up for proposals, SE-184 Improved Pointers is 
>> ready for community review, and I encourage everyone to look it over and 
>> provide feedback. Thank you!
>> >  
>> >
> 
> 
> Excellent. Thanks for patiently iterating on this. I know it's time consuming.
> 
> > add a default value of 1 to all size parameters on
> > UnsafeMutablePointer and applicable size parameters on
> > UnsafeMutableRawPointer
> 
> I'm generally ok with this if you have seen the benefit of it in real
> code. However, I do not think any `repeating:` methods should have a
> default count.
> 
> Actually, i believe initialize(to:count:) is currently the one method that 
> already has a default count. That’s probably because the standard library 
> calls this method with a count argument of 1 more than any other memorystate 
> method. I don’t know if this decision was only made for the sake of the 
> stdlib or if it had an API justification.

Right you are. I had just noticed that none of the other `repeating` APIs had a 
default. But if this default argument simplifies real code patterns then I’m 
fine with it.

> > UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)
> 
> Well, I think it's somewhat ridiculous for users to write this every time 
> they allocate a buffer:
> 
> `UnsafeMutableRawBufferPointer.allocate(bytes: size, alignedTo: 
> MemoryLayout.alignment)`
> 
> If anyone reading the code is unsure about the Swift API's alignment
> guarantee, it's trivial to check the API docs.
> 
> You could introduce a clearly documented default `alignedTo`
> argument. The reason I didn't do that is that the runtime won't
> respect it anyway. But I think it would be fair to go ahead with the
> API and file a bug against the runtime.
> 
> Default argument of MemoryLayout.alignment is the way to go but as you 
> said i don’t know if that is actually allowed/works. An alternative is to 
> have two allocate methods each, one that takes an alignment argument and one 
> that doesn’t (and aligns to pointer alignment) but that feels inelegant. 
> Default arguments would be better.

Default argument makes sense to me too. Then the raw buffer pointer and regular 
raw pointer APIs can be consistent with each other.

Runtime bug: https://bugs.swift.org/browse/SR-5664

> > and initializeMemory(as:at:repeating:count:),
> > initializeMemory(as:from:count:)
> > moveInitializeMemory(as:from:count:), and
> > bindMemory(to:count:) to UnsafeMutableRawBufferPointer
> 
> I think you should move the raw pointer changes to a separate bullet point.
> 
> Presumably the raw buffer capacity must match or exceed count * stride?
> 
> -Andy
> 
> The reason the raw buffer pointers don’t fill in their own size is the 
> destination type might not line up with the raw buffer size, and then there’s 
> questions of rounding and whatnot. bindMemory(to:count:) would also have to 
> perform integer division or some other defined behavior. Though if people 
> think computing the strided count inside the buffer pointer method is the 
> right way to go, I’m open to that.

No, I think your API is fine. I just wanted to clarify that we will trap if the 
raw buffer is too small to fit the requested elements.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-184 Improved Pointers

[Andrew Trick via swift-evolution]

> On Aug 8, 2017, at 9:52 AM, Taylor Swift via swift-evolution 
>  wrote:
> 
> Since Swift 5 just got opened up for proposals, SE-184 Improved Pointers is 
> ready for community review, and I encourage everyone to look it over and 
> provide feedback. Thank you!
>   
> >


Excellent. Thanks for patiently iterating on this. I know it's time consuming.

> This differs from the previous draft of this proposal, in that this
> expansion is more additive and less source-breaking, preserving the
> much of the sized API present on UnsafeMutablePointer.

Yay!

> For the binary operations assign(from:), moveAssign(from:),
> moveInitialize(from:), and initialize(from:), it is assumed that the
> other buffer pointer contains at least as many elements as self does.

Uh-oh! This should be consistent with
UnsafeRawBufferPointer.copy(from:bytes:) (the raw version of
assign/initialize(from:)). There we assume no data is dropped and
allow an uninitalized buffer tail. What was your rationale for the
opposite choice and how can we reconcile these APIs?

> add a default value of 1 to all size parameters on
> UnsafeMutablePointer and applicable size parameters on
> UnsafeMutableRawPointer

I'm generally ok with this if you have seen the benefit of it in real
code. However, I do not think any `repeating:` methods should have a
default count.

> avoids the contradictory and inconsistent use of count to represent a byte 
> quantity

Background: Whether you consider an API consistent depends on how you
prioritize the guidelines. Here you've taken guidelines that I started
to use for new raw pointer APIs and given them higher priority than
other guidelines, in this case having the `count` initializer label match
the name of the public property being initialized. I think your change
is an improvement, but there was nothing accidental about the previous
API.

> UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)

Well, I think it's somewhat ridiculous for users to write this every time they 
allocate a buffer:

`UnsafeMutableRawBufferPointer.allocate(bytes: size, alignedTo: 
MemoryLayout.alignment)`

If anyone reading the code is unsure about the Swift API's alignment
guarantee, it's trivial to check the API docs.

You could introduce a clearly documented default `alignedTo`
argument. The reason I didn't do that is that the runtime won't
respect it anyway. But I think it would be fair to go ahead with the
API and file a bug against the runtime.

> and initializeMemory(as:at:repeating:count:),
> initializeMemory(as:from:count:)
> moveInitializeMemory(as:from:count:), and
> bindMemory(to:count:) to UnsafeMutableRawBufferPointer

I think you should move the raw pointer changes to a separate bullet point.

Presumably the raw buffer capacity must match or exceed count * stride?

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 19, 2017, at 11:13 AM, Taylor Swift  wrote:
> 
> What about `value:`?
> 
> `ptr.initialize(value: value)`
> `ptr.initialize(value: value, count: 13)`
> `ptr.initialize(as: UInt16.self, at: 0, value: value, count: 13)`

Works for me if it’s reasonably consistent with the rest of the stdlib. I don’t 
know why `value` was avoided in the first place.
-Andy

> On Wed, Jul 19, 2017 at 12:01 PM, Andrew Trick  > wrote:
> 
>> On Jul 18, 2017, at 9:42 PM, Taylor Swift > > wrote:
>> 
>> How do we feel about changing the label to `repeated:`, even in cases where 
>> `count:` is 1?
>> 
>> This would mean that calls would read like this: `ptr.initialize(repeated: 
>> value)`
> 
> Right. Given the default `count` of 1, it’s probably best to stick with `to: 
> value`.
> -Andy
> 
>> 
>> A grep through the stdlib shows that this is by far the most common use case:
>> 
>> swift-source/swift$ grep initialize\(to: . -r
>> ./stdlib/private/SwiftPrivatePthreadExtras/SwiftPrivatePthreadExtras.swift:  
>>   result.initialize(to: block(arg))
>> ./stdlib/private/StdlibUnittest/OpaqueIdentityFunctions.swift:  
>> ptr.initialize(to: x)
>> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
>> anyPointer.initialize(to: any)
>> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
>> fn.initialize(to: ThickFunction0(function: function))
>> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
>> fn.initialize(to: ThickFunction1(function: function))
>> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
>> fn.initialize(to: ThickFunction2(function: function))
>> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
>> fn.initialize(to: ThickFunction3(function: function))
>> ./stdlib/private/SwiftPrivate/ShardedAtomicCounter.swift:  (shards + 
>> i).initialize(to: 0)
>> ./stdlib/public/SDK/Foundation/NSError.swift:out.initialize(to: bridged)
>> ./stdlib/public/core/UnsafeRawPointer.swift.gyb://   
>> (self.assumingMemoryBound(to: T.self) + i).initialize(to: source[i])
>> ./stdlib/public/core/UnsafeRawPointer.swift.gyb:  //   .initialize(to: 
>> (source + i).move())
>> ./stdlib/public/core/UnsafeRawPointer.swift.gyb:  //   
>> (--dst).initialize(to: (--src).move())
>> ./stdlib/public/core/Arrays.swift.gyb:  p.initialize(to: repeatedValue)
>> ./stdlib/public/core/Arrays.swift.gyb:(_buffer.firstElementAddress + 
>> oldCount).initialize(to: newElement)
>> ./stdlib/public/core/Arrays.swift.gyb:p.initialize(to: x)
>> ./stdlib/public/core/Arrays.swift.gyb:  p.initialize(to: newValues[q])
>> ./stdlib/public/core/Arrays.swift.gyb:(base + 
>> newCount).initialize(to: next)
>> ./stdlib/public/core/AnyHashable.swift:result.initialize(to: value)
>> ./stdlib/public/core/AnyHashable.swift:result.initialize(to: value)
>> ./stdlib/public/core/AnyHashable.swift:  target.initialize(to: 
>> AnyHashable(value))
>> ./stdlib/public/core/ArrayCast.swift:  target.initialize(to: 
>> _arrayForceCast(source.pointee))
>> ./stdlib/public/core/ArrayCast.swift:target.initialize(to: result)
>> ./stdlib/public/core/ContiguousArrayBuffer.swift:  (resultPtr + 
>> i).initialize(to: _bridgeAnythingToObjectiveC(p[i]))
>> ./stdlib/public/core/ContiguousArrayBuffer.swift:p.initialize(to: 
>> source[i])
>> ./stdlib/public/core/ContiguousArrayBuffer.swift:p.initialize(to: 
>> element)
>> ./stdlib/public/core/ArrayBuffer.swift:  result.initialize(to: 
>> result.pointee)
>> ./stdlib/public/core/HashedCollections.swift.gyb:  target.initialize(to: 
>> _setDownCast(source.pointee))
>> ./stdlib/public/core/HashedCollections.swift.gyb:target.initialize(to: 
>> result)
>> ./stdlib/public/core/HashedCollections.swift.gyb:  target.initialize(to: 
>> _dictionaryDownCast(source.pointee))
>> ./stdlib/public/core/HashedCollections.swift.gyb:target.initialize(to: 
>> result)
>> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
>> i).initialize(to: k)
>> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
>> toEntryAt).initialize(to: (from.keys + at).move())
>> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
>> i).initialize(to: k)
>> ./stdlib/public/core/HashedCollections.swift.gyb:(values + 
>> i).initialize(to: v)
>> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
>> toEntryAt).initialize(to: (from.keys + at).move())
>> ./stdlib/public/core/HashedCollections.swift.gyb:(values + 
>> toEntryAt).initialize(to: (from.values + at).move())
>> ./stdlib/public/core/ReflectionLegacy.swift:  out.initialize(to: 
>> String(reflecting: x))
>> 
>> ./stdlib/public/core/ManagedBuffer.swift:p.headerAddress.initialize(to: 
>> initHeaderVal)
>> ./stdlib/public/core/ManagedBuffer.swift:  $0.initialize(to: 
>> ./stdlib/public/core/ArrayBufferProtocol.swift:

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 18, 2017, at 9:42 PM, Taylor Swift  wrote:
> 
> How do we feel about changing the label to `repeated:`, even in cases where 
> `count:` is 1?
> 
> This would mean that calls would read like this: `ptr.initialize(repeated: 
> value)`

Right. Given the default `count` of 1, it’s probably best to stick with `to: 
value`.
-Andy
> 
> A grep through the stdlib shows that this is by far the most common use case:
> 
> swift-source/swift$ grep initialize\(to: . -r
> ./stdlib/private/SwiftPrivatePthreadExtras/SwiftPrivatePthreadExtras.swift:   
>  result.initialize(to: block(arg))
> ./stdlib/private/StdlibUnittest/OpaqueIdentityFunctions.swift:  
> ptr.initialize(to: x)
> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
> anyPointer.initialize(to: any)
> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
> fn.initialize(to: ThickFunction0(function: function))
> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
> fn.initialize(to: ThickFunction1(function: function))
> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
> fn.initialize(to: ThickFunction2(function: function))
> ./stdlib/private/SwiftReflectionTest/SwiftReflectionTest.swift:  
> fn.initialize(to: ThickFunction3(function: function))
> ./stdlib/private/SwiftPrivate/ShardedAtomicCounter.swift:  (shards + 
> i).initialize(to: 0)
> ./stdlib/public/SDK/Foundation/NSError.swift:out.initialize(to: bridged)
> ./stdlib/public/core/UnsafeRawPointer.swift.gyb://   
> (self.assumingMemoryBound(to: T.self) + i).initialize(to: source[i])
> ./stdlib/public/core/UnsafeRawPointer.swift.gyb:  //   .initialize(to: 
> (source + i).move())
> ./stdlib/public/core/UnsafeRawPointer.swift.gyb:  //   
> (--dst).initialize(to: (--src).move())
> ./stdlib/public/core/Arrays.swift.gyb:  p.initialize(to: repeatedValue)
> ./stdlib/public/core/Arrays.swift.gyb:(_buffer.firstElementAddress + 
> oldCount).initialize(to: newElement)
> ./stdlib/public/core/Arrays.swift.gyb:p.initialize(to: x)
> ./stdlib/public/core/Arrays.swift.gyb:  p.initialize(to: newValues[q])
> ./stdlib/public/core/Arrays.swift.gyb:(base + 
> newCount).initialize(to: next)
> ./stdlib/public/core/AnyHashable.swift:result.initialize(to: value)
> ./stdlib/public/core/AnyHashable.swift:result.initialize(to: value)
> ./stdlib/public/core/AnyHashable.swift:  target.initialize(to: 
> AnyHashable(value))
> ./stdlib/public/core/ArrayCast.swift:  target.initialize(to: 
> _arrayForceCast(source.pointee))
> ./stdlib/public/core/ArrayCast.swift:target.initialize(to: result)
> ./stdlib/public/core/ContiguousArrayBuffer.swift:  (resultPtr + 
> i).initialize(to: _bridgeAnythingToObjectiveC(p[i]))
> ./stdlib/public/core/ContiguousArrayBuffer.swift:p.initialize(to: 
> source[i])
> ./stdlib/public/core/ContiguousArrayBuffer.swift:p.initialize(to: element)
> ./stdlib/public/core/ArrayBuffer.swift:  result.initialize(to: 
> result.pointee)
> ./stdlib/public/core/HashedCollections.swift.gyb:  target.initialize(to: 
> _setDownCast(source.pointee))
> ./stdlib/public/core/HashedCollections.swift.gyb:target.initialize(to: 
> result)
> ./stdlib/public/core/HashedCollections.swift.gyb:  target.initialize(to: 
> _dictionaryDownCast(source.pointee))
> ./stdlib/public/core/HashedCollections.swift.gyb:target.initialize(to: 
> result)
> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
> i).initialize(to: k)
> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
> toEntryAt).initialize(to: (from.keys + at).move())
> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
> i).initialize(to: k)
> ./stdlib/public/core/HashedCollections.swift.gyb:(values + 
> i).initialize(to: v)
> ./stdlib/public/core/HashedCollections.swift.gyb:(keys + 
> toEntryAt).initialize(to: (from.keys + at).move())
> ./stdlib/public/core/HashedCollections.swift.gyb:(values + 
> toEntryAt).initialize(to: (from.values + at).move())
> ./stdlib/public/core/ReflectionLegacy.swift:  out.initialize(to: 
> String(reflecting: x))
> 
> ./stdlib/public/core/ManagedBuffer.swift:p.headerAddress.initialize(to: 
> initHeaderVal)
> ./stdlib/public/core/ManagedBuffer.swift:  $0.initialize(to: 
> ./stdlib/public/core/ArrayBufferProtocol.swift:(elements + 
> j).initialize(to: newValues[i])
> ./stdlib/public/core/Sequence.swift:ptr.initialize(to: x)
> ./stdlib/public/core/HeapBuffer.swift:  $0.initialize(to: 
> Header(initializer))
> ./stdlib/public/core/UnsafeBitMap.swift:values.initialize(to: 0, count: 
> numberOfWords)
> ./stdlib/public/core/String.swift:resultStorage.initialize(to: 
> ./test/api-digester/stdlib-stable.json:  "printedName": 
> "initialize(to:count:)",
> ./test/Generics/slice_test.swift:(newbase + i).initialize(to: 
> (base+i).move())
> ./test/Generics/slice_test.swift:(base+length).initialize(to: elem)
> 

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 18, 2017, at 11:36 AM, Taylor Swift  wrote:
> 
> I'm not sure removing the need for implicit casts is a goal. I did
> that with the pointer `init` methods, but now I think that should be
> cleaned up to reduce API surface. I think smaller API surface wins in
> these cases. Is there a usability issue you're solving?
> 
> Yes, I can imagine initializing a mutable pointer to some values, and then 
> wanting to use that pointer as a source to initialize more buffers. Having to 
> convert a mutable pointer to an immutable pointer is annoying because a 
> function that takes an immutable pointer obviously shouldn’t care if the 
> pointer could be mutated anyway. It’s like having to rebind a `var` variable 
> to a `let` constant before passing it as any non-inout argument to a 
> function, since function parameters are immutable. At any rate, this only 
> applies to two out of the seven memorystate operations, so comparably, it’s 
> not a big API expansion at all.

The conversion you’re talking about should be handled by the compiler.

public func get(_ p: UnsafePointer) -> T {
  return p.pointee
}

public func foo(p: UnsafeMutablePointer) {
  _ = get(p)
}

Or are you thinking of a different use case?

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 18, 2017, at 11:36 AM, Taylor Swift  wrote:
> 
> > fix the ordering of the arguments in 
> > initializeMemory(as:at:count:to:)
> 
> I think this ordering was an attempt to avoid confusion with binding
> memory where `to` refers to a type. However, it should be consistent
> with `UnsafePointer.initialize`, so we need to pick one of those to
> change.
> 
> This would be a non-issue had we just been consistent with the rest of the 
> stdlib and named this argument `repeating:` instead of `to:`. But 
> `ptr.initialize(repeating: 255, count: 100)` doesn’t read quite as naturally 
> in English as `ptr.initialize(to: 255, count: 100)` which is why I left this 
> idea out of the proposal. Now that you mention the problem with 
> `initializeMemory(as:at:count:to:)`, it might be a good idea to add 
> this rename back into it.

I think `repeating` is much more clear.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 17, 2017, at 10:06 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> I’ve drafted a new version of the unsafe pointer proposal based on feedback 
> I’ve gotten from this thread. You can read it here 
> .
> 
> ~~~
> Swift’s pointer types are an important interface for low-level memory 
> manipulation, but the current API design is not very safe, consistent, or 
> convenient. Many memory methods demand a capacity: or count: argument, 
> forcing the user to manually track the size of the memory block, even though 
> most of the time this is either unnecessary, or redundant as buffer pointers 
> track this information natively. In some places, this design turns 
> UnsafePointers into outright DangerousPointers, leading users to believe that 
> they have allocated or freed memory when in fact, they have not.
> 
> The current API suffers from inconsistent naming, poor usage of default 
> argument values, missing methods, and excessive verbosity, and encourages 
> excessively unsafe programming practices. This proposal seeks to iron out 
> these inconsistencies, and offer a more convenient, more sensible, and less 
> bug-prone API for Swift pointers.
> 
> The previous draft 
>  of this 
> proposal was relatively source-breaking, calling for a separation of 
> functionality between singular pointer types and vector (buffer) pointer 
> types. This proposal instead separates functionality between 
> internally-tracked length pointer types and externally-tracked length pointer 
> types. This results in an equally elegant API with about one-third less 
> surface area.
> 
>  >
> 
> ~~~
> 

> remove the capacity parameter from deallocate(capacity:) and 
> deallocate(bytes:alignedTo:)

That's probably for the best.

> add unsized memory methods to UnsafeMutableBufferPointer

Yay!

> add an assign(to:count:) method to UnsafeMutablePointer and an assign(to:) 
> method to UnsafeMutableBufferPointer

Sure.

> add a default value of 1 to all size parameters on UnsafeMutablePointer and 
> applicable
> size parameters on UnsafeMutableRawPointer

I'm not opposed to it.

> rename copyBytes(from:count:) to copy(from:bytes:)

LGTM in the interest of consistency. I should not have caved on this the first 
time around.

> bytes refers to, well, a byte quantity that is not assumed to be initialized.
> capacity refers to a strided quantity that is not assumed to be initialized.
> count refers to a strided quantity that is assumed to be initialized.

That's how I see it.

> rename count in UnsafeMutableRawBufferPointer.allocate(count:) to bytes and 
> add an
> alignedTo parameter to make it 
> UnsafeMutableRawBufferPointer.allocate(bytes:alignedTo:)

Memory allocation is an issue unto itself. I generally prefer your
proposed API. However...

1. Larger-than-pointer alignments aren't currently respected.

2. Users virtually never want to specify the alignment explicitly. They
   just want platform alignment. Unfortunately, there's no reasonable
   "maximal" alignment to use as a default. I think pointer-alignment
   is an excellent default guarantee.

3. The current allocation builtins seem to presume that
   allocation/deallocation can be made more efficient if the user code
   specifies alignment at deallocation. I don't think
   UnsafeRawBufferPointer should expose that to the user, so I agree
   with your proposal. In fact, I think aligned `free` should be
   handled within the Swift runtime.

Resolving these issues requires changes to the Swift runtime API and
implementation. This might be a good time to revisit that design, but
it might slow down the rest of the proposal.

> fix the ordering of the arguments in 
> initializeMemory(as:at:count:to:)

I think this ordering was an attempt to avoid confusion with binding
memory where `to` refers to a type. However, it should be consistent
with `UnsafePointer.initialize`, so we need to pick one of those to
change.

> add the sized memorystate functions withMemoryRebound Result>(to:count:_:) to
> UnsafeMutableBufferPointer, and initializeMemory(as:at:to:count:),
> initializeMemory(as:from:count:) 
> moveInitializeMemory(as:from:count:),
> and bindMemory(to:count:) to UnsafeMutableRawBufferPointer

Yay!

> add mutable overloads to non-vacating memorystate method arguments

I'm not sure removing the need for implicit casts is a goal. I did
that with the pointer `init` methods, but now I think that should be
cleaned up to reduce API surface. I think smaller API surface wins in
these cases. Is there a usability issue you're solving?

> add a init(mutating:) initializer to UnsafeMutableBufferPointer

Yes, finally.

> remove initialize(from:) from UnsafeMutablePointer

Yep.

> 

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 14, 2017, at 7:24 PM, Taylor Swift  wrote:
> 
> For the sake of argument, I’m gonna claim that instead, 
> UnsafeMutableBufferPointer is the low level necessity, and 
> UnsafeMutablePointer is the low level convenience structure.
> 
> Suppose that every block of memory has a starting point, and a length. An 
> UnsafeBufferPointer represents that, since all an UnsafeBufferPointer is, is 
> a start address, and a length value, bundled together into one variable 
> instead of two. Then we can say that UnsafePointer is a special case of 
> UnsafeBufferPointer, where the length value is unknown.
> 
> Mind you I didn’t say that the length value equals 1. But it might as well 
> be. Because when the length is unknown, the only index where you can 
> reasonably expect there to be a valid value is index zero. You wouldn’t feel 
> confident trying to do something to the memory address 3 strides down from 
> the pointer. If you did feel confident doing that, well now you have a length 
> value. The length value is 4. Your regular Pointer is now a BufferPointer.
> 
> The only reason any of this matters, is because all of our memory and 
> memorystate -related functions take size arguments. You have to fill in the 
> size argument with something. That something is the buffer `count`. Even if 
> you don’t actually know the number for the `count`, you still have to supply 
> some value, in which case the “only” sensible choice is “1”. That’s why you 
> could argue that UnsafePointer is just a special case of a buffer pointer 
> where `count` is 1. The UnsafePointer API demands information that only 
> BufferPointers know. You could design an API where buffer pointers are the 
> only pointers that exist, and everything would still work fine. Just because 
> they have high-level capabilities doesn’t mean they can’t do everything plain 
> pointers can do just as efficiently. A plain pointer takes up one word of 
> storage, and you use another word of storage yourself to track the size. A 
> buffer pointer stores the two words next to each other. You could ignore the 
> size word and track the size yourself, taking up three words of storage, but 
> that would just be silly.
> 
> But wait! I don’t need to track the size, you say! The pointer just points to 
> one item! I just need the address word, not the size word! Well, yeah. That’s 
> why I proposed a sizeless singular pointer API, so you don’t have to go 
> around plugging “1”s everywhere and save yourself a word of storage. You can 
> argue that’s the actual high-level API, since it abstracts away the length 
> stuff. But if “1” isn’t your answer to the question “on how many instances 
> should this method operate on”, go back to paragraph 3 and convince yourself 
> that what you really have is an buffer pointer, not a plain pointer.
> 
> Now I thought of an exception to this, like when you’re assigning 8-bit RGBA 
> values to an image buffer and it might make sense to write something like 
> this in the current API:
> 
> let size:Int = height * width << 2
> let base = UnsafeMutablePointer.allocate(capacity: size) 
> defer 
> {
> base.deallocate(capacity: -314159) // anything goes
> }
> var pixel:UnsafeMutablePointer = base
> while pixel < base + size 
> {
> pixel.initialize(from: bg_color_rgba, count: 4)
> pixel += 4
> }
> 
> And it’s convenient to be able to initialize 4 instances at a time without 
> creating a 4-count buffer pointer. But this doesn’t really contradict my 
> point if you think carefully. The groups of 4 RGBA values are your atom here, 
> you’re not really working on 4 instances at a time with a stride 1 UInt8 big, 
> you’re working on 1 single instance at a time with a stride 4 UInt8s big. 
> Writing this is currently painful without the sized UnsafeMutablePointer API, 
> but I think this is a deficiency of the UnsafeMutableBufferPointer API and 
> that of its slice type, not a reason for keeping the sized 
> UnsafeMutablePointer API, which if you ask me is a hack to get around the 
> fact that we can’t use buffer pointer slices easily.
> 
> Anyway, I mention all of this only because everyone seems convinced that 
> UnsafePointer is supposed to be “edgier” than UnsafeBufferPointer. If you 
> have ideas to make this better, by all means share them

UnsafeBufferPointer was developed to add safety and convenience on top of 
UnsafePointer. That’s just a historical fact, not necessarily detracting from 
your argument.

The problem with your use case above is that UnsafePointer doesn’t get the 
debug-mode bounds checks.

There’s an argument to be made for keeping the UnsafePointer API complete (as a 
single point of truth for the memory model), and an argument to be made for 
removing it (it’s not the *right* API to use in nearly all cases). I would be 
fine with either approach and people can argue it out. At the moment I would 
lean toward not breaking source for something that isn’t a critical problem for 

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 14, 2017, at 9:33 AM, Taylor Swift  wrote:
> 
> How would you feel about:
> 
> struct UnsafeMutableRawBufferPointer
> {
> 
> --- static func allocate(count:Int) -> UnsafeMutableRawBufferPointer
> +++ static func allocate(bytes:Int, alignedTo:Int) -> 
> UnsafeMutableRawBufferPointer
> func deallocate()
> +++ func bindMemory(to:Element.Type, capacity:Int)
> +++ func copy(from:UnsafeRawBufferPointer, bytes:Int)
> +++ func initializeMemory(as:Element.Type, at:Int, to:Element, 
> count:Int)
> +++ func initializeMemory(as:Element.Type, 
> from:UnsafeBufferPointer, count:Int)
> +++ func moveInitializeMemory(as:Element.Type, 
> from:UnsafeMutableBufferPointer, count:Int
> }
> 
> “bytes”= 8 bit quantities (don’t @ me we’re assuming 8 bit bytes)
> “capacity” = strided quantities, not assumed to be initialized 
> “count”= strided quantities, assumed to be initialized
> 
> It’s also worth nothing that a lot of what the proposal tries to add to 
> UnsafeBufferPointer is already present in UnsafeMutableRawPointer like a 
> sizeless deallocate() and a sizeless copyBytes(from:).
> 
> Although I’m not sure what’s going on with the latter one 
> …lol
>  swiftdoc

Purely in terms of label names, what you have above is perfectly fine.

You’re going to have a problem adding the alignment constraint to buffer 
pointer, but that’s another topic.

-Andy

> On Fri, Jul 14, 2017 at 1:57 AM, Andrew Trick  > wrote:
> 
>> On Jul 13, 2017, at 10:30 PM, Taylor Swift > > wrote:
>> 
>> I’m confused I thought we were talking about the naming choices for the 
>> argument labels in those functions. I think defining and abiding by 
>> consistent meanings for `count`, `capacity`, and `bytes` is a good idea, and 
>> it’s part of what this proposal tries to accomplish. Right now half the time 
>> we use `count` to refer to “bytes” and half the time we use it to refer to 
>> “instances”. The same goes for the word “capacity”. This is all laid out in 
>> the document:
>> 
>> “““
>> Finally, the naming and design of some UnsafeMutableRawPointer members 
>> deserves to be looked at. The usage of capacity, bytes, and count as 
>> argument labels is wildly inconsistent and confusing. In 
>> copyBytes(from:count:), count refers to the number of bytes, while in 
>> initializeMemory(as:at:count:to:) and 
>> initializeMemory(as:from:count:), count refers to the number of strides. 
>> Meanwhile bindMemory(to:capacity:) uses capacity to refer to this 
>> quantity. The always-problematic deallocate(bytes:alignedTo) method and 
>> allocate(bytes:alignedTo:) type methods use bytes to refer to 
>> byte-quantities. Adding to the confusion, UnsafeMutableRawBufferPointer 
>> offers an allocate(count:) type method (the same signature method we’re 
>> trying to add to UnsafeMutableBufferPointer), except the count in this 
>> method refers to bytes. This kind of API naming begets stride bugs and makes 
>> Swift needlessly difficult to learn.
>> ”””
>> 
>> The only convenience methods this proposal is trying to add is the 
>> functionality on the buffer pointer types. There seems to be broad support 
>> for adding this functionality as no one has really opposed that part of the 
>> proposal yet. Any other new methods like `UnsafeMutablePointer.assign(to:)` 
>> are there for API consistency.
>> 
>> This proposal also calls for getting rid of one of those “redundant 
>> initializers” :)
> 
> Since we’re not bike-shedding the specifics yet, I’ll just give you some 
> background.
> 
> We would ultimately like APIs that allocate and initialize in one go. It’s 
> important that the current lower-level (dangerous) APIs make a clear 
> distinction between initialized and uninitialized memory to avoid confusing 
> them with future (safer) APIs. `capacity` always refers to memory that may be 
> uninitialized. I think that’s very clear and helpful.
> 
> In the context of pointers `count` should always be in strides. For raw 
> pointers, that happens to be the same as as `bytes`.
> 
> I initially proposed copy(bytes:from:), but someone thought that `bytes` in 
> this particular context did not properly convey the "count of bytes" as 
> opposed to the source of the bytes. You’re right, that’s inconsistent with 
> allocate/deallocate(bytes:), because allocateBytes(count:) would be silly. 
> Just be aware that the inconsistency is a result of over-thinking and 
> excessive bike shedding to the detriment of something that looks nice and is 
> easy to remember.
> 
> I should also point out that the inconsistencies in functionality across 
> pointer types, in terms of collection support and other convenience, is also 
> known but was deliberately stripped from proposals as “additive”.
> 
> -Andy
> 

___

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 13, 2017, at 10:30 PM, Taylor Swift  wrote:
> 
> I’m confused I thought we were talking about the naming choices for the 
> argument labels in those functions. I think defining and abiding by 
> consistent meanings for `count`, `capacity`, and `bytes` is a good idea, and 
> it’s part of what this proposal tries to accomplish. Right now half the time 
> we use `count` to refer to “bytes” and half the time we use it to refer to 
> “instances”. The same goes for the word “capacity”. This is all laid out in 
> the document:
> 
> “““
> Finally, the naming and design of some UnsafeMutableRawPointer members 
> deserves to be looked at. The usage of capacity, bytes, and count as argument 
> labels is wildly inconsistent and confusing. In copyBytes(from:count:), count 
> refers to the number of bytes, while in initializeMemory(as:at:count:to:) 
> and initializeMemory(as:from:count:), count refers to the number of 
> strides. Meanwhile bindMemory(to:capacity:) uses capacity to refer to this 
> quantity. The always-problematic deallocate(bytes:alignedTo) method and 
> allocate(bytes:alignedTo:) type methods use bytes to refer to 
> byte-quantities. Adding to the confusion, UnsafeMutableRawBufferPointer 
> offers an allocate(count:) type method (the same signature method we’re 
> trying to add to UnsafeMutableBufferPointer), except the count in this method 
> refers to bytes. This kind of API naming begets stride bugs and makes Swift 
> needlessly difficult to learn.
> ”””
> 
> The only convenience methods this proposal is trying to add is the 
> functionality on the buffer pointer types. There seems to be broad support 
> for adding this functionality as no one has really opposed that part of the 
> proposal yet. Any other new methods like `UnsafeMutablePointer.assign(to:)` 
> are there for API consistency.
> 
> This proposal also calls for getting rid of one of those “redundant 
> initializers” :)

Since we’re not bike-shedding the specifics yet, I’ll just give you some 
background.

We would ultimately like APIs that allocate and initialize in one go. It’s 
important that the current lower-level (dangerous) APIs make a clear 
distinction between initialized and uninitialized memory to avoid confusing 
them with future (safer) APIs. `capacity` always refers to memory that may be 
uninitialized. I think that’s very clear and helpful.

In the context of pointers `count` should always be in strides. For raw 
pointers, that happens to be the same as as `bytes`.

I initially proposed copy(bytes:from:), but someone thought that `bytes` in 
this particular context did not properly convey the "count of bytes" as opposed 
to the source of the bytes. You’re right, that’s inconsistent with 
allocate/deallocate(bytes:), because allocateBytes(count:) would be silly. Just 
be aware that the inconsistency is a result of over-thinking and excessive bike 
shedding to the detriment of something that looks nice and is easy to remember.

I should also point out that the inconsistencies in functionality across 
pointer types, in terms of collection support and other convenience, is also 
known but was deliberately stripped from proposals as “additive”.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 13, 2017, at 6:16 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> I am not very familiar with the inner workings of the standard library, 
> however I maintain a few libraries which make extensive use of Swift 
> pointers, such as https://github.com/kelvin13/maxpng 
>  which makes extensive use of 
> Unsafe_Pointers. I also write a lot of code that interfaces with C APIs 
> like Cairo and OpenGL. Most of the ideas in the original proposal came from 
> me dealing with the current Swift pointer APIs in my own code. For example I 
> find myself writing this bit of code 
> 
> let buffer = UnsafeMutableBufferPointer(start: 
> UnsafeMutablePointer.allocate(capacity: byteCount), count: byteCount)
> defer
> {
> buffer.baseAddress?.deallocate(capacity: buffer.count)
> }
> 
> far more than I would like to. While this proposal doesn’t solve every 
> problem with Swift pointers — for example, we need a UMBP initializer that 
> takes an immutable buffer pointer before we are really able to write a lot of 
> examples more concisely, it takes us a great deal closer to being able to 
> write things like 
> 
> UnsafeMutablePointer(mutating: 
> self.zero_line.baseAddress!).deallocate(capacity: self.zero_line.count)
> 
> as 
> 
> UnsafeMutableBufferPointer(mutating: self.zero_line).deallocate()

You should not need to do this at all. A pointer does not need to be mutable to 
deallocate the memory. That’s a bug in the UnsafePointer API.

-Andy

> 
> On Thu, Jul 13, 2017 at 2:47 PM, Dave Abrahams via swift-evolution 
> > wrote:
> 
> on Wed Jul 12 2017, Taylor Swift  > wrote:
> 
> > Hi all, I’ve written up a proposal to modify the unsafe pointer API for
> > greater consistency, safety, and ease of use.
> >
> > ~~~
> >
> > Swift currently offers two sets of pointer types — singular pointers such
> > as UnsafeMutablePointer, and vector (buffer) pointers such as UnsafeMutable
> > *Buffer*Pointer. This implies a natural separation of tasks the two kinds
> > of pointers are meant to do. For example, buffer pointers implement
> > Collection conformance, while singular pointers do not.
> >
> > However, some aspects of the pointer design contradict these implied roles.
> > It is possible to allocate an arbitrary number of instances from a type
> > method on a singular pointer, but not from a buffer pointer. The result of
> > such an operation returns a singular pointer, even though a buffer pointer
> > would be more appropriate to capture the information about the *number* of
> > instances allocated. It’s possible to subscript into a singular pointer,
> > even though they are not real Collections. Some parts of the current design
> > turn UnsafePointers into downright *Dangerous*Pointers, leading users to
> > believe that they have allocated or freed memory when in fact, they have
> > not.
> >
> > This proposal seeks to iron out these inconsistencies, and offer a more
> > convenient, more sensible, and less bug-prone API for Swift pointers.
> >
> >  > >
> 
> I have no problem with this direction in principle; it sounds like a
> good idea.  However, before affirming any particular design I would like
> to see the results of any such proposal applied to a fairly large body
> of code that uses Unsafe[XXX]Pointer today in diverse ways (such as the
> Swift standard library itself), to demonstrate that it does in fact
> improve the code in practice.
> 
> Cheers,
> 
> --
> -Dave
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-evolution 
> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 13, 2017, at 6:55 PM, Taylor Swift  wrote:
> 
> 
> 
> On Thu, Jul 13, 2017 at 6:56 PM, Andrew Trick  > wrote:
> 
>> On Jul 12, 2017, at 12:16 PM, Taylor Swift via swift-evolution 
>> > wrote:
>> 
>> Hi all, I’ve written up a proposal to modify the unsafe pointer API for 
>> greater consistency, safety, and ease of use.
>> 
>> ~~~
>> 
>> Swift currently offers two sets of pointer types — singular pointers such as 
>> UnsafeMutablePointer, and vector (buffer) pointers such as 
>> UnsafeMutableBufferPointer. This implies a natural separation of tasks the 
>> two kinds of pointers are meant to do. For example, buffer pointers 
>> implement Collection conformance, while singular pointers do not.
>> 
>> However, some aspects of the pointer design contradict these implied roles. 
>> It is possible to allocate an arbitrary number of instances from a type 
>> method on a singular pointer, but not from a buffer pointer. The result of 
>> such an operation returns a singular pointer, even though a buffer pointer 
>> would be more appropriate to capture the information about the number of 
>> instances allocated. It’s possible to subscript into a singular pointer, 
>> even though they are not real Collections. Some parts of the current design 
>> turn UnsafePointers into downright DangerousPointers, leading users to 
>> believe that they have allocated or freed memory when in fact, they have not.
>> 
>> This proposal seeks to iron out these inconsistencies, and offer a more 
>> convenient, more sensible, and less bug-prone API for Swift pointers.
>> 
>> > >
>> 
>> ~~~
>> 
> 
> Thanks for taking time to write this up.
> 
> General comments:
> 
> UnsafeBufferPointer is an API layer on top of UnsafePointer. The role
> of UnsafeBufferPointer is direct memory access sans lifetime
> management with Collection semantics. The role of UnsafePointer is
> primarily C interop. Those C APIs should be wrapped in Swift APIs that
> take UnsafeBufferPointer whenever the pointer represents a C array. I
> suppose making UnsafePointer less convenient would push developers
> toward UnsafeBufferPointer. I don't think that's worth outright
> breaking source, but gradual deprecation of convenience methods, like
> `susbscript` might be acceptable.
> 
> Gradual deprecation is exactly what I am proposing. As the document states 
> ,
>  the only methods which should be marked immediately as unavailable are the 
> `deallocate(capacity:)` methods, for safety and source compatibility reasons. 
> Removing `deallocate(capacity:)` now and forcing a loud compiler error 
> prevents catastrophic *silent* source breakage in the future, or worse, from 
> having to *support our own bug*.
>  
> 
> I have mixed feelings about stripping UnsafePointer of basic
> functionality. Besides breaking source, doing that would be
> inconsistent with its role as a lower API layer. The advantage would
> just be descreasing API surface area and forcing developers to use a
> higher-level API.
> 
> UnsafePointer is as much a high level API as UnsafeBufferPointer is.

No it isn’t. We don’t have support for importing certain function signatures as 
taking UnsafeBufferPointer and UnsafePointer doesn't conform to Collection even 
though it nearly always represents an array.

> You wouldn’t create a buffer pointer of length 1 just so you can “stick with 
> the high level API”. UnsafePointer and UnsafeBufferPointer are two tools that 
> do related but different things and they can exist at whatever abstract level 
> you need them at. After all, UnsafeBufferPointer is nothing but an 
> UnsafePointer? with a length value attached to it. If you’re allocating more 
> than one instance of memory, you almost certainly need to track the length of 
> the buffer anyway.

You could call this a proposal to "make unsafe pointer APIs easier to use 
safely". I just want to put an end to the fallacy that the buffer type is for 
multiple values and the plain old pointer represents single instances.

> The additive changes you propose are fairly obvious. See [SR-3088]
> UnsafeMutableBufferPointer doesn't have an allocating init.
> 
> I haven't wanted to waste review cycles on small additive
> changes. It may make sense to batch them up into one coherent
> proposal. Here are a few more to consider.
> 
> - [SR-3929] UnsafeBufferPointer should have init from mutable
> - [SR-4340] UnsafeBufferPointer needs a withMemoryRebound method
> - [SR-3087] No way to arbitrarily initialise an Array's storage
> 
> The feature requests you mention are all very valuable, however with 
> Michael’s point about fixing the memorystate API’s, the size of this 

Re: [swift-evolution] Pitch: Improved Swift pointers

[Andrew Trick via swift-evolution]

> On Jul 12, 2017, at 12:16 PM, Taylor Swift via swift-evolution 
>  wrote:
> 
> Hi all, I’ve written up a proposal to modify the unsafe pointer API for 
> greater consistency, safety, and ease of use.
> 
> ~~~
> 
> Swift currently offers two sets of pointer types — singular pointers such as 
> UnsafeMutablePointer, and vector (buffer) pointers such as 
> UnsafeMutableBufferPointer. This implies a natural separation of tasks the 
> two kinds of pointers are meant to do. For example, buffer pointers implement 
> Collection conformance, while singular pointers do not.
> 
> However, some aspects of the pointer design contradict these implied roles. 
> It is possible to allocate an arbitrary number of instances from a type 
> method on a singular pointer, but not from a buffer pointer. The result of 
> such an operation returns a singular pointer, even though a buffer pointer 
> would be more appropriate to capture the information about the number of 
> instances allocated. It’s possible to subscript into a singular pointer, even 
> though they are not real Collections. Some parts of the current design turn 
> UnsafePointers into downright DangerousPointers, leading users to believe 
> that they have allocated or freed memory when in fact, they have not.
> 
> This proposal seeks to iron out these inconsistencies, and offer a more 
> convenient, more sensible, and less bug-prone API for Swift pointers.
> 
>  >
> 
> ~~~
> 

Thanks for taking time to write this up.

General comments:

UnsafeBufferPointer is an API layer on top of UnsafePointer. The role
of UnsafeBufferPointer is direct memory access sans lifetime
management with Collection semantics. The role of UnsafePointer is
primarily C interop. Those C APIs should be wrapped in Swift APIs that
take UnsafeBufferPointer whenever the pointer represents a C array. I
suppose making UnsafePointer less convenient would push developers
toward UnsafeBufferPointer. I don't think that's worth outright
breaking source, but gradual deprecation of convenience methods, like
`susbscript` might be acceptable.

I have mixed feelings about stripping UnsafePointer of basic
functionality. Besides breaking source, doing that would be
inconsistent with its role as a lower API layer. The advantage would
just be descreasing API surface area and forcing developers to use a
higher-level API.

The additive changes you propose are fairly obvious. See [SR-3088]
UnsafeMutableBufferPointer doesn't have an allocating init.

I haven't wanted to waste review cycles on small additive
changes. It may make sense to batch them up into one coherent
proposal. Here are a few more to consider.

- [SR-3929] UnsafeBufferPointer should have init from mutable
- [SR-4340] UnsafeBufferPointer needs a withMemoryRebound method
- [SR-3087] No way to arbitrarily initialise an Array's storage

Point by point:

> drop the capacity parameter from UnsafeMutablePointer.allocate() and 
> deallocate().

I do not agree with removing the capacity parameter and adding a
single-instance allocation API. UnsafePointer was not designed for
single instances, it was primarily designed for C-style arrays. I
don't see the value in providing a different unsafe API for single
vs. multiple values.

I agree the primary allocation API should be
UnsafeMutableBufferPointer.allocate(capacity:). There is an argument
to be made for removing UnsafeMutablePointer.allocate(capacity:)
entirely. But, as Michael Ilseman pointed out, that would involve
reevaluating several other members of the UnsafePointer API. I think
it's reasonable for UnsafePointer to retain all its functionality as a
lower level API.

I don't understand what is misleading about
UnsafePointer.deallocate(capacity:). It *is* inconvenienent for the
user to keep track of memory capacity. Presumably that was done so
either the implementation can move away from malloc/free or some sort
of memory tracking can be implemented on the standard library
side. Obviously, UnsafeBufferPointer.deallocate() would be cleaner in
most cases.

> add an allocate(count:) type method to UnsafeMutableBufferPointer

`capacity` should be used for allocating uninitialized memory not
`count`. `count` should only refer to a number of initialized objects!

> add a deallocate() instance method to UnsafeMutableBufferPointer

Yes, of course! I added a mention of that in SR-3088.

> remove subscripts from UnsafePointer and UnsafeMutablePointer

It's often more clear to perform arithmetic on C array indices rather
than pointers. That said, I'm happy to push developers to use
UnsafeBufferPointer whenever that have a known capacity. To me, this
is a question of whether the benefit of making a dangerous thing less
convenient is worth breaking source compatibility.

-Andy

___
swift-evolution mailing list

Re: [swift-evolution] Yet another fixed-size array spitball session

[Andrew Trick via swift-evolution]

> On Jun 2, 2017, at 2:20 AM, Brent Royal-Gordon via swift-evolution 
>  wrote:
> 
>> On May 28, 2017, at 11:37 PM, Daryle Walker via swift-evolution 
>> > wrote:
>> 
>> Static-Sized Arrays
> 
> My preference would still be to build this from four separate features:
> 
> 1. Magic tuple conformances: We already want to be able to automatically 
> conform tuples to protocols like Equatable, Hashable, and Comparable. These 
> can all be compiler magic; they don't have to be definable in userspace.
> 
> 2. Conform tuples to Collection: The Element type should be the most specific 
> common supertype of the tuple's elements. If all the elements are the same 
> type, it would be that type. The Index and IndexDistance types should be Int.
> 
> 3. Conform same-type tuples to MutableCollection: If all elements are the 
> same type, you can also modify the values. (If their types vary in any way, 
> however, it would not be safe to allow mutations, since you could assign the 
> wrong type to an element.)
> 
> 3. Add sugar for a tuple of N identical elements: Probably something like `4 
> * Int`, but opinions can vary.
> 
> This solution avoids adding another structural type to the language or 
> introducing non-type generic parameters. It also addresses other needs: 1 and 
> 2 are desirable features in their own right which address other use cases in 
> addition to this one. And it's nicely incremental, which is always a plus.
> 
> -- 
> Brent Royal-Gordon
> Architechies

+1. Sounds great to me as long as the type system can handle it cleanly.
-Andy
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Don't require & for UnsafeRawPointer

[Andrew Trick via swift-evolution]

> On May 18, 2017, at 12:38 PM, Anders Kierulf  wrote:
> 
>> And you don’t want to create a temp copy:
>> 
>>  var tupleMemory = letTuple
>>  get(, at: 2)
>> 
>> In this case, the letTuple->UnsafeRawPointer conversion is likely going to 
>> create that copy anyway in order to give the tuple a memory address. A 
>> slightly more compelling example would be:
>> 
>> struct S {
>>  var tuple: (Int, Int, Int, Int, Int, Int)
>> }
>> 
>> func foo(s: S) -> Int {
>>  var tupleMemory = s.tuple // was s.t, should be s.tuple [AK]
>>  return get(, at: 2) // fails: wrong type
>> }
>> 
>> Are you more concerned that the copy won't be optimized away or that you 
>> need the extra line of code?
> 
> My main concern is performance. In my code, the tuple is often 380 words 
> long, so a copy completely kills performance. This part of my code is 
> performance critical, which is why I can’t just use Swift’s standard Array 
> type.


I agree with your proposed language changes, but there’s really a larger issue. 
Until we have move-only types and ‘shared’ argument conventions, I’m afraid 
that passing around a large struct, or even implicitly converting it to a 
pointer creates copies. `let` doesn’t give you a no-copy guarantee. If those 
copies aren’t already optimized at -O, it’s possible the optimizer could be 
improved to handle your cases. However, I don’t think you should rely on that. 
For now, until the ownership work is further along, the best way to avoid 
copies is to embrace mutability. Declare your large value types as `var` and 
always pass them `inout`.

I just noticed this bug, which has shows why the type system can make a copy 
impossible to avoid.
https://bugs.swift.org/browse/SR-4581 
If you don’t care about conforming to UnsafeMutableCollection and implementing 
subscript { get } conformance, maybe you can avoid that problem.

Another option is to use indirect reference-counted storage for your fixed 
array, just like Array, so incidental copies aren’t expensive.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Don't require & for UnsafeRawPointer

[Andrew Trick via swift-evolution]

> On May 18, 2017, at 7:23 AM, Anders Kierulf  wrote:
> 
>> Thanks for refusing to let your pitch die. Something should eventually be 
>> done here and it's good to get feedback. The only reason to bring this up in 
>> Swift 4 is if we decide to outlaw some code pattern that's already in use. 
>> If this ends up just an additive convenience request, than it can be a bug 
>> report for now and come back in Swift 5. There have been a couple +1's 
>> already but I don't know whether it's a serious problem or just an annoyance.
> 
> For me, it’s a serious annoyance. A single wrong `mutating` in an API would 
> be a minor issue, but the problem is that it infects the rest of the code 
> like a plague. It’s bad enough that I’ve delayed work on that project, hoping 
> for a resolution.

Ok. I hear you.

To be clear, the only case that’s really giving you grief is this one right?

  func get(_ pointer: UnsafeRawPointer, at index: Int) -> Int

  _ = get(letTuple, at: 2) // fails: wrong type

And you don’t want to create a temp copy:

  var tupleMemory = letTuple
  get(, at: 2)

In this case, the letTuple->UnsafeRawPointer conversion is likely going to 
create that copy anyway in order to give the tuple a memory address. A slightly 
more compelling example would be:

struct S {
  var tuple: (Int, Int, Int, Int, Int, Int)
}

func foo(s: S) -> Int {
  var tupleMemory = s.t
  return get(, at: 2) // fails: wrong type
}

Are you more concerned that the copy won't be optimized away or that you need 
the extra line of code?

… I forgot to mention. Regarding this line:

  let a = pointer.bindMemory(to: Int.self, capacity: 6)

If the tuple memory is always viewed as (Int, Int...), then you can use 
assumeMemoryBound(to:) and don't need to specify capacity.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Don't require & for UnsafeRawPointer

[Andrew Trick via swift-evolution]

Thanks for refusing to let your pitch die. Something should eventually be done 
here and it's good to get feedback. The only reason to bring this up in Swift 4 
is if we decide to outlaw some code pattern that's already in use. If this ends 
up just an additive convenience request, than it can be a bug report for now 
and come back in Swift 5. There have been a couple +1's already but I don't 
know whether it's a serious problem or just an annoyance. 

Side note: don’t worry, simply declaring everything ‘var’ and passing them 
‘inout’ shouldn’t break exclusive memory access. If you pass ‘inout’ to a 
non-mutable pointer, it’s just considered a “read” access.

I'll speculate a bit and make some suggestions...

> On Apr 20, 2017, at 12:10 PM, Anders Kierulf via swift-evolution 
>  wrote:
> 
> Summary: Currently, only mutable values can be passed to UnsafeRawPointer, 
> except for a special case for arrays. That special case should be 
> generalized, allowing any values to be passed to UnsafeRawPointer without 
> using &.
> 
> The following code shows the inconsistency in passing values to 
> UnsafeRawPointer:
> 
> var varArray = [Int](repeating: 0, count: 6)
> var varTuple = (0, 0, 0, 0, 0, 0)
> 
> let letArray = [Int](repeating: 0, count: 6)
> let letTuple = (0, 0, 0, 0, 0, 0)
> 
> func get(_ pointer: UnsafeRawPointer, at index: Int) -> Int {
>let a = pointer.bindMemory(to: Int.self, capacity: 6)
>return a[index]
> }
> 
> // Array can be passed directly, automatically takes address.
> _ = get(varArray, at: 2) // okay
> _ = get(letArray, at: 2) // okay

This works out because a special implicit conversion is doing the work for us. 
I think the special case is all about lightweight C interop with Swift Array.

Let's not change this.

> // When explicitly taking address, can only pass mutable variables.
> _ = get(, at: 2) // okay, but seems inconsistent
> _ = get(, at: 2) // fails: & not allowed

I'll speculate that the type checker has a special case for `` for two 
reasons:

1. To prevent users from accidentally exposing the Array struct rather than its 
storage as a C pointer. That would be confusing to debug.

2. We need a mutable analog to the non-inout immutable case above, for 
lightweight C interop.

We allow implicit UnsafeMutablePointer to UnsafePointer, so, I don't think 
there's anything inconsistent about `` here.

I also think it's fine that we don't allow ``.

Let's not change this.

> // Passing tuple instead of array fails.
> _ = get(varTuple, at: 2) // fails: wrong type
> _ = get(letTuple, at: 2) // fails: wrong type

Refusing to compile these cases would probably seem normal if we didn't support 
the first non-inout array case.

I think we should allow both cases as a new, additive type system feature.

> // Adding & to pass a tuple only works for mutable values. Having to
> // pass the address using & means that any methods calling this must
> // be mutating, even though they don't mutate.
> _ = get(, at: 2) // okay, but forces mutating
> _ = get(, at: 2) // fails: cannot pass immutable value

Well, the `` case only works by accident for raw pointers, but typed 
pointers will have the wrong type!

See [SR-3590] Implicitly convert  to UnsafePointer.

The only reason I haven't pushed for this in Swift 4, along with several other 
important usability issues with UnsafePointer, is that (I thought) it's 
additive and I haven't wanted to compete for review bandwidth in this release 
cycle.

However, come to think of it, this will break some very unlikely code:

func foo(_ p: UnsafeMutablePointer<(Int, Int, Int)>) {
  p[0].0 = 42
}

var a = (0, 1, 2)
foo()
print(a)

I think that's fair if migration is provided.

> Passing a value to an UnsafeRawPointer parameter should not require use of &, 
> as that forces all code calling it to be mutating. Having a special case for 
> array also doesn't make sense, as the idea of UnsafeRawPointer is that we're 
> interpreting that memory content as whatever we want. Logged as SR-4649.

Splitting hairs: the array case is really a special case and it does make 
sense. I'm *also* proposing a similar special case for homogeneous tuples.

> Proposal:
> - Never require & when passing value to UnsafeRawPointer.
> - Always require & when passing value to UnsafeMutableRawPointer.

+1 to your basic proposal. But I think each type system change needs to be more 
clearly spelled out. Probably with a separate bug for each.

- Support non-inout homogeneous tuple argument conversion to 
UnsafePointer.
(additive)

- Support non-homgenous non-inout tuple argument conversion to UnsafeRawPointer.
(additive)

- Support inout homogeneous tuple argument conversion to 
UnsafeMutablePointer.
See [SR-3590] Implicitly convert  to UnsafePointer.
(source breaking in ridiculous cases)

- [SR-1956] `withUnsafePointer` shouldn't take its argument as `inout`
(additive... we don't need to ban the inout syntax)

The problem I have with [SR-4649] "Don't 

Re: [swift-evolution] Proposed amendment to SE-0138: Normalize UnsafeRawBufferPointer Slices

[Andrew Trick via swift-evolution]

> On Mar 23, 2017, at 7:34 PM, Karl Wagner <razie...@gmail.com> wrote:
> 
> Oh, one more thing (about this specific change): If we do this, we should add 
> an “offset” parameter to UnsafeMutableBufferPointer.[move]initialize/assign 
> (with a default of 0 for source compatibility). Otherwise, it becomes awkward 
> to initialise a region of a buffer from another buffer.
> 
> What I want to write:
> buffer.suffix(from: filled).initialize(from: newData)
> 
> If SubSequence is not another unsafe pointer, I’d have to do this:
> buffer.baseAddress!.advanced(by: filled).initialize(from: newData)  // 
> Warning: deprecated in Swift 4.0

Oh no. Don't do that. You're supposed to do:

UnsafeMutableRawBufferPointer(rebasing: buffer.suffix(from: 
filled)).initialize(from: newData)

I don't blame you for not wanting to write out the type name.

> 
> So instead, we should have the ability to write this:
> buffer.initialize(startingAt: filled, from: newData) // awkward labels, but 
> src compat…

I agree that's worth doing as a result of changing the slice type. You
shouldn't need to "rebase" the buffer each time you want to initialize
a region. My concern is that there are a handful of APIs to fix
(initialize, initializeMemory, copyBytes, load). We should handle them
all in one proposal, and we may want time to bikeshed the API. I'm
unconvinced this is worth pursuing for Swift 4. Can you file a bug?

> 
> But yeah, this post just reminded me that there are a number of small 
> consistency tweaks we could make to the unsafe-buffer API.

Yeah, but these are convenience issues, not correctness issues.

I think these are the ones that are important, obvious, and trivial
enough that might make sense in Swift 4.

** UnsafeBufferPointer should have init from mutable
https://bugs.swift.org/browse/SR-3929

** UnsafeMutableBufferPointer doesn't have an allocating init
https://bugs.swift.org/browse/SR-3088

** UnsafeBufferPointer needs a withMemoryRebound method
https://bugs.swift.org/browse/SR-4340

** Implicit Conversion:  to UnsafePointer
https://bugs.swift.org/browse/SR-3590

-Andy

> 
> - Karl
> 
>> On 24 Mar 2017, at 03:22, Karl Wagner <karl.sw...@springsup.com 
>> <mailto:karl.sw...@springsup.com>> wrote:
>> 
>> The convenience initialiser should exist on all of the unsafe buffers, not 
>> just the raw (untyped) ones.
>> 
>> I’ve run in to this problem a few times, and I think it would get worse if 
>> we adopted a ContiguouslyStored protocol to formalise accessing the 
>> raw-pointers of generic collections. It would mean that you couldn’t write 
>> code that works with UnsafeRawBufferPointer/Data/DispatchData generically, 
>> or with UnsafeBufferPointer/Array.
>> 
>> Also, there seem to be some implicit conversions for the unsafe-pointer 
>> types, but UMBP -> UBP requires an awkward initialiser. We should introduce 
>> an implicit conversion for that case or add an “immutable” computed property 
>> to UMBP.
>> 
>> And while we’re on the subject, memory allocation/deallocation functions are 
>> weirdly dispersed. In order to allocate an UnsafeMutableBufferPointer, 
>> for instance, you have to do:
>> 
>> var buffer: UnsafeMutableBufferPointer
>> init(length: Int) {
>>   let b  = UnsafeMutablePointer.allocate(capacity: length)
>>   buffer = UnsafeMutableBufferPointer(start: b, count: length)
>> }
>> 
>> Also, the deallocate API feels weird - since it deallocates n items from the 
>> head of the pointer, it is a consuming operation and I feel like it should 
>> return a new pointer (with @discardableResult). Once you’ve deallocated a 
>> memory address, you can never re-allocate that specific location so there is 
>> no reason to know about it any more.
>> 
>> - Karl
>> 
>>> On 21 Mar 2017, at 03:21, Andrew Trick via swift-evolution 
>>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>>> 
>>> This proposal amends SE-0138: Normalize UnsafeRawBufferPointer Slices
>>> to fix a design bug: https://github.com/apple/swift-evolution/pull/651 
>>> <https://github.com/apple/swift-evolution/pull/651>
>>> 
>>> The issue was discussed on swift-evolution in Nov/Dec:
>>> See [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers
>>> https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html
>>>  
>>> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html>
>>> 
>>> The implementation of this fix is in PR #8222:
>>> https://github.com/apple/swift/pull/8222 
>>> &l

Re: [swift-evolution] Proposed amendment to SE-0138: Normalize UnsafeRawBufferPointer Slices

[Andrew Trick via swift-evolution]

This is all valuable feedback. I also have a bunch of convenience APIs in
mind but haven't been pushing them yet because:

- the Swift 4 schedule is tight

- I don't want to speculatively add API surface until enough users
  have had experience with the feature. Maybe there are better ideas.
…

> On Mar 23, 2017, at 7:22 PM, Karl Wagner <razie...@gmail.com> wrote:
> 
> The convenience initialiser should exist on all of the unsafe buffers, not 
> just the raw (untyped) ones.

I think it's fair to add the UnsafeBuffer.init(rebasing:) initializer
as part of this SE-0138 amendment. It's not so much a new API as a
consistency fix. I’ll update the current proposal.

> I’ve run in to this problem a few times, and I think it would get worse if we 
> adopted a ContiguouslyStored protocol to formalise accessing the raw-pointers 
> of generic collections. It would mean that you couldn’t write code that works 
> with UnsafeRawBufferPointer/Data/DispatchData generically, or with 
> UnsafeBufferPointer/Array.
> 
> Also, there seem to be some implicit conversions for the unsafe-pointer 
> types, but UMBP -> UBP requires an awkward initialiser. We should introduce 
> an implicit conversion for that case or add an “immutable” computed property 
> to UMBP.

UnsafeBufferPointer should have init from mutable. We already had a bug for 
that.
Thanks for reminding me: https://bugs.swift.org/browse/SR-3929

I think that needs a new proposal, but it's a trivial addition. I can work on 
that.

I agree that implicit conversions would be nice but does merit some
discussion and involves a bit of type system work. You have to be a
bit careful with implicit conversion because of potentially ambigous
overloads. Implicit conversion is primarily for C interop. Eventually
I do think it would be nice to optionally import some pointer+length
arguments as UnsafeBuffer. We just don't do it now. Regardless of
interop, this would also be handy way to pass an inout argument as a
buffer without wrapping it in a closure.

Your specific concern seems to be about the mutable -> immutable
conversion being implicit though. But that just comes down to writing out
the type name. As bad as the type name is, to me that's not worth an
evolution proposal at the moment. I suggest filing a bug for any
implicit conversion issues, or `immutable` property suggestions for now.

> And while we’re on the subject, memory allocation/deallocation functions are 
> weirdly dispersed. In order to allocate an UnsafeMutableBufferPointer, for 
> instance, you have to do:
> 
> var buffer: UnsafeMutableBufferPointer
> init(length: Int) {
>   let b  = UnsafeMutablePointer.allocate(capacity: length)
>   buffer = UnsafeMutableBufferPointer(start: b, count: length)
> }

We've had a bug open on this since introducing raw buffers:
https://bugs.swift.org/browse/SR-3088

This is obvious enough that I can probably sneak it into Swift 4 if I just get 
around to writing the proposal.

> Also, the deallocate API feels weird - since it deallocates n items from the 
> head of the pointer, it is a consuming operation and I feel like it should 
> return a new pointer (with @discardableResult). Once you’ve deallocated a 
> memory address, you can never re-allocate that specific location so there is 
> no reason to know about it any more.

Now we're outside UnsafeBufferPointer territory.
UnsafePointer.deallocate(capacity:) needs to be passed the
same capacity as allocate. If that's confusing, please file a
documentation bug.

Or you may be referring to UnsafePointer.deinitialize(count:). That
returns a raw pointer to the same memory address to indicate that you
can now initialize the same memory as a different type.

With move-only types, I think we'll be able to revisit some of the
pointer initialize/deinitialize API to be more robust.

-Andy

> 
> - Karl
> 
>> On 21 Mar 2017, at 03:21, Andrew Trick via swift-evolution 
>> <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>> 
>> This proposal amends SE-0138: Normalize UnsafeRawBufferPointer Slices
>> to fix a design bug: https://github.com/apple/swift-evolution/pull/651 
>> <https://github.com/apple/swift-evolution/pull/651>
>> 
>> The issue was discussed on swift-evolution in Nov/Dec:
>> See [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers
>> https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html
>>  
>> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html>
>> 
>> The implementation of this fix is in PR #8222:
>> https://github.com/apple/swift/pull/8222
>> 
>> Fix: Change Unsafe[Mutable]RawBufferPointer's SubSequence type
>> 
>> Original: Unsafe[Mutable]RawBufferPointer.SubSequence = 
>> Unsafe[

[swift-evolution] Proposed amendment to SE-0138: Normalize UnsafeRawBufferPointer Slices

[Andrew Trick via swift-evolution]

This proposal amends SE-0138: Normalize UnsafeRawBufferPointer Slices
to fix a design bug: https://github.com/apple/swift-evolution/pull/651

The issue was discussed on swift-evolution in Nov/Dec:
See [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html

The implementation of this fix is in PR #8222:
https://github.com/apple/swift/pull/8222

Fix: Change Unsafe[Mutable]RawBufferPointer's SubSequence type

Original: Unsafe[Mutable]RawBufferPointer.SubSequence = 
Unsafe[Mutable]RawBufferPointer

Fixed: Unsafe[Mutable]RawBufferPointer.SubSequence = 
[Mutable]RandomAccessSlice

This is a source breaking bug fix that only applies to
post-3.0.1. It's extremely unlikely that any Swift 3 code would rely
on the SubSequence type beyond the simple use case of passing a
raw buffer subrange to an another raw buffer argument:

`takesRawBuffer(buffer[i..

Re: [swift-evolution] [Proposal] Change (Dispatch)Data.withUnsafeBytes to use UnsafeMutableBufferPointer

[Andrew Trick via swift-evolution]

> On Dec 27, 2016, at 12:16 AM, Karl via swift-evolution 
>  wrote:
> 
> Looking for feedback before submitting a PR: 
> https://github.com/karwa/swift-evolution/blob/corelibs-unsafebytes/proposals/-corelibs-unsafebytes.md
>  
> 
> 
> —
> 
> Change (Dispatch)Data.withUnsafeBytes to use UnsafeMutableBufferPointer
> 
> Proposal: SE- 
> 
> Authors: Karl Wagner 
> Review Manager: TBD
> Status: Awaiting review
> During the review process, add the following fields as needed:
> 
> Decision Notes: Rationale 
> , Additional Commentary 
> 
> Bugs: SR- , SR- 
> 
> Previous Revision: 1 
> 
> Previous Proposal: SE- 
> 
>  
> Introduction
> 
> The standard library's Array and ContiguousArray types expose the method 
> withUnsafeBytes, which allows you to view their contents as a contiguous 
> collection of bytes. The core libraries Foundation and Dispatch contain types 
> which wrap some allocated data, but their withUnsafeBytes method only allows 
> you to view the contents as a pointer to a contiguous memory location of a 
> given type.
> 
> Swift-evolution thread: Discussion thread topic for that proposal 
> 
>  
> Motivation
> 
> The current situation makes it awkward to write generic code. Personally, I 
> use the following extension in my projects to sort the naming confusion out:
> 
> protocol ContiguousByteCollection {
>   func withUnsafeBytes(_ body: (UnsafeRawBufferPointer) throws -> T) 
> rethrows -> T
> }
> 
> // stdlib types are fine.
> extension Array: ContiguousByteCollection {}
> extension ArraySlice: ContiguousByteCollection {}
> extension ContiguousArray: ContiguousByteCollection {}
> 
> // corelibs types give us a pointer, should be: { pointer, count }
> #if canImport(Dispatch)
>   import Dispatch
> 
>   extension DispatchData : ContiguousByteCollection {
> func withUnsafeBytes(_ body: (UnsafeRawBufferPointer) throws -> T) 
> rethrows -> T {
>   return try withUnsafeBytes { try body(UnsafeRawBufferPointer(start: $0, 
> count: count)) }
> }
>   }
> #endif
> 
> #if canImport(Foundation)
>   import Foundation
> 
>   extension Data : ContiguousByteCollection {
> func withUnsafeBytes(_ body: (UnsafeRawBufferPointer) throws -> T) 
> rethrows -> T {
>   return try withUnsafeBytes { try body(UnsafeRawBufferPointer(start: $0, 
> count: count)) }
> }
>   }
> #endif
> Conceptually, the corelibs types are untyped regions of memory, and it would 
> make sense for them to adopt the UnsafeRawBufferPointer model.
> 
>  
> Proposed
>  solution
> 
> The proposed solution would be to deprecate the current methods on 
> (Dispatch)Data (with 2 generic parameters), and replace them with methods 
> with identical signatures to Array (with 1 generic parameter).
> 
> To be deprecated:
> 
> public func withUnsafeBytes(_ body: 
> (UnsafePointer) throws -> ResultType) rethrows -> ResultType
> Replaced with:
> 
> public func withUnsafeBytes(_ body: (UnsafeRawBufferPointer) throws -> R) 
> rethrows -> R
>  
> 
Thanks Karl. Good observation.

I proposed exactly this API along with a few other UnsafeRawBufferPointer 
compatibility API’s during SE-0183. Look for Tony to follow up on this.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0147: Move UnsafeMutablePointer.initialize(from:) to UnsafeMutableBufferPointer

[Andrew Trick via swift-evolution]

> On Dec 12, 2016, at 6:30 PM, Ben Cohen  wrote:
> 
> 
>> On Dec 12, 2016, at 1:32 PM, Andrew Trick > > wrote:
>> 
>>> 
>>> On Dec 7, 2016, at 10:07 PM, Douglas Gregor via swift-evolution 
>>> > wrote:
>>> 
>>> Hello Swift community,
>>> 
>>> The review of SE-0147 "Move UnsafeMutablePointer.initialize(from:) to 
>>> UnsafeMutableBufferPointer" begins now and runs through December 12, 2016. 
>>> The proposal is available here:
>>> 
>>> https://github.com/apple/swift-evolution/blob/master/proposals/0147-move-unsafe-initialize-from.md
>>>  
>>> 
>>> Reviews are an important part of the Swift evolution process. All reviews 
>>> should be sent to the swift-evolution mailing list at
>>> 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>>> or, if you would like to keep your feedback private, directly to the review 
>>> manager. When replying, please try to keep the proposal link at the top of 
>>> the message:
>>> 
>>> Proposal link:
>>> 
>>> https://github.com/apple/swift-evolution/blob/master/proposals/0147-move-unsafe-initialize-from.md
>>>  
>>> 
>> For UnsafeMutableRawBufferPointer.initializeMemory:
>> 
>> We have this doc comment:
>> 
>>   /// Returns an iterator to any elements of `source` that didn't fit in the 
>>   /// buffer, and an index into the buffer one past the last byte written.
>> 
>> Which is consistent with the PR https://github.com/apple/swift/pull/5718 
>> 
>> 
>> + public func initializeMemory(
>> +as: S.Iterator.Element.Type, from source: S
>> +  ) -> (unwritten: S.Iterator, initializedUpTo: Index) {
>> 
>> However, the proposal reads:
>> 
>> public func initializeMemory(
>>  as: S.Iterator.Element.Type, from source: S
>>   ) -> (unwritten: S.Iterator, initialized: 
>> UnsafeMutableBufferPointer)
>> 
>> Which API did we end up deciding on?
>> 
>> (unwritten:, initialized:) makes sense to me, but I can’t remember if we 
>> ditched that approach for some reason (e.g. consistency with 
>> UnsafeMutableBufferPointer).
>> 
>> -Andy
> 
> The proposed API in the proposal is right, the comment above it should have 
> been updated. The intention is for the raw buffer version to return an 
> initialized typed buffer not an index into the raw buffer. While this means 
> it’s inconsistent with the typed equivalent, an index into the raw buffer 
> isn’t all that useful as usually you want to operate on typed values after 
> you’ve initialized them, and the calculation to get to the raw buffer index 
> from the typed buffer if you need it is easy i.e. initialized.count * the 
> stride of the type.
> 
> (I haven’t yet updated the PR to reflect this change yet either, was waiting 
> to see if there was more feedback first to incorporate)

That’s good. Those are the reasons the proposed API makes more sense to me too.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers

[Andrew Trick via swift-evolution]

> On Dec 9, 2016, at 11:50 AM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Fri Dec 09 2016, Andrew Trick  wrote:
> 
>>> On Dec 9, 2016, at 10:27 AM, Dave Abrahams via swift-evolution
>>  wrote:
>>> 
>>> 
>>> on Thu Dec 08 2016, Xiaodi Wu >> > wrote:
>>> 
>> 
 On Thu, Dec 8, 2016 at 6:53 PM, Ben Cohen via swift-evolution <
 swift-evolution@swift.org> wrote:
 
> 
> On Dec 8, 2016, at 4:35 PM, Jordan Rose via swift-evolution <
> swift-evolution@swift.org> wrote:
> 
> Um, Sequence doesn’t have a subscript (or indexes). Sequences are
> single-pass. So if this is important, it needs to stay a Collection.
> 
> 
> Just because something fulfills one of the requirements of a Collection
> does not mean it should be one. It needs to tick all the boxes before its
> allowed to be elevated.
> 
> But it’s still allowed to have subscripts (UnsafePointer has subscripting
> but isn’t a collection) or be multi-pass (strides are multiples but are
> only sequences). That’s OK
> 
> In this case, yes it’s multi-pass, yes it has a subscript, but no it isn’t
> a collection because it doesn’t meet the requirements for slicing i.e. 
> that
> indices of the slice be indices of the parent.
> (relatedly… it appears this requirement is documented on the concrete
> Slice type rather than on Collection… which is a documentation bug we
> should fix).
> 
 
 If this is indeed a requirement for Collection, then my vote would be for
 Nate's option #1 and Andy's option #2, to give UnsafeRawBufferPointer a
 Slice type that fulfills the requirement. It's the smallest change,
 preserves the use of integer indices, and preserves what Andy stated as the
 desired use case of making it easy for users to switch out code written for
 [UInt8].
 
 I'm not sure I fully understand yet why Dave finds the idea of Collection
 conformance fishy, 
>>> 
>>> Because the memory can easily be already bound to another type than
>>> UInt8, and there's no obvious reason why UInt8 should be privileged as a
>>> type you can get out of a raw buffer without binding the memory.
>> 
>> I strongly disagree with that statement. The overwhelmingly common use
>> case for raw buffers is to view them as a sequence of UInt8 *without*
>> binding the type.  Generally, at the point that you're dealing with a
>> raw buffer it's impossible to (re)bind memory because you don't know
>> what type it holds. 
> 
> Oh, you can't just rebind to UInt8 because that's not defined as
> universally compatible with all data.  OK, sorry.
> 
>> The reason it's so important to have an UnsafeRawBufferPointer data
>> type is precisely so that users don't need mess about with binding
>> memory. It's easy to get that wrong even when it's possible.
>> 
>> The only reason that UInt8 is special is that when users create
>> temporary typed buffers for bytes (e.g. they sometimes want a growable
>> array or just don't want to bother with manual allocation) they always
>> use UInt8 as the element type.
>> 
>> That said, we could easily divide these concerns into two types as
>> you suggested. A raw buffer, which doesn't have any special UInt8
>> features, and a RawBytes collection that handles both buffer slicing
>> and UInt8 interoperability.
> 
> But, now that I think of it, that wouldn't really solve any problems,
> would it?


A new Collection type doesn't solve any practical problems. It does solve a
conceptual problem if you think that a raw buffer is not *inherently* a
collection of bytes. There is an elegance in separating the raw buffer
semantics from the byte collection semantics, but that elegance does
not simplify anything for users--it's just more abstraction to figure
out. Certainly, the most straightforward way to fix this is to simply
change raw buffer's slice type, so I'm inclined to favor that
approach. Creating a new collection type would involve
rethinking/redesigning some of the related APIs.

Also note that I'm leaning toward slice -> buffer conversion via an
unlabeled initializer because I think it's the most obvious with least
API surface.

I don't think we absolutely need a new proposal for this easy fix,
since it's not really introducing a new API. The additional
initializer merely allows code that used to work to be migrated via a
fixit.

Here's a quick summary. If there aren't any strong objections, I'll
post an ammendment to the original proposal along with a PR for more
formal review.

Proposed ammendment to SE-0138:


Fix: Change Unsafe${Mutable}RawBufferPointer's SubSequnce type

Original: Unsafe${Mutable}RawBufferPointer.SubSequence = 
Unsafe${Mutable}RawBufferPointer

Fixed: 

Re: [swift-evolution] [Review] SE-0147: Move UnsafeMutablePointer.initialize(from:) to UnsafeMutableBufferPointer

[Andrew Trick via swift-evolution]

> On Dec 7, 2016, at 10:07 PM, Douglas Gregor via swift-evolution 
>  wrote:
> 
> Hello Swift community,
> 
> The review of SE-0147 "Move UnsafeMutablePointer.initialize(from:) to 
> UnsafeMutableBufferPointer" begins now and runs through December 12, 2016. 
> The proposal is available here:
> 
> https://github.com/apple/swift-evolution/blob/master/proposals/0147-move-unsafe-initialize-from.md
>  
> 
> Reviews are an important part of the Swift evolution process. All reviews 
> should be sent to the swift-evolution mailing list at
> 
> https://lists.swift.org/mailman/listinfo/swift-evolution 
> 
> or, if you would like to keep your feedback private, directly to the review 
> manager. When replying, please try to keep the proposal link at the top of 
> the message:
> 
> Proposal link:
> 
> https://github.com/apple/swift-evolution/blob/master/proposals/0147-move-unsafe-initialize-from.md
>  
> 
For UnsafeMutableRawBufferPointer.initializeMemory:

We have this doc comment:

  /// Returns an iterator to any elements of `source` that didn't fit in the 
  /// buffer, and an index into the buffer one past the last byte written.

Which is consistent with the PR https://github.com/apple/swift/pull/5718

+ public func initializeMemory(
+as: S.Iterator.Element.Type, from source: S
+  ) -> (unwritten: S.Iterator, initializedUpTo: Index) {

However, the proposal reads:

public func initializeMemory(
 as: S.Iterator.Element.Type, from source: S
  ) -> (unwritten: S.Iterator, initialized: 
UnsafeMutableBufferPointer)

Which API did we end up deciding on?

(unwritten:, initialized:) makes sense to me, but I can’t remember if we 
ditched that approach for some reason (e.g. consistency with 
UnsafeMutableBufferPointer).

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers

[Andrew Trick via swift-evolution]

> On Dec 9, 2016, at 10:27 AM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Thu Dec 08 2016, Xiaodi Wu  > wrote:
> 
>> On Thu, Dec 8, 2016 at 6:53 PM, Ben Cohen via swift-evolution <
>> swift-evolution@swift.org> wrote:
>> 
>>> 
>>> On Dec 8, 2016, at 4:35 PM, Jordan Rose via swift-evolution <
>>> swift-evolution@swift.org> wrote:
>>> 
>>> Um, Sequence doesn’t have a subscript (or indexes). Sequences are
>>> single-pass. So if this is important, it needs to stay a Collection.
>>> 
>>> 
>>> Just because something fulfills one of the requirements of a Collection
>>> does not mean it should be one. It needs to tick all the boxes before its
>>> allowed to be elevated.
>>> 
>>> But it’s still allowed to have subscripts (UnsafePointer has subscripting
>>> but isn’t a collection) or be multi-pass (strides are multiples but are
>>> only sequences). That’s OK
>>> 
>>> In this case, yes it’s multi-pass, yes it has a subscript, but no it isn’t
>>> a collection because it doesn’t meet the requirements for slicing i.e. that
>>> indices of the slice be indices of the parent.
>>> (relatedly… it appears this requirement is documented on the concrete
>>> Slice type rather than on Collection… which is a documentation bug we
>>> should fix).
>>> 
>> 
>> If this is indeed a requirement for Collection, then my vote would be for
>> Nate's option #1 and Andy's option #2, to give UnsafeRawBufferPointer a
>> Slice type that fulfills the requirement. It's the smallest change,
>> preserves the use of integer indices, and preserves what Andy stated as the
>> desired use case of making it easy for users to switch out code written for
>> [UInt8].
>> 
>> I'm not sure I fully understand yet why Dave finds the idea of Collection
>> conformance fishy, 
> 
> Because the memory can easily be already bound to another type than
> UInt8, and there's no obvious reason why UInt8 should be privileged as a
> type you can get out of a raw buffer without binding the memory.

I strongly disagree with that statement. The overwhelmingly common use
case for raw buffers is to view them as a sequence of UInt8 *without*
binding the type.  Generally, at the point that you're dealing with a
raw buffer it's impossible to (re)bind memory because you don't know
what type it holds. The reason it's so important to have an
UnsafeRawBufferPointer data type is precisely so that users don't need
mess about with binding memory. It's easy to get that wrong even when
it's possible.

The only reason that UInt8 is special is that when users create
temporary typed buffers for bytes (e.g. they sometimes want a growable
array or just don't want to bother with manual allocation) they always
use UInt8 as the element type.

That said, we could easily divide these concerns into two types as
you suggested. A raw buffer, which doesn't have any special UInt8
features, and a RawBytes collection that handles both buffer slicing
and UInt8 interoperability.

-Andy

> 
>> but I'm comfortable with a type that's clearly labeled as unsafe not
>> being fully footgun-proof.
>> 
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>>> 
>>> 
> 
> -- 
> -Dave
> ___
> swift-evolution mailing list
> swift-evolution@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-evolution 
> 
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers

[Andrew Trick via swift-evolution]

> On Dec 9, 2016, at 10:22 AM, Dave Abrahams <dabrah...@apple.com> wrote:
> 
> 
> on Thu Dec 08 2016, Jordan Rose  wrote:
> 
>>> On Dec 8, 2016, at 16:22, Andrew Trick via swift-evolution 
>>> <swift-evolution@swift.org> wrote:
>>> In practice, it needs to be able to interoperate with [UInt8] and be 
>>> interchangeable in the same
>> generic context.
>>> e.g. `byteBuffer += rawBuffer[payloadIndex..<endIndex]` is typical.
>>> I think Sequence is sufficient for that purpose.
>> 
>> Um, Sequence doesn’t have a subscript (or indexes). Sequences are 
>> single-pass. So if this is
>> important, it needs to stay a Collection.
> 
> Yes.  But I don't see why the raw buffer should be a collection in the first 
> place.  Why not
> 
> byteBuffer += rawBuffer.bytes[payloadIndex..<endIndex]
> 
> ?
> 
> -- 
> -Dave

`RawBytes : Collection` could be a view over the raw buffer with `SubSequence : 
RandomAccessSlice` if that's what you mean.

I hadn’t considered that just because it’s yet another type that needs to be 
introduced. Now interface authors need to decide whether they want to take a 
collection of bytes or a buffer. (In a non-generic context, the correct answer 
is to pass the buffer, not the collection--the collection would be just a 
convenient temporary view).

We still need an initializer or extension to handle the expected use case of 
nested buffers:

extension RandomAccessSlice where Base == RawBytes {
var rebased: UnsafeRawBufferPointer {
return UnsafeRawBufferPointer(start: base.baseAddress, count: count)
}
}

I would even be willing to eliminate raw buffer subscripting altogether, which 
I think is what you're getting at, since this isn't too awful:

`buffer.bytes[i]`

Pro: More explicit division between raw memory semantics and Collection 
semantics.

Con: Raw buffer users need to juggle two different types and know how convert 
between them.

-Andy
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers

[Andrew Trick via swift-evolution]

> On Dec 8, 2016, at 5:44 PM, Xiaodi Wu via swift-evolution 
>  wrote:
> 
> On Thu, Dec 8, 2016 at 6:53 PM, Ben Cohen via swift-evolution 
> > wrote:
> 
>> On Dec 8, 2016, at 4:35 PM, Jordan Rose via swift-evolution 
>> > wrote:
>> 
>> Um, Sequence doesn’t have a subscript (or indexes). Sequences are 
>> single-pass. So if this is important, it needs to stay a Collection.
>> 
> 
> Just because something fulfills one of the requirements of a Collection does 
> not mean it should be one. It needs to tick all the boxes before its allowed 
> to be elevated.
> 
> But it’s still allowed to have subscripts (UnsafePointer has subscripting but 
> isn’t a collection) or be multi-pass (strides are multiples but are only 
> sequences). That’s OK
> 
> In this case, yes it’s multi-pass, yes it has a subscript, but no it isn’t a 
> collection because it doesn’t meet the requirements for slicing i.e. that 
> indices of the slice be indices of the parent.
> (relatedly… it appears this requirement is documented on the concrete Slice 
> type rather than on Collection… which is a documentation bug we should fix).
> 
> If this is indeed a requirement for Collection, then my vote would be for 
> Nate's option #1 and Andy's option #2, to give UnsafeRawBufferPointer a Slice 
> type that fulfills the requirement. It's the smallest change, preserves the 
> use of integer indices, and preserves what Andy stated as the desired use 
> case of making it easy for users to switch out code written for [UInt8].

Ok, but there needs to be an easy way in a nongeneric context to convert from a 
Slice into an URBP (with normalized byte offsets).

Does anyone object to adding an initializer for this? Any suggestions on 
naming? Do we need an argument label? etc?

UnsafeRawBufferPointer(_ : Slice)

as in:

let region = UnsafeRawBufferPointer(buffer[i..

Re: [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers

[Andrew Trick via swift-evolution]

> On Dec 8, 2016, at 4:54 PM, Jordan Rose  wrote:
> 
> 
>> On Dec 8, 2016, at 16:53, Ben Cohen > > wrote:
>> 
>> 
>>> On Dec 8, 2016, at 4:35 PM, Jordan Rose via swift-evolution 
>>> > wrote:
>>> 
>>> Um, Sequence doesn’t have a subscript (or indexes). Sequences are 
>>> single-pass. So if this is important, it needs to stay a Collection.
>>> 
>> 
>> Just because something fulfills one of the requirements of a Collection does 
>> not mean it should be one. It needs to tick all the boxes before its allowed 
>> to be elevated.
>> 
>> But it’s still allowed to have subscripts (UnsafePointer has subscripting 
>> but isn’t a collection) or be multi-pass (strides are multiples but are only 
>> sequences). That’s OK
>> 
>> In this case, yes it’s multi-pass, yes it has a subscript, but no it isn’t a 
>> collection because it doesn’t meet the requirements for slicing i.e. that 
>> indices of the slice be indices of the parent.
>> (relatedly… it appears this requirement is documented on the concrete Slice 
>> type rather than on Collection… which is a documentation bug we should fix).
> 
> Ah, right, thank you. Retracted.

Let me restate, because I think Jordan's question was valid given my statement.

It would be *nice* for raw buffers to be Collection because they’re 
meant to be a replacement for code that is typically written for [UInt8], and 
anything you can do with an array that applies to raw buffers is covered by 
Collection.

However, I don’t expect the raw buffer to be used in a generic context except 
being passed to utilities that copy the bytes out. That will either be done by 
directly iterating over the collection or invoking some other API that could 
take a Sequence. The most important is probably Array.append(contentsOf:), 
which is moving over to Sequence. However, we would also need to change 
UnsafeRawBufferPointer(copyBytes:), NSData(replaceSubrange:), and whatever else 
I haven't thought of. That's a small disadvantage to this solution.

I'm also a little concerned that Sequence is immutable, so generic code has no 
way to copy bytes into the buffer.

My bigger concern is still that the range subscript’s inconsistent behavior may 
still lead to bugs in practice in nongeneric code.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers

[Andrew Trick via swift-evolution]

> On Dec 8, 2016, at 12:50 PM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Thu Dec 08 2016, Ben Cohen  wrote:
> 
>>> On Dec 2, 2016, at 8:27 PM, Nate Cook  wrote:
>>> 
 On Dec 2, 2016, at 2:12 PM, Ben Cohen via swift-evolution
 >
 wrote:
>> 
 
> On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution
> >
> wrote:
> 
> 3) Make all buffer pointers their own slices but use a different
> index type. If the indices were just wrapped pointers, that would
> handle the index sharing without needing an additional property on
> the buffer. We could also maintain integer-based stridable
> conformance (which greatly simplifies index arithmetic), since the
> indices would just offset by a byte for raw buffers or a stride
> for typed buffers.
> 
 
 Unfortunately, switching to non-integer indices would change this
 from being mildly source-breaking to being extremely
 source-breaking, as there’s lots of code out there using buffers
 today indexing them with integers (including integer literals).
 
 The big win with UnsafeBufferPointer having an integer index is
 it’s a drop-in replacement for arrays, so when you hit a
 performance problem using an array you can quickly switch to using
 a buffer under most circumstances instead without having to change
 much of your code – including code that uses for i in
 0..>> 
>>> It is definitely very source-breaking, though with relatively simple fixits:
>>> 
>>> buf[0] ---> buf[buf.startIndex]
>>> buf[3] ---> buf[buf.startIndex + 3]
>>> buf[i] ---> buf[buf.startIndex + i]
>>> 
>>> Any integer arithmetic happening outside the subscript could be left
>>> unchanged. If that cost isn't worth the benefit, then making
>>> UnsafeRawBufferPointer use Slice as its slice type is probably the
>>> best way to resolve that issue.
>>> 
>>> Nate
>> 
>> The fixits aren’t quite that simple for slices, though:
>> 
>>  let slice = buf[3..<6]
>>  slice[3] —> slice[slice.startIndex + 0] // fixit would somehow need to 
>> know this is 0 not 3
>>  slice[i] —> slice[slice.startIndex + ??] // or even need to
>> know this is, erm, I haven’t had enough coffee this morning
>> 
>> The other downside is it would thwart speculatively switching an Array
>> to an UnsafeBuffer to see if that was a bottleneck, then switching
>> back.
>> 
>>> On Dec 1, 2016, at 11:33 PM, Nate Cook via swift-evolution 
>>>  wrote:
>>> 
>>> 1) Switch to using Slice as a wrapper for UnsafeRawBufferPointer.
>>> 
>> 
>> Based on the above, it seems like this is the least bad option, and we
>> need to do this ASAP as currently UnsafeRawBufferPointer is
>> non-compliant with the requirements of slicing and needs changing
>> before it’s more widely adopted.
> 
> Or we could say that UnsafeRawBufferPointer isn't a Collection.  Making
> it a Collection in the first place has always seemed suspect to me.
> 
> -- 
> -Dave

UnsafeRawBufferPointer does not need to be a Collection, but should at least be 
a Sequence. It is a Collection now simply because it fits the criteria 
(nondestructively accessed and subscriptable).

In practice, it needs to be able to interoperate with [UInt8] and be 
interchangeable in the same generic context.

e.g. `byteBuffer += rawBuffer[payloadIndex.. out-of-bounds

To some extent, this ship has sailed. I can see a few of options now:

1. We make UnsafeRawBufferPointer a Sequence and just live with that 
inconsistency. Users need discern the importance of range subscript's 

Re: [swift-evolution] [Pitch] Move UnsafeMutablePointer.initialize(from:) to UnsafeMutableBufferPointer

[Andrew Trick via swift-evolution]

> On Nov 30, 2016, at 4:48 PM, Ben Cohen via swift-evolution 
>  wrote:
> 
> 
> Hi swift-evolution,
> 
> Below is a draft proposal for a change to facilitate fixing some memory 
> safety issues in the standard library.
> 
> If you’re interested in the implementation, PRs can be found here: [stdlib] 
> [WIP] Eliminate version of Array.append(contentsOf:) that takes a Collection 
>  and here: [stdlib] [WIP] Add 
> UnsafeRawBufferPointer.initialize(as:from:) 
> 
> (they need a bit of performance work before they’re ready to land)
> 
> Introduction
> 
> The version of UnsafeMutablePointer.initialize(from:) that takes a Collection 
> should be deprecated in favour of a new method on UnsafeMutableBufferPointer 
> that takes a Sequence, with a goal of improving memory safety and enabling 
> faster initialization of memory from sequences. Similarly, 
> UnsafeMutableRawPointer.initializeMemory(as:from:) should be deprecated in 
> favour of a new UnsafeMutableRawBufferPointer.initialize(as:from:).
> 
>  
> 
+1 on this proposal.

(looks like I had forgotten to reply to the list)

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Proposal: Allow explicit type parameter specification in generic function call

[Andrew Trick via swift-evolution]

> On Nov 28, 2016, at 10:11 PM, Douglas Gregor via swift-evolution 
>  wrote:
> 
>> 
>> On Nov 21, 2016, at 3:05 PM, Ramiro Feria Purón via swift-evolution 
>> > wrote:
>> 
>> Problem:
>> 
>> Currently, it is not possible to be explicit about the generic parameters 
>> (type parameters) in a generic function call. Type parameters are inferred 
>> from actual parameters:
>> 
>> func f(_ t: T) {
>> 
>> //..
>> }
>> 
>> f(5)// T inferred to be Int
>> f("xzcvzxcvx")  // T inferred to be string 
>> 
>> If no type parameter is involved in the formal parameters, the type 
>> parameter needs to be used somehow as part of the return type. For example:
>> 
>> func g(_ x: Int) -> [T] {
>> 
>> var result: [T] = []
>> 
>> //..
>> 
>> return result
>> }
>> 
>> In such cases, the type parameters must be inferrable from the context:
>> 
>> g(7)// Error: T cannot be inferred
>> let array = g(7)// Error: T cannot be inferred
>> let array: [String] = g(7)  // Ok: T inferred to be String
>> let array = g(7)// Error: Cannot explicitly specialise 
>> generic function
>> 
>> 
>> 
>> Proposed Solution:
>> 
>> Allow explicit type parameters in generic function call:
>> 
>> let _ = g(7)// Ok
>> 
>> 
>> 
>> Motivation:
>> 
>> Consider the following contrived example:
>> 
>> class Vehicle {
>> var currentSpeed = 0
>> //..
>> }
>> 
>> class Bicycle: Vehicle {
>> //..
>> }
>> 
>> class Car: Vehicle {
>> //..
>> }
>> 
>> @discardableResult
>> func processAll(in vehicles: [Vehicle], condition: (Vehicle) -> 
>> Bool) -> [T] {
>> 
>> var processed: [T] = []
>> 
>> for vehicle in vehicles {
>> guard let t = vehicle as? T, condition(vehicle) else { continue }
>> //..
>> processed.append(t)
>> }
>> 
>> return processed
>> }
>> 
>> func aboveSpeedLimit(vehicle: Vehicle) -> Bool {
>> return vehicle.currentSpeed >= 100
>> }
>> 
>> 
>> let processedVehicles = processAll(in: vehicles, condition: aboveSpeedLimit) 
>>// Uh, T inferred to be Vehicle!
>> let processedCars: [Car] = processAll(in: vehicles, condition: 
>> aboveSpeedLimit) // T inferred to be Car
>> processAll(in: vehicles, condition: aboveSpeedLimit)
>>// This should be allowed under this proposal
>> 
>> 
>> Notes:
>> 
>> If necessary, the (real life) Swift code that lead to the proposal could be 
>> shared.
> 
> This seems completely reasonable to me. I had always expected us to implement 
> this feature, but we never got around to it, and it wasn’t a high priority 
> because one can always use type inference. Additionally, there were a few 
> places where we originally thought we wanted this feature, but prefer the 
> more-explicit form where the user is required to explicitly pass along a 
> metatype. unsafeBitCast is one such case:
> 
>   func unsafeBitCast(_ x: T, to: U.Type) -> U
> 
> Even if we had the ability to provide explicit type arguments, we would *not* 
> want to change this signature to
> 
>   func unsafeBitCast(_ x: T) -> U // bad idea
> 
> because while it makes the correct usage slightly cleaner:
> 
>   unsafeBitCast(something)   // slightly prettier, but…

Angle brackets in function calls are hideous. This is objectively more clear 
and much prettier IMO:

  unsafeBitCast(something, to: Int)

> it would enable type inference to go wild with unsafe casts:
> 
>   foo(unsafeBitCast(something))   // just cast it to.. whatever   
> 
> which is… not great.
> 
> I’d like one bit of clarification in the proposal. Right now, one is not 
> permitted to have a type parameter in a generic function that isn’t used 
> somewhere in its signature, e.g.,
> 
>   func f() -> Void { … }   // error: T is not part of the signature of 
> f()
> 
> This restriction is obvious in today’s Swift, because there is absolutely no 
> way one could ever use this function. With your proposed extension, it would 
> be possible to use this function. Does the restriction remain or is it lifted?
> 
> Personally, I’d like the restriction to stay, because it feels like such 
> functions fall into the same camp as unsafeBitCast: if the type parameter 
> affects how the function operates but is *not* part of its signature, then it 
> should be expressed like a normal parameter (of a metatype). It also helps 
> provide better diagnostics when changing a generic function to no longer 
> require one of its type parameters.

+1 for required type parameters being normal parameters.

I think the case mentioned in the proposal reads much better as:

  processAll(in: vehicles, as: Bicycle, condition: aboveSpeedLimit)

If angle brackets can be limited to generic definitions and type names, that’s 
a great accomplishment.

-Andy

> 
> And, as Dave notes, it’s effectively 

Re: [swift-evolution] [swift-dev] Rebinding UnsafePointer makes it mutable

[Andrew Trick via swift-evolution]

> On Sep 19, 2016, at 11:44 PM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Mon Sep 19 2016, Andrew Trick  wrote:
> 
>>> On Sep 19, 2016, at 1:24 AM, Martin R via swift-dev  
>>> wrote:
>>> 
>>> I noticed that both UnsafePointer and UnsafeMutablePointer have the 
>>> identical method
>>> 
>>>   public func withMemoryRebound(to: T.Type, capacity count: Int, 
>>> _ body: (UnsafeMutablePointer) throws -> Result) rethrows -> Result
>> 
>>> 
>>> so that rebinding an immutable pointer gives you a _mutable_ pointer. That 
>>> is different from what
>>> 
>>>   Unsafe[Mutable]Pointer {
>>> func withMemoryRebound(to: T.Type, capacity count: Int,
>>>   _ body: (Unsafe[Mutable]Pointer) throws -> ()) rethrows
>>>   }
>>> 
>>> in 
>>> https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md
>>>  indicates. Perhaps I am misunderstanding something. Shouldn't rebinding an 
>>> UnsafePointer result in an UnsafePointer again?
>>> 
>>> Martin
>> 
>> I think you’re right about that. I didn’t notice the discrepancy until
>> source breaking changes were frozen and was concerned that fixing it
>> would be more restrictive.
>> 
>> Some users may migrate their code to:
>> 
>> constPtr.withMemoryRebound(to: T.self, capacity: 1) {
>> takesMutablePointer($0)
>> }
>> 
>> We probably want them to be more explicit:
>> 
>> constPtr.withMemoryRebound(to: T.self, capacity: 1) {
>> takesMutablePointer(UnsafeMutablePointer(mutating: $0))
>> }
>> 
>> We could possibly justify correcting this in Swift 3 though on these grounds:
>> 
>> - It’s effectively a bug considering that the proposal and
>> implementation are inconsistent.
> 
> It's definitely a bug, IMO.
> 
>> - There is a correct way write the code that will continue to work
>> before and after fixing the bug.  
>> - A simple fixit will tell them to add the “mutating” label.
>> 
>> If not, it’s something I was already planning to roll into Swift 4.

If there’s no objection, I’ll just go ahead with a fix on the 3.0 branch since 
it was already covered by SE-0107.
-Andy


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-users] Problem with COW optimization

[Andrew Trick via swift-evolution]

> On Sep 19, 2016, at 3:18 AM, Adrian Zubarev via swift-evolution 
>  wrote:
> 
> I think I just found a solution to my problem:
> 
> /// - Parameter child: The new `child` to add to the `children` array.
> public mutating func add(_ child: Element) {
> 
>let clonedChildReference = Reference(cloning: child.reference)
>let index = self.reference.children.endIndex
>  
>self.mutableInsert(clonedChildReference, at: index, isNotOwnReference: 
> child.reference !== self.reference)
> }
>  
> /// Warning: Pass only clonded nodes of type Element to this function!
> private mutating func mutableInsert(_ node: XML.Node, at index: Int, 
> isNotOwnReference: Bool) {
> 
>// * If `self.reference` is NOT uniquely referenced and `node` is a String,
>//   we should rebuilt own reference.
>// * If `self.reference` is NOT uniquely referenced and `node` is a 
> Reference
>//   `isNotOwnReference` is true, we should rebuilt own reference.
>// * If `self.reference` is NOT uniquely referenced and `node` is a clone  
>//   reference to `self.reference` where is `isNotOwnReference` is false, 
> we
>//   should check if there are more than **two** strong references to 
> rebuild
>//   own reference, otherwise it's an implementation artifact and we can 
> keep
>//   old reference (any `node` of type Reference is cloned before it's 
> added  
>//   to the child array).
>let isNotKnownUniquelyReferenced = 
> !isKnownUniquelyReferenced()
>  
>var shouldRebuildReference = false
>  
>switch node.kind {
>  
>case .element(_):
>   let hasMoreThanTwoStrongReferences = (CFGetRetainCount(self.reference) 
> - 1) > 2
>   shouldRebuildReference = (isNotKnownUniquelyReferenced && 
> isNotOwnReference) || hasMoreThanTwoStrongReferences
>  
>case .text(_):
>   shouldRebuildReference = isNotKnownUniquelyReferenced
>}
>  
>if shouldRebuildReference {
>  
>   self.reference = Reference(cloning: self.reference, wholeTree: true)
>}
> 
>self.reference.insert(node, at: index)
> }
> I’m using CFGetRetainCount(self.reference) to catch that implementation 
> artifact.
> 
If people are resorting to CFGetRetainCount, then we have a problem. The 
optimizer is not under any obligation to bump the refcount to 2.

There must be a better way to handle this. Rather than passing an
'isNotOwnReference' flag, I think you should determine whether a clone
is needed before passing the node into mutableInsert.

You effectively want an API like root.addSelf().

-Andy
> 
> 
> 
> -- 
> Adrian Zubarev
> Sent with Airmail
> 
> Am 19. September 2016 um 09:59:24, Adrian Zubarev 
> (adrian.zuba...@devandartist.com ) 
> schrieb:
> 
>> Hello Dave,
>> 
>> thank you for trying to help me. I’ll try to explain the issue with some 
>> more details.
>> 
>> First here is some code:
>> 
>> extension XML {
>>   
>> public struct Element {
>> 
>> // public for testing   
>> public var reference: Reference
>> 
>> public var name: String { return self.reference.name }
>> 
>> public var children: [Element] {
>>   
>> return self.reference.children.flatMap {
>>   
>> guard case .element(let element) = $0.kind else { return nil 
>> }
>> return Element(wrapping: element)
>> }
>> }
>>   
>> public init(name: String) {
>>   
>> self.reference = Reference(name: name)
>> }
>> 
>> public mutating func add(_ child: Element) {
>>   
>> self.mutableInsert(Reference(cloning: child.reference), at: 
>> self.reference.children.endIndex)
>> }
>>   
>> // Ignore XML.Node, it's a String or Reference
>> // Parameter `Node` is assumed to be a clone of a reference passed 
>> to `add` or `insert` method.
>> private mutating func mutableInsert(_ node: XML.Node, at index: Int) 
>> {
>>   
>> // Clone own reference all way up to the root
>> if !isKnownUniquelyReferenced() {
>>   
>> self.reference = Reference(cloning: self.reference, 
>> wholeTree: true)
>> }
>>   
>> // Extract a reference or just insert a string as a child
>> guard case .element(let nodeReference) = node.kind else {
>>   
>> self.reference.insert(node, at: index)
>> return
>> }
>>   
>> // Check for possible debelopment bug
>> if nodeReference === self.reference {
>>   
>> fatalError("wrong usage of `mutableInsert` function")
>> }
>>   
>> 

Re: [swift-evolution] [swift-dev] Rebinding UnsafePointer makes it mutable

[Andrew Trick via swift-evolution]

> On Sep 19, 2016, at 1:24 AM, Martin R via swift-dev  
> wrote:
> 
> I noticed that both UnsafePointer and UnsafeMutablePointer have the identical 
> method
> 
>public func withMemoryRebound(to: T.Type, capacity count: Int, 
> _ body: (UnsafeMutablePointer) throws -> Result) rethrows -> Result
> 
> so that rebinding an immutable pointer gives you a _mutable_ pointer. That is 
> different from what
> 
>Unsafe[Mutable]Pointer {
>  func withMemoryRebound(to: T.Type, capacity count: Int,
>_ body: (Unsafe[Mutable]Pointer) throws -> ()) rethrows
>}
> 
> in 
> https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md
>  indicates. Perhaps I am misunderstanding something. Shouldn't rebinding an 
> UnsafePointer result in an UnsafePointer again?
> 
> Martin

I think you’re right about that. I didn’t notice the discrepancy until source 
breaking changes were frozen and was concerned that fixing it would be more 
restrictive.

Some users may migrate their code to:

constPtr.withMemoryRebound(to: T.self, capacity: 1) {
 takesMutablePointer($0)
}

We probably want them to be more explicit:

constPtr.withMemoryRebound(to: T.self, capacity: 1) {
 takesMutablePointer(UnsafeMutablePointer(mutating: $0))
}

We could possibly justify correcting this in Swift 3 though on these grounds:

- It’s effectively a bug considering that the proposal and implementation are 
inconsistent.
- There is a correct way write the code that will continue to work before and 
after fixing the bug.
- A simple fixit will tell them to add the “mutating” label.

If not, it’s something I was already planning to roll into Swift 4.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeRawBufferPointer

[Andrew Trick via swift-evolution]

> On Sep 10, 2016, at 5:53 PM, Andrew Trick  wrote:
> 
> https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md
> 
> The review period has been extended until September 14. The 
> UnsafeRawBufferPointer type name is settled, but we still need to come up 
> with an answer for the name of the new closure taking functions:
> 
> withXyz() should normally reveal the closure argument type as Xyz. That's why 
> I originally proposed UnsafeBytes as the type name. Now that we've decided to 
> use the descriptive type instead we have a problem…

I was totally wrong about this policy. In closure-taking “withXyz" functions, 
“Xyz" should reveal the role of the closure argument, not its type. We do not 
need to repeat type information.

We have strong agreement to leave the proposed `withUnsafeBytes {…}` name as it 
stands.

Note that `withRawBytes` was a strong contender, but at this time it's more 
important to consistently follow the convention for using `Unsafe` in the 
closure name whenever the closure argument is unsafe (e.g. you can't return it 
from the closure). We may want to revisit this logic later (in some sense 
Unsafe is redundant), but when we do that, we also need to reevaluate all of 
our withUnsafe APIs. Furthermore, we would want to change Foundation Data's API 
to be consistent. These are bigger debates that can be deferred.

-Andy

> 
> In this code, it's obvious that a sequence of bytes is being appended to an 
> array.
> 
> var buffer = [UInt8]()
> withUnsafeBytes(of: ) {
>  buffer += $0
> }
> 
> In the following version, the closure argument type is obvious, which is 
> nice, but otherwise it's borderline unreadable, and doesn't describe what's 
> actually happenning. How can we tell that a sequence of bytes will be 
> appended?
> 
> var buffer = [UInt8]()
> withUnsafeRawBufferPointer(to: ) {
>  buffer += $0
> }
> 
> The mutable version really stretches the limits of descriptively naming 
> things, and still doesn't say anything about a byte sequence:
> 
> withUnsafeMutableRawBufferPointer(to: ) {
>  readHeader(into: $0)
> }
> 
> -Andy
> 
>> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>>  wrote:
>> 
>> 
>> on Thu Sep 01 2016, Andrew Trick  wrote:
>> 
>>> I’m resending this for Review Manager Dave A. because the announce list is 
>>> dropping his messages...
>>> 
>>> Hello Swift community,
>>> 
>>> The review of "UnsafeBytes" begins now and runs through September
>>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>>> allowing developers to continue working with collections of UInt8 values,
>>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>>> direct manipulation of raw pointers or reasoning about binding memory.
>>> 
>>> The proposal is available here:
>>> 
>>> >>  
>>> >
>>> 
>>> * What is your evaluation of the proposal?
>> 
>> I strongly support inclusion of the feature, but I have issues with the
>> name.  It seems to me that in order to fit into the standard library, it
>> should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name
>> conveys something important, and for the same reasons we're using
>> Unsafe[Mutable]BufferPointer instead of UnsafeMutableElements, we should
>> stick to the scheme:
>> 
>> - “Unsafe,” because you can break memory safety with this tool
>> 
>> - “Raw,” because the fundamental model is that of “raw,” rather than
>> “typed,” memory.
>> 
>> - “Buffer,” because it works on a series of contiguous elements of known
>> length.
>> 
>> - “Pointer,” because it has reference semantics!  When you pass one of
>> these things around by value, you're not passing the bytes; you're
>> passing a shared reference to the bytes.
>> 
>>> * Is the problem being addressed significant enough to warrant a
>>>  change to Swift?
>> 
>> Yes, and it fills an important funcationality gap now that we have the
>> unsafe pointer model nailed down.
>> 
>>> 
>>> * Does this proposal fit well with the feel and direction of Swift?
>> 
>> Yes, except for the name.
>> 
>>> 
>>> * If you have used other languages or libraries with a similar
>>> feature, how do you feel that this proposal compares to those?  
>> 
>> I don't think any other language distinguishes raw from typed memory in
>> this way.
>> 
>>> * How much effort did you put into your review? A glance, a quick
>>> reading, or an in-depth study?
>> 
>> Enough ;-)
>> 
>> -- 
>> -Dave, posting as a reviewer, not a review manager
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 

Re: [swift-evolution] SE-0138 UnsafeRawBufferPointer

[Andrew Trick via swift-evolution]

> On Sep 11, 2016, at 2:29 PM, Rick Mann <rm...@latencyzero.com> wrote:
> 
> Thanks, Andrew. In my case, the Data reads from a file, and since the raw 
> access is wrapped around the LZMA decompression, I think it should be safe 
> (no one else is accessing the data at that time).
> 
> I'll just wait for Foundation.Data to be updated and update my code then.

Sure, enumerateBytes is fine for you. To be clear, it would only be a problem 
if Data did not own the memory (bytesNoCopy:), allowing the same memory to be 
accessed as a non-UInt8 type.
-Andy

> 
>> On Sep 10, 2016, at 19:33 , Andrew Trick <atr...@apple.com> wrote:
>> 
>> 
>>> On Sep 10, 2016, at 6:23 PM, Rick Mann via swift-evolution 
>>> <swift-evolution@swift.org> wrote:
>>> 
>>> Coincidentally, I just wrote my first Swift code to use UnsafePointer<>. I 
>>> was wrapping the LZMA API to decompress LZMA data. It's a C API that works 
>>> by pointing to an input buffer and and output buffer, and then calling a 
>>> function that decompresses what it can given those two buffers (and their 
>>> lengths).
>>> 
>>> I treated them as UnsafePointer, but really they're raw, in the 
>>> sense that they are not a collection of a single element, just a collection 
>>> of bytes.
>>> 
>>> My wrapper's interface to LZMA uses Data instances. I don't see a way of 
>>> getting from Data to UnsafeRawBufferPointer in Xcode 8 GM seed (which makes 
>>> sense, given that this is still in progress). But I also didn't see a way 
>>> to get to UnsafeRawPointer; should there be?
>> 
>> There should be and there isn't. It used to be Data.bytes, but it was just 
>> deprecated. In the current state of limbo, you just do this:
>> 
>> return data.withUnsafeBytes { bytes: UnsafeBufferPointer in … }
>> 
>> and that binds Data’s memory to UInt8. It fine in practice as long as Data 
>> owns its memory (not using bytesNoCopy). Otherwise whoever else uses the 
>> memory should also view it as either raw or UInt8, or they should bind 
>> memory each time they access it.
>> 
>>> Will something be added to Data when SE-0138 is finalized? I guess that's 
>>> not for Swift 3 but 3.x? 
>> 
>> Yes. It just takes a little more time to evolve the Data API.
>> 
>> -Andy
>> 
>>> Thanks, and sorry if I'm hijacking the thread a bit with this.
>>> 
>>>> On Sep 10, 2016, at 17:53 , Andrew Trick via swift-evolution 
>>>> <swift-evolution@swift.org> wrote:
>>>> 
>>>> https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md
>>>> 
>>>> The review period has been extended until September 14. The 
>>>> UnsafeRawBufferPointer type name is settled, but we still need to come up 
>>>> with an answer for the name of the new closure taking functions:
>>>> 
>>>> withXyz() should normally reveal the closure argument type as Xyz. That's 
>>>> why I originally proposed UnsafeBytes as the type name. Now that we've 
>>>> decided to use the descriptive type instead we have a problem...
>>>> 
>>>> In this code, it's obvious that a sequence of bytes is being appended to 
>>>> an array.
>>>> 
>>>> var buffer = [UInt8]()
>>>> withUnsafeBytes(of: ) {
>>>> buffer += $0
>>>> }
>>>> 
>>>> In the following version, the closure argument type is obvious, which is 
>>>> nice, but otherwise it's borderline unreadable, and doesn't describe 
>>>> what's actually happenning. How can we tell that a sequence of bytes will 
>>>> be appended?
>>>> 
>>>> var buffer = [UInt8]()
>>>> withUnsafeRawBufferPointer(to: ) {
>>>> buffer += $0
>>>> }
>>>> 
>>>> The mutable version really stretches the limits of descriptively naming 
>>>> things, and still doesn't say anything about a byte sequence:
>>>> 
>>>> withUnsafeMutableRawBufferPointer(to: ) {
>>>> readHeader(into: $0)
>>>> }
>>>> 
>>>> -Andy
>>>> 
>>>>> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>>>>> <swift-evolution@swift.org> wrote:
>>>>> 
>>>>> 
>>>>> on Thu Sep 01 2016, Andrew Trick <swift-evolution@swift.org> wrote:
>>>>> 
>>>>>> I’m resending this for Review Manager Dave A. b

Re: [swift-evolution] SE-0138 UnsafeRawBufferPointer

[Andrew Trick via swift-evolution]

> On Sep 11, 2016, at 3:07 AM, Ben Rimmington  wrote:
> 
> 
>> On 11 Sep 2016, at 01:53, Andrew Trick wrote:
>> 
>> https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md
>> 
>> The review period has been extended until September 14. The 
>> UnsafeRawBufferPointer type name is settled, but we still need to come up 
>> with an answer for the name of the new closure taking functions:
>> 
>> withXyz() should normally reveal the closure argument type as Xyz. That's 
>> why I originally proposed UnsafeBytes as the type name. Now that we've 
>> decided to use the descriptive type instead we have a problem...
> 
> Does the `enumerateBytes` method (of Foundation.Data and DispatchData) also 
> need an UnsafeRawBufferPointer version?


I think it should only have an UnsafeRawBufferPointer version. If the user 
wants to bind memory, they should do that explicitly. I’ve made the likely 
changes to Data on a branch:
https://github.com/atrick/swift/commit/19968405608fa326eb7ad5ffed5fcd9a78b0f0a5 


There are enough changes to Data that I think it deserves a separate proposal 
and discussion thread. It’s useful to look ahead at how the Data API should 
look but I’m trying to get language-level changes accepted first (in some 
sense, Unsafe constructs are part of the language even if they don’t require 
compiler changes).

Also keep in mind, adding UnsafeRawBufferPointer does not make Data any less 
usable today. We just need to get core support in place so we can have a 
discussion about Foundation.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeRawBufferPointer

[Andrew Trick via swift-evolution]

> On Sep 10, 2016, at 6:23 PM, Rick Mann via swift-evolution 
> <swift-evolution@swift.org> wrote:
> 
> Coincidentally, I just wrote my first Swift code to use UnsafePointer<>. I 
> was wrapping the LZMA API to decompress LZMA data. It's a C API that works by 
> pointing to an input buffer and and output buffer, and then calling a 
> function that decompresses what it can given those two buffers (and their 
> lengths).
> 
> I treated them as UnsafePointer, but really they're raw, in the sense 
> that they are not a collection of a single element, just a collection of 
> bytes.
> 
> My wrapper's interface to LZMA uses Data instances. I don't see a way of 
> getting from Data to UnsafeRawBufferPointer in Xcode 8 GM seed (which makes 
> sense, given that this is still in progress). But I also didn't see a way to 
> get to UnsafeRawPointer; should there be?

There should be and there isn't. It used to be Data.bytes, but it was just 
deprecated. In the current state of limbo, you just do this:

  return data.withUnsafeBytes { bytes: UnsafeBufferPointer in … }

and that binds Data’s memory to UInt8. It fine in practice as long as Data owns 
its memory (not using bytesNoCopy). Otherwise whoever else uses the memory 
should also view it as either raw or UInt8, or they should bind memory each 
time they access it.

> Will something be added to Data when SE-0138 is finalized? I guess that's not 
> for Swift 3 but 3.x? 

Yes. It just takes a little more time to evolve the Data API.

-Andy

> Thanks, and sorry if I'm hijacking the thread a bit with this.
> 
>> On Sep 10, 2016, at 17:53 , Andrew Trick via swift-evolution 
>> <swift-evolution@swift.org> wrote:
>> 
>> https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md
>> 
>> The review period has been extended until September 14. The 
>> UnsafeRawBufferPointer type name is settled, but we still need to come up 
>> with an answer for the name of the new closure taking functions:
>> 
>> withXyz() should normally reveal the closure argument type as Xyz. That's 
>> why I originally proposed UnsafeBytes as the type name. Now that we've 
>> decided to use the descriptive type instead we have a problem...
>> 
>> In this code, it's obvious that a sequence of bytes is being appended to an 
>> array.
>> 
>> var buffer = [UInt8]()
>> withUnsafeBytes(of: ) {
>> buffer += $0
>> }
>> 
>> In the following version, the closure argument type is obvious, which is 
>> nice, but otherwise it's borderline unreadable, and doesn't describe what's 
>> actually happenning. How can we tell that a sequence of bytes will be 
>> appended?
>> 
>> var buffer = [UInt8]()
>> withUnsafeRawBufferPointer(to: ) {
>> buffer += $0
>> }
>> 
>> The mutable version really stretches the limits of descriptively naming 
>> things, and still doesn't say anything about a byte sequence:
>> 
>> withUnsafeMutableRawBufferPointer(to: ) {
>> readHeader(into: $0)
>> }
>> 
>> -Andy
>> 
>>> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>>> <swift-evolution@swift.org> wrote:
>>> 
>>> 
>>> on Thu Sep 01 2016, Andrew Trick <swift-evolution@swift.org> wrote:
>>> 
>>>> I’m resending this for Review Manager Dave A. because the announce list is 
>>>> dropping his messages...
>>>> 
>>>> Hello Swift community,
>>>> 
>>>> The review of "UnsafeBytes" begins now and runs through September
>>>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>>>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>>>> allowing developers to continue working with collections of UInt8 values,
>>>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>>>> direct manipulation of raw pointers or reasoning about binding memory.
>>>> 
>>>> The proposal is available here:
>>>> 
>>>> <https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md
>>>>  
>>>> <https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md>>
>>>> 
>>>> * What is your evaluation of the proposal?
>>> 
>>> I strongly support inclusion of the feature, but I have issues with the
>>> name.  It seems to me that in order to fit into the standard library, it
>>> should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name
>>> conveys something i

Re: [swift-evolution] SE-0138 UnsafeRawBufferPointer

[Andrew Trick via swift-evolution]

https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsaferawbufferpointer.md

The review period has been extended until September 14. The 
UnsafeRawBufferPointer type name is settled, but we still need to come up with 
an answer for the name of the new closure taking functions:

withXyz() should normally reveal the closure argument type as Xyz. That's why I 
originally proposed UnsafeBytes as the type name. Now that we've decided to use 
the descriptive type instead we have a problem...

In this code, it's obvious that a sequence of bytes is being appended to an 
array.

var buffer = [UInt8]()
withUnsafeBytes(of: ) {
  buffer += $0
}

In the following version, the closure argument type is obvious, which is nice, 
but otherwise it's borderline unreadable, and doesn't describe what's actually 
happenning. How can we tell that a sequence of bytes will be appended?

var buffer = [UInt8]()
withUnsafeRawBufferPointer(to: ) {
  buffer += $0
}

The mutable version really stretches the limits of descriptively naming things, 
and still doesn't say anything about a byte sequence:

withUnsafeMutableRawBufferPointer(to: ) {
  readHeader(into: $0)
}

-Andy

> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Thu Sep 01 2016, Andrew Trick  wrote:
> 
>> I’m resending this for Review Manager Dave A. because the announce list is 
>> dropping his messages...
>> 
>> Hello Swift community,
>> 
>> The review of "UnsafeBytes" begins now and runs through September
>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>> allowing developers to continue working with collections of UInt8 values,
>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>> direct manipulation of raw pointers or reasoning about binding memory.
>> 
>> The proposal is available here:
>> 
>> >  
>> >
>> 
>> * What is your evaluation of the proposal?
> 
> I strongly support inclusion of the feature, but I have issues with the
> name.  It seems to me that in order to fit into the standard library, it
> should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name
> conveys something important, and for the same reasons we're using
> Unsafe[Mutable]BufferPointer instead of UnsafeMutableElements, we should
> stick to the scheme:
> 
> - “Unsafe,” because you can break memory safety with this tool
> 
> - “Raw,” because the fundamental model is that of “raw,” rather than
>  “typed,” memory.
> 
> - “Buffer,” because it works on a series of contiguous elements of known
>  length.
> 
> - “Pointer,” because it has reference semantics!  When you pass one of
>  these things around by value, you're not passing the bytes; you're
>  passing a shared reference to the bytes.
> 
>> * Is the problem being addressed significant enough to warrant a
>>   change to Swift?
> 
> Yes, and it fills an important funcationality gap now that we have the
> unsafe pointer model nailed down.
> 
>> 
>> * Does this proposal fit well with the feel and direction of Swift?
> 
> Yes, except for the name.
> 
>> 
>> * If you have used other languages or libraries with a similar
>> feature, how do you feel that this proposal compares to those?  
> 
> I don't think any other language distinguishes raw from typed memory in
> this way.
> 
>> * How much effort did you put into your review? A glance, a quick
>> reading, or an in-depth study?
> 
> Enough ;-)
> 
> -- 
> -Dave, posting as a reviewer, not a review manager
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Review] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 2, 2016, at 7:36 PM, Karl via swift-evolution 
>  wrote:
> 
>> * Does this proposal fit well with the feel and direction of Swift?
> 
> Yes, but I’m not sure about the “Unsafe” part of “UnsafeBytes” — what is 
> unsafe about getting the byte-representation of a value?
> 
> As I understand it, “Raw” is what we use for “untyped”, so might I suggest 
> “RawBytes"?

It’s annoying, but a strict requirement in Swift that any “unsafe” operation be 
marked with the word “Unsafe” either at that point or in some enclosing scope. 
Unsafe broadly refers to various forms of memory unsafety. Unsafe bytes is 
actually safe with respect to pointer aliasing but it is unsafe because

1. It points into memory that it does not own. The developer must do something 
to to manage the memory’s lifetime.

2. It does not perform bounds-checking in release mode.

In both those respects, it’s just like UnsafeBufferPointer.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 3, 2016, at 3:36 PM, Drew Crawford  wrote:
> 
> 
> On September 2, 2016 at 2:36:43 AM, Andrew Trick (atr...@apple.com 
> ) wrote:
> 
>> After thinking about this for a moment, I like the approach of extending 
>> UnsafeBytes with release-mode bounds checked versions of subscript, load, 
>> and storeBytes.
> 
> I agree with this, I think it's mostly a question of naming and defaults.  My 
> concern here is letting a swift developer accidentally write heartbleed, 
> which we can't actually prevent, but we can make it harder.
> 
> IMO 
> 
> 1.  There should be clear consistency in the checked-ness of the API surface. 
>  Agree that checked iterator makes no sense, but I think the most important 
> thing is to avoid creating a job interview trivia game where `set` is checked 
> but `store` is unchecked, spot the bug in this function.
> 
> 2.  For consistency with UnsafeBufferPointer it may make the most sense to 
> just ship unchecked or ship an opt-in checked wrapper.  I believe however 
> that the existing precedent is all wrong on this point, and I'd like to see 
> us revisit this question across both interfaces in Swift 4, but I don't want 
> to lay out a whole case here that should be its own thread.
> 
I generally agree with what you said. I think the vague plan is later in Swift 
4 to ship a bounds-checked variant of both UnsafeBufferPointer and UnsafeBytes 
(or  UnsafeRawBufferPointer if you prefer).

I don’t want to eliminate the debug-mode checks though. I did try to make it 
clear in the comments that bounds-checking only applied to debug mode, so 
developers should not accidentally become too reliant on them.

So, the only question is whether the UnsafeBytes.copyBytes() API should have 
debug or release-mode checks. My decision to keep the stronger checks here was 
probabilistic—it seems unlikely to be a performance issue but likely to catch 
most buffer overruns. But I agree that it is inconsistent, especially if we 
plan to introduce a release bounds-checked variant later. We don’t want 
developers to begin relying on that check. I’m leaning toward dropping it down 
to a debug-mode check.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 2, 2016, at 11:14 AM, Dave Abrahams via swift-evolution 
>  wrote:
> 
> 
> on Thu Sep 01 2016, Andrew Trick  wrote:
> 
>> I’m resending this for Review Manager Dave A. because the announce list is 
>> dropping his messages...
>> 
>> Hello Swift community,
>> 
>> The review of "UnsafeBytes" begins now and runs through September
>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>> allowing developers to continue working with collections of UInt8 values,
>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>> direct manipulation of raw pointers or reasoning about binding memory.
>> 
>> The proposal is available here:
>> 
>> >  
>> >
>> 
>> * What is your evaluation of the proposal?
> 
> I strongly support inclusion of the feature, but I have issues with the

Clearly, otherwise you wouldn't have announced it 4 times ;)

> name.  It seems to me that in order to fit into the standard library, it
> should be called Unsafe[Mutable]RawBufferPointer.  Each part of the name

Well, that's natural from a stdlib designer's viewpoint. It is almost
identical functionality, but it also exposes the UnsafeRawPointer API
for loading and storing arbitrary types. This naming issue was
discussed for a couple weeks on swift-evolution. Let's see if I can
recap inline with your comments.

> conveys something important, and for the same reasons we're using
> Unsafe[Mutable]BufferPointer instead of UnsafeMutableElements, we should
> stick to the scheme:
> 
> - “Unsafe,” because you can break memory safety with this tool

OK. Let's not drop that one!

> - “Raw,” because the fundamental model is that of “raw,” rather than
>  “typed,” memory.

To me, bytes only exist in memory. Accessing a byte, as opposed to some
in-memory type, is always a raw access.

> - “Buffer,” because it works on a series of contiguous elements of known
>  length.

To me, bytes always represent a contiguous chunk of raw memory. The
term implies that we're dealing with memory layout, as opposed to just
some opaque chunk of data, which is I think what Foundation Data is for.

> - “Pointer,” because it has reference semantics!  When you pass one of
>  these things around by value, you're not passing the bytes; you're
>  passing a shared reference to the bytes.

Unsafe means that this value doesn't own the memory. I agree with you
that reference semantics are important, and we need to clearly
distinguished this from something like Data. I just think Unsafe is
enough for the name.

UnsafeMutableRawBufferPointer does not actually convey that it can be
viewed as a collection of 8-bit values, which is fairly important.

Now that I've satisfied my pedantic side, let's look at it from the developer's 
side.
To me it's a question of whether a longer or shorter name is more meaningful in
the natural setting of users' source code:

func foo(bytes: UnsafeMutableRawBufferPointer)

withUnsafeMutableRawBufferPointer(to: ) {
  foo(bytes: $0)
}
---
func foo(bytes: UnsafeMutableBytes)

withUnsafeBytes(of: ) {
  write(bytes: $0)
}

I don't think the longer name is more descriptive. I do think the
shorter name is more intuitive and meaningful.

UnsafeMutableRawPointer is already too long to be recognizable to
users. A benefit of UnsafeBytes is that the most developers won't need
to know how to work directly with raw pointers. So the name doesn’t
need to evoke them.

-Andy
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [swift-evolution-announce] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 2, 2016, at 9:31 AM, Tony Parker via swift-evolution 
>  wrote:
> 
> Hi Andy, Dave,
> 
> I have two major objections to this proposal.
> 
> The first is timing. It is too late for us to evaluate this correctly for 
> Swift 3.
> 
> The second is that this API overlaps too much with Foundation’s struct Data. 
> We should be standardizing on a small number of common types, so that 
> developers do not need to find ways to translate one API output into another 
> API’s input.
> 
> I think we should instead focus on what needs to be added to struct Data (and 
> other API, like Stream) to fill this role. The proposal touches only touches 
> on this briefly, but in my opinion it is the direction we should pursue — and 
> for Swift 4 or perhaps some Swift 3 update.
> 
> - Tony

Foundation Data needs an interface to UnsafePointers. UnsafeBytes
perfectly fits that need. The same is true of any Stream or
BinaryFormat abstraction that we design in the future. In fact,
migrating Data itself to the raw pointer changes in Swift 3 is enough
motivation to add UnsafeBytes.

It's really unfortunate that Data's interface can't take advantage of
UnsafeBytes in Swift 3 because it would help with adoption of
Data. There is currently a design flaw whenever a client of Data
doesn't know the memory's original type. I'm not happy about that, but
I think we can live with it for a while.

The issue at hand is the large amount of Swift code out there working
with UnsafePointers. We urgently need to provide a migration strategy
for that code. Obviously, we don't have an urgent need to migrate code
already using Data, so improving it's interface can wait until Swift 4.

I've seen many attempts to migrate to Swift 3 (this has been my job
for the past month). I can claim with certainty that if we don't
give developers a natural way to replace their UnsafePointer,
we are going to be left with a lot of incorrect Swift code.

Let me make it clear that there is no overlap between Data and UnsafeBytes.

For public APIs, UnsafeBytes is meant to replace those functions that
currently take (UnsafePointer, Int) including Data's own
interfaces. Otherwise, we're strongly encouraging users to write
incorrect code on the client side. Won't NSStream, for example,
continue to to support UnsafePointer for those developers who need it?
If the developer does not need UnsafePointers, that's great, and
that's what we should continue striving for. But when developers are
using UnsafePointer, we need a natural way to use it correctly.

For general Swift code, Unsafe means something special and
important. The ultimate goal of the standard library and frameworks is
that application developers never need to do something Unsafe. If
they do, it needs to be explicitly marked Unsafe. If we are promoting
Foundation Data as the right way to solve problems for app developers,
then it needs to *not* be Unsafe.

UnsafeBytes simply provides a missing bridge between Unsafe pointers
and safe APIs like Data. By definition, Data and UnsafeBytes use cases
don't overlap. You either need to use Unsafe pointers because you're
programming at the systems level, or existing (safe) libraries do the
job. In practice, there are points at which these worlds meet.

So...

- Today we need UnsafeBytes so that we can migrate existing Swift
  code correctly to a well-defined memory model.

- In the future we need UnsafeBytes to safely implement the transitions
  between "systems code" and "application code”.

-Andy

> 
>> On Sep 1, 2016, at 3:18 PM, Andrew Trick > > wrote:
>> 
>> I’m resending this for Review Manager Dave A. because the announce list is 
>> dropping his messages...
>> 
>> Hello Swift community,
>> 
>> The review of "UnsafeBytes" begins now and runs through September
>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>> allowing developers to continue working with collections of UInt8 values,
>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>> direct manipulation of raw pointers or reasoning about binding memory.
>> 
>> The proposal is available here:
>> 
>>  
>> >  
>> >
>> 
>> Reviews are an important part of the Swift evolution process. All reviews
>> should be sent to the swift-evolution mailing list at
>> 
>>  > >
>> 
>> or, if you would like to keep your feedback private, directly to the
>> review manager. When replying, please try to keep the proposal link at
>> the top of the message:
>> 
>> Proposal link:
>>  > 

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

> On Sep 1, 2016, at 5:37 PM, Andrew Trick  wrote:
> 
> The proposal is available here:
> 
>  
>   
> >
> 
>> On Sep 1, 2016, at 4:59 PM, Drew Crawford > > wrote:
>> I'm possibly one of the larger users of raw byte stuff in Swift as I 
>> maintain an entire client/server network protocol stack in Swift userspace, 
>> similar in spirit to one of the examples drawn out a lot longer.  Grepping 
>> my code produces over 200 individual uses of unsafe byte accesses.
>> 
>> I definitely agree that the problem is significant enough to warrant a 
>> last-minute change.
>> 
>> To a first approximation I agree with all the implementation choices.  The 
>> naming, the choice of UInt8, length tracking, and debug-bounds checking are 
>> all correct IMO.  We have been using something similar for a long time 
>> internally [have you been reading my code? :-) ] so I can speak from 
>> experience that the basic plan here is sound.
>> 
>> One thing I would like to see is an (opt-in) release-mode-bounds-check.  
>> Networking is a core use case for this feature, but when you are reading 
>> from a socket, production is where you need a guard against out-of-bounds UB 
>> the most.  If we can't solve it for Swift 3, affected users can write a 
>> wrapper to implement the boundscheck, but I think we should at very least 
>> take it up again for Swift 4.
>> 
>> Drew
> 
> In my current implementation:
> https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
>  
> 
> 
> The bounds checks in `copyBytes(from:)` are release mode preconditions.
> 
> The bounds checks for `subscript`, `load(as:)`, and `storeBytes(of:as:)` are 
> debug only because it’s likely they occur in some loop that could be covered 
> by a single bounds check. By extension, the sequence iterator is only bounds 
> checked in debug mode.
> 
> One possibility would be different names for the bounds checked forms of 
> those methods: getByte(atOffset:), setByte(atOffset:), 
> load(fromCheckedOffset:as:), storeBytes(of:toCheckedOffset:as:). Along with 
> some kind of bounds checked Iterator.
> 
> I don’t think makes a lot of sense as generic Collection though. 
> Alternatively, we just have an UnsafeBoundsCheckedBytes wrapper.
> 
> This would a good thing to experiment with in your project. We may be able to 
> follow-up with a Swift 4 proposal. The important thing now is to determine 
> whether the proposed Swift 3 design will make that wrapper difficult in any 
> way.

After thinking about this for a moment, I like the approach of extending 
UnsafeBytes with release-mode bounds checked versions of subscript, load, and 
storeBytes. It’s not actually meaningful to have a bounds checked iterator for 
UnsafeBytes. A wrapper would only be useful to guard against accidentally 
circumventing the bounds checks, but I’m not sure that’s really helpful in 
practice. It seems that a framework would want to provide more abstract Socket 
I/O or network message abstractions and those wrappers would just call the 
bounds checked version of the UnsafeBytes APIs.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

The proposal is available here:

 
<https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md
 
<https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md>>

> On Sep 1, 2016, at 4:59 PM, Drew Crawford <d...@sealedabstract.com> wrote:
> I'm possibly one of the larger users of raw byte stuff in Swift as I maintain 
> an entire client/server network protocol stack in Swift userspace, similar in 
> spirit to one of the examples drawn out a lot longer.  Grepping my code 
> produces over 200 individual uses of unsafe byte accesses.
> 
> I definitely agree that the problem is significant enough to warrant a 
> last-minute change.
> 
> To a first approximation I agree with all the implementation choices.  The 
> naming, the choice of UInt8, length tracking, and debug-bounds checking are 
> all correct IMO.  We have been using something similar for a long time 
> internally [have you been reading my code? :-) ] so I can speak from 
> experience that the basic plan here is sound.
> 
> One thing I would like to see is an (opt-in) release-mode-bounds-check.  
> Networking is a core use case for this feature, but when you are reading from 
> a socket, production is where you need a guard against out-of-bounds UB the 
> most.  If we can't solve it for Swift 3, affected users can write a wrapper 
> to implement the boundscheck, but I think we should at very least take it up 
> again for Swift 4.
> 
> Drew

In my current implementation:
https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb
 
<https://github.com/atrick/swift/blob/unsafebytes/stdlib/public/core/UnsafeBytes.swift.gyb>

The bounds checks in `copyBytes(from:)` are release mode preconditions.

The bounds checks for `subscript`, `load(as:)`, and `storeBytes(of:as:)` are 
debug only because it’s likely they occur in some loop that could be covered by 
a single bounds check. By extension, the sequence iterator is only bounds 
checked in debug mode.

One possibility would be different names for the bounds checked forms of those 
methods: getByte(atOffset:), setByte(atOffset:), load(fromCheckedOffset:as:), 
storeBytes(of:toCheckedOffset:as:). Along with some kind of bounds checked 
Iterator.

I don’t think makes a lot of sense as generic Collection though. Alternatively, 
we just have an UnsafeBoundsCheckedBytes wrapper.

This would a good thing to experiment with in your project. We may be able to 
follow-up with a Swift 4 proposal. The important thing now is to determine 
whether the proposed Swift 3 design will make that wrapper difficult in any way.

-Andy
> On September 1, 2016 at 5:19:02 PM, Andrew Trick via swift-evolution 
> (swift-evolution@swift.org <mailto:swift-evolution@swift.org>) wrote:
> 
>> I’m resending this for Review Manager Dave A. because the announce list is 
>> dropping his messages...
>> 
>> Hello Swift community,
>> 
>> The review of "UnsafeBytes" begins now and runs through September
>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>> allowing developers to continue working with collections of UInt8 values,
>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>> direct manipulation of raw pointers or reasoning about binding memory.
>> 
>> The proposal is available here:
>> 
>>  
>> <https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md
>>  
>> <https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md>>
>> 
>> Reviews are an important part of the Swift evolution process. All reviews
>> should be sent to the swift-evolution mailing list at
>> 
>>  <https://lists.swift.org/mailman/listinfo/swift-evolution 
>> <https://lists.swift.org/mailman/listinfo/swift-evolution>>
>> 
>> or, if you would like to keep your feedback private, directly to the
>> review manager. When replying, please try to keep the proposal link at
>> the top of the message:
>> 
>> Proposal link:
>>  <https://lists.swift.org/mailman/listinfo/swift-evolution 
>> <https://lists.swift.org/mailman/listinfo/swift-evolution>>
>> 
>> What goes into a review?
>> 
>> The goal of the review process is to improve the proposal under review
>> through constructive criticism and, eventually, determine the direction of
>> Swift. When writing your review, here are some questions you might want to
>> answer in your review:
>> 
>>  * What is your evaluation of the proposal?
>>  * Is the problem being addressed significant enough to

[swift-evolution] SE-0138 UnsafeBytes

[Andrew Trick via swift-evolution]

I’m resending this for Review Manager Dave A. because the announce list is 
dropping his messages...

Hello Swift community,

The review of "UnsafeBytes" begins now and runs through September
7th. This late addition to Swift 3 is a follow-up to SE-0107:
UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
allowing developers to continue working with collections of UInt8 values,
but now doing so via a type safe API. The UnsafeBytes API will not require 
direct manipulation of raw pointers or reasoning about binding memory.

The proposal is available here:

 
>

Reviews are an important part of the Swift evolution process. All reviews
should be sent to the swift-evolution mailing list at

 >

or, if you would like to keep your feedback private, directly to the
review manager. When replying, please try to keep the proposal link at
the top of the message:

Proposal link:
 >

What goes into a review?

The goal of the review process is to improve the proposal under review
through constructive criticism and, eventually, determine the direction of
Swift. When writing your review, here are some questions you might want to
answer in your review:

 * What is your evaluation of the proposal?
 * Is the problem being addressed significant enough to warrant a
   change to Swift?
 * Does this proposal fit well with the feel and direction of Swift?
 * If you have used other languages or libraries with a similar
   feature, how do you feel that this proposal compares to those?
 * How much effort did you put into your review? A glance, a quick
   reading, or an in-depth study?

More information about the Swift evolution process is available at

 >

Thank you,

-Dave Abrahams
Review Manager___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [late pitch] UnsafeBytes proposal

[Andrew Trick via swift-evolution]

> On Aug 23, 2016, at 9:56 PM, Chris Lattner via swift-evolution 
>  wrote:
> 
> 
>> On Aug 23, 2016, at 5:28 PM, Trent Nadeau via swift-evolution 
>>  wrote:
>> 
>> For an architecture to be C-compatible, its byte size must be 8 bits. Given 
>> the need to run C everywhere and that almost all OSes are written in C, it's 
>> a very safe assumption that a byte equals 8 bits. At this point, I think the 
>> only thing where that's not true are certain micro-controllers for which 
>> there are specialized compilers.
> 
> Right, Swift depends on Clang.  Clang assumes that the target machine has 
> 8-bit-byte-addressable memory.  I don’t see any reason that Swift should 
> pretend to work on the (only historically interesting) systems that had 9-bit 
> (or any other weird number) addressable units.

To be fair to Jason, I think he just wants the language to be explicit about a 
byte being an 8-bit numeric value.

That’s not quite what I’m trying to convey though. I want bytes to refer only 
to the in-memory layout of values of any type in 8-bit chunks per the ABI. 
Typically, loading a byte without imposing any type on the in-memory value 
should produce a UInt8 value. Swift should not have a Byte type because it 
would contradict this meaning and serve no purpose.

I’ve been working with developers who need to parse or stream binary data. It’s 
a common use case. What I’m proposing perfectly matches the intuition of these 
developers who are already using the same concepts and terminology that I am.

-Andy

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [late pitch] UnsafeBytes proposal

[Andrew Trick via swift-evolution]

> On Aug 19, 2016, at 12:43 PM, Xiaodi Wu  wrote:
> 
> On Fri, Aug 19, 2016 at 2:32 PM, Karl via swift-evolution 
> > wrote:
> 
>> On 19 Aug 2016, at 19:35, Andrew Trick > > wrote:
>> 
>> 
>>> On Aug 16, 2016, at 7:13 PM, Karl via swift-evolution 
>>> > wrote:
>>> 
>>> 
 On 16 Aug 2016, at 01:14, David Sweeris via swift-evolution 
 > wrote:
 
> On Aug 15, 2016, at 13:55, Michael Ilseman via swift-evolution 
> > wrote:
 
> 
> It seems like there’s a potential for confusion here, in that people may 
> see “UInt8” and assume there is some kind of typed-ness, even though the 
> whole point is that this is untyped. Adjusting the header comments 
> slightly might help:
> 
> 
> /// A non-owning view of raw memory as a collection of bytes.
> ///
> /// Reads and writes on memory via `UnsafeBytes` are untyped operations 
> that
> /// do no require binding the memory to a type. These operations are 
> expressed 
> /// in terms of `UInt8`, though the underlying memory is untyped.
> 
> …
> 
> You could go even further towards hinting this fact with a `typealias 
> Byte = UInt8`, and use Byte throughout. But, I don’t know if that’s 
> getting too excessive.
 
 I don't think that's too excessive at all. I might even go further and say 
 that we should call it "Untyped" instead of "Byte", to really drive home 
 the point (many people see "byte" and think "8-bit int", which is merely a 
 side effect of CPUs generally not having support for types *other* than 
 ints and floats, rather than a reflection of the true "type" of the data).
 
 - Dave Sweeris
 ___
 swift-evolution mailing list
 swift-evolution@swift.org 
 https://lists.swift.org/mailman/listinfo/swift-evolution 
 
>>> ‘Byte’ is sufficient, I think.
>>> 
>>> In some sense, it is typed as bytes. It reflects the fact that anything 
>>> that is representable to the computer must be expressible as a sequence of 
>>> bits (the same way we have string de/serialisation — which of course is not 
>>> to say that the byte representation is good for serialisation purposes). 
>>> “withUnsafeBytes” can be seen as doing a reversible type conversion the 
>>> same way LosslessStringConvertible does; only in this case the conversion 
>>> is free.
>> 
>> Yes. Byte clearly refers to a value's in-memory representation. But 
>> typealias Byte = UInt8 would imply the opposite of what needs to be 
>> conveyed. The name Byte refers to raw memory being accessed, not the value 
>> being returned by the collection. The in-memory value's bytes are loaded 
>> from memory and reinterpreted as UInt8 values. UInt8 is the correct type for 
>> the value after it is loaded. Calling the collection’s element type Byte 
>> sends the wrong message. e.g. [Byte] or UnsafePointer would be 
>> nonsense.
>> 
>> Keep in mind the important use case is code that needs to work with a 
>> collection of UInt8 values without knowing the type of the values in memory. 
>> This makes it intuitive and convenient to implement correctly without 
>> needing to reason about the Swift-specific notions of raw vs. typed pointers 
>> and binding memory to a type.
>> 
>> The documentation should be fixed to clarify that the in-memory value is not 
>> the same as the loaded value.
>> 
>> -Andy
> 
> Well, a byte is a numerical type as much as a UInt8 is. We attach meaning to 
> it (e.g. a memory location), but it’s just a number.
> 
> But I thought what Andy's saying is that he's proposing to standardize the 
> usage of the word byte to mean raw memory and not a number?

That’s right. That’s exactly how the name “bytes” is being used in APIs and 
method names. A byte is not itself a number but it is common practice to 
reinterpret a byte as a number in [0,256). IMO this isn’t a problem that needs 
to be fixed.

> Perhaps it shouldn’t be a typealias then (if the alias would have some kind 
> of impure semantics), but its own type which is exactly the same as UInt8. 
> Typing raw memory accesses with `Byte` to indicate that the number was read 
> from raw memory is a good idea for type-safety IMO.
> 
> You’d wonder if we could have initialisers for other integer types which take 
> a fixed-size array of `Byte`s - e.g. UInt16(_: [2 * Byte]). That wouldn’t 
> make as much sense with two UInt8s.

You would always go through memory to reinterpret the bits. There’s nothing 
wrong with this if you know the underlying pointer is aligned:

  

Re: [swift-evolution] [late pitch] UnsafeBytes proposal

[Andrew Trick via swift-evolution]

> On Aug 16, 2016, at 7:13 PM, Karl via swift-evolution 
>  wrote:
> 
> 
>> On 16 Aug 2016, at 01:14, David Sweeris via swift-evolution 
>> > wrote:
>> 
>>> On Aug 15, 2016, at 13:55, Michael Ilseman via swift-evolution 
>>> > wrote:
>> 
>>> 
>>> It seems like there’s a potential for confusion here, in that people may 
>>> see “UInt8” and assume there is some kind of typed-ness, even though the 
>>> whole point is that this is untyped. Adjusting the header comments slightly 
>>> might help:
>>> 
>>> 
>>> /// A non-owning view of raw memory as a collection of bytes.
>>> ///
>>> /// Reads and writes on memory via `UnsafeBytes` are untyped operations that
>>> /// do no require binding the memory to a type. These operations are 
>>> expressed 
>>> /// in terms of `UInt8`, though the underlying memory is untyped.
>>> 
>>> …
>>> 
>>> You could go even further towards hinting this fact with a `typealias Byte 
>>> = UInt8`, and use Byte throughout. But, I don’t know if that’s getting too 
>>> excessive.
>> 
>> I don't think that's too excessive at all. I might even go further and say 
>> that we should call it "Untyped" instead of "Byte", to really drive home the 
>> point (many people see "byte" and think "8-bit int", which is merely a side 
>> effect of CPUs generally not having support for types *other* than ints and 
>> floats, rather than a reflection of the true "type" of the data).
>> 
>> - Dave Sweeris
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>> 
> ‘Byte’ is sufficient, I think.
> 
> In some sense, it is typed as bytes. It reflects the fact that anything that 
> is representable to the computer must be expressible as a sequence of bits 
> (the same way we have string de/serialisation — which of course is not to say 
> that the byte representation is good for serialisation purposes). 
> “withUnsafeBytes” can be seen as doing a reversible type conversion the same 
> way LosslessStringConvertible does; only in this case the conversion is free.

Yes. Byte clearly refers to a value's in-memory representation. But typealias 
Byte = UInt8 would imply the opposite of what needs to be conveyed. The name 
Byte refers to raw memory being accessed, not the value being returned by the 
collection. The in-memory value's bytes are loaded from memory and 
reinterpreted as UInt8 values. UInt8 is the correct type for the value after it 
is loaded. Calling the collection’s element type Byte sends the wrong message. 
e.g. [Byte] or UnsafePointer would be nonsense.

Keep in mind the important use case is code that needs to work with a 
collection of UInt8 values without knowing the type of the values in memory. 
This makes it intuitive and convenient to implement correctly without needing 
to reason about the Swift-specific notions of raw vs. typed pointers and 
binding memory to a type.

The documentation should be fixed to clarify that the in-memory value is not 
the same as the loaded value.

-Andy___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [late pitch] UnsafeBytes proposal

[Andrew Trick via swift-evolution]

Thanks i'll try to clarify in the comments that The memory is read out as UInt8 
but the memory is untyped. 

Andy

> On Aug 15, 2016, at 11:55 AM, Michael Ilseman <milse...@apple.com> wrote:
> 
> It seems like there’s a potential for confusion here, in that people may see 
> “UInt8” and assume there is some kind of typed-ness, even though the whole 
> point is that this is untyped. Adjusting the header comments slightly might 
> help:
> 
> 
> /// A non-owning view of raw memory as a collection of bytes.
> ///
> /// Reads and writes on memory via `UnsafeBytes` are untyped operations that
> /// do no require binding the memory to a type. These operations are 
> expressed 
> /// in terms of `UInt8`, though the underlying memory is untyped.
> 
> …
> 
> You could go even further towards hinting this fact with a `typealias Byte = 
> UInt8`, and use Byte throughout. But, I don’t know if that’s getting too 
> excessive.
> 
> 
>>> On Aug 13, 2016, at 9:34 AM, Andrew Trick via swift-evolution 
>>> <swift-evolution@swift.org> wrote:
>>> 
>>> 
>>> On Aug 13, 2016, at 7:12 AM, Félix Cloutier <felix...@yahoo.ca> wrote:
>>> 
>>> And then, we can't really use UnsafeBufferPointer for the purpose of 
>>> UnsafeBytes because we want to expose a different API. Is that right?
>> 
>> UnsafeBufferPointer should be used in the same situation that 
>> UnsafePointer is used for any T. A view over an array of UInt8 that can 
>> bypasses release bounds checks and can interoperate with C.
>> 
>> UnsafeBufferPointer should not be used to erase the memory’s pointee 
>> type.
>> 
>> UnsafeBytes erases the pointee type and gives algorithms a collection of 
>> bytes to work with. It turns out to be an important use case that I very 
>> much want to distinguish from the UnsafeBufferPointer use case. I don’t want 
>> to present users with a false analogy to UnsafeBufferPointer.
>> 
>> -Andy
>> 
>>> 
>>>>> Le 13 août 2016 à 01:44:28, Andrew Trick <atr...@apple.com> a écrit :
>>>>> 
>>>>> 
>>>>>> On Aug 13, 2016, at 12:17 AM, Brent Royal-Gordon 
>>>>>> <br...@architechies.com> wrote:
>>>>>> 
>>>>>> On Aug 12, 2016, at 9:34 PM, Andrew Trick <atr...@apple.com> wrote:
>>>>>> 
>>>>>> That matrix is the correct starting point. UnsafeRawBufferPointer would 
>>>>>> be in the lower right. But it would be nothing more than a raw pointer 
>>>>>> with length. It wouldn’t be a collection of anything. UnsafeBytes is a 
>>>>>> powerful abstraction on top of what we just called 
>>>>>> UnsafeRawBufferPointer. It is a collection of typed elements `UInt8`. 
>>>>> 
>>>>> But how is that different from UnsafeBufferPointer? Put another way, what 
>>>>> is it about the UnsafeRawPointer -> UnsafeBytes relationship that isn't 
>>>>> true about UnsafePointer -> UnsafeBufferPointer, and that therefore 
>>>>> justifies the different name?
>>>> 
>>>> 
>>>> Giving UnsafeRawPointer a memory size doesn’t imply a collection of any 
>>>> specific type. You’re supposed to used bindMemory(to:capacity:) to get a 
>>>> collection out of it. Giving UnsafeBytes a name analogous to 
>>>> UnsafeBufferPointer only exposes that subtle difference, which is actually 
>>>> irrelevant. In the common case, users don’t need to know how 
>>>> UnsafeRawPointer works, so why start with that analogy?
>>>> 
>>>> The use cases justify the name. `UnsafeBytes` is what developers have been 
>>>> trying to get all along with `UnsafeBufferPointer`. The concept 
>>>> already exists to developers, but we have failed to give them a distinct, 
>>>> simple, and intuitive name for it, not to mention a correct implementation.
>>>> 
>>>> -Andy
>>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution