Re: [swift-users] Need some help understanding test behavior.

[Andrew Trick via swift-users]

This sounds like a question for the swift-dev list, directed toward whomever 
has been working on linux/powerpc support...

> On Dec 14, 2017, at 3:25 AM, Atul Sowani via swift-users 
>  wrote:
> 
> Hi,
> 
> I am trying to understand behavior of Swift test case Sanitizers/tsan.swift. 
> When I run Swift test suite, this test case generates corresponding test 
> script as follows:
> 
> set -o pipefail;{ 
> /root/swift-source/buildbot_incremental/swift-linux-powerpc64le/bin/swiftc 
> -target powerpc64le-unknown-linux-gnu  -module-cache-path 
> '/tmp/swift-testsuite-clang-module-cacheGEeVqc' 
> /root/swift-source/swift/test/Sanitizers/tsan.swift -target 
> powerpc64le-unknown-linux-gnu -g -sanitize=thread -o 
> /root/swift-source/buildbot_incremental/swift-linux-powerpc64le/test-linux-powerpc64le/Sanitizers/Output/tsan.swift.tmp_tsan-binary;
>  } &&
> { not env TSAN_OPTIONS="abort_on_error=0"  
> /root/swift-source/buildbot_incremental/swift-linux-powerpc64le/test-linux-powerpc64le/Sanitizers/Output/tsan.swift.tmp_tsan-binary
>  2>&1 | /root/swift-source/swift/utils/PathSanitizingFileCheck --sanitize 
> 'BUILD_DIR=/root/swift-source/buildbot_incremental/swift-linux-powerpc64le' 
> --sanitize 'SOURCE_DIR=/root/swift-source/swift' --use-filecheck 
> '/root/swift-source/buildbot_incremental/llvm-linux-powerpc64le/./bin/FileCheck'
>  /root/swift-source/swift/test/Sanitizers/tsan.swift; }
> 
> The first line in this script is failing (actually crashing) on my machine 
> and I am trying to analyze it. What I have seen is this first line generates 
> following command:
> 
> /root/swift-source/buildbot_incremental/swift-linux-powerpc64le/bin/swift 
> -frontend -c -primary-file 
> /root/swift-source/swift/test/Sanitizers/tsan.swift -target 
> powerpc64le-unknown-linux-gnu -disable-objc-interop -g -module-cache-path 
> /tmp/swift-testsuite-clang-module-cacheGEeVqc -sanitize=thread 
> -emit-module-doc-path /tmp/tsan-d095e5.swiftdoc -color-diagnostics 
> -module-name main -emit-module-path /tmp/tsan-d095e5.swiftmodule -o 
> /tmp/tsan-d095e5.o
> 
> This crashes bitterly. Having extremely new with Swift, I was trying to toy 
> with this command and found that if I omit "-frontend -c -primary-file" 
> arguments and keep everything else same, it just works. So effectively I am 
> using this "modified" command:

If you want to see the full command line from a “swiftc” invocation, append 
“-###” to the command line.

I don’t know what effect -primary-file would have when compiling a single 
source.

> /root/swift-source/buildbot_incremental/swift-linux-powerpc64le/bin/swift 
> /root/swift-source/swift/test/Sanitizers/tsan.swift -target 
> powerpc64le-unknown-linux-gnu -disable-objc-interop -g -module-cache-path 
> /tmp/swift-testsuite-clang-module-cacheGEeVqc -sanitize=thread 
> -emit-module-doc-path /tmp/tsan-d095e5.swiftdoc -color-diagnostics 
> -module-name main -emit-module-path /tmp/tsan-d095e5.swiftmodule -o 
> /tmp/tsan-d095e5.o
> 
> As per my understanding so far, -frontend -c is invoking "C" frontend, I 
> don't know much about the "-primary_file". By removing "-frontend -c" I am 
> probably suppressing invocation of "C" frontend. But they what is used to 
> complete the execution?

“-c” just means “compile" down to a .o file. The language will be inferred from 
the file extension I believe.

> 
> I would like to know if tweaking makes any sense first of all, and if so, how 
> can I go about debugging this issue?

I have no idea what could be going wrong since I don’t work on tsan or linux, 
but I suspect that the compile step is succeeding and the link step is failing 
(-### will also show you the linker command line). The linker should tell you 
what symbol is missing. Then you can compare the symbols to those the x86 build 
provides and find out why they’re not available on powerpc.

-Andy

> 
> Thanks,
> Atul.
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Tuple of two or more instances to withExtendedLifetime

[Andrew Trick via swift-users]

Yes, withExtendedLifetime is a generic API that guarantees the lifetime of any 
non-trivial value. i.e. if the tuple contains a reference, the lifetime of that 
reference will be extended.

That said, there is a theoretical bug in the optimizer that you will be much 
more likely to hit in practice if you pass a tuple to this API. I still think 
it’s extremely unlikely that the  code will be miscompiled though, and I’m sure 
it will be fixed shortly ;)

https://bugs.swift.org/browse/SR-6608: DeadCodeElimination removes fix_lifetime

-Andy

> On Dec 13, 2017, at 10:05 AM, Jordan Rose  wrote:
> 
> Good question and good thought! I believe the answer is yes, this will work 
> as intended, but I'd like to get someone more versed in SIL than me to verify 
> that. Andy?
> 
> Jordan
> 
> 
>> On Dec 13, 2017, at 01:28, Jens Persson via swift-users 
>>  wrote:
>> 
>> There are no restrictions on the type of the first argument to 
>> withExtendedLiftetime, even though it makes sense only for reference types.
>> 
>> So both the below variants will compile, and my question is if the first one 
>> will work as expected (extending the lifetime of both a and b) or if the 
>> second variant must be used instead.
>> 
>> 1:
>> withExtendedLifetime((a, b)) {
>>...
>> }
>> 
>> 2:
>> withExtendedLifetime(a) {
>>withExtendedLifetime(b) {
>>...
>>}
>> }
>> 
>> /Jens
>> 
>> ___
>> swift-users mailing list
>> swift-users@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-users
> 

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Why does the withUnsafeMutableBufferPointer closure take an inout parameter?

[Andrew Trick via swift-users]

> On Oct 12, 2017, at 3:52 AM, Geordie Jay via swift-users 
>  wrote:
> 
> 
> Guillaume Lessard via swift-users  > schrieb am Mi. 11. Okt. 2017 um 23:49:
> A lot of the MutableCollection implementation is in protocol extensions 
> (swapAt, for example.)
> 
> Should an additional version be written just for the Unsafe*BufferPointer 
> types?
> 
> Makes sense to me, given the examples above. It doesn’t seem to be a high 
> priority task though, and one one suited to a contributor pull request rather 
> than taking resources away from the core team.
> 
> Would this kind of change need to go through swift evolution or is it a “no 
> brainer”?
> 
> Geordie

I’ll just point out that it’s already the case that methods defined in 
Unsafe*BufferPointer that write to memory are “nonmutating”. So I think it’s 
both a “no brainer” and needs to go through evolution.

-Andy

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Memory Address of value types and reference types

[Andrew Trick via swift-users]

> On Sep 12, 2017, at 9:55 AM, somu subscribe via swift-users 
>  wrote:
> 
> Hi Quinn,
> 
> Thanks for the reply,
> 
> It is an iOS Swift project (uses Foundation, UIKit, CloudKit and other native 
> frameworks) in which I would like to check the memory address for debugging 
> (and out of enthusiasm). There is no C code I am using.
> 
> I have some asynchronous call back functions from CloudKit frameworks which 
> return CKUserIdentity objects.
> 
> So thought it would be nice if I could print the memory address of 
> CKUserIdentity objects and to check if it was unique.
> 
> And there are some other custom Swift Structs which I would like to know the 
> memory address of.
> 
> Thanks and regards,
> Muthu

For classes, use the Unmanaged API as Quinn’s suggested.

Your structs, tuples, and enums only have an address during mutation. So, for 
example, if you wrap all of your code in a function that takes the variable 
`inout`, you’ll see a consistent address within a single call to that function. 
There’s an implicit cast from `inout` to Unsafe[Mutable]Pointer arguments, so 
you can inspect the pointer value...

func foo(p: Unsafe[Mutable]Pointer) { print(p) }

foo()

As you noticed, between calls to `foo` you could see a different address.

If you really want to give your structs an “identity” you would need to wrap 
them in a class.

-Andy

>> On 12 Sep 2017, at 10:35 PM, Quinn The Eskimo! via swift-users 
>>  wrote:
>> 
>> 
>> On 12 Sep 2017, at 13:44, somu subscribe via swift-users 
>>  wrote:
>> 
>>> 1. Is the above shown the correct way to get reference type memory address ?
>>> 2. What Is it the correct way to get value type memory address ?
>> 
>> It’s hard to answer that without knowing how you’re intended to use these 
>> techniques.  If you can explain more about where you’re going with this, I 
>> should be able to make some specific recommendations.
>> 
>> For example, if you’re goal were to pass a Swift object to a C API that 
>> takes a callback function pointer and a ‘cookie’ value, and hen calls that 
>> function with that cookie (like the `qsort_r` function), the to/from opaque 
>> mechanism provider by `Unmanaged` is the recommended way to pass a Swift 
>> object through such a cookie value.
>> 
>>> 3. Is it possible to get the memory address of immutable value type 
>>> (declared as let)
>> 
>> No, because such values don’t necessarily exist in memory.
>> 
>> Share and Enjoy
>> --
>> Quinn "The Eskimo!"
>> Apple Developer Relations, Developer Technical Support, Core OS/Hardware
>> 
>> 
>> ___
>> swift-users mailing list
>> swift-users@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-users
> 
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Law of Exclusivity runtime false positive?

[Andrew Trick via swift-users]

> On Jul 30, 2017, at 5:11 AM, David Hart via swift-users 
>  wrote:
> 
>> 
>> On 28 Jul 2017, at 18:55, Joe Groff > > wrote:
>> 
>>> 
>>> On Jul 28, 2017, at 12:06 AM, David Hart via swift-users 
>>> > wrote:
>>> 
>>> Hello,
>>> 
>>> Indeed, I had reduced the code too much. John McCall was kind enough to 
>>> have a look and here’s the offending code:
>>> 
>>> func layoutHorizontally(leftRect: inout CGRect, rightRect: inout CGRect) {
>>> let totalWidth = imageRect.size.width + titleRect.size.width + 
>>> contentSpacing
>>> rightRect.origin.x = leftRect.maxX + contentSpacing
>>> }
>>> 
>>> The problem is that imageRect and titleRect are referenced both directly 
>>> and in the inout parameters.
>>> 
>>> But there’s something I’m not understanding. I went back to the ownership 
>>> manifesto and re-read the law of exclusivity to make sure I understand it:
>>> 
>>> If a storage reference expression evaluates to a storage reference that is 
>>> implemented by a variable, then the formal access duration of that access 
>>> may not overlap the formal access duration of any other access to the same 
>>> variable unless both accesses are reads.
>>> 
>>> So I tried to write a test to trigger the runtime error, but was 
>>> unsuccessful:
>>> 
>>> class MyClass {
>>> var p: CGRect = .zero
>>> }
>>> 
>>> func trigger(a: MyClass, b: MyClass) {
>>> a.p = b.p
>>> }
>>> 
>>> let m = MyClass()
>>> trigger(a: m, b: m)
>>> 
>>> Here, a storage reference expression (a.p) evaluates to a storage reference 
>>> that is implemented by a variable (the p property of an instance m of 
>>> MyClass) and its formal access duration (the trigger function) overlaps the 
>>> formal access duration of another access to the same variable (through the 
>>> b.p storage reference expression) and both accesses are not reads (a.p is 
>>> on the LHS of an assignment).
>>> 
>>> Why does this not trigger the Law of Exclusivity?
>> 
>> `b.p` is loaded before `a.p` is written to, so the accesses do not overlap 
>> even when a === b. Something like swap(, ) (using the Swift 3 global 
>> definition of 'swap') would trigger an exclusivity trap when a === b, since 
>> the access to storage passed as an inout argument needs to last for the 
>> duration of the call.
> 
> Thanks for the explanation! It’s starting to make sense :) Is it possible to 
> trigger the exclusivity trap without an inout argument?
> 
>> -Joe


Users are probably more likely to run into problems with mutating methods, 
which are implicitly `inout`: 

struct S {
  var x = 1

  mutating func updateX(_ getOtherS: ()->S) {
x += getOtherS().x
  }
}

class C {
  var s = S()
}

let c = C()
c.s.updateX( { c.s } )

-Andy___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Passing Data to a f(void *) function

[Andrew Trick via swift-users]

> On Jun 30, 2017, at 9:14 AM, Joe Groff via swift-users 
>  wrote:
> 
> 
>> On Jun 30, 2017, at 7:40 AM, Martin R via swift-users 
>>  wrote:
>> 
>> I have a C function 
>> 
>>   void myfunc(const void *ptr);
>> 
>> which is imported to Swift as
>> 
>>   func myfunc(_ ptr: UnsafeRawPointer!)
>> 
>> This compiles and runs without problems:
>> 
>>   let data = Data(bytes: [1, 2, 3, 4])
>>   data.withUnsafeBytes { (ptr) in myfunc(ptr) } // (A)
>> 
>> and the type of `ptr` is inferred as `UnsafePointer`. But adding an 
>> explicit type
>> annotation produces a compiler warning:
>> 
>>   data.withUnsafeBytes { (ptr: UnsafePointer) in myfunc(ptr) } // (B)
>>   // warning: UnsafePointer has been replaced by UnsafeRawPointer
>> 
>> which is understandable in the view of "SE-0107 UnsafeRawPointer API".
>> 
>> The "Fix-it" replaces `UnsafePointer` by `UnsafeRawPointer`, and that 
>> does not
>> compile anymore:
>> 
>>   data.withUnsafeBytes { (ptr: UnsafeRawPointer) in myfunc(ptr) } // (C)
>>   // error: cannot convert value of type 'Void' to closure result type '_'
>> 
>> because there is no `withUnsafeBytes()` method taking a 
>> `(UnsafeRawPointer)->ResultType`
>> closure.
> 
> This is a bug. The Data API ought to be using UnsafeRawPointer here.
> 
> -Joe

I was going for UnsafeRawBufferPointer. Here’s a prototype from Sept ‘16:
https://github.com/atrick/swift/commit/796b08fa5635a7d61cbb6bf0718178e0ce326e4f

I created a JIRA to track this and related issues: 
https://bugs.swift.org/browse/SR-5363

-Andy
___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Ugliness bridging Swift String to char *

[Andrew Trick via swift-users]

> On Mar 1, 2017, at 7:08 PM, Kenny Leung via swift-users 
>  wrote:
> 
> Hi Jordan.
> 
> Thanks for the lengthy answer.
> 
>> On Mar 1, 2017, at 6:21 PM, Jordan Rose > > wrote:
>> 
>> Hey, Kenny. The const vs non-const part is important, since both the 
>> implicit conversion and cString(using:) are allowed to return a pointer to 
>> the internal data being used by the String, and modifying that would be 
>> breaking the rules (and could potentially cause a crash). In the case of 
>> 'fieldSep', it's unlikely that anyone is going to mutate the contents of the 
>> string; it was probably just the original author (you?) not bothering to be 
>> const-correct. If you control this struct, a better fix would be to use 
>> 'const char *' for the field.
> 
> This is the PostgreSQL client library, so I don’t really want to change it. 
> (Although the source is available. Maybe I should submit a patch…)
> 
>> That said, that's not the main isuse. The implicit conversion from String to 
>> UnsafePointer is only valid when the string is used as a function 
>> argument, because the conversion might need to allocate temporary storage. 
>> In that case, it’s important to know when it’s safe to deallocate that 
>> storage. For a function call, that’s when the call returns, but for storing 
>> into a struct field it’s completely unbounded. So there’s no implicit 
>> conversion there.
>> 
>> If you’re willing to limit your use of the pointer value to a single block 
>> of code, you can use withCString 
>> :
>> 
>> myString.withCString {
>>   var opt :PQprintOpt = PQprintOpt()
>>   opt.fieldSep = UnsafeMutablePointer(mutating: $0)
>>   // use 'opt'
>> }
> 
> Unfortunately, this is not always an option, since there are multiple char * 
> files in PQprintOpt. Or could I just nest .withCString calls? Looks ugly, but 
> might work, I guess.

Here are some basic recommendations for converting between String and C string 
representations:
https://swift.org/migration-guide/se-0107-migrate.html#common-use-cases

The best ways to convert Swift to C strings are:

1. Pass the Swift string as a function argument of type
   Unsafe[Mutable]Pointer.

2. Use `String.withCString` to create a block of Swift code that can
   access the C string.

That doesn't help you if you want to store a pointer to the CString in a 
property without defining its scope. In that case, you just need to explicitly 
copy the string.

I like Jordan's recommendation for calling strdup. That's the canonical way of 
creating a new C string with its own lifetime.

Guillaume's explanation with withMemoryRebound(to:) is also correct, if you 
want to work at that level.

We probably should provide an API that handles the special case of String 
literals so you don't need to copy. As Jordan suggested, that we could simply 
add a `cstring` property to StaticString. Feel free to file a bug for that so 
it isn't forgotten.

-Andy___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] unsafeBitCast to Unimplemented Class

[Andrew Trick via swift-users]

> On Feb 6, 2017, at 8:51 PM, Dave Abrahams  wrote:
> 
> 
> on Mon Feb 06 2017, Andrew Trick  wrote:
> 
>> Is a missing declaration a use case that needs to be supported?
> 
> I couldn't say.
> 
>> Wouldn’t it be more proper to use selector based dispatch in those
>> cases?
> 
> Example, please?  I don't know what that means, though I probably should.
> 
> -- 
> -Dave

I phrased that as a question because I'm the last person who should be giving 
advice here... What I had in mind is this:

if ([self isKindOfClass:NSClassFromString(@“Bar”)]) {
  self.perform(@selector(FakeBarProtocol.foo))
}

It's not type safe, but it's a lot better than outright lying about the 
reference's dynamic type.

Of course, the root problem is that Bar's declaration is unavailable, and 
that's not a normal, expected thing.

-Andy

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] unsafeBitCast to Unimplemented Class

[Andrew Trick via swift-users]

> On Feb 5, 2017, at 9:01 AM, Dave Abrahams via swift-users 
>  wrote:
> 
> 
> on Sat Feb 04 2017, Saagar Jha  > wrote:
> 
>> Thanks–your not only did you method work, it had the side effect of
>> obviating the need for a Bridging Header. 
> 
> Uh, wait: this doesn't add up.  If you needed a bridging header before,
> surely it was so that you could get a declaration for Bar?  If that's
> the case, you shouldn't be using this protocol hack.
> 
>> One more thing: what happens if self isn’t a Bar (crash, I’m
>> guessing)? 
> 
> No, it should just be using ObjC method dispatch (a.k.a. duck-typing)
> under the covers.  If you have a method with the right signature, things
> just work.

I don’t want to encourage reinterpreting a reference to `Bar` as a reference to 
`BarProtocol`, when `Bar` does not conform to the protocol according to the 
type system. Yes, it works for ObjC dispatch, and it’s done as an internal 
workaround in a couple places, but I would hesitate to say that’s it’s 
generally supported.

Is a missing declaration a use case that needs to be supported? Wouldn’t it be 
more proper to use selector based dispatch in those cases?

-Andy

>> Is there a way to compare the type of self, other than using `is`
>> (which has the same issue as unsafeBitCast in that I don’t have the
>> declaration for it)?
> 
> I think you might be able to use something like 
> 
>  class_getName(type(of: x))

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] isKnownUniquelyReferenced really a mutating function?

[Andrew Trick via swift-users]

> On Jan 25, 2017, at 10:53 AM, Edward Connell via swift-users 
>  wrote:
> 
> I am implementing a custom Array type for a distributed memory system.
> 
> I have a DataView struct which represents the array to the user as a value 
> type, and a backing store class object referenced by the DataView that does 
> all the replication. Write accesses are mutating so they don't cause a 
> problem.
> 
> However read only accesses are not mutating and there lies the problem. If 
> the storage object is uniquely referenced, then I can do my business without 
> taking a synchronization lock, otherwise I need to take a lock before syncing 
> memory.
> 
> I can "always" take a lock to work around this, but most of the time it isn't 
> necessary and I care about performance.
> 
> I think there should be a "read only pass by reference"
> 
> I just pulled this comment from the source code. I was under the impression 
> that isKnownUniquelyReferenced is thread safe, but the comments implies that 
> it isn't??

If you implement the copy-on-write backing store correctly, as done with 
ContiguousArray, then you don’t need to worry about multiple threads sharing 
the same storage. That’s not a race because none of the threads can mutate the 
storage without first making a local copy.

So, if you pass the DataView object off to another thread by value, then you’ll 
be fine. Each thread will have it’s own logical copy of the data. 
`isUniquelyReferenced` will return `false` for any thread that has a copy.

Now, if the DataView struct is itself a shared object, either by virtue of 
being stored in a global or a class property, then you have to worry about 
races. Any thread calling `isUniquelyReferenced` on that object should have 
exclusive access to the object at that time. In other words, 
`isUniquelyReferenced` should be thought of as a “write” to the location that 
stores the reference to your storage. In that sense, `inout` is again 
semantically correct even though it doesn’t change the value.

It sounds like you want to use the refcount of the data storage as 
synchronization on the data value itself which doesn’t work. If the data value 
is in a class property, that class  needs its own synchronization. Or just 
always pass DataView by value and don’t store it in shared class properties.

Hope that makes sense!

-Andy

> 
> --
> /// If the instance passed as `object` is being accessed by multiple threads
> /// simultaneously, this function may still return `true`. Therefore, you must
> /// only call this function from mutating methods with appropriate thread
> /// synchronization. That will ensure that `isKnownUniquelyReferenced(_:)`
> /// only returns `true` when there is really one accessor, or when there is a
> /// race condition, which is already undefined behavior.
> ///
> /// - Parameter object: An instance of a class. This function does *not* 
> modify
> ///   `object`; the use of `inout` is an implementation artifact.
> 
> 
> 
> On Wed, Jan 25, 2017 at 10:31 AM, Joe Groff  > wrote:
> 
> > On Jan 25, 2017, at 10:20 AM, Edward Connell via swift-users 
> > > wrote:
> >
> > I have a data structure that calls isKnownUniquelyReferenced on a member. 
> > It forces everything to be marked as mutating because of the inout 
> > parameter, however the parameter is never mutated, it is just read right?? 
> > The reason it is inout is because a read only reference is required.
> >
> > If it is truly not mutating, is there some way around this so I don't have 
> > to mark everything in the caller chain as mutating also? It's kind of 
> > annoying...
> 
> In Swift's current model, `isKnownUniquelyReferenced` needs to be inout 
> because that's currently the only way to assert unique access to an object in 
> memory, since read-only rvalues are otherwise freely copyable so the result 
> of the uniqueness check would be only momentarily valid at best. What are you 
> trying to do that requires using it on a nonmutable value? There may be 
> another way to go about it.
> 
> -Joe
> 
> 
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] How much memory does withMemoryRebound bind

[Andrew Trick via swift-users]

> On Dec 29, 2016, at 2:03 PM, Guillaume Lessard via swift-users 
>  wrote:
> 
> Hi Etan,
> 
> `withMemoryRebound` does not copy memory.
> The proposal for UnsafeRawPointer contains information about the memory model 
> (as related to pointers):
> https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md
> 
> (also, the method is defined in the following file:
> https://github.com/apple/swift/blob/master/stdlib/public/core/UnsafePointer.swift.gyb)
> 
> The capacity parameter of withMemoryRebound allows rebinding a contiguous 
> buffer at once; it might be nice if it had a default value of 1.
> 
> Cheers,
> Guillaume Lessard

Yep. To further reassure you, `withMemoryRebound` could not be implemented as a 
copy without breaking various semantics.

Specifically, `forwardPointer` below is semantically equivalent to calling `f` 
directly.

func forwardPointer(_ p: UnsafePointer, to f: (UnsafePointer) -> 
()) {
  p.withMemoryRebound(to: Int32.self, capacity: 1) {
$0.withMemoryRebound(to: Int64.self, capacity: 1) {
  f($0)
}
  }
}

[For an instant, the high bytes are initialized to untyped raw bits. The 
documentation skirts around this case, but the model is consistent and 
verifiable.]

The `capacity` label serves to distinguish this cast from the familiar C 
pointer cast where it’s customary to dereference multiple elements from the 
resulting typed pointer without specifying the array size. A default capacity=1 
would be convenient but misleading.

-Andy
___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Atomics and Memory Fences in Swift

[Andrew Trick via swift-users]

> On Nov 30, 2016, at 5:40 AM, Anders Ha via swift-users 
>  wrote:
> 
> Hi guys
> 
> I have recently started adopting lock-free atomics with memory fences, but it 
> seems Swift at this moment does not have any native instruments.
> 
> Then I read a thread in the Apple Developer Forum 
> (https://forums.developer.apple.com/thread/49334 
> ), which an Apple staff 
> claimed that all imported atomic operations are "not guaranteed to be 
> atomic". But for my tests with all optimizations enabled (-Owholemodule and 
> -O), the OSAtomic primitives and stdatomic fences do not seem going wild.
> 
> Is these `atomic_*` and `OSAtomic*` primitives really unsafe in Swift as 
> claimed? It doesn't seem like the Swift compiler would reorder memory 
> accesses around a C function call that it wouldn't be able to see through.

Did you get an answer to this? I’m not sure what led you to believe the 
primitives are unsafe in Swift. Importing them doesn’t change their semantics.

-Andy

> P.S. Is any of these primitives available on Linux? It seems the glibc 
> modulemap does not export an `stdatomic` submodule at all.
> 
> Best Regards,
> Anders
> 
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 4, 2016, at 12:16 AM, Rien  wrote:
> 
> Thanks Any, most informative.
> 
> So the pointer “gateway’s” are in fact ephemeral. That is good for 
> performance.
> 
> As to the low level interfaces, are you aware of any effort that addresses 
> the POSIX socket functions?
> (Things like ‘addrinfo')

I've seen some discussion on mailing lists and forums.  I included a little 
blurb in the 3.0 migration guide.  Search for socket API helpers in this page. 

https://swift.org/migration-guide/se-0107-migrate.html#common-use-cases

Andy

> 
> Regards,
> Rien
> 
> Site: http://balancingrock.nl
> Blog: http://swiftrien.blogspot.com
> Github: http://github.com/Swiftrien
> Project: http://swiftfire.nl
> 
> 
> 
> 
>>> On 04 Nov 2016, at 06:24, Andrew Trick  wrote:
>>> 
>>> 
 On Nov 3, 2016, at 7:41 AM, Rien via swift-users  
 wrote:
 
 On 03 Nov 2016, at 15:16, Manfred Schubert via swift-users 
  wrote:
 
 
> Am 02.11.2016 um 18:37 schrieb Rien :
> 
>>> 
>>> var rawPtr = UnsafeMutableRawPointer.allocate(bytes: 2, alignedTo: 0)
>>> 
>>> var widePtr = rawPtr.bindMemory(to: Int16.self, capacity: 1)
>>> 
>>> widePtr.pointee = 32
>>> 
>>> var narrowPtr = rawPtr.bindMemory(to: UInt8.self, capacity: 2)
>>> 
>>> narrowPtr[0] = 16
>>> narrowPtr[1] = 255
>>> 
>>> print(widePtr.pointee)
>> 
>> This compiles and runs as expected, but it should not be allowed if I 
>> understand things correctly. So shouldn’t it be a compile time error or 
>> crash at runtime? If not, what do I get over how it was before where I 
>> was casting to a typed pointer?
> 
> Why do you think it should not be allowed.
> AFAICS everything is correct.
 
 If I understand the documentation correctly, this should not be allowed, 
 because it’s not allowed to access memory as different types at the same 
 time. It needs to be „bound“ to the type first, and can only be bound to 
 one type at a time, so all access as another type must be encapsulated 
 within a closure.
 
>>> 
>>> Ah, but that is not the case.
>>> 
>>> It is important to differentiate between the “gateway” to the memory and 
>>> the memory area itself.
>>> Different programming languages/compilers have different approaches, but I 
>>> believe that Swift allocates a struct for every gateway.
>>> widePtr and narrowPtr are two different gateways. They refer to different 
>>> struct's. But the struct for each of them refers to the same memory area.
>> 
>> In the Swift memory model, a pointer value is substitutable with any other 
>> pointer of the same value. And for efficiency, pointer values are 
>> implemented as addresses. This was an intentional design decision. For 
>> example, this is well-defined:
>> 
>> func foo(rawPtr: UnsafeRawPointer, ptrT: UnsafePointer) {
>>  if rawPtr == UnsafeRawPointer(ptrT) {
>>assert(rawPtr.assumingMemoryBound(to: T.self).pointee == ptrT.pointee)
>>  }
>> }
>> 
>> Note that assumingMemoryBound(to:) is essentially a nice way of doing 
>> unsafeBitCast, but intentional and verifiable.
>> 
>>> Example: widePtr can be located at address 0x1000, narrowPtr can be 
>>> allocated at address 0x2000 while the memory area both refer to (the raw 
>>> memory) can be at address 0x3000
>> 
>> Swift strict aliasing means that unrelated typed accesses to memory cannot 
>> overlap in their underlying raw memory.
>> 
>> Our UnsafePointer value representation is not realistically ever going to 
>> depend on the Pointee type.
>> 
>>> Every access to the raw memory via a gateway must follow the rules for that 
>>> gateway.
>> 
>>> Note: Some compilers may only create ephemeral gateways, but as long as the 
>>> compiler checks that the acces follows the correct rules, that is not a 
>>> problem.
>>> 
>>> This is not only very useful, but it also opens the door to better 
>>> interfaces to some low level Unix APIs.
>> 
>> I think we would want a Swift interface on top of those APIs that doesn’t 
>> rely on type punning.
>> 
>> A Swift pointer value itself doesn’t provide a gateway. It is really 
>> bindMemory(to:capacity:) or withMemoryRebound(to:capacity:) that control 
>> whether typed access is well-defined at some point in the program.
>> 
>> In native Swift code, interacting with the memory binding APIs will be 
>> extremely rare. Type punning is not a “normal” activity outside of use cases 
>> like network I/O and binary file formats. In those cases it probably makes 
>> more sense to use a raw pointer directly rather than binding memory to a 
>> type.
>> 
>> -Andy
>> 
>>> 
>>> 
 
 Manfred
 ___
 swift-users mailing list
 swift-users@swift.org
 https://lists.swift.org/mailman/listinfo/swift-users
>>> 
>>> Regards,
>>> Rien
>>> 
>>> Site: 

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 3, 2016, at 7:41 AM, Rien via swift-users  
> wrote:
> 
>> On 03 Nov 2016, at 15:16, Manfred Schubert via swift-users 
>> > wrote:
>> 
>> 
>>> Am 02.11.2016 um 18:37 schrieb Rien >> >:
>>> 
> 
> var rawPtr = UnsafeMutableRawPointer.allocate(bytes: 2, alignedTo: 0)
> 
> var widePtr = rawPtr.bindMemory(to: Int16.self, capacity: 1)
> 
> widePtr.pointee = 32
> 
> var narrowPtr = rawPtr.bindMemory(to: UInt8.self, capacity: 2)
> 
> narrowPtr[0] = 16
> narrowPtr[1] = 255
> 
> print(widePtr.pointee)
 
 This compiles and runs as expected, but it should not be allowed if I 
 understand things correctly. So shouldn’t it be a compile time error or 
 crash at runtime? If not, what do I get over how it was before where I was 
 casting to a typed pointer?
>>> 
>>> Why do you think it should not be allowed.
>>> AFAICS everything is correct.
>> 
>> If I understand the documentation correctly, this should not be allowed, 
>> because it’s not allowed to access memory as different types at the same 
>> time. It needs to be „bound“ to the type first, and can only be bound to one 
>> type at a time, so all access as another type must be encapsulated within a 
>> closure.
>> 
> 
> Ah, but that is not the case.
> 
> It is important to differentiate between the “gateway” to the memory and the 
> memory area itself.
> Different programming languages/compilers have different approaches, but I 
> believe that Swift allocates a struct for every gateway.
> widePtr and narrowPtr are two different gateways. They refer to different 
> struct's. But the struct for each of them refers to the same memory area.

In the Swift memory model, a pointer value is substitutable with any other 
pointer of the same value. And for efficiency, pointer values are implemented 
as addresses. This was an intentional design decision. For example, this is 
well-defined:

func foo(rawPtr: UnsafeRawPointer, ptrT: UnsafePointer) {
  if rawPtr == UnsafeRawPointer(ptrT) {
assert(rawPtr.assumingMemoryBound(to: T.self).pointee == ptrT.pointee)
  }
}

Note that assumingMemoryBound(to:) is essentially a nice way of doing 
unsafeBitCast, but intentional and verifiable.

> Example: widePtr can be located at address 0x1000, narrowPtr can be allocated 
> at address 0x2000 while the memory area both refer to (the raw memory) can be 
> at address 0x3000

Swift strict aliasing means that unrelated typed accesses to memory cannot 
overlap in their underlying raw memory.

Our UnsafePointer value representation is not realistically ever going to 
depend on the Pointee type.

> Every access to the raw memory via a gateway must follow the rules for that 
> gateway.

> Note: Some compilers may only create ephemeral gateways, but as long as the 
> compiler checks that the acces follows the correct rules, that is not a 
> problem.
> 
> This is not only very useful, but it also opens the door to better interfaces 
> to some low level Unix APIs.

I think we would want a Swift interface on top of those APIs that doesn’t rely 
on type punning.

A Swift pointer value itself doesn’t provide a gateway. It is really 
bindMemory(to:capacity:) or withMemoryRebound(to:capacity:) that control 
whether typed access is well-defined at some point in the program.

In native Swift code, interacting with the memory binding APIs will be 
extremely rare. Type punning is not a “normal” activity outside of use cases 
like network I/O and binary file formats. In those cases it probably makes more 
sense to use a raw pointer directly rather than binding memory to a type.

-Andy

> 
> 
>> 
>> Manfred
>> ___
>> swift-users mailing list
>> swift-users@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-users
> 
> Regards,
> Rien
> 
> Site: http://balancingrock.nl 
> Blog: http://swiftrien.blogspot.com 
> Github: http://github.com/Swiftrien 
> Project: http://swiftfire.nl 
> 
> 
> 
> 
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 3, 2016, at 7:16 AM, Manfred Schubert via swift-users 
>  wrote:
> 
> 
>> Am 02.11.2016 um 18:37 schrieb Rien :
>> 
 
 var rawPtr = UnsafeMutableRawPointer.allocate(bytes: 2, alignedTo: 0)
 
 var widePtr = rawPtr.bindMemory(to: Int16.self, capacity: 1)
 
 widePtr.pointee = 32
 
 var narrowPtr = rawPtr.bindMemory(to: UInt8.self, capacity: 2)
 
 narrowPtr[0] = 16
 narrowPtr[1] = 255
 
 print(widePtr.pointee)
>>> 
>>> This compiles and runs as expected, but it should not be allowed if I 
>>> understand things correctly. So shouldn’t it be a compile time error or 
>>> crash at runtime? If not, what do I get over how it was before where I was 
>>> casting to a typed pointer?
>> 
>> Why do you think it should not be allowed.
>> AFAICS everything is correct.
> 
> If I understand the documentation correctly, this should not be allowed, 
> because it’s not allowed to access memory as different types at the same 
> time. It needs to be „bound“ to the type first, and can only be bound to one 
> type at a time, so all access as another type must be encapsulated within a 
> closure.

Yes, this would work

  withMemoryRebound(to: UInt16.self, capacity: 1) { print($0.pointee) }

So would this

  rawPtr.bindMemory(to: Int16.self, capacity: 1)
  print(widePtr.pointee)

Or this

  rawPtr.load(as: UInt16.self)

-Andy

> 
> 
> Manfred
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 2, 2016, at 10:37 AM, Rien via swift-users  
> wrote:
> 
>>> var rawPtr = UnsafeMutableRawPointer.allocate(bytes: 2, alignedTo: 0)
>>> 
>>> var widePtr = rawPtr.bindMemory(to: Int16.self, capacity: 1)
>>> 
>>> widePtr.pointee = 32
>>> 
>>> var narrowPtr = rawPtr.bindMemory(to: UInt8.self, capacity: 2)
>>> 
>>> narrowPtr[0] = 16
>>> narrowPtr[1] = 255
>>> 
>>> print(widePtr.pointee)
>> 
>> This compiles and runs as expected, but it should not be allowed if I 
>> understand things correctly. So shouldn’t it be a compile time error or 
>> crash at runtime? If not, what do I get over how it was before where I was 
>> casting to a typed pointer?
> 
> Why do you think it should not be allowed.
> AFAICS everything is correct.
> Are you referring to the multiple interpretation of the raw memory? That is 
> entirely intentional, indeed one of the main purposes.

Allowing type punning is one of the main purposes of raw memory access (via a 
raw pointer). But in this code, widePtr.pointee is a typed UInt16 load from 
memory that is bound to UInt8. Loading a UInt16 without rebinding memory first 
requires a raw load such as:

rawPtr.load(as: UInt16.self)

-Andy___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 2, 2016, at 10:32 AM, Manfred Schubert via swift-users 
>  wrote:
> 
> Am 01.11.2016 um 21:43 schrieb Michael Ilseman :
>> 
>> This is more so a semantic distinction rather than some kind of physical 
>> operation. The memory is not altered, but all reads and writes to that 
>> memory location have to be through the “bound type”. If it’s “bound” to some 
>> type T, you must only read and write through values of type T, and not some 
>> unrelated type.
> 
> So is „binding memory to a type“ like declaring the intent to the compiler 
> that this memory is accessed as a certain type?

Yes.

>> [1] 
>> https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md#memory-model-explanation
> 
> Re-reading this, I guess I don’t even understand the need for memory being 
> initialized and deinitialized. I understand that it is useful to have a 
> defined initial value, but why is that a requirement for raw memory?

Memory is initialized whenever it holds a value. Memory does not need an 
“initial” value, so I’m not sure I understand the question.

API’s for reading memory do require that the memory be initialized. After all, 
they claim to return a valid value.

Deferring initialization is convenient for some contiguous data structures, but 
generally dealing with uninitialized memory is expected to be rare. About the 
only things you can do with uninitialized memory are initialize it, deallocate 
it, or copy it with C memcpy.

> If I receive raw memory from outside of Swift, would this already be „bound“ 
> and „initialized“? Many raw data can be interpreted in multiple ways, so it 
> cannot be bound to any particular type, but it is initialized with meaningful 
> data already.

If a C interface returns a pointer to uninitialized memory (e.g. malloc), then 
Swift code cannot assume that memory is bound to any type.

If a C interface returns a pointer to initialized memory, then it will be 
returning a typed pointer and indicating the capacity. Swift code can safely 
assume the memory is bound to that type.

There is a subtle issue here. I’m reluctant to bring it up because it really 
isn’t the user’s job to understand. But since we’re on the topic… It’s 
theoretically possible for a C interface to return a pointer to the same memory 
as different, unrelated, types via different functions. This is valid in C 
because of language-specific rules regarding aliasing and layout of structs, 
but would not otherwise be valid in Swift. For this reason, the compiler will 
be more conservative about aliasing of imported C types. Again, it’s not 
something the user needs to worry about.

> And what does deinitialize actually do? How could a row of Ints be 
> deinitialized for example? Or is it zeroing the raw bytes of the memory?

By definition, deinitializing a value of trivial type, such as Int, does 
absolutely nothing other than semantically “mark” the memory as deinitialized 
(it’s a nop in the implementation).

Deinitializing nontrivial values decrements reference counts, runs 
deinitializers, and what not.

-Andy

> Manfred
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 2, 2016, at 10:07 AM, Manfred Schubert via swift-users 
>  wrote:
> 
> Am 01.11.2016 um 21:40 schrieb Andrew Trick :
>> 
>> I’m not sure I like the “prepares the memory” language myself. Binding 
>> memory communicates to the compiler that the memory locations are safe for 
>> typed access. Nothing happens at runtime--until someone writes a type safety 
>> sanitizer.
> 
> So nothing happens at runtime, and also nothing appears to happen at compile 
> time.

Nothing observable happens at the call to ptrT = bindMemory(to: T.self, …). 
It's effect on the program is making the subsequent ptrT.pointee a well-defined 
operation.

> If I try this code:
> 
>> var rawPtr = UnsafeMutableRawPointer.allocate(bytes: 2, alignedTo: 0)
>> 
>> var widePtr = rawPtr.bindMemory(to: Int16.self, capacity: 1)
>> 
>> widePtr.pointee = 32
>> 
>> var narrowPtr = rawPtr.bindMemory(to: UInt8.self, capacity: 2)
>> 
>> narrowPtr[0] = 16
>> narrowPtr[1] = 255
>> 
>> print(widePtr.pointee)
> 
> This compiles and runs as expected, but it should not be allowed if I 
> understand things correctly. So shouldn’t it be a compile time error or crash 
> at runtime? If not, what do I get over how it was before where I was casting 
> to a typed pointer?

print(widePtr.pointee) is undefined. It may execute just the way you expect. In 
practice, it may also be reordered with the assignments to narrowPtr. Our 
optimizer is designed to do that but by chance isn’t in this version of the 
compiler. In theory anything can happen.

The real danger of undefined behavior of course is that the program will behave 
as you expect without warning when you develop the code, and future versions of 
the compiler may change the behavior in unpredictable ways.

This API will make it easier to develop static diagnostics that catch undefined 
behavior and much easier to write a runtime sanitizer. Neither of those things 
exist yet. It might be easy to catch obvious cases like this, but the value of 
catching just the obvious cases is limited.

>> It affects the abstract state of the memory location, independent of the 
>> pointer variable used to access that memory. Binding memory returns a typed 
>> pointer for convenience and clarity, but there’s nothing special about that 
>> particular pointer value.
> 
> If it were not returning a typed pointer, what would it actually do?

It would bind memory. Returning a typed pointer actually has nothing to do with 
its semantics.

That fact that it’s one of the only ways to get a typed pointer from a raw 
pointer forces the programmer to bind the memory’s type before performing typed 
access on the memory.

-Andy

> 
> 
> Manfred
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 2, 2016, at 3:50 PM, Dave Abrahams  wrote:
> 
>> The original poster seemed to have the impression that the operation
>> of binding memory itself might affect program state, 
> 
> Formally speaking, it does!

Oh boy. I keep failing at this. How does one formally distinguish between 
semantic state and actual state?
-Andy
___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] What is "binding" memory?

[Andrew Trick via swift-users]

> On Nov 1, 2016, at 11:55 AM, Manfred Schubert via swift-users 
>  wrote:
> 
> The "UnsafeRawPointer Migration" guide talks about "binding memory to a type“ 
> as if that was a well known term. I have never heard of it yet though, and 
> googling it returns no relevant results. I do not understand what binding 
> memory is supposed to do.
> 
> The migration guide says "Binding uninitialized memory to a type prepares the 
> memory to store values of that type“, but clearly raw memory does not need to 
> be prepared (and cannot be) to hold any arbitrary type and value.
> 
> So what is this for, what does it actually do, and to whom is it done (the 
> raw pointer, or the typed pointer which is returned, or the raw memory)?
> 
> 
> Manfred

Hi Manfred,

At the top of the migration guide is a link to the memory model explanation:
https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md#memory-model-explanation

"A memory location's bound type is an abstract, dynamic property of the memory 
used to formalize type safety.”

I’m not sure I like the “prepares the memory” language myself. Binding memory 
communicates to the compiler that the memory locations are safe for typed 
access. Nothing happens at runtime--until someone writes a type safety 
sanitizer. It affects the abstract state of the memory location, independent of 
the pointer variable used to access that memory. Binding memory returns a typed 
pointer for convenience and clarity, but there’s nothing special about that 
particular pointer value.

Initialized memory is always bound to some type. A rawpointer can be used to 
access that memory without knowing its bound type.

-Andy
___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] How to malloc in Swift 3

[Andrew Trick via swift-users]

> On Sep 23, 2016, at 2:23 PM, Joe Groff  wrote:
> 
> 
>> On Sep 23, 2016, at 2:20 PM, Jens Persson  wrote:
>> 
>> What is the difference between:
>> ptr.storeBytes(of: x, toByteOffset: offset, as: type(of: x))
>> ptr.advanced(by: offset).assumingMemoryBound(to: type(of: x)).pointee = x
>> ?
>> I noticed that the former traps if storing to a misaligned offset while the 
>> latter is happy to do that, and I saw it mentioned as a requirement in the 
>> documentation, but other than that I'm not sure what would be the pros and 
>> cons of using the former / latter?
> 
> cc'ing Andy, who's the expert on this. AIUI, the former does not semantically 
> bind the memory to the type being stored—informally, it has "memcpy 
> semantics"—whereas the latter *will* bind the memory to a type, which will 
> require all other loads and stores derived from the same pointer to remain of 
> the same type. Neither API officially supports unaligned loads or stores yet; 
> if one crashes and the other doesn't, that's an accident.
> 
> -Joe

storeBytes(of:as:) is an untyped memory operation. e.g. you could use it store 
a UInt32 to an Float’s location without binding memory.

assumingMemoryBound(to:) gives you a typed pointer, that you the programmer 
must guarantee is the correct type for that memory location. If you use this to 
get a UInt32 pointer into a Float’s location, you get undefined behavior as 
soon as you access the pointee.

storeBytes traps on misaligned access because memory is being reinterpreted 
making it easy to violate the alignment precondition.

UnsafePointer.pointee never checked the alignment precondition because 
normally you wouldn’t need to and it’s supposed to be zero overhead.

assumingMemoryBound(to:) does not check alignment because it isn’t undefined 
behavior until the pointer is accessed and it’s supposed to be zero overhead.

Basically assumingMemoryBound(to:) is the one backdoor that we have for force 
casting pointers. The “assuming” should clue the programmer in that they really 
need to know what they’re doing before using it.

https://swift.org/migration-guide/se-0107-migrate.html#api-for-binding-memory-types-and-pointer-conversion
 


-Andy___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Function to unsafe pointer and back

[Andrew Trick via swift-users]

> On Sep 2, 2016, at 9:49 AM, Jordan Rose  wrote:
> 
> (One place we are still weak is in converting between C function references 
> and UnsafeRawPointer—there’s no dedicated API to do this. I’m not sure we 
> have a recommendation in the rare cases when this is necessary. Andy?)

Good question. I don’t know how to do off the top of my head that without 
passing a function typed value off to C as a function pointer and returning a 
void*.

-Andy___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Function to unsafe pointer and back

[Andrew Trick via swift-users]

> On Sep 1, 2016, at 12:37 PM, Lou Zell via swift-users  
> wrote:
> 
> As to your real question, what’s your high-level goal?  Swift doesn’t really 
> do pointers to functions [1] but it does provide lots of other excellent 
> ‘treat code as data’ features.  If you can explain more about your goal, 
> perhaps we can direct you to a better path.
> ​
> Curiosity got the better of me on this one - there's no higher level goal 
> other than improved understanding.  I was playing around with function 
> currying in the repl and that's what lead me to those experiments.  
> 
> The article, and the preceding one in the series, has plenty for me to work 
> with.  Thank you!

That’s an awesome article, but I don’t think you need to understand any of it! 
I’m not an expert in this either, but here’s what I can see from your code...

It looks like you’re trying to capture the address of a function that was JIT’d 
for the purpose of evaluating a statement in the repl. That address might not 
be valid in the next statement.

Also, in the first case the UnsafePointer points to local memory that holds the 
function value. In the second case, you’re trying to load a function value from 
the address of the function body. So a level of indirection is missing.

You can’t take an address to a function in Swift. You can assign a variable to 
the function, as you did in the first case:

var x = doNothing

And view the address of that variable as an argument, only for the duration of 
the call, as you did in the first case:

call()

-Andy

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Data(bytesNoCopy:count:deallocator)

[Andrew Trick via swift-users]

> On Aug 23, 2016, at 8:40 AM, Stéphane Lizeray via swift-users 
>  wrote:
> 
> Hello,
> 
> I create an UsafeMutableRawPointer using the allocate method.
> 
> Later on I want to create a Data struct from this pointer using the 
> bytesNoCopy initializer. Which deallocator should I pass?
> 
> It looks like this:
> 
> let retPointer = UnsafeMutableRawPointer.allocate(bytes: size, alignedTo: 
> MemoryLayout.alignment)
> ….
> 
> let d = Data(bytesNoCopy: retPointer, count: 
> size,deallocator:Deallocator.free)

Hi Stéphane,

This should work:

let retPointer = UnsafeMutableRawPointer.allocate(bytes: size, alignedTo: 
MemoryLayout.alignment)

let d = Data(bytesNoCopy: retPointer, count: size, deallocator: .custom({ (ptr, 
size) in
  ptr.deallocate(bytes: size, alignedTo: 1)
}))

-Andy

> 
> Thanks,
> 
> Stéphane
> 
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Pointer conversions between different sockaddr types

[Andrew Trick via swift-users]

> On Aug 20, 2016, at 4:02 AM, Michael Ferenduros via swift-users 
>  wrote:
> 
>> 
>> On 20 Aug 2016, at 07:25, Andrew Trick > > wrote:
>> 
>>> 
>>> On Aug 19, 2016, at 4:49 PM, Michael Ferenduros via swift-users 
>>> > wrote:
>>> 
>>> 
> On Aug 18, 2016, at 12:28 AM, Quinn The Eskimo! via swift-users 
> http://swift.org/>> wrote: 
> 
> 
> On 17 Aug 2016, at 18:55, Martin R via swift-users  swift.org > wrote: 
> 
>> - Are both solutions correct, should one be preferred, or are both 
>> wrong? 
> 
> Your `withMemoryRebound` solution is correct. 
 Absolutely, withMemoryRebound is always safe. You can use it whenever you 
 just need to reinterpret memory at a call site and know the number of 
 values stored that memory location. In this case it’s “easy" because 
 you’re dealing a single sockaddr_in. The UnsafeRawPointer proposal is the 
 definitive reference 
 https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md
  
 
  
 >
  But this migration guide is more approachable… it’s still WIP: 
 https://gist.github.com/atrick/0283ae0e284610fd21ad6ed3f454a585 
 
>>> 
>>> I’m running into the same issues, which is making me wonder 
>>> withMemoryRebound - socket functions expect an UnsafePointer, but 
>>> the thing pointed to can actually be larger than a sockaddr (eg. 
>>> sockaddr_in6 or sockaddr_storage). Is withMemoryRebound safe to use to cast 
>>> pointers to differently-sized structures? In the case of 
>>> UnsafeMutablePointer it seems like you’re lying to the API about 
>>> what memory may be modified… But if withMemoryRebound isn’t about providing 
>>> that guarantee, what does it actually do vs eg. a dumb cast via 
>>> OpaquePointer?
>> 
>> Good point. I replied too hastily, but you’re about to find out why I didn’t 
>> go into details…
>> 
>> As you can see 
>> ,
>>  withMemoryRebound(to: T.self, capacity: count) actually does this:
>> - (re)binds memory to `count` `T` values
>> - executes the closure
>> - (re)binds memory to `count` `Pointee` values
>> 
>> For `count == 1` and `MemoryLayout.size < MemoryLayout.size` 
>> this is valid and safe for "compatible" structs. The precondition on 
>> `withMemoryRebound` states that `T` must be layout compatible with `Pointee`.
>>   
>>   aggregates (tuples, array storage, and structs), are layout
>>   compatible with larger aggregates of the same kind if their common
>>   elements are mutually layout compatible.
>> 
>> I should really update withMemoryRebound's precondition to read:
>> 
>>   If `count == 1` then `T` must be layout compatible with
>>   `Pointee`. If `count > 1`, then `T` must be mutually layout
>>   compatible with `Pointee`.
>> 
>> You may be wondering why it's necessary to bind and rebind memory if the 
>> structures are "compatible". In general, you could have Swift code inside 
>> the closure that accesses `sockaddr` and swift code outside that accesses 
>> `sockaddr_in6`. We don't have a formal rule that says those two types are 
>> "related", so that could be miscompiled. "Related" is a type system concept 
>> that has to do with child/subclass relationships, "compatible" is an ABI 
>> concept that has to do with in-memory representation of types.
>> 
>> That's the theory. But in practice, the optimizer is going to have to treat 
>> sockaddr and sockaddr_in6 as related types simply because they are both 
>> imported from C and are allowed to alias in C land. In fact, if they were 
>> defined in Swift they wouldn't even be compatible because their overlapping 
>> members are not mutually compatible.
>> 
>> Even if one of the types were not imported, an opaque pointer cast would 
>> still work in this particular case because Swift code is never actually 
>> dereferencing the sockaddr pointer. It's just being passed off to an 
>> external C API.
>> 
>> So, the only real reason not to use an opaque pointer cast in this case is 
>> that someone reading your code likely won't understand what makes it valid 
>> and why they can't do the same thing in their code.
> 
> Excellent, thanks for the explanation.
> 
> I just realised I was somehow misreading ‘rebound’ to imply something 
> trampoline-like which made worry about copying / indirection :)

 The whole point of that method was to avoid punning!

-Andy

___

Re: [swift-users] Pointer conversions between different sockaddr types

[Andrew Trick via swift-users]

> On Aug 19, 2016, at 4:49 PM, Michael Ferenduros via swift-users 
>  wrote:
> 
> 
>>> On Aug 18, 2016, at 12:28 AM, Quinn The Eskimo! via swift-users 
>>>  wrote: 
>>> 
>>> 
>>> On 17 Aug 2016, at 18:55, Martin R via swift-users >> swift.org> wrote: 
>>> 
 - Are both solutions correct, should one be preferred, or are both wrong? 
>>> 
>>> Your `withMemoryRebound` solution is correct. 
>> Absolutely, withMemoryRebound is always safe. You can use it whenever you 
>> just need to reinterpret memory at a call site and know the number of values 
>> stored that memory location. In this case it’s “easy" because you’re dealing 
>> a single sockaddr_in. The UnsafeRawPointer proposal is the definitive 
>> reference 
>> https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md
>>  
>> 
>>  But this migration guide is more approachable… it’s still WIP: 
>> https://gist.github.com/atrick/0283ae0e284610fd21ad6ed3f454a585
> 
> I’m running into the same issues, which is making me wonder withMemoryRebound 
> - socket functions expect an UnsafePointer, but the thing pointed 
> to can actually be larger than a sockaddr (eg. sockaddr_in6 or 
> sockaddr_storage). Is withMemoryRebound safe to use to cast pointers to 
> differently-sized structures? In the case of UnsafeMutablePointer 
> it seems like you’re lying to the API about what memory may be modified… But 
> if withMemoryRebound isn’t about providing that guarantee, what does it 
> actually do vs eg. a dumb cast via OpaquePointer?

Good point. I replied too hastily, but you’re about to find out why I didn’t go 
into details…

As you can see 
,
 withMemoryRebound(to: T.self, capacity: count) actually does this:
- (re)binds memory to `count` `T` values
- executes the closure
- (re)binds memory to `count` `Pointee` values

For `count == 1` and `MemoryLayout.size < MemoryLayout.size` this 
is valid and safe for "compatible" structs. The precondition on 
`withMemoryRebound` states that `T` must be layout compatible with `Pointee`.
  
  aggregates (tuples, array storage, and structs), are layout
  compatible with larger aggregates of the same kind if their common
  elements are mutually layout compatible.

I should really update withMemoryRebound's precondition to read:

  If `count == 1` then `T` must be layout compatible with
  `Pointee`. If `count > 1`, then `T` must be mutually layout
  compatible with `Pointee`.

You may be wondering why it's necessary to bind and rebind memory if the 
structures are "compatible". In general, you could have Swift code inside the 
closure that accesses `sockaddr` and swift code outside that accesses 
`sockaddr_in6`. We don't have a formal rule that says those two types are 
"related", so that could be miscompiled. "Related" is a type system concept 
that has to do with child/subclass relationships, "compatible" is an ABI 
concept that has to do with in-memory representation of types.

That's the theory. But in practice, the optimizer is going to have to treat 
sockaddr and sockaddr_in6 as related types simply because they are both 
imported from C and are allowed to alias in C land. In fact, if they were 
defined in Swift they wouldn't even be compatible because their overlapping 
members are not mutually compatible.

Even if one of the types were not imported, an opaque pointer cast would still 
work in this particular case because Swift code is never actually dereferencing 
the sockaddr pointer. It's just being passed off to an external C API.

So, the only real reason not to use an opaque pointer cast in this case is that 
someone reading your code likely won't understand what makes it valid and why 
they can't do the same thing in their code.

Now, what about Martin's first version:

  UnsafeRawPointer($0).assumingMemoryBound(to: sockaddr_in.self)

The code that this generates is just as safe or unsafe as your opaque pointer 
cast except that you're now making an untrue assertion about the memory type 
(per the precondition on the assumingMemoryBound API 
).
 A code verification tool--like a hypothetcial pointer sanitizer--could call 
you out on this.

 - Can the same be achieved simpler? 
>>> 
>>> Not without introducing a layer of abstraction. 
>>> 
>>> In my case I introduced an abstract `Address` type (basically a wrapper 
>>> around `sockaddr_storage`) and then added a method to that object which 
>>> calls a closure with the right parameters (actually, multiple such methods, 
>>> depending on whether I’m calling something like `connect` which takes an 
>>> address, or `getpeername`, which returns one). This approach concentrates 
>>> all the ugly in one place, making the rest of my BSD Sockets 

Re: [swift-users] Pointer conversions between different sockaddr types

[Andrew Trick via swift-users]

> On Aug 18, 2016, at 12:28 AM, Quinn The Eskimo! via swift-users 
>  wrote:
> 
> 
> On 17 Aug 2016, at 18:55, Martin R via swift-users  
> wrote:
> 
>> - Are both solutions correct, should one be preferred, or are both wrong?
> 
> Your `withMemoryRebound` solution is correct.

Absolutely, withMemoryRebound is always safe. You can use it whenever you just 
need to reinterpret memory at a call site and know the number of values stored 
that memory location. In this case it’s “easy" because you’re dealing a single 
sockaddr_in.

The UnsafeRawPointer proposal is the definitive reference
https://github.com/apple/swift-evolution/blob/master/proposals/0107-unsaferawpointer.md
 


But this migration guide is more approachable… it’s still WIP:
https://gist.github.com/atrick/0283ae0e284610fd21ad6ed3f454a585

>> - Can the same be achieved simpler?
> 
> Not without introducing a layer of abstraction.
> 
> In my case I introduced an abstract `Address` type (basically a wrapper 
> around `sockaddr_storage`) and then added a method to that object which calls 
> a closure with the right parameters (actually, multiple such methods, 
> depending on whether I’m calling something like `connect` which takes an 
> address, or `getpeername`, which returns one).  This approach concentrates 
> all the ugly in one place, making the rest of my BSD Sockets code much 
> cleaner.

This is an annoying UpdatePointer migration case because it falls under the 
category of misbehaving C APIs that we deliberately don't want to encourage in 
Swift.

The only good answer is to provide a Swift wrapper on top of the socket API as 
Quinn has done. It would be nice to post that code at some point so users of 
the socket API can copy-paste into their project.

-Andy___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] NSData and UnsafePointer

[Andrew Trick via swift-users]

> On Jul 16, 2016, at 5:28 AM, J.E. Schotsman via swift-users 
>  wrote:
> 
> A mysterious bug has got me thinking about using UnsafePointer with 
> NSData (Swift 2).
> 
> Is this safe:
> 
> let data:NSData = …
> let dataStart = UnsafePointer(data:NSDAta.bytes)
> 
> myProcessdata1(dataStart,data.length)
> 
> … (no more references to data)

I don’t know what the recommended idiom is or if the syntax has changed from 
Swift 2 to 3, but I would do something like this:

withExtendedLifetime(data) {
  let dataStart = UnsafePointer(data.bytes)
  myProcessdata1(dataStart,data.length)
}

UnsafePointers aren’t meant to keep things alive.

There’s also the problem of potentially casting your UnsafePointer to something 
other than CChar, but that’s not really the issue at hand.

> And this:
> 
> let data:NSData = …
> myProcessdata2(data)
> 
> … (no more references to data)
> 
> func myProcessdata2( data:NSData )
> {
> let dataStart = UnsafePointer(data:NSData.bytes)
> myProcessdata1(dataStart,data.length)
> }
> 
> In the latter case I would hope that data remains alive until the function 
> myProcessdata2 returns. But does it?

I think the latter case has the same problems as the former.

-Andy

> 
> TIA,
> 
> Jan E.
> 
> 
> ___
> swift-users mailing list
> swift-users@swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Re: [swift-users] Unsafe(Mutable)Pointer (suc)predecessor and advancedBy functions

[Andrew Trick via swift-users]

> On May 26, 2016, at 9:59 AM, Adrian Zubarev via swift-users 
>  wrote:
> 
> I’ve got one more questions about Unsafe(Mutable)Pointer. I know that I’m 
> able to access memory that might not belong to me. 
> 
> My question is:
> 
> Can I trust these functions that they will return a pointer to some memory 
> when I allocate more than one object AND when I’m moving only inside that 
> range?
> 
> 
Yes.
> public func successor() -> UnsafeMutablePointer
> public func predecessor() -> UnsafeMutablePointer
> public func advancedBy(n: Int) -> UnsafeMutablePointer
> UnsafeMutablePointer.alloc(4) when I advance only in range of [0,1,2,3] 
> am I safe or could I get a pointer to memory that does not belong to me?
> 
> 
UnsafeMutablePointer.alloc(N) creates a single object in memory that holds N 
consecutive T values. Each value resides at index*strideof(T.self) bytes beyond 
the allocated pointer where index is valid in the range 0.. Example:
> 
> // imagine this is some memory portion,
> // where x is memory that does not belong to me
> // and 0 is moemory free to use
>  
> […, x, 0, 0, 0 x, 0, x, …]
>  
> // now I want to allocate 4 objects  
> // variant A:
>  
> […, x, MY1, MY2, MY3, x, MY4, x, …]
>  
> // my pointer will sit at `MY1` and if I advance by 2 I'll get `x`
> // can this happen to me?
>  
> // variant B:
> // Unsafe(Mutable)Pointer will ensure that I always get memory tied together  
> // (or the above functions will skip memory that doesn't belong to me??):
>  
>  
> […, x, MY1, MY2, MY3, MY4 x, …]  
> So which is right?
> 
> 
> 
> 
> -- 
> Adrian Zubarev
> Sent with Airmail
> 
> ___
> swift-users mailing list
> swift-users@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-users 
> 

___
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users