Re: [swift-evolution] Swift ABI Stability Manifesto

[John McCall via swift-evolution]

> On Jan 26, 2017, at 4:04 PM, Michael Ilseman via swift-evolution 
>  wrote:
> Weak references will most likely not be bitwise-movable on all platforms.
> 
> In order for a weak reference to be nil-ed out, the weak reference itself 
> must, conceptually, be referenced. There may be many convoluted schemes that 
> can serve as alternatives, but a straight-forward (and pragmatic) approach 
> would make the weak reference address-only. That is, something elsewhere has 
> taken and stored its address so that it can nil-out the weak reference when 
> the objects that it references is destroyed. At that point, it can’t just be 
> bitwise-moved, as that would result in a dangling reference to the weak 
> reference. One approach to implementing this is a global table mapping class 
> instance addresses to weak reference addresses that need to be nil-ed out. 
> Alternatively, the class instances could store a list of the addresses of 
> weak reference to themselves. Either way, if the address of the weak 
> reference changes, something somewhere else in memory must be updated. Thus, 
> they are not bitwise-movable.
> 
> This is not true for unowned references, which can be implemented without 
> their address being taken, e.g. by a separate ref count.

Right.  We wanted to guarantee loadability for unowned references in general 
but had to walk it back because of ObjC interop; we could still make the 
stronger guarantee on a non-ObjC platform.  But I'm fine with treating all weak 
references as memory-pinned even though our current native implementation 
doesn't technically require it.

The big issue is whether we can completely define away memory-pinned types, and 
in my mind, even if we could do that today, it would be extraordinarily 
short-sighted to assume it forever.

John.


> 
> 
>> On Jan 25, 2017, at 10:30 PM, Guillaume Lessard 
>>  wrote:
>> 
>> 
>>> On 25 janv. 2017, at 14:16, Michael Ilseman via swift-evolution 
>>>  wrote:
>>> 
>>> An example of a type that is neither trivial nor bitwise movable is a 
>>> struct containing a weak reference. Weak references are tracked in a side 
>>> table so that they can be nil-ed out when the referenced object is 
>>> destroyed. When moving an object of such type from one address to another, 
>>> the side table must be updated to refer to the weak reference's new address.
>> 
>> Isn’t this specifically a description of @objc weak references? Would weak 
>> references with no compatibility mode (e.g. on Linux) be bitwise-movable?
>> 
>> Thanks for the illuminating document!
>> 
>> Guillaume Lessard
>> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Swift ABI Stability Manifesto

[Dave Abrahams via swift-evolution]

on Thu Jan 26 2017, Michael Ilseman  wrote:

> Weak references will most likely not be bitwise-movable on all platforms.
>
> In order for a weak reference to be nil-ed out, the weak reference
> itself must, conceptually, be referenced. There may be many convoluted
> schemes that can serve as alternatives, 
> but a straight-forward (and pragmatic) approach would make the weak
> reference address-only. 

I don't think adding a level of indirection is convoluted, but it does
bring its own problems.

> That is, something elsewhere has taken and stored its address so that
> it can nil-out the weak reference when the objects that it references
> is destroyed. At that point, it can’t just be bitwise-moved, as that
> would result in a dangling reference to the weak reference. One
> approach to implementing this is a global table mapping class instance
> addresses to weak reference addresses that need to be nil-ed
> out. Alternatively, the class instances could store a list of the
> addresses of weak reference to themselves. Either way, if the address
> of the weak reference changes, something somewhere else in memory must
> be updated. Thus, they are not bitwise-movable.
>
> This is not true for unowned references, which can be implemented without 
> their address being taken,
> e.g. by a separate ref count.
>
>> On Jan 25, 2017, at 10:30 PM, Guillaume Lessard 
>>  wrote:
>> 
>> 
>>> On 25 janv. 2017, at 14:16, Michael Ilseman via swift-evolution 
>>> 
> wrote:
>>> 
>>> An example of a type that is neither trivial nor bitwise movable is
>>> a struct containing a weak reference. Weak references are tracked
>>> in a side table so that they can be nil-ed out when the referenced
>>> object is destroyed. When moving an object of such type from one
>>> address to another, the side table must be updated to refer to the
>>> weak reference's new address.
>> 
>> Isn’t this specifically a description of @objc weak references? Would weak 
>> references with no
> compatibility mode (e.g. on Linux) be bitwise-movable?
>> 
>> Thanks for the illuminating document!
>> 
>> Guillaume Lessard
>> 
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-- 
-Dave

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Swift ABI Stability Manifesto

[Michael Ilseman via swift-evolution]

Weak references will most likely not be bitwise-movable on all platforms.

In order for a weak reference to be nil-ed out, the weak reference itself must, 
conceptually, be referenced. There may be many convoluted schemes that can 
serve as alternatives, but a straight-forward (and pragmatic) approach would 
make the weak reference address-only. That is, something elsewhere has taken 
and stored its address so that it can nil-out the weak reference when the 
objects that it references is destroyed. At that point, it can’t just be 
bitwise-moved, as that would result in a dangling reference to the weak 
reference. One approach to implementing this is a global table mapping class 
instance addresses to weak reference addresses that need to be nil-ed out. 
Alternatively, the class instances could store a list of the addresses of weak 
reference to themselves. Either way, if the address of the weak reference 
changes, something somewhere else in memory must be updated. Thus, they are not 
bitwise-movable.

This is not true for unowned references, which can be implemented without their 
address being taken, e.g. by a separate ref count.


> On Jan 25, 2017, at 10:30 PM, Guillaume Lessard  
> wrote:
> 
> 
>> On 25 janv. 2017, at 14:16, Michael Ilseman via swift-evolution 
>>  wrote:
>> 
>> An example of a type that is neither trivial nor bitwise movable is a struct 
>> containing a weak reference. Weak references are tracked in a side table so 
>> that they can be nil-ed out when the referenced object is destroyed. When 
>> moving an object of such type from one address to another, the side table 
>> must be updated to refer to the weak reference's new address.
> 
> Isn’t this specifically a description of @objc weak references? Would weak 
> references with no compatibility mode (e.g. on Linux) be bitwise-movable?
> 
> Thanks for the illuminating document!
> 
> Guillaume Lessard
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] Swift ABI Stability Manifesto

[Michael Ilseman via swift-evolution]

> On Jan 26, 2017, at 3:12 AM, Dale Buckley  
> wrote:
> 
>> I put together a document compiling many conversations with many people 
>> about Swift’s ABI and what needs to happen prior to ABI stability. It is 
>> meant to be a blueprint for the project on how to approach and achieve ABI 
>> stability, as well as a resource to the community. It can be viewed fully 
>> rendered (and updated) 
>> athttps://github.com/apple/swift/blob/master/docs/ABIStabilityManifesto.md
>> 
>> Below is the raw markdown:
>> 
>> # Swift ABI Stability Manifesto
>> 
>> * Authors: [Michael Ilseman](https://github.com/milseman) (compiled through 
>> conversations with many others)
>> 
>> ## Introduction
>> 
>> ### The Big Picture
>> 
>> One of the top priorities for Swift right now is compatibility across future 
>> Swift versions. Compatibility aims at accomplishing two goals:
>> 
>> 1. *Source compatibility* means that newer compilers can compile code 
>> written in an older version of Swift. This aims to reduce the migration pain 
>> that Swift developers face when migrating to a newer Swift version. Without 
>> source compatibility, projects face version-lock where all source code in a 
>> project and its packages must be written in the same version of Swift. With 
>> source compatibility, package authors will be able to maintain a single code 
>> base across multiple Swift versions while allowing their users to use a 
>> newer version of Swift.
>> 2. *Binary framework&runtime compatibility* enables the distribution of 
>> frameworks in a binary form that works across multiple Swift versions. 
>> Binary frameworks include both a *Swift module file*, which communicates 
>> source-level information of the framework's API, and a *shared library*, 
>> which provides the compiled implementation that is loaded at runtime. Thus, 
>> there are two necessary goals for binary framework compatibility:
>> * *Module format stability* stabilizes the module file, which is the 
>> compiler's representation of the public interfaces of a framework. This 
>> includes API declarations and inlineable code. The module file is used by 
>> the compiler for necessary tasks such as type checking and code generation 
>> when compiling client code using a framework.
>> * *ABI stability* enables binary compatibility between applications and 
>> libraries compiled with different Swift versions. It is the focus of the 
>> rest of this document.
>> 
>> This document is an exploration and explanation of Swift's ABI alongside the 
>> goals and investigations needed before declaring Swift's ABI stable. It is 
>> meant to be a resource to the community as well as a declaration of the 
>> direction of Swift's ABI.
>> 
>> Throughout this document there will be references to issues in Swift's 
>> [issue tracking system](https://bugs.swift.org) denoted by "SR-". These 
>> references track open engineering and design tasks for Swift's ABI.
>> 
>> ### What Is ABI?
>> 
>> At runtime, Swift program binaries interact with other libraries and 
>> components through an ABI. ABI is Application Binary Interface, or the 
>> specification to which independently compiled binary entities must conform 
>> to be linked together and executed. These binary entities must agree on many 
>> low level details: how to call functions, how their data is represented in 
>> memory, and even where their metadata is and how to access it.
>> 
>> ABI is per-platform, as it is a low level concern influenced by both the 
>> architecture and the OS. Most platform vendors define a "standard ABI" which 
>> is used for C code and built on by C-family languages. Swift, however, is a 
>> very different language from C and has its own per-platform ABI. While most 
>> of this document is platform-agnostic, platform-specific concerns have 
>> influenced details of the design and implementation of Swift's ABI. For 
>> details on each platform's standard ABI, refer to the 
>> [Appendix](#platform-abis).
>> 
>> ### What Is ABI Stability?
>> 
>> ABI stability means locking down the ABI to the point that future compiler 
>> versions can produce binaries conforming to the stable ABI. Once an ABI is 
>> stable, it tends to persist for the rest of the platform's lifetime due to 
>> ever-increasing mutual dependencies.
>> 
>> ABI stability only affects invariants of externally visible public 
>> interfaces and symbols. Internal symbols, conventions, and layout can 
>> continue to change without breaking the ABI. For example, future compilers 
>> are free to change the calling conventions for internal function calls so 
>> long as the public interfaces are preserved.
>> 
>> Decisions about the ABI will have long-term ramifications and may limit the 
>> ways in which the language can grow and evolve in the future. Future Swift 
>> versions can add new, orthogonal aspects to the ABI, but any inefficiencies 
>> or inflexibilities present when stability is declared will (effectively) 
>> persist forever for that pl

[swift-evolution] Swift ABI Stability Manifesto

[Michael Ilseman via swift-evolution]

I put together a document compiling many conversations with many people about 
Swift’s ABI and what needs to happen prior to ABI stability. It is meant to be 
a blueprint for the project on how to approach and achieve ABI stability, as 
well as a resource to the community. It can be viewed fully rendered (and 
updated) at 
https://github.com/apple/swift/blob/master/docs/ABIStabilityManifesto.md 
 

Below is the raw markdown:

# Swift ABI Stability Manifesto

* Authors: [Michael Ilseman](https://github.com/milseman) (compiled through 
conversations with many others)

## Introduction

### The Big Picture

One of the top priorities for Swift right now is compatibility across future 
Swift versions. Compatibility aims at accomplishing two goals:

1. *Source compatibility* means that newer compilers can compile code written 
in an older version of Swift. This aims to reduce the migration pain that Swift 
developers face when migrating to a newer Swift version. Without source 
compatibility, projects face version-lock where all source code in a project 
and its packages must be written in the same version of Swift. With source 
compatibility, package authors will be able to maintain a single code base 
across multiple Swift versions while allowing their users to use a newer 
version of Swift.
2. *Binary framework & runtime compatibility* enables the distribution of 
frameworks in a binary form that works across multiple Swift versions. Binary 
frameworks include both a *Swift module file*, which communicates source-level 
information of the framework's API, and a *shared library*, which provides the 
compiled implementation that is loaded at runtime. Thus, there are two 
necessary goals for binary framework compatibility:
* *Module format stability* stabilizes the module file, which is the 
compiler's representation of the public interfaces of a framework. This 
includes API declarations and inlineable code. The module file is used by the 
compiler for necessary tasks such as type checking and code generation when 
compiling client code using a framework.
* *ABI stability* enables binary compatibility between applications and 
libraries compiled with different Swift versions. It is the focus of the rest 
of this document.

This document is an exploration and explanation of Swift's ABI alongside the 
goals and investigations needed before declaring Swift's ABI stable. It is 
meant to be a resource to the community as well as a declaration of the 
direction of Swift's ABI.

Throughout this document there will be references to issues in Swift's [issue 
tracking system](https://bugs.swift.org) denoted by "SR-". These references 
track open engineering and design tasks for Swift's ABI.

### What Is ABI?

At runtime, Swift program binaries interact with other libraries and components 
through an ABI. ABI is Application Binary Interface, or the specification to 
which independently compiled binary entities must conform to be linked together 
and executed. These binary entities must agree on many low level details: how 
to call functions, how their data is represented in memory, and even where 
their metadata is and how to access it.

ABI is per-platform, as it is a low level concern influenced by both the 
architecture and the OS. Most platform vendors define a "standard ABI" which is 
used for C code and built on by C-family languages. Swift, however, is a very 
different language from C and has its own per-platform ABI. While most of this 
document is platform-agnostic, platform-specific concerns have influenced 
details of the design and implementation of Swift's ABI. For details on each 
platform's standard ABI, refer to the [Appendix](#platform-abis). 

### What Is ABI Stability?

ABI stability means locking down the ABI to the point that future compiler 
versions can produce binaries conforming to the stable ABI. Once an ABI is 
stable, it tends to persist for the rest of the platform's lifetime due to 
ever-increasing mutual dependencies.

ABI stability only affects invariants of externally visible public interfaces 
and symbols. Internal symbols, conventions, and layout can continue to change 
without breaking the ABI. For example, future compilers are free to change the 
calling conventions for internal function calls so long as the public 
interfaces are preserved.

Decisions about the ABI will have long-term ramifications and may limit the 
ways in which the language can grow and evolve in the future. Future Swift 
versions can add new, orthogonal aspects to the ABI, but any inefficiencies or 
inflexibilities present when stability is declared will (effectively) persist 
forever for that platform.

ABI changes that are new and orthogonal are called *ABI-additive* changes. 
ABI-additive changes may be taken advantage of when the minimum targeted Swift 
version supports them. This allows us to extend or progressively lock dow