Hi, I'm on vacation and can't quite answer in depth, but this proposal encourages all the wrong things in terms of performance. We know from 10 years of GCD that mixing async/sync, while possible, yields significant performance issues, and I'd rather not have something at the core of the language encouraging it.
Actors seem like a much better way of solving these problems IMO. -Pierre > On Nov 16, 2017, at 1:50 PM, Christopher Heath via swift-evolution > <swift-evolution@swift.org> wrote: > > Good evening all, > > I had a little idea and thought I’d pitch it. So here goes! > > Synthesizing Concurrency aims to provide a foundation by which, regardless of > concurrency implementation (however current proposal uses GCD examples for > design), concurrency can be synthesized by the compiler and remove many lines > of boilerplate code. Offering benefits to Application, Server and OS > Developers alike; with easy and intelligent concurrency conformance. Also, > should the community decide to head in another direction with concurrency > this should be a relatively mappable idea to any Synchronization Primitive > with a little work. > > Thanks for looking, and I hope you like and want to contribute to the > discussion of the Synthesizing Concurrency proposal. I’ve added a printout > below. Please give it a read or check out the gist: > https://gist.github.com/XNUPlay/a0d6f6c0afdb3286e324c480cb5c4290 > <https://gist.github.com/XNUPlay/a0d6f6c0afdb3286e324c480cb5c4290> > Chris > > Printout : > Synthesizing Concurrency > Proposal: SE-NNNN > <https://gist.github.com/XNUPlay/a0d6f6c0afdb3286e324c480cb5c4290> > Author: Christopher Heath: XNUPlay <https://github.com/xnuplay> > Review Manager: TBD > Status: Pending Discussion > Implementation: Awaiting Implementation > Decision Notes: TBD > > Introduction > Developers have to write large amounts of boilerplate code to support > concurrency in complex types. This proposal offers a way for the compiler to > automatically synthesize conformance to a high-level Concurrent protocol to > reduce concurrent boilerplate code, in a set of scenarios where generating > the correct implementation is known to be possible. > Specifically: > It aims to provide a high-level Swift protocol that offers opt-in concurrency > support for any conformable type > It aims to provide a well-defined set of thread-safe, concurrent > implementations for types, their properties, and methods. > It aims to provide a language/library compatible implementation with deadlock > prevention. > > Motivation > Building robust types in Swift can involve writing significant boilerplate > code to support concurrency. By eliminating the complexity for the users, we > make Concurrent types much more appealing to users and allow them to use > their own types in optimized concurrent and parallel environments that > require thread safety with no added effort on their part (beyond declaring > the conformance). > Concurrency is typically not pervasive across many types, and for each one > users must implement the concurrent code such that it performs some form of > synchronization to prevent unexpected behavior. > > Note: Due to it's current status in Swift and use in the Runtime > <https://github.com/apple/swift/blob/master/stdlib/public/runtime/Once.cpp#L31>, > examples are written in Grand Central Dispatch > <https://github.com/apple/swift-corelibs-libdispatch> > > // Concurrent Protocol - Dispatch Example > protocol Concurrent { > // Synthesized Property > var internalQueue: DispatchQueue { get } > } > > What's worse is that if any functions or properties are added, removed, or > changed, they must each have their own concurrency code and since it must be > manually written, it's possible to get it wrong, either by omission or > typographical error (async vs. sync). > Likewise, it becomes necessary when one wishes to modify an existing type > that implements concurrency to do so without introducing bottlenecks or > different forms of synchronization for some functions and not others, this > leads to illegible and inefficient code that may defeat the purpose of > implementing concurrency. > Crafting high-performance, readable concurrency code can be difficult and > inconvenient to write. > Swift already derives conformance to a number of protocols, automatically > synthesizing their inner-workings when possible. Since there is precedent for > synthesized conformances in Swift, we propose extending it to concurrency in > predictable circumstances. > > Proposed solution > In general, we propose that a type synthesize conformance to Concurrent as > long as the compiler has reasonable insight into the type. We describe the > specific conditions under which these conformances are synthesized below, > followed by the details of how the conformance requirements are implemented. > > Requesting synthesis is opt-in > Users must opt-in to automatic synthesis by declaring their type as > Concurrent without implementing any of its requirements. This conformance > must be part of the original type declaration and not on an extension (see > Synthesis in extensions below for more on this). > Any type that declares such conformance and satisfies the conditions below > will cause the compiler to synthesize an implementation of an internalQueue > and async or sync for all properties and methods on that type. > Making the synthesis opt-in—as opposed to automatic derivation without an > explicit declaration—provides a number of benefits: > The syntax for opting in is natural; there is no clear analogue in Swift > today for having a type opt out of a feature. > It requires users to make a conscious decision about the public API surfaced > by their types. Types cannot accidentally "fall into" conformances that the > user does not wish them to; a type that does not initially support Concurrent > can be made to at a later date, but the reverse is a potentially breaking > change. > The conformances supported by a type can be clearly seen by examining its > source code; nothing is hidden from the user. > We reduce the work done by the compiler and the amount of code generated by > not synthesizing conformances that are not desired and not used. > As will be discussed later, explicit conformance significantly simplifies the > implementation for recursive types. > > Overriding synthesized conformances > Any user-provided implementations of an internalQueue and use of async or > sync will override the default implementations that would be provided by the > compiler. > > Defining conditions where synthesis is allowed > For example take the struct below, which contains all-kinds of properties; > variable, constant, and computed. > struct Person { > var name: String // Variable Property > let birthday: Date // Constant Property > var age: Int { // Computed Property > /* - */ > } > } > > Synthesized Requirements > > Constant Properties > Constants are always accessed asynchronously. > A Constant is guaranteed to be immutable and therefore able to be read from > any thread without concern for unexpected mutation. > The compiler sees this Constant as storage for a value. > > // Compiler View - of a Constant Property > struct Person { > /* Variable Property */ > > // Constant Property > let birthday: Date { > get { > return underlying_Birthday_Date_Value_Storage > } > } > > /* Computed Property */ > } > > After opting-in to the Concurrent protocol, the compiler synthesizes this > implementation, adding an asynchronous access point to any Constant on a > Concurrent type. > > // Compiler View - of a Constant Property on a Concurrent type > struct Person: Concurrent { > /* Variable Property */ > > // Constant Property > let birthday: Date { > get { > internalQueue.async { // Immediately returns on calling thread > return underlying_Birthday_Date_Value_Storage > } > } > } > > /* Computed Property */ > } > > Synthesized requirements for Variable Properties > Variables are always accessed synchronously. > A Variable is mutable and therefore each thread must schedule writes and > reads separately out of concern for possible mutation. > Just like a Constant, the compiler sees this Variable as storage for a value. > > // Compiler View - of a Variable Property > struct Person { > // Variable Property > var name: String { > get { > return underlying_Name_String_Value_Storage > } > set (newValue) { > underlying_Name_String_Value_Storage = newValue > } > } > > /* Constant Property */ > /* Computed Property */ > } > > Here the compiler synthesizes synchronous access (read or write) to any > Variable on a Concurrent type. > > // Compiler View - of a Variable Property on a Concurrent type > struct Person: Concurrent { > // Variable Property > var name: String { > get { > internalQueue.sync { // Wait to ensure all mutation has finished > return underlying_Name_String_Value_Storage > } > } > set (newValue) { > internalQueue.sync { // Schedule this mutation to happen, in order > underlying_Name_String_Value_Storage = newValue > } > } > } > > /* Constant Property */ > /* Computed Property */ > } > > Synthesized requirements for Computed Properties > Computed Properties are always accessed synchronously. > A Computed Property is essentially a function that gets called to create a > value from other values. These other values can be mutable and therefore each > thread must schedule writes and reads separately out of concern for possible > mutation. (Note: If a computed property only accesses Constants, it should > probably be a one-time set Constant; Swift <https://github.com/apple/swift> > could use a few proposals in this area.) > > // Compiler View - of a Variable Property > struct Person { > /* Variable Property */ > /* Constant Property */ > > // Computed Property > var age: Int { > // Compute age from birthday; return > } > } > > Here the compiler synthesizes synchronous access (read or write) to any > Computed Property on a Concurrent type. > > // Compiler View - of a Variable Property on a Concurrent type > struct Person: Concurrent { > /* Variable Property */ > /* Constant Property */ > > // Computed Property > var age: Int { > internalQueue.sync { // Wait to ensure all mutation has finished > // Compute age from birthday; return > } > } > } > > Considerations for recursive types and abstraction > By making the synthesized conformances opt-in, recursive types have their > requirements fall into place with no extra effort. In any cycle belonging to > a recursive type, every type in that cycle must declare its conformance > explicitly. If a type does so but cannot have its conformance synthesized > because it does not satisfy the conditions above, then it is simply an error > for that type and not something that must be detected earlier by the compiler > in order to reason about all the other types involved in the cycle. (On the > other hand, if conformance were implicit, the compiler would have to fully > traverse the entire cycle to determine eligibility, which would make > implementation much more complex). > With respect to abstraction, the idea that a synchronous function or property > can access another synchronous function or property, introduces a problem: > Deadlocking. > > The Deadlock Problem > Or just Deadlocking, is a problem where a complex program cannot continue > execution because one or more threads is waiting on a resource to become > available or for another task to complete. > Anytime a synchronous function or property accesses another synchronous > function or property; this is defined as a Deadlock, because the first cannot > finish without the second being run and the second cannot execute without the > first being finished. > > Solving the Deadlock Problem > In complex functions where any number of synchronous and asynchronous calls > can happen inside a larger scope it is required that the compiler know how to > handle compilation of such functions, that may access many concurrent objects > through a multitude of calls. Much like Automatic Reference Counting > <https://en.wikipedia.org/wiki/Automatic_Reference_Counting> increments and > decrements a counter to determine whether an object should be marked for > deallocation, we suggest that during compilation a call or set of calls is > handled by evaluating their concurrent requirements. > I.e. When a call nests as such: > > // Compiler View - of a Complex Function on a Concurrent type > > func heavyLift() { > syncFunction() // 1 Sync > > async() // 1 Async > > syncSomeFunction() // 2 Sync > syncSomeOtherFunction() // 3 Sync > > asyncSomeFunction() // 2 Async > asyncSomeOtherFunction() // 3 Async > } > > The compiler should implement a non-modified function, exactly as it would > today, and wrap each usage in-scope with an asynchronous or synchronous > requirement. > Specifically: > If a higher-level function accesses only asynchronous functions or properties > internally, that function can be executed in-order as a single asynchronous > call and inlining access to all non-modified calls. > The same is true of synchronous functions or properties. They can be executed > in-order as a single synchronous call and inlining access to existing > non-modified calls. > If at any point a function or property, accesses an asynchronous and > synchronous call then that function must be run as a single synchronous call. > > Implementation details > Deadlock Prevention is then inherent by synthesis. The following example > explains this through a chunk of modified, disassembled Swift code. > > // Disassembly View - of an Integer Assignment without Thread-Safety > int __T07Project14ConcurrentTypeC21functionWithoutSafetyySi5value_tF(int > arg0) { // Standard > _swift_beginAccess(__T07Project6objectSiv, &var_30, 0x1, 0x0); > *__T07Project6objectSiv = arg0; > rax = _swift_endAccess(&var_30); > return rax; > } > > // Disassembly View - of an Integer Assignment with Asynchronous Access > int __T07Project14ConcurrentTypeC17functionWithAsyncySi5value_tF(int arg0) { > // AsyncCall > _swift_beginAccess(r13 + 0x10, &var_30, 0x0, 0x0); > _swift_endAccess(&var_30, &var_30, 0x0, 0x0); > rax = _swift_rt_swift_allocObject(); > > // Call the Standard Function > __T07Project14ConcurrentTypeC21functionWithoutSafetyySi5value_tF(int arg0) > > var_60 = __NSConcreteStackBlock; > var_98 = _Block_copy(&var_60); > var_A0 = > __T0So13DispatchQueueC0A0E5asyncySo0A5GroupCSg5group_AC0A3QoSV3qosAC0A13WorkItemFlagsV5flagsyyXB7executetFfA_(&var_60, > 0x18, __NSConcreteStackBlock); > var_A1 = > __T0So13DispatchQueueC0A0E5asyncySo0A5GroupCSg5group_AC0A3QoSV3qosAC0A13WorkItemFlagsV5flagsyyXB7executetFfA0_(); > > __T0So13DispatchQueueC0A0E5asyncySo0A5GroupCSg5group_AC0A3QoSV3qosAC0A13WorkItemFlagsV5flagsyyXB7executetF(var_A0, > var_A1 & 0xff, __NSConcreteStackBlock, > __T0So13DispatchQueueC0A0E5asyncySo0A5GroupCSg5group_AC0A3QoSV3qosAC0A13WorkItemFlagsV5flagsyyXB7executetFfA1_(&var_60, > 0x18), var_98); > rax = _swift_rt_swift_release(rax, var_A1 & 0xff); > return rax; > } > > // Disassembly View - of an Integer Assignment with Synchronous Access > int __T07Project14ConcurrentTypeC16functionWithSyncySi5value_tF(int arg0) { > // SyncCall > _swift_beginAccess(r13 + 0x10, &var_28, 0x0, 0x0); > _swift_endAccess(&var_28, &var_28, 0x0, &var_28); > rax = _swift_rt_swift_allocObject(); > > // Call the Standard Function > __T07Project14ConcurrentTypeC21functionWithoutSafetyySi5value_tF(int arg0) > > var_58 = __NSConcreteStackBlock; > var_90 = _Block_copy(&var_58); > _swift_rt_swift_release(rax, 0x18); > dispatch_sync(*(r13 + 0x10), var_90); > rax = _Block_release(var_90); > return rax; > } > > Here are 3 functions, one which assigns a value to an integer without any > safety, like a non-concurrent type. As well as two more which call that > function using asynchronous and synchronous access respectively. > The compiler determines which access should be used in a given scope, and > places that scope inside a synchronous or asynchronous call. > Take a function which assigns this integer twice asynchronously: > > // Standard Call > +---------------------------------------+ > | TwoAsyncs | > | +-------------+ +-------------+ | > | | AsyncCall | | AsyncCall | | > | +-------------+ +-------------+ | > +---------------------------------------+ > One might reason that this function would fire-and-forget those calls, but > instead the compiler is rectifying them as a single async. > > // Single Asynchronous Call > +---------------------------------------+ > | TwoAsyncs (Actually) | > | +-------------+ +-------------+ | > | | Standard | | Standard | | > | +-------------+ +-------------+ | > +---------------------------------------+ > This behavior is the same for synchronous-only functions; however, instead of > rehashing lets look at the more interesting complex case. We start with this: > > // Standard Call > +---------------------------------------+ > | AnyAsync/SyncCombination | > | +-------------+ +-------------+ | > | | AsyncCall | | SyncCall | | > | +-------------+ +-------------+ | > +---------------------------------------+ > But in actuality the compiler has composed a single synchronous call, since > there is a sync call at any point in the function. > > // Single Synchronous Call > +---------------------------------------+ > | AnyAsync/SyncCombination (Actually) | > | +-------------+ +-------------+ | > | | Standard | | Standard | | > | +-------------+ +-------------+ | > +---------------------------------------+ > Let's do one more for added clarity. > > +-------------------------------------------------------------------------------------+ > | Combination > | > | +---------------------------------------+ > +---------------------------------------+ | > | | TwoAsyncs | | AnyAsync/SyncCombination > | | > | | +-------------+ +-------------+ | | +-------------+ > +-------------+ | | > | | | AsyncCall | | AsyncCall | | | | AsyncCall | | > SyncCall | | | > | | +-------------+ +-------------+ | | +-------------+ > +-------------+ | | > | +---------------------------------------+ > +---------------------------------------+ | > > +-------------------------------------------------------------------------------------+ > Becomes: > > +-------------------------------------------------------------------------------------+ > | Combination //Sync > | > | +---------------------------------------+ > +---------------------------------------+ | > | | TwoAsyncs //Async | | AnyAsync/SyncCombination > //Sync | | > | | +-------------+ +-------------+ | | +-------------+ > +-------------+ | | > | | | Standard | | Standard | | | | Standard | | > Standard | | | > | | +-------------+ +-------------+ | | +-------------+ > +-------------+ | | > | +---------------------------------------+ > +---------------------------------------+ | > > +-------------------------------------------------------------------------------------+ > > We've already made great decisions about thread-safety by implementing > SE-0035 > <https://github.com/apple/swift-evolution/blob/master/proposals/0035-limit-inout-capture.md> > for Value Types > SE-0035 Limiting Inout Capture > <https://github.com/apple/swift-evolution/blob/master/proposals/0035-limit-inout-capture.md> > actually provides us with a proof-of-concept as to why Value Types should > not be asynchronously mutated inside a closure. > > Source compatibility > By making the conformance opt-in, this is a purely additive change that > should not affect existing code and should be easily applicable to stdlib > types. > Some current types using Grand Central Dispatch > <https://github.com/apple/swift-corelibs-libdispatch> should be audited for > recursive-implementation, if a user wishes to replace their own > implementation with this synthesized one. > > Effect on ABI stability > This feature is purely additive and should not change ABI. > (Additionally, see Explicit Manglings for Sync/Async below for more on this.) > > Effect on API resilience > N/A. > > Alternatives considered > In order to realistically scope this proposal, we considered but ultimately > deferred the following items, some of which could be proposed additively in > the future. > > Synthesis in extensions > Requirements will be synthesized only for protocol conformances that are part > of the type declaration itself; conformances added in extensions will not be > synthesized. > However, to align with Codable in the context of SR-4920 > <https://bugs.swift.org/browse/SR-4920>, we will also currently forbid > synthesized requirements in extensions in the same file; this specific case > can be revisited later for all derived conformances. > > Explicit Manglings > Because accesses are compiled and their async or sync wrappers are > deterministic from use case, it may be useful to create specific Manglings; > this is optional. > > Embedding or Building Dispatch > Dispatch already provides us with a very powerful and exacting standard for > concurrency in Swift, it would be even more useful if embedded directly in > the runtime with replacements like that currently in the Runtime > <https://github.com/apple/swift/blob/master/stdlib/public/runtime/Once.cpp#L31>. > I feel as if most of the reason this would be frowned upon is a Swift desire > for style, code cleanliness and some hope of a 'better' (whatever that means > to you) solution. > Yet, we could add the existing library to the Runtime or even rewrite Grand > Central Dispatch <https://github.com/apple/swift-corelibs-libdispatch> as a > Swift project and embed it in the Standard Library. > Ideally, this would be deferred to a separate Swift Evolution Proposal. > > Keyword Overrides > It is worth mentioning a Keyword could be used for overriding a function and > defining explicit behavior as sync or async. However, this opens the door to > misuse and incorrect code that can only be debugged at runtime with TSAN. And > while we love TSAN: > TSAN how I love thee. Let me non atomically count the ways… > - Philippe > Hausler > This is not a good idea. > > Acknowledgments > Thanks to everyone in the Swift Community working to make it an even more > vibrant place. And especially to those who worked on SE-0166 > <https://github.com/apple/swift-evolution/blob/master/proposals/0166-swift-archival-serialization.md> > and SE-0185 > <https://github.com/apple/swift-evolution/blob/master/proposals/0185-synthesize-equatable-hashable.md>. > Whom might notice large parts of a shared ideal, that made this proposal > much easier to write. > > _______________________________________________ > swift-evolution mailing list > swift-evolution@swift.org > https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution
