Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 27, 2016, at 11:52 PM, Charlie Monroe via swift-evolution 
>  wrote:
> 
> 
>> On Jun 27, 2016, at 9:10 PM, Christopher Kornher > > wrote:
>> 
>> 
>>> On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution 
>>> > wrote:
>>> 
 All object references used within a closure must unwrap successfully for 
 the closure to execute.
>>> I agree with the logic of this proposal, but this is the confusing part or 
>>> a part that I slightly disagree with.
>>> 
>>> By this logic, the block won't be invoked if all captured variables can't 
>>> be unwrapped, which is definitely confusing to the newcomer (to whom this 
>>> is partially targetted as you've mentioned) if he puts a breakpoint in it 
>>> and doesn't get executed even when he's explicitely invoking it somewhere.
>>> 
>>> On the other hand, when it crashes, it gives him some idea that something's 
>>> wrong.
>> 
>> Tooling could alleviate some of this mystery. For example:
>> 
>> 1) In Xcode and other GUIs, closures that will not be executed could be 
>> greyed-out, perhaps with the reason overlaid on the closure perhaps this 
>> would only happen when a breakpoint was set within the closure. Perhaps the 
>> app could break on the on breakpoints within non-executing closures and 
>> notify the user that the closure did not execute.
>> 
>> 2) Debugger console commands could be added to describe the execution state 
>> of closures in scope and closure variables.
>> 
>> 3) Debug apps could bottleneck all closures through a function that that can 
>> be break-pointed. Breakpoints could be edited to filter on specific 
>> closures. 
>> 
>> 4) Some runtime switches could be added to enable verbose login modes for 
>> closures.
> 
> This can solve issues that you may have with a 2KLOC project that does 
> nothing but waits for the user to click on something. When you have a project 
> of 100KLOC, running at 100% CPU on a background thread, I don't think you'd 
> debug anything since console would be flooded by thousands of log messages…

This technique is used with CoreData, OS X Bindings, and probably other 
frameworks and yes, it is often a tool of last resort. Tools like grep and 
spunk are often needed to make use of the output, but this is an effective 
technique, if not a very efficient one. I much prefer interactive debugger 
support. One advantage of global logging options is that they can be to used 
over long periods, on remote machines to capture infrequent errors. 

> 
>> 
>> I don’t think that this is an insurmountable problem. There are many 
>> features of modern applications that are difficult to debug.
>> 
>>> 
>>> 
 
 I believe that these are safe, sensible and understandable rules that will 
 eliminate the need for capture lists for many closures. What do you think?
 
 
 ___
 swift-evolution mailing list
 swift-evolution@swift.org 
 https://lists.swift.org/mailman/listinfo/swift-evolution 
 
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 28, 2016, at 6:53 AM, ad...@wheerd.de wrote:
> 
> Am 27.06.2016 20:55, schrieb Christopher Kornher:
>>> On Jun 27, 2016, at 2:45 AM, Manuel Krebber via swift-evolution 
>>>  wrote:
>>> On 06/26/2016 09:10 PM, Christopher Kornher via swift-evolution wrote:
 The core proposal:
 ——
 Closures capturing object references should automatically capture all
 object references as weak.
>>> In my code, most closures are used in a functional programming capacity,
>>> e.g. with map(), reduce, etc. Hence, most closures are non-escaping and
>>> local, where strong capture is the desired way. Otherwise I would have
>>> to litter everything with optional unwrapping or add the explicit
>>> capture definition which would both make the code less readable in my
>>> opinion.
>> I thought about this some more and it makes sense to treat
>> non-escaping closures are they are treated now. This might increase
>> the burden on the compiler especially because these closures may not
>> be declared inline. This would be far more straightforward than having
>> to worry about changes to object existence within one or more
>> invocations of a @nonescaping closure, especially in multi-threaded
>> code.
>> I do not think that this would be a significant change for developers
>> in practice. Any developer who would try to rely upon object
>> references changing within the application of a @nonescaping closure
>> would probably have read the manual very carefully :)
> 
> I am unsure whether treating closures differently depending on a @nonescaping 
> attribute on a function is a good idea.
> The problem is that, theoretically, the closure might be stored in a variable 
> or used with different functions. Then how do you decide whether the default 
> is strong or weak capture?

The compiler/runtime (in theory at least) knows how a closure is going to be 
applied. I am not a compiler writer, but two versions of the closure could be 
created when necessary and the appropriate one could be used depending upon the 
context.

> 
> Example:
> 
> func f1(_ f : (Int) -> Int) {
> ...
> }
> 
> func f2(_ f : @noescape  (Int) -> Int) {
> ...
> }
> 
> let foo = Foo()
> let f : (Int) -> Int = {
>  return foo.x
> }
> 
> f1(f)
> f2(f)
> 
> In addition different default capture behaviour dependant on context could be 
> confusing for developers I think.

The changes in behavior in the application of @noescaping closures are very 
much] edge case and are as likely to result from actions in other threads as 
anything else. So Capturing references as strong by default makes @noescaping 
closures more predictable and eliminates the potential for really some really 
subtle bug and race conditions.

For the same reason, implicit “let guards” in escaping closures should be 
applied early to improve predicability. Developers wanting to modify reference 
counts within closures are free to use traditional capture lists and capture 
and un-capture references as needed.

Memory management within closures should not be the default behavior, but it 
should be possible and discoverable. 


> 
 1) Closures with object references could be simplified further by
 implicitly including ‘let’ guards for all object references:
>>> This sounds good for closures without return value, but how would you
>>> handle closures with non-optional non-void return values?
>> Good point.
>> 1) The simplest solution would be to have these closures require
>> capture lists to some or all object references. Given the prevalence
>> of the weak/strong dance, calling these out in some way probably
>> should be done anyway.
>> 2) Another option would be to eliminate the implicit nil check guards,
>> making all object references optional. The compiler would force these
>> to be unwrapped, so users would be guided to do the right thing.
>> 3) Just always weakly capturing all object references would treat all
>> closures uniformly.
> 
> I think I favour 2) out of these, even though it still means the code will 
> potentially be littered with ?. or let guards...
> 
> Kind regards, Manuel
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Manuel Krebber via swift-evolution]

Am 27.06.2016 20:55, schrieb Christopher Kornher:
On Jun 27, 2016, at 2:45 AM, Manuel Krebber via swift-evolution 
 wrote:


On 06/26/2016 09:10 PM, Christopher Kornher via swift-evolution wrote:

The core proposal:
——

Closures capturing object references should automatically capture all
object references as weak.
In my code, most closures are used in a functional programming 
capacity,
e.g. with map(), reduce, etc. Hence, most closures are non-escaping 
and

local, where strong capture is the desired way. Otherwise I would have
to litter everything with optional unwrapping or add the explicit
capture definition which would both make the code less readable in my
opinion.


I thought about this some more and it makes sense to treat
non-escaping closures are they are treated now. This might increase
the burden on the compiler especially because these closures may not
be declared inline. This would be far more straightforward than having
to worry about changes to object existence within one or more
invocations of a @nonescaping closure, especially in multi-threaded
code.

I do not think that this would be a significant change for developers
in practice. Any developer who would try to rely upon object
references changing within the application of a @nonescaping closure
would probably have read the manual very carefully :)




I am unsure whether treating closures differently depending on a 
@nonescaping attribute on a function is a good idea.
The problem is that, theoretically, the closure might be stored in a 
variable or used with different functions. Then how do you decide 
whether the default is strong or weak capture?


Example:

func f1(_ f : (Int) -> Int) {
...
}

func f2(_ f : @noescape  (Int) -> Int) {
...
}

let foo = Foo()
let f : (Int) -> Int = {
  return foo.x
}

f1(f)
f2(f)

In addition different default capture behaviour dependant on context 
could be confusing for developers I think.



1) Closures with object references could be simplified further by
implicitly including ‘let’ guards for all object references:

This sounds good for closures without return value, but how would you
handle closures with non-optional non-void return values?


Good point.

1) The simplest solution would be to have these closures require
capture lists to some or all object references. Given the prevalence
of the weak/strong dance, calling these out in some way probably
should be done anyway.

2) Another option would be to eliminate the implicit nil check guards,
making all object references optional. The compiler would force these
to be unwrapped, so users would be guided to do the right thing.

3) Just always weakly capturing all object references would treat all
closures uniformly.



I think I favour 2) out of these, even though it still means the code 
will potentially be littered with ?. or let guards...


Kind regards, Manuel

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Charlie Monroe via swift-evolution]

> On Jun 27, 2016, at 9:10 PM, Christopher Kornher  wrote:
> 
> 
>> On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution 
>> > wrote:
>> 
>>> All object references used within a closure must unwrap successfully for 
>>> the closure to execute.
>> I agree with the logic of this proposal, but this is the confusing part or a 
>> part that I slightly disagree with.
>> 
>> By this logic, the block won't be invoked if all captured variables can't be 
>> unwrapped, which is definitely confusing to the newcomer (to whom this is 
>> partially targetted as you've mentioned) if he puts a breakpoint in it and 
>> doesn't get executed even when he's explicitely invoking it somewhere.
>> 
>> On the other hand, when it crashes, it gives him some idea that something's 
>> wrong.
> 
> Tooling could alleviate some of this mystery. For example:
> 
> 1) In Xcode and other GUIs, closures that will not be executed could be 
> greyed-out, perhaps with the reason overlaid on the closure perhaps this 
> would only happen when a breakpoint was set within the closure. Perhaps the 
> app could break on the on breakpoints within non-executing closures and 
> notify the user that the closure did not execute.
> 
> 2) Debugger console commands could be added to describe the execution state 
> of closures in scope and closure variables.
> 
> 3) Debug apps could bottleneck all closures through a function that that can 
> be break-pointed. Breakpoints could be edited to filter on specific closures. 
> 
> 4) Some runtime switches could be added to enable verbose login modes for 
> closures.

This can solve issues that you may have with a 2KLOC project that does nothing 
but waits for the user to click on something. When you have a project of 
100KLOC, running at 100% CPU on a background thread, I don't think you'd debug 
anything since console would be flooded by thousands of log messages...

> 
> I don’t think that this is an insurmountable problem. There are many features 
> of modern applications that are difficult to debug.
> 
>> 
>> 
>>> 
>>> I believe that these are safe, sensible and understandable rules that will 
>>> eliminate the need for capture lists for many closures. What do you think?
>>> 
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 27, 2016, at 2:35 PM, L. Mihalkovic via swift-evolution 
>  wrote:
> 
> 
> 
> Regards
> (From mobile)
> 
> On Jun 27, 2016, at 10:18 PM, Dennis Lysenko via swift-evolution 
> > wrote:
> 
>> My 2c:
>> 
>> This proposal is made more appealing to me because it is not simply a 
>> 'beginners will get confused' issue. 
>> 
>> I have written tens of thousands of lines of Swift from Swift 1 to the Swift 
>> 3 preview and I still can't shake occasionally accidentally capturing `self` 
>> strongly when I, for example, assign a closure as a listener/action to a UI 
>> component.
> 
> The problem of is that it is easier for an app to survive an accidental extra 
> capture than it is for it the survive a missing one.. So if this is really 
> for beginners, then it is much better to keep the current behavior which will 
> lead them to have leaky apps, than it is to give them apps whre objects will 
> vanish from under their feet... granted neither is good to begin with.

Not always. large memory leaks are often fatal in iOS because of the limited 
memory footprint. On more constrained devices the problem would e worse.

All of the great feedback has made me realize that my original proposal was 
more UIKit centric that I thought. One size does not fit all. Any errors in 
closure memory management can be fatal and the challenge is to optimize the 
experience for all programming domains and styles.


> 
> 
>> 
>> To spin off of Christopher's last email, my proposal is thus: we could 
>> include the original proposal (without numbered addendums) and use tooling, 
>> but have it alleviate the burden another way: have the tooling insert 
>> `[strong self]` by default when autocompleting a block.
>> 
>> BTW, @Manuel Krebber: this proposal would not affect your run-of-the-mill 
>> map statement. array.map { object in object.property } requires no explicit 
>> capture since object is intuitively strongly captured there.
>> 
>> What do we think? Worth discussing?
>> 
>> On Mon, Jun 27, 2016 at 12:10 PM Christopher Kornher via swift-evolution 
>> > wrote:
>>> On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution 
>>> > wrote:
>>> 
 All object references used within a closure must unwrap successfully for 
 the closure to execute.
>>> I agree with the logic of this proposal, but this is the confusing part or 
>>> a part that I slightly disagree with.
>>> 
>>> By this logic, the block won't be invoked if all captured variables can't 
>>> be unwrapped, which is definitely confusing to the newcomer (to whom this 
>>> is partially targetted as you've mentioned) if he puts a breakpoint in it 
>>> and doesn't get executed even when he's explicitely invoking it somewhere.
>>> 
>>> On the other hand, when it crashes, it gives him some idea that something's 
>>> wrong.
>> 
>> Tooling could alleviate some of this mystery. For example:
>> 
>> 1) In Xcode and other GUIs, closures that will not be executed could be 
>> greyed-out, perhaps with the reason overlaid on the closure perhaps this 
>> would only happen when a breakpoint was set within the closure. Perhaps the 
>> app could break on the on breakpoints within non-executing closures and 
>> notify the user that the closure did not execute.
>> 
>> 2) Debugger console commands could be added to describe the execution state 
>> of closures in scope and closure variables.
>> 
>> 3) Debug apps could bottleneck all closures through a function that that can 
>> be break-pointed. Breakpoints could be edited to filter on specific 
>> closures. 
>> 
>> 4) Some runtime switches could be added to enable verbose login modes for 
>> closures.
>> 
>> I don’t think that this is an insurmountable problem. There are many 
>> features of modern applications that are difficult to debug.
>> 
>>> 
>>> 
 
 I believe that these are safe, sensible and understandable rules that will 
 eliminate the need for capture lists for many closures. What do you think?
 
 
 ___
 swift-evolution mailing list
 swift-evolution@swift.org 
 https://lists.swift.org/mailman/listinfo/swift-evolution 
 
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>> 

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 27, 2016, at 2:35 PM, L. Mihalkovic via swift-evolution 
>  wrote:
> 
> 
> 
> Regards
> (From mobile)
> 
> On Jun 27, 2016, at 10:18 PM, Dennis Lysenko via swift-evolution 
> > wrote:
> 
>> My 2c:
>> 
>> This proposal is made more appealing to me because it is not simply a 
>> 'beginners will get confused' issue. 
>> 
>> I have written tens of thousands of lines of Swift from Swift 1 to the Swift 
>> 3 preview and I still can't shake occasionally accidentally capturing `self` 
>> strongly when I, for example, assign a closure as a listener/action to a UI 
>> component.
> 
> The problem of is that it is easier for an app to survive an accidental extra 
> capture than it is for it the survive a missing one.. So if this is really 
> for beginners, then it is much better to keep the current behavior which will 
> lead them to have leaky apps, than it is to give them apps whre objects will 
> vanish from under their feet... granted neither is good to begin with.

For many cases, that is exactly the behavior you want. In my experience, the 
instances of closures being the only live reference to objects not being 
constructed by the closure are rare. 

To clarify, the proposed implicit guard statements would create strong 
references once the closure starts executing, so referenced objects would not 
disappear during the execution of the closure. 


>> To spin off of Christopher's last email, my proposal is thus: we could 
>> include the original proposal (without numbered addendums) and use tooling, 
>> but have it alleviate the burden another way: have the tooling insert 
>> `[strong self]` by default when autocompleting a block.
>> 
>> BTW, @Manuel Krebber: this proposal would not affect your run-of-the-mill 
>> map statement. array.map { object in object.property } requires no explicit 
>> capture since object is intuitively strongly captured there.
>> 
>> What do we think? Worth discussing?
>> 
>> On Mon, Jun 27, 2016 at 12:10 PM Christopher Kornher via swift-evolution 
>> > wrote:
>>> On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution 
>>> > wrote:
>>> 
 All object references used within a closure must unwrap successfully for 
 the closure to execute.
>>> I agree with the logic of this proposal, but this is the confusing part or 
>>> a part that I slightly disagree with.
>>> 
>>> By this logic, the block won't be invoked if all captured variables can't 
>>> be unwrapped, which is definitely confusing to the newcomer (to whom this 
>>> is partially targetted as you've mentioned) if he puts a breakpoint in it 
>>> and doesn't get executed even when he's explicitely invoking it somewhere.
>>> 
>>> On the other hand, when it crashes, it gives him some idea that something's 
>>> wrong.
>> 
>> Tooling could alleviate some of this mystery. For example:
>> 
>> 1) In Xcode and other GUIs, closures that will not be executed could be 
>> greyed-out, perhaps with the reason overlaid on the closure perhaps this 
>> would only happen when a breakpoint was set within the closure. Perhaps the 
>> app could break on the on breakpoints within non-executing closures and 
>> notify the user that the closure did not execute.
>> 
>> 2) Debugger console commands could be added to describe the execution state 
>> of closures in scope and closure variables.
>> 
>> 3) Debug apps could bottleneck all closures through a function that that can 
>> be break-pointed. Breakpoints could be edited to filter on specific 
>> closures. 
>> 
>> 4) Some runtime switches could be added to enable verbose login modes for 
>> closures.
>> 
>> I don’t think that this is an insurmountable problem. There are many 
>> features of modern applications that are difficult to debug.
>> 
>>> 
>>> 
 
 I believe that these are safe, sensible and understandable rules that will 
 eliminate the need for capture lists for many closures. What do you think?
 
 
 ___
 swift-evolution mailing list
 swift-evolution@swift.org 
 https://lists.swift.org/mailman/listinfo/swift-evolution 
 
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution 
>> 
>> 

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 27, 2016, at 2:51 PM, Russ Bishop via swift-evolution 
>  wrote:
> 
>> 
>> On Jun 27, 2016, at 12:35 AM, Christopher Kornher > > wrote:
>> 
>> 
>>> On Jun 26, 2016, at 4:25 PM, Russ Bishop >> > wrote:
>>> 
>>> 
 On Jun 26, 2016, at 12:10 PM, Christopher Kornher via swift-evolution 
 > wrote:
 
 I may be too late for Swift 3, but I am planning to propose changes to the 
 default behavior for closures capturing object references. The 
 introduction of Swift Playgrounds has raised the importance of simplifying 
 the coding of leak-free, crash-free closures. New developers should not 
 have to understand closure memory management to start writing useful and 
 correct code.
 
 The topic of the closure weak/strong dance has been discussed on this list 
 before. This proposal differs from previous proposals in that it will 
 eliminate the dance altogether by default. I am very interested in hearing 
 others’ opinions as to whether the benefits outweigh the costs of various 
 options.
>>> 
>>> The problem is that strong reference capture is probably the far more 
>>> common case.
>> 
>> Strong reference capture has not been more common in carefully written code 
>> in my experience. Swift is starting to be used for many different problem 
>> domains, so your experience may be different. Any examples of real-world 
>> code would be greatly appreciated.
> 
> In my experience reference cycles are almost always caused by capturing self 
> strongly.
> 
>> 
>> Sometimes closures contain the only references to objects, but this is not 
>> common in code that I have seen. It would be extremely rare for strong 
>> references to be required for all the references in closure with multiple 
>> references (the current default behavior). Long closures can reference many 
>> objects and it is all too easy to leave a reference out of the capture list 
>> and create a reference cycle. I believe that this pattern should be 
>> called-out (see below).
>> 
>> Strong references are occasionally needed to ensure operations are performed 
>> when objects would otherwise be reclaimed, but I have not seen many of these 
>> cases. This pattern could be more common in other frameworks. I believe that 
>> this pattern should be called-out (see below).
>> 
>> Multiple capture rules for closures can be supported if desired. The 
>> ```[required…``` capture qualifier in the original email is one way todo 
>> this. The question mark could be used in a way analogous to `try?` to 
>> identify closures using the proposed rules: 
>> 
>> ```let a:()->Void = {…}?```
>> or 
>> ``` let a:()->Void = ?{…}```
>> etc.
>> 
>> This would obviously add more complexity but would still be an improvement, 
>> I believe.
> 
> Possibly, though this inevitably ends up with the C++ lambda rules more or 
> less. To put it another way, why would Swift introduce a new way of handling 
> this instead of just requiring all closures to specify whether they want 
> strong default, weak default, or whatever and completely eliminate any 
> automatic behavior? Unfortunately that makes creating closures much more 
> annoying.

I took a quick look at C++ capture lists. Yeah, we don’t want those. But 
another default behavior for captures does make sense for Swift I think. More 
to come.

>> 
>> 
>>> If you wanted to say that @noescape closures capture references strongly by 
>>> default, while escaping closures capture them weakly by default that may be 
>>> closer to reasonable for most situations but now you have magic behavior 
>>> with an even greater cognitive overhead. (I wonder if it would be more 
>>> common that @noescape captures strongly, escaping captures self weak by 
>>> default but other objects strongly… of course that would have even worse 
>>> cognitive overhead.)
>> 
>> I did consider treating self differently, but this leads to some very 
>> strange cases when delegation, factories and other patterns are considered. 
>> 
>> This email was implicitly referring to escaping uses of closures. The same 
>> closure can be used as escaping or non-escaping. The Swift documentation 
>> states:
>> 
>>  "Marking a closure with @noescape lets the compiler make more aggressive 
>> optimizations because it knows more information about the closure’s 
>> lifespan."
>> 
>> @noescape is essentially a hint to the compiler. Optimizers would be free to 
>> use strong or unowned references if they can determine that it is safe to do 
>> so without altering behavior. 
> 
> The majority of all closures are passed to non-escaping functional methods 
> like map, filter, etc.

 Not in the iOS programming that I do most of the time, but I am sure that the 
is often the case for other types of development. I rarely use object 
references in 

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[L. Mihalkovic via swift-evolution]

Regards
(From mobile)

> On Jun 27, 2016, at 10:18 PM, Dennis Lysenko via swift-evolution 
>  wrote:
> 
> My 2c:
> 
> This proposal is made more appealing to me because it is not simply a 
> 'beginners will get confused' issue. 
> 
> I have written tens of thousands of lines of Swift from Swift 1 to the Swift 
> 3 preview and I still can't shake occasionally accidentally capturing `self` 
> strongly when I, for example, assign a closure as a listener/action to a UI 
> component.

The problem of is that it is easier for an app to survive an accidental extra 
capture than it is for it the survive a missing one.. So if this is really for 
beginners, then it is much better to keep the current behavior which will lead 
them to have leaky apps, than it is to give them apps whre objects will vanish 
from under their feet... granted neither is good to begin with.


> 
> To spin off of Christopher's last email, my proposal is thus: we could 
> include the original proposal (without numbered addendums) and use tooling, 
> but have it alleviate the burden another way: have the tooling insert 
> `[strong self]` by default when autocompleting a block.
> 
> BTW, @Manuel Krebber: this proposal would not affect your run-of-the-mill map 
> statement. array.map { object in object.property } requires no explicit 
> capture since object is intuitively strongly captured there.
> 
> What do we think? Worth discussing?
> 
> On Mon, Jun 27, 2016 at 12:10 PM Christopher Kornher via swift-evolution 
>  wrote:
 On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution 
  wrote:
 
 All object references used within a closure must unwrap successfully for 
 the closure to execute.
>>> I agree with the logic of this proposal, but this is the confusing part or 
>>> a part that I slightly disagree with.
>>> 
>>> By this logic, the block won't be invoked if all captured variables can't 
>>> be unwrapped, which is definitely confusing to the newcomer (to whom this 
>>> is partially targetted as you've mentioned) if he puts a breakpoint in it 
>>> and doesn't get executed even when he's explicitely invoking it somewhere.
>>> 
>>> On the other hand, when it crashes, it gives him some idea that something's 
>>> wrong.
>> 
>> Tooling could alleviate some of this mystery. For example:
>> 
>> 1) In Xcode and other GUIs, closures that will not be executed could be 
>> greyed-out, perhaps with the reason overlaid on the closure perhaps this 
>> would only happen when a breakpoint was set within the closure. Perhaps the 
>> app could break on the on breakpoints within non-executing closures and 
>> notify the user that the closure did not execute.
>> 
>> 2) Debugger console commands could be added to describe the execution state 
>> of closures in scope and closure variables.
>> 
>> 3) Debug apps could bottleneck all closures through a function that that can 
>> be break-pointed. Breakpoints could be edited to filter on specific 
>> closures. 
>> 
>> 4) Some runtime switches could be added to enable verbose login modes for 
>> closures.
>> 
>> I don’t think that this is an insurmountable problem. There are many 
>> features of modern applications that are difficult to debug.
>> 
>>> 
>>> 
 
 I believe that these are safe, sensible and understandable rules that will 
 eliminate the need for capture lists for many closures. What do you think?
 
 
 ___
 swift-evolution mailing list
 swift-evolution@swift.org
 https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>>> ___
>>> swift-evolution mailing list
>>> swift-evolution@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Dennis Lysenko via swift-evolution]

My 2c:

This proposal is made more appealing to me because it is not simply a
'beginners will get confused' issue.

I have written tens of thousands of lines of Swift from Swift 1 to the
Swift 3 preview and I still can't shake occasionally accidentally capturing
`self` strongly when I, for example, assign a closure as a listener/action
to a UI component.

To spin off of Christopher's last email, my proposal is thus: we could
include the original proposal (without numbered addendums) and use tooling,
but have it alleviate the burden another way: have the tooling insert
`[strong self]` by default when autocompleting a block.

BTW, @Manuel Krebber: this proposal would not affect your run-of-the-mill
map statement. array.map { object in object.property } requires no explicit
capture since object is intuitively strongly captured there.

What do we think? Worth discussing?

On Mon, Jun 27, 2016 at 12:10 PM Christopher Kornher via swift-evolution <
swift-evolution@swift.org> wrote:

> On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
>All object references used within a closure must unwrap successfully
>for the closure to execute.
>
> I agree with the logic of this proposal, but this is the confusing part or
> a part that I slightly disagree with.
>
> By this logic, the block won't be invoked if all captured variables can't
> be unwrapped, which is definitely confusing to the newcomer (to whom this
> is partially targetted as you've mentioned) if he puts a breakpoint in it
> and doesn't get executed even when he's explicitely invoking it somewhere.
>
> On the other hand, when it crashes, it gives him some idea that
> something's wrong.
>
>
> Tooling could alleviate some of this mystery. For example:
>
> 1) In Xcode and other GUIs, closures that will not be executed could be
> greyed-out, perhaps with the reason overlaid on the closure perhaps this
> would only happen when a breakpoint was set within the closure. Perhaps the
> app could break on the on breakpoints within non-executing closures and
> notify the user that the closure did not execute.
>
> 2) Debugger console commands could be added to describe the execution
> state of closures in scope and closure variables.
>
> 3) Debug apps could bottleneck all closures through a function that that
> can be break-pointed. Breakpoints could be edited to filter on specific
> closures.
>
> 4) Some runtime switches could be added to enable verbose login modes for
> closures.
>
> I don’t think that this is an insurmountable problem. There are many
> features of modern applications that are difficult to debug.
>
>
>
>
>
>
> I believe that these are safe, sensible and understandable rules that will
> eliminate the need for capture lists for many closures. What do you think?
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 26, 2016, at 11:22 PM, Charlie Monroe via swift-evolution 
>  wrote:
> 
>> All object references used within a closure must unwrap successfully for the 
>> closure to execute.
> I agree with the logic of this proposal, but this is the confusing part or a 
> part that I slightly disagree with.
> 
> By this logic, the block won't be invoked if all captured variables can't be 
> unwrapped, which is definitely confusing to the newcomer (to whom this is 
> partially targetted as you've mentioned) if he puts a breakpoint in it and 
> doesn't get executed even when he's explicitely invoking it somewhere.
> 
> On the other hand, when it crashes, it gives him some idea that something's 
> wrong.

Tooling could alleviate some of this mystery. For example:

1) In Xcode and other GUIs, closures that will not be executed could be 
greyed-out, perhaps with the reason overlaid on the closure perhaps this would 
only happen when a breakpoint was set within the closure. Perhaps the app could 
break on the on breakpoints within non-executing closures and notify the user 
that the closure did not execute.

2) Debugger console commands could be added to describe the execution state of 
closures in scope and closure variables.

3) Debug apps could bottleneck all closures through a function that that can be 
break-pointed. Breakpoints could be edited to filter on specific closures. 

4) Some runtime switches could be added to enable verbose login modes for 
closures.

I don’t think that this is an insurmountable problem. There are many features 
of modern applications that are difficult to debug.

> 
> 
>> 
>> I believe that these are safe, sensible and understandable rules that will 
>> eliminate the need for capture lists for many closures. What do you think?
>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 27, 2016, at 2:45 AM, Manuel Krebber via swift-evolution 
>  wrote:
> 
> On 06/26/2016 09:10 PM, Christopher Kornher via swift-evolution wrote:
>> The core proposal:
>> ——
>> 
>> Closures capturing object references should automatically capture all
>> object references as weak.
> In my code, most closures are used in a functional programming capacity,
> e.g. with map(), reduce, etc. Hence, most closures are non-escaping and
> local, where strong capture is the desired way. Otherwise I would have
> to litter everything with optional unwrapping or add the explicit
> capture definition which would both make the code less readable in my
> opinion.

I thought about this some more and it makes sense to treat non-escaping 
closures are they are treated now. This might increase the burden on the 
compiler especially because these closures may not be declared inline. This 
would be far more straightforward than having to worry about changes to object 
existence within one or more invocations of a @nonescaping closure, especially 
in multi-threaded code.

I do not think that this would be a significant change for developers in 
practice. Any developer who would try to rely upon object references changing 
within the application of a @nonescaping closure would probably have read the 
manual very carefully :)


>> 1) Closures with object references could be simplified further by
>> implicitly including ‘let’ guards for all object references:
> This sounds good for closures without return value, but how would you
> handle closures with non-optional non-void return values?

Good point.

1) The simplest solution would be to have these closures require capture lists 
to some or all object references. Given the prevalence of the weak/strong 
dance, calling these out in some way probably should be done anyway.

2) Another option would be to eliminate the implicit nil check guards, making 
all object references optional. The compiler would force these to be unwrapped, 
so users would be guided to do the right thing.

3) Just always weakly capturing all object references would treat all closures 
uniformly.

> Also I think that explicit error handling in case of "expired" objects
> is safer than silent failure (by not executing the closure).


I believe that it is decision that should be left up to the author of the 
closure. Sometimes it is, sometimes it isn’t.

> 
> Kind regards, Manuel
> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Manuel Krebber via swift-evolution]

On 06/26/2016 09:10 PM, Christopher Kornher via swift-evolution wrote:
> The core proposal:
> ——
>
> Closures capturing object references should automatically capture all
> object references as weak.
In my code, most closures are used in a functional programming capacity,
e.g. with map(), reduce, etc. Hence, most closures are non-escaping and
local, where strong capture is the desired way. Otherwise I would have
to litter everything with optional unwrapping or add the explicit
capture definition which would both make the code less readable in my
opinion.
> 1) Closures with object references could be simplified further by
> implicitly including ‘let’ guards for all object references:
This sounds good for closures without return value, but how would you
handle closures with non-optional non-void return values?
Also I think that explicit error handling in case of "expired" objects
is safer than silent failure (by not executing the closure).

Kind regards, Manuel


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Christopher Kornher via swift-evolution]

> On Jun 26, 2016, at 4:25 PM, Russ Bishop  wrote:
> 
> 
>> On Jun 26, 2016, at 12:10 PM, Christopher Kornher via swift-evolution 
>>  wrote:
>> 
>> I may be too late for Swift 3, but I am planning to propose changes to the 
>> default behavior for closures capturing object references. The introduction 
>> of Swift Playgrounds has raised the importance of simplifying the coding of 
>> leak-free, crash-free closures. New developers should not have to understand 
>> closure memory management to start writing useful and correct code.
>> 
>> The topic of the closure weak/strong dance has been discussed on this list 
>> before. This proposal differs from previous proposals in that it will 
>> eliminate the dance altogether by default. I am very interested in hearing 
>> others’ opinions as to whether the benefits outweigh the costs of various 
>> options.
> 
> The problem is that strong reference capture is probably the far more common 
> case.

Strong reference capture has not been more common in carefully written code in 
my experience. Swift is starting to be used for many different problem domains, 
so your experience may be different. Any examples of real-world code would be 
greatly appreciated.

Sometimes closures contain the only references to objects, but this is not 
common in code that I have seen. It would be extremely rare for strong 
references to be required for all the references in closure with multiple 
references (the current default behavior). Long closures can reference many 
objects and it is all too easy to leave a reference out of the capture list and 
create a reference cycle. I believe that this pattern should be called-out (see 
below).

Strong references are occasionally needed to ensure operations are performed 
when objects would otherwise be reclaimed, but I have not seen many of these 
cases. This pattern could be more common in other frameworks. I believe that 
this pattern should be called-out (see below).

Multiple capture rules for closures can be supported if desired. The 
```[required…``` capture qualifier in the original email is one way todo this. 
The question mark could be used in a way analogous to `try?` to identify 
closures using the proposed rules: 

```let a:()->Void = {…}?```
or 
``` let a:()->Void = ?{…}```
etc.

This would obviously add more complexity but would still be an improvement, I 
believe.


> If you wanted to say that @noescape closures capture references strongly by 
> default, while escaping closures capture them weakly by default that may be 
> closer to reasonable for most situations but now you have magic behavior with 
> an even greater cognitive overhead. (I wonder if it would be more common that 
> @noescape captures strongly, escaping captures self weak by default but other 
> objects strongly… of course that would have even worse cognitive overhead.)

I did consider treating self differently, but this leads to some very strange 
cases when delegation, factories and other patterns are considered. 

This email was implicitly referring to escaping uses of closures. The same 
closure can be used as escaping or non-escaping. The Swift documentation states:

 "Marking a closure with @noescape lets the compiler make more aggressive 
optimizations because it knows more information about the closure’s lifespan."

@noescape is essentially a hint to the compiler. Optimizers would be free to 
use strong or unowned references if they can determine that it is safe to do so 
without altering behavior. 

> No matter what, without a garbage collector you are stuck with sub-optimal 
> solutions for fixing reference cycles. I could imagine a language feature 
> that automatically detected trivial cycles (A -> B -> A) but anything more 
> complex just becomes a form of garbage collection anyway.

Object networks are difficult enough with ARC without dozens of closures with 
unnecessary strongly captured references. The new Xcode tools will be a huge 
help with leaks, but they should not be required.

> 
> I don’t think there is a way to square this circle. Either you have one 
> “automagic” behavior that is wrong for some cases (whether strong or weak is 
> the default), or you require everyone to spam their closures with explicit 
> capture annotations even if there’s a default “capture everything strongly” 
> variant (see C++ lambdas).

Yes, we are discussing tradeoffs. Writing correct code with closures will 
always require care.I believe that is is better to have safe, leak free code by 
default than not. I also believe that capturing strong references by default 
can probably lead to at least as many unexpected behaviors as capturing weak 
references. I don’t think that many developers would expect, for example, to 
see zombie view controllers that have not been associated with an active view 
hierarchy for weeks because a closure is holding on to a strong reference.

> 
>> 
>> Use of ‘unowned’ 
>> 
>> 

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Charlie Monroe via swift-evolution]

> All object references used within a closure must unwrap successfully for the 
> closure to execute.
I agree with the logic of this proposal, but this is the confusing part or a 
part that I slightly disagree with.

By this logic, the block won't be invoked if all captured variables can't be 
unwrapped, which is definitely confusing to the newcomer (to whom this is 
partially targetted as you've mentioned) if he puts a breakpoint in it and 
doesn't get executed even when he's explicitely invoking it somewhere.

On the other hand, when it crashes, it gives him some idea that something's 
wrong.


> 
> I believe that these are safe, sensible and understandable rules that will 
> eliminate the need for capture lists for many closures. What do you think?
> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Russ Bishop via swift-evolution]

> On Jun 26, 2016, at 12:10 PM, Christopher Kornher via swift-evolution 
>  wrote:
> 
> I may be too late for Swift 3, but I am planning to propose changes to the 
> default behavior for closures capturing object references. The introduction 
> of Swift Playgrounds has raised the importance of simplifying the coding of 
> leak-free, crash-free closures. New developers should not have to understand 
> closure memory management to start writing useful and correct code.
> 
> The topic of the closure weak/strong dance has been discussed on this list 
> before. This proposal differs from previous proposals in that it will 
> eliminate the dance altogether by default. I am very interested in hearing 
> others’ opinions as to whether the benefits outweigh the costs of various 
> options.

The problem is that strong reference capture is probably the far more common 
case.

If you wanted to say that @noescape closures capture references strongly by 
default, while escaping closures capture them weakly by default that may be 
closer to reasonable for most situations but now you have magic behavior with 
an even greater cognitive overhead. (I wonder if it would be more common that 
@noescape captures strongly, escaping captures self weak by default but other 
objects strongly… of course that would have even worse cognitive overhead.)

No matter what, without a garbage collector you are stuck with sub-optimal 
solutions for fixing reference cycles. I could imagine a language feature that 
automatically detected trivial cycles (A -> B -> A) but anything more complex 
just becomes a form of garbage collection anyway.

I don’t think there is a way to square this circle. Either you have one 
“automagic” behavior that is wrong for some cases (whether strong or weak is 
the default), or you require everyone to spam their closures with explicit 
capture annotations even if there’s a default “capture everything strongly” 
variant (see C++ lambdas).

> 
> 
> Use of ‘unowned’ 
> 
> 
> I now routinely create closures that capture `self` and other object 
> references as ‘weak’ even if I think that I feel that `unowned ` would be 
> safe. This may not be the absolutely most performant solution, but it is 
> straightforward and robust.

I agree and our team has adopted the rule that use of unowned is not allowed 
unless the declaration is private and there is profiler proof that it 
represents a performance problem, and if used warning comments must be placed 
in the code. Weak is almost never a performance problem and eliminates the risk 
of a crash, so it is highly preferable to unowned.

I’d go so far as to say unowned should be removed; let the user use 
Unmanaged if they need to capture an unowned reference. They’re entering 
expert territory anyway.



> 
> The core proposal:
> ——
> 
> Closures capturing object references should automatically capture all object 
> references as weak.
> 

This becomes a form of the Objective-C messages-to-nil-do-nothing problem where 
you don’t crash but your closure doesn’t do any work (or does half the work!) 
because the reference(s) are/become nil. It doesn’t save you from reasoning 
about object lifetime because it is just as easy for the closure to capture the 
last reference to an object or for the lifetime to differ from the closure 
lifetime. You’re just trading reference cycles for a different problem.


> 
> 
> 2) Some of the magic in #1 could be eliminated by introducing a new capture 
> type:  ‘required’ to specify ‘weak guarded’ captures, allowing the example 
> closure to be written:
> 

This has been debated before. I support the idea of a “required” capture 
specifier but IIRC the core team was not supportive of the idea because it adds 
another layer of magic on the existing magic (object deallocated magically 
means your closure never executes).

Russ

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] An upcoming proposal for simplifying leak free, safe closures.

[Charles Srstka via swift-evolution]

I like it. Accidental capture of a variable is probably one of the primary 
causes of retain cycles in modern Swift code. Requiring the capture to be 
explicit would take a lot of the surprise out of it and force the developer to 
think about the capture semantics.

+1.

Charles

> On Jun 26, 2016, at 2:10 PM, Christopher Kornher via swift-evolution 
>  wrote:
> 
> I may be too late for Swift 3, but I am planning to propose changes to the 
> default behavior for closures capturing object references. The introduction 
> of Swift Playgrounds has raised the importance of simplifying the coding of 
> leak-free, crash-free closures. New developers should not have to understand 
> closure memory management to start writing useful and correct code.
> 
> The topic of the closure weak/strong dance has been discussed on this list 
> before. This proposal differs from previous proposals in that it will 
> eliminate the dance altogether by default. I am very interested in hearing 
> others’ opinions as to whether the benefits outweigh the costs of various 
> options.
> 
> I have found that Swift’s capture lists and rules are a bit of a mystery to 
> many experienced developers, even though Swift’s closure capture rules are 
> very similar to those of Objective-C. Capture lists are probably thought of 
> as opaque incantations by many new Swift developers if they are aware of them 
> at all. Capture lists should, ideally, not be needed for sensible and safe 
> default behavior.
> 
> This discussion is iOS / OS X centric and uses terms from those domains, but 
> these issues should be applicable to almost any codebase that uses closures 
> capturing object references.
> 
> Capture lists are required by the fact that object references are captured as 
> `strong` by default, often leading to strong reference cycles and memory 
> leaks.
> 
> Use of ‘unowned’ 
> 
> Many examples of using closures capture self as `unowned`. Often, this 
> pattern does not scale well beyond simple examples. iOS and MacOS 
> applications with dynamic UIs, for example, switch between numerous views and 
> view controllers. These objects are dereferenced and reclaimed when they are 
> no longer needed. Closures capturing these objects as `unowned` crash when 
> the references are accessed.
> 
> Unfortunately, ‘unowned’ captures are tempting because they eliminate `guard` 
> and `if let` constructs and avoid littering code with optional unwrapping. 
> They are also slightly more performant, but this difference is probably 
> negligible in most application code.
> 
> Capturing escaping object references as `unowned` is only safe when object 
> lifetimes are perfectly understood. In complex systems, it is difficult to 
> predict execution order. Even if object lifetimes are perfectly understood 
> when code is originally written, seemingly innocuous changes to complex 
> systems can negate original assumptions.
> 
> For these reasons, I believe that capturing object references as `unowned` 
> should be considered an advanced optimization technique.
> 
> I now routinely create closures that capture `self` and other object 
> references as ‘weak’ even if I think that I feel that `unowned ` would be 
> safe. This may not be the absolutely most performant solution, but it is 
> straightforward and robust.
> 
> 
> The core proposal:
> ——
> 
> Closures capturing object references should automatically capture all object 
> references as weak.
> 
> The closure defined in:
> 
> ```
> class ClosureOwner2 {
> var aClosure: (() -> Void)?
> 
> func aMethod() {
> aClosure = { [weak self] in
>self?.someOtherMethod()
>}
> }
> 
> func someOtherMethod(){}
> }
> ```
> 
> would normally be written as: 
> 
> ```
> aClosure = {
>self?.someOtherMethod()
> }
> ```
> Closures that can be optimized to safely capture object references as 
> `unowned` can use the current syntax.
> 
> Swift 2 closure without explicit capture lists for object references will not 
> compile.
> 
> Capturing strong object references can be very useful in certain 
> circumstances and have a straightforward syntax:
> 
> ```
> aClosure = { [strong self] in
>self.someOtherMethod()
> }
> ```
> 
> 
> Alternatives / Modifications / Improvements(?):
> —
> 
> 1) Closures with object references could be simplified further by implicitly 
> including ‘let’ guards for all object references: 
>  
> aClosure = {
>// This is included implicitly at the top of the closure:
>   // guard let strongSelf = self else { return }
> 
> /*strongSelf*/ self.someOtherMethod()
>   print( “This will not appear when self is nil.” )
> 
>   … other uses of strongSelf…
> 
> }
> 
> This would have the effect of making the execution of the closure dependent 
> upon the successful unwrapping of all of its object