Re: [swift-dev] Continuation-based versus nested-function-based materializeForSet

[Joe Groff via swift-dev]

> On Nov 1, 2016, at 3:32 PM, John McCall  wrote:
> 
>> On Oct 31, 2016, at 12:22 PM, Joe Groff via swift-dev  
>> wrote:
>> We currently abstract over mutable property accesses using what I’ll call a 
>> continuation-based model–the materializeForSet accessor is called before an 
>> inout access, and returns a continuation callback that must be called when 
>> the inout access is completed. I think a nested-function-based model, if 
>> designed correctly, has the potential to be more efficient and easier to 
>> implement.
> 
> Well, I'm not totally convinced about either of those two points, but it's 
> definitely something we need to talk about.
> 
> Let me try to break this down.  I think I'm going to repeat you a lot in 
> different words.
> 
> We have a coroutine-esque problem:
>  0. Outer is executing.
>  1. Outer calls Inner.
>  2. Inner executes for a while.
>  3. Inner passes control back to Outer without abandoning its execution 
> context.
>  4. Outer executes for awhile.
>  5. Outer passes control back to Inner, potentially having changed its own 
> execution context.
>  6. Inner executes for awhile.
>  7. Inner finally abandons its execution context and passes control back to 
> Outer.
>  8. Outer is executing.
> 
> Note that there might be multiple static points for (3) and (5), each 
> requiring a different resumption point (and creating different context 
> changes) at (5) and (7) respectively.
> 
> (I'm going to use the terms "standard spilling", "subfunction spilling", and 
> "transfer spilling" without defining them.  I'll come back to this.)
> 
> I. Lowerings
> 
> Currently, as you say, we do this with a sort of CPS conversion where Inner 
> is split into multiple sub-functions.  The initial function is passed a 
> (necessarily fixed-size, opaque to Outer) buffer; at (3), it saves its 
> context into that buffer and returns a sub-function that Outer will invoke 
> (passing the same buffer) at point (5); at (5), that sub-function restores 
> its context from that buffer; and finally the sub-function returns normally 
> at (7).  This has two advantages: Outer can reason locally about the data 
> flow from (0) to (4) to (8), and Inner can return a null pointer at (3) to 
> indicate that there's no work to do in (6) and thus save an indirect branch 
> (at the cost of a local conditional branch).  However, if Inner has 
> interesting context to save at (3), that requires extra code to manage that 
> data flow, and the amount of data to save can overflow the buffer and 
> therefore require malloc.  Total branching cost: the original call at (1), 
> the easily-predicted return at (3), the conditional branch at (5), the 
> possible indirect call at (5), and the easily-predicted return at (7).  Total 
> save/restore cost: standard spilling at (1-3) and (5-7), transfer spilling at 
> (3-5), possible malloc.
> 
> The simplest alternative proposal is a callback conversion, extracting (4) 
> from Outer as a sub-function which is passed to Inner.  Inner now gets an 
> ordinary stack frame to save its context, so its data flow and control flow 
> are easier for the backend to reason about, and malloc is not required.  
> However, the data/control flow in Outer is significantly obscured; most 
> importantly, different static points for (5) may need to resume to different 
> instructions at (7) (which the continuation lowering implicitly handles by 
> duplicating the call in (5)), and Outer's context will be modified at various 
> points from (1-7).  Total branching cost: the original call at (1), the 
> indirect call at (3), the easily-predicted return at (5), the 
> easily-predicted return at (7), the possible switch-out at (7).  Total 
> save/restore cost: subfunction spilling at (1-7), standard spilling at (3-5), 
> only stack allocations.
> 
> Note that the call at (5) needs to duplicated for continuation lowering in 
> exactly those cases where we need a switch-out at (7) in callback conversion.
> 
> Your observation is that, if Outer needs to perform a nested access with this 
> sequence at (4), a naive callback conversion will have a lot of branches.  
> Let the outer access be (A), with its steps being (A.0-8), and the inner 
> access be (B), with its steps being (B.0-8), so we have (A.4) := (B.0-8).  If 
> the nesting is simple — i.e. there is no code in B.0 or B.8 — and the 
> initiation sequences are compatible — i.e. you can convince (A.3) to directly 
> make the original call for (B.1) — then it is unnecessary to extract a 
> sub-function for (A.4), and instead (A.1) can just set up the chain of calls. 
>  Total branching cost: the original call at (A.1), the indirect call at (A.3 
> / B.1), the indirect call at (B.3), the easily-predicted return at (B.5), the 
> easily-predicted return at (B.7 / A.5), the easily-predicted return at (A.7), 
> the possible switch-out at (A.7).  Total save/restore cost: subfunction 
> spilling at (A.1-7), standard 

Re: [swift-dev] Continuation-based versus nested-function-based materializeForSet

[Joe Groff via swift-dev]

> On Nov 2, 2016, at 11:38 AM, John McCall  wrote:
> 
>> On Nov 2, 2016, at 9:05 AM, Joe Groff via swift-dev  
>> wrote:
>>> On Nov 1, 2016, at 9:23 PM, Slava Pestov  wrote:
>>> 
 
 On Nov 1, 2016, at 11:00 AM, Jordan Rose via swift-dev 
  wrote:
 
 - Does this help us with the nested dictionary CoW problem? 
 `foo["bar"]["baz"] += 1`
>>> 
>>> My understanding is that this problem arises because we don’t have 
>>> ‘optional addressors’. A dictionary lookup might return nil. If addressors 
>>> had a way to return an optional wrapping a pointer, and optional evaluation 
>>> supported this, we could do in-place updates of this kind. For Apple 
>>> people: I have a radar that I think describes this problem 
>>> (rdar://17509619).
>> 
>> I think our long-term goal here is to let the property definition fully 
>> control the `inout` access with a coroutine-like definition, something like:
>> 
>> var foo: T {
>> get { return /*value for read-only access*/ }
>> set { _foo = /*write-only access*/ }
>> mutate {
>>   var tmp = prepareForMutation()
>>   yield  // continue nested inout access
>>   reconcileMutation(tmp)
>> }
>> }
>> 
>> This should let the dictionary implementation do whatever it needs to 
>> present a moved Optional value to the chained access. If Dictionary can't 
>> store Optionals directly inline in all cases, we ought to still be able to 
>> rely on enforced inout uniqueness to get away with moving in and out of a 
>> temporary.
> 
> It's tricky, because if Dictionary is going to support non-movable values, 
> then we also want the non-mutating accessor to be able to return a 
> Borrowed, which it can't do if that representation has to be a 
> pointer to an Optional.  It can returned an Optional just 
> fine, of course.
> 
> This relates to the earlier conversation we had about borrowed tuples.  We 
> may have to make the representation of a Borrowed potentially different 
> from just a T*, and that really stinks.

Being able to model an access as Optional or Optional 
might fit better with the move-only types model for dictionaries, and would be 
useful for other things like optional protocol members and properties found on 
AnyObject. Dictionaries have the "delete" operation to shoehorn into the `inout 
Optional` model and give `foo["bar"] = nil` meaning, but there are other 
mutation interfaces for which that isn't true. (Even for dictionaries, I've 
always felt it was a bit of a hack.)

-Joe___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568

[David P Grove via swift-corelibs-dev]

I think the root problem is that some of the dispatch tests, including this
one, are trying to test that timers/events happen within a "reasonable"
interval of time from when they are supposed to.   If deadlines are missed
too badly, then the test is marked as failing.

If we completely disabled all such tests, we run the risk of breaking the
timer/event subsystem of libdispatch and not noticing.

However, on the CI system there is unpredictable load, so "reasonable" is
hard to specify and with some probability, deadlines will be missed no
matter how loosely they are set.

If this test is failing frequently and the infrastructure does not support
rerunning  or ignoring a flagged "flakey" test then disabling it is
probably the expedient option.

Probably what we need to do is strip down to (1) a very reliable subset of
the dispatch tests that are run on most CI jobs and a (2) more
comprehensive set of tests that is run in a separate job just for
libdispatch testing.  Job (2) can afford to include tests with low
probability failures because only the dispatch team would need to pay
attention to them and decide if they were real failures or not.  Is
something like this possible to set up?

--dave



From:   Doug Coleman via swift-corelibs-dev

To: Jordan Rose 
Cc: swift-dev , swift-corelibs-dev

Date:   11/02/2016 01:48 PM
Subject:Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure:
0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568
Sent by:swift-corelibs-dev-boun...@swift.org



It has also been reported as a dup here:
https://bugs.swift.org/browse/SR-3110

It’s important that we fix this one so we stop wasting five minutes of many
peoples’ time several times a day.

Maybe we should disable it until it’s fixed?

  On Nov 2, 2016, at 10:46 AM, Jordan Rose 
  wrote:

  Adding swift-corelibs-dev, since that’s where the Dispatch folks hang
  out.

On Nov 2, 2016, at 10:44, Doug Coleman via swift-dev <
swift-...@swift.org> wrote:

https://bugs.swift.org/browse/SR-2931


  On Nov 2, 2016, at 10:43 AM, no-re...@swift.org wrote:

  [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10
  [#8568]

   
 Build URL: 
https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ 

   
  Project:  oss-swift-incremental-RA-linux-ubuntu-15_10 
   

   
  Date of   Wed, 02 Nov 2016 10:17:41 -0700 
   
   build:   
   

   
   Build25 min  
   
 duration:  
   

   

  Tests:



 Name: Swift(linux-x86_64)  
 Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s)   



 Name: Swift-Unit   
 Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s) 





  Changes
Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by
practicalswift:


[swiftc (94 vs. 5180)] Add crasher in ?
  add:
  
validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift

___
swift-dev mailing list
swift-...@swift.org

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568

[Doug Coleman via swift-dev]

It has also been reported as a dup here: https://bugs.swift.org/browse/SR-3110

It’s important that we fix this one so we stop wasting five minutes of many 
peoples’ time several times a day.

Maybe we should disable it until it’s fixed?

> On Nov 2, 2016, at 10:46 AM, Jordan Rose  wrote:
> 
> Adding swift-corelibs-dev, since that’s where the Dispatch folks hang out.
> 
>> On Nov 2, 2016, at 10:44, Doug Coleman via swift-dev > > wrote:
>> 
>> https://bugs.swift.org/browse/SR-2931 
>> 
>> 
>>> On Nov 2, 2016, at 10:43 AM, no-re...@swift.org  
>>> wrote:
>>> 
>>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10 [#8568]
>>> 
>>> Build URL:  
>>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ 
>>> 
>>> Project:oss-swift-incremental-RA-linux-ubuntu-15_10
>>> Date of build:  Wed, 02 Nov 2016 10:17:41 -0700
>>> Build duration: 25 min
>>> Tests:
>>> 
>>> Name: Swift(linux-x86_64)
>>> Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s)
>>> Name: Swift-Unit
>>> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
>>> 
>>> Changes
>>> 
>>> Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by practicalswift:
>>> [swiftc (94 vs. 5180)] Add crasher in ?
>>> 
>>> add: 
>>> validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568

[Doug Coleman via swift-corelibs-dev]

It has also been reported as a dup here: https://bugs.swift.org/browse/SR-3110

It’s important that we fix this one so we stop wasting five minutes of many 
peoples’ time several times a day.

Maybe we should disable it until it’s fixed?

> On Nov 2, 2016, at 10:46 AM, Jordan Rose  wrote:
> 
> Adding swift-corelibs-dev, since that’s where the Dispatch folks hang out.
> 
>> On Nov 2, 2016, at 10:44, Doug Coleman via swift-dev > > wrote:
>> 
>> https://bugs.swift.org/browse/SR-2931 
>> 
>> 
>>> On Nov 2, 2016, at 10:43 AM, no-re...@swift.org  
>>> wrote:
>>> 
>>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10 [#8568]
>>> 
>>> Build URL:  
>>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ 
>>> 
>>> Project:oss-swift-incremental-RA-linux-ubuntu-15_10
>>> Date of build:  Wed, 02 Nov 2016 10:17:41 -0700
>>> Build duration: 25 min
>>> Tests:
>>> 
>>> Name: Swift(linux-x86_64)
>>> Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s)
>>> Name: Swift-Unit
>>> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
>>> 
>>> Changes
>>> 
>>> Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by practicalswift:
>>> [swiftc (94 vs. 5180)] Add crasher in ?
>>> 
>>> add: 
>>> validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift
>> 
>> ___
>> swift-dev mailing list
>> swift-...@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 

___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568

[Jordan Rose via swift-dev]

Adding swift-corelibs-dev, since that’s where the Dispatch folks hang out.

> On Nov 2, 2016, at 10:44, Doug Coleman via swift-dev  
> wrote:
> 
> https://bugs.swift.org/browse/SR-2931 
> 
> 
>> On Nov 2, 2016, at 10:43 AM, no-re...@swift.org wrote:
>> 
>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10 [#8568]
>> 
>> Build URL:   
>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ 
>> 
>> Project: oss-swift-incremental-RA-linux-ubuntu-15_10
>> Date of build:   Wed, 02 Nov 2016 10:17:41 -0700
>> Build duration:  25 min
>> Tests:
>> 
>> Name: Swift(linux-x86_64)
>> Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s)
>> Name: Swift-Unit
>> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
>> 
>> Changes
>> 
>> Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by practicalswift:
>> [swiftc (94 vs. 5180)] Add crasher in ?
>> 
>> add: 
>> validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-corelibs-dev] [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568

[Jordan Rose via swift-corelibs-dev]

Adding swift-corelibs-dev, since that’s where the Dispatch folks hang out.

> On Nov 2, 2016, at 10:44, Doug Coleman via swift-dev  
> wrote:
> 
> https://bugs.swift.org/browse/SR-2931 
> 
> 
>> On Nov 2, 2016, at 10:43 AM, no-re...@swift.org wrote:
>> 
>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10 [#8568]
>> 
>> Build URL:   
>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ 
>> 
>> Project: oss-swift-incremental-RA-linux-ubuntu-15_10
>> Date of build:   Wed, 02 Nov 2016 10:17:41 -0700
>> Build duration:  25 min
>> Tests:
>> 
>> Name: Swift(linux-x86_64)
>> Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s)
>> Name: Swift-Unit
>> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
>> 
>> Changes
>> 
>> Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by practicalswift:
>> [swiftc (94 vs. 5180)] Add crasher in ?
>> 
>> add: 
>> validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift
> 
> ___
> swift-dev mailing list
> swift-...@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 15.10 (master) #8568

[Doug Coleman via swift-dev]

https://bugs.swift.org/browse/SR-2931


> On Nov 2, 2016, at 10:43 AM, no-re...@swift.org wrote:
> 
> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-15_10 [#8568]
> 
> Build URL:
> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-15_10/8568/ 
> 
> Project:  oss-swift-incremental-RA-linux-ubuntu-15_10
> Date of build:Wed, 02 Nov 2016 10:17:41 -0700
> Build duration:   25 min
> Tests:
> 
> Name: Swift(linux-x86_64)
> Failed: 0 test(s), Passed: 8487 test(s), Total: 8487 test(s)
> Name: Swift-Unit
> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
> 
> Changes
> 
> Commit 3575ee1954ea68f3d569c1f4481ca850d9fe87a3 by practicalswift:
> [swiftc (94 vs. 5180)] Add crasher in ?
> 
> add: 
> validation-test/compiler_crashers/28463-child-source-range-not-contained-within-its-parent-guard-stmt.swift

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Continuation-based versus nested-function-based materializeForSet

[Joe Groff via swift-dev]

> On Nov 1, 2016, at 9:23 PM, Slava Pestov  wrote:
> 
>> 
>> On Nov 1, 2016, at 11:00 AM, Jordan Rose via swift-dev  
>> wrote:
>> 
>> - Does this help us with the nested dictionary CoW problem? 
>> `foo["bar"]["baz"] += 1`
> 
> My understanding is that this problem arises because we don’t have ‘optional 
> addressors’. A dictionary lookup might return nil. If addressors had a way to 
> return an optional wrapping a pointer, and optional evaluation supported 
> this, we could do in-place updates of this kind. For Apple people: I have a 
> radar that I think describes this problem (rdar://17509619).

I think our long-term goal here is to let the property definition fully control 
the `inout` access with a coroutine-like definition, something like:

var foo: T {
  get { return /*value for read-only access*/ }
  set { _foo = /*write-only access*/ }
  mutate {
var tmp = prepareForMutation()
yield  // continue nested inout access
reconcileMutation(tmp)
  }
}

This should let the dictionary implementation do whatever it needs to present a 
moved Optional value to the chained access. If Dictionary can't store Optionals 
directly inline in all cases, we ought to still be able to rely on enforced 
inout uniqueness to get away with moving in and out of a temporary.

-Joe
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-corelibs-dev] Server APIs: networking and security kick-off meetings

[Chris Bailey via swift-corelibs-dev]

In order to try to schedule kick-off meetings for the Server APIs focus 
areas, I've set up Doodle polls for the networking and security kick-offs, 
with the aim of trying to hold those this week:
Server APIs: Networking sub-team kickoff
http://doodle.com/poll/b5vy7s8vs9pbcb7y
Server APIs: Security sub-team kickoff  
http://doodle.com/poll/6xc84g5kfgv3ebv5
we'll follow up with the HTTP kick-off once we have these two scheduled.

The Doodle poll lets you set what time(s) you'd be able to make a call - 
we'll use either Webex or Google Hangouts for the meeting, so you'll be 
able to use your network connection to take part. We won't be recording 
the call, but we will publish full agenda and minutes to GitHub 
afterwards.

If you'd like to take part, but can't make any of the available times, 
fill in your name and select "Cannot make it". If there's enough people 
that cannot make it, we'll postpone to a later date.

Chris___
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev