Re: [swift-dev] Testing, validation, and many-core machines

[Michael Gottesman via swift-dev]

> On Jan 17, 2018, at 8:23 AM, David Zarzycki via swift-dev 
>  wrote:
> 
> Hello,
> 
> In docs/Testing.md, “long test” is acknowledged but never defined. When 
> should tests be marked “long_test”? I’m asking because there are arguably a 
> number of latent long_test tests that are exposed on many-core machines.

I think about it this way. A long test is a test that we want to run, but would 
bog down PR testing or are enough of a validation test that the trade-off in 
between signal/PR testing delays is found to be lacking. That being said I 
think of it as an approximation.

> 
> Speaking of many-core machines, there really isn’t a practical difference 
> between smoke/primary testing and secondary/validation testing. Both test 
> suites rush through 95-99% of the tests in very short order, and then wait a 
> relatively long time for the remaining tests to finish. Can we move some of 
> the exhaustive/stress tests in the “primary” suite to the validation suite? 
> Likewise, can we move some of the short/functional tests in the validation 
> directory to the primary test directory? That would match people’s 
> expectations better, would it not?

Do you have specific test? Again, it isn't just about test time. It is about 
the trade-off in between signal and length of time.

> 
> Dave
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] JIRA down

[Michael Gottesman via swift-dev]

K

Sent from my iPhone

> On Jan 8, 2018, at 7:57 AM, Mishal Shah  wrote:
> 
> bugs.swift.org is back online.
> 
> Please also email swift-infrastruct...@swift.org in the future. 
> 
> Thanks,
> Mishal Shah
> 
>> On Jan 7, 2018, at 4:35 PM, Michael Gottesman  wrote:
>> 
>> The JIRA looks like it is down...
>> 
>> Michael
> 
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] JIRA down

[Michael Gottesman via swift-dev]

The JIRA looks like it is down...

Michael
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

> On Jan 2, 2018, at 7:00 PM, Joe Groff <jgr...@apple.com> wrote:
> 
> 
> 
>> On Dec 30, 2017, at 11:23 AM, Michael Gottesman via swift-dev 
>> <swift-dev@swift.org> wrote:
>> 
>> No. A parameter's convention is ABI. You don't want to change ABI related 
>> things like that via escape analysis since it means that as a function 
>> changes, due to the optimizer, the ABI can change =><=. *BUT* recently Adam 
>> Nemet has added support for LLVM's opt remark infrastructure to Swift. 
>> Something like that /could/ provide suggestions when it is run that this 
>> might be a profitable parameter to change from +0 to +1 or vis-a-versa. Then 
>> the user could make that change via annotation. Keep in mind that this would 
>> most likely be an ABI breaking change.
> 
> ABI only matters at public entry points for ABI-stable binaries. We can do 
> this sort of analysis freely within modules, or in a hypothetical 
> multi-module-optimization mode.

Sure. We were talking in the general case though which does include public 
entry points of ABI-stable binaries. So my statement was correct.

Michael

> 
> -Joe

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Weird dyn_cast code gen / performance problem

[Michael Gottesman via swift-dev]

I posted a quick comment there + the llvm ir. It seems like something that 
could be a simple SimplifyInstructions transformation.

Michael

> On Jan 1, 2018, at 6:37 PM, David Zarzycki  wrote:
> 
> 
> 
>> On Jan 1, 2018, at 18:20, John McCall  wrote:
>> 
>>> 
>>> On Jan 1, 2018, at 5:51 PM, David Zarzycki via swift-dev 
>>>  wrote:
>>> 
>>> Hi Michael,
>>> 
>>> I reduced it down to a simple test case. I was wrong about this requiring 
>>> two or more dyn_casts. This actually affects any C++ code that uses the “if 
>>> (auto x = y(z))” convention. What follows is the reduction (compiled with 
>>> “clang++ -O3 -c” if it matters):
>>> 
>>> // Uncomment the next line to see the expected code gen (albeit not inlined)
>>> //__attribute__((used,noinline))
>>> int *x(void *arg) {
>>>   return ((long long)arg & 1) ? (int *)arg : nullptr;
>>> }
>>> 
>>> int test(void *arg) {
>>>   if (auto y = x(arg))
>>> return *y;
>>>   return 42;
>>> }
>>> 
>>> It seems like inlining ‘x’ causes the compiler to effectively generate the 
>>> following pseudo-code:
>>> 
>>> int test(void *arg) {
>>>   if (arg != nullptr)
>>> if (arg & 1)
>>>   return *arg;
>>>   return 42;
>>> }
>>> 
>>> Which is surprising in multiple ways and (as far as I can tell) difficult 
>>> to workaround without lots of source churn.
>>> 
>>> Where should I file a bug?
>> 
>> bugs.llvm.org would be best.  Including both your reduced test case and the 
>> fact that it was reduced from dyn_cast patterns should make them sit up and 
>> take notice.
> 
> Thanks John,
> 
> In case anybody wants to cherry-pick the fix once it is found:
>   https://bugs.llvm.org/show_bug.cgi?id=35790
> 
>> John.
>> 
>>> 
>>> Dave
>>> 
>>> 
 On Jan 1, 2018, at 13:10, David Zarzycki via swift-dev 
  wrote:
 
 I don’t have the IR handy. You can easily generate it for yourself though. 
 Just drop the following into any file (I use swift/lib/AST/Type.cpp) and 
 recompile swift.
 
 Decl *my_test_function(Type t) {
   return t->getClassOrBoundGenericClass();
 }
 
 
> On Jan 1, 2018, at 12:53, Michael Gottesman  wrote:
> 
> Do you have the llvm-ir handy?
> 
>> On Jan 1, 2018, at 11:30 AM, David Zarzycki via swift-dev 
>>  wrote:
>> 
>> Hello,
>> 
>> I noticed recently that the code gen of 
>> CanType::getClassOrBoundGenericClass() could be better and along the way 
>> I found a clang/LLVM bug. Where exactly, I do not know, although my bet 
>> is the LLVM optimizer.
>> 
>> When more than one dyn_cast() happens in a row, LLVM/clang emits 
>> redundant and pointless nullptr checks. Both Apple clang-900.0.39.2 and 
>> clang/llvm top-of-tree generate essentially the same code:
>> 
>> <+35>: movb   0x8(%rbx), %cl  ; getKind()
>> <+38>: testq  %rbx, %rbx  ; XXX - nullptr check after deref is 
>> pointless
>> <+41>: je 0x1377df6   ; <+54>
>> <+43>: cmpb   $0x12, %cl  ; isa()
>> <+46>: jne0x1377df6   ; <+54>
>> <+48>: addq   $0x10, %rbx ; (void*)this + offsetof(ClassType, 
>> TheDecl)
>> <+52>: jmp0x1377e06   ; <+70>
>> <+54>: xorl   %eax, %eax  ; the default return value (nullptr)
>> <+56>: testq  %rbx, %rbx  ; XXX - another pointless nullptr check?
>> <+59>: je 0x1377e09   ; <+73>
>> <+61>: cmpb   $0x29, %cl  ; isa()
>> <+64>: jne0x1377e09   ; <+73>
>> <+66>: addq   $0x18, %rbx ; (void*)this + 
>> offsetof(BoundGenericClassType, TheDecl)
>> <+70>: movq   (%rbx), %rax; load the decl pointer
>> <+73>: popq   %rbx
>> <+74>: retq   
>> 
>> I’ve tried adding different “nonnull” spellings in various parts of both 
>> Swift and LLVM’s casting machinery, but with no luck. The only thing 
>> that seems to work is to create a free function that takes a non-null 
>> “const TypeBase *” parameter and then have 
>> CanType::getClassOrBoundGenericClass() call that.
>> 
>> FWIW – I *suspect* this is because LLVM’s casting machinery internally 
>> converts traditional pointers into C++ references before ultimately 
>> calling classof().
>> 
>> Before I file a bug against clang/llvm, might I be missing something? 
>> Can anybody think of a good workaround?
>> 
>> Dave
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 
 
 ___
 swift-dev mailing list
 swift-dev@swift.org
 https://lists.swift.org/mailman/listinfo/swift-dev
>>> 
>>> ___
>>> swift-dev mailing list
>>> swift-dev@swift.org
>>> 

Re: [swift-dev] Weird dyn_cast code gen / performance problem

[Michael Gottesman via swift-dev]

Do you have the llvm-ir handy?

> On Jan 1, 2018, at 11:30 AM, David Zarzycki via swift-dev 
>  wrote:
> 
> Hello,
> 
> I noticed recently that the code gen of 
> CanType::getClassOrBoundGenericClass() could be better and along the way I 
> found a clang/LLVM bug. Where exactly, I do not know, although my bet is the 
> LLVM optimizer.
> 
> When more than one dyn_cast() happens in a row, LLVM/clang emits redundant 
> and pointless nullptr checks. Both Apple clang-900.0.39.2 and clang/llvm 
> top-of-tree generate essentially the same code:
> 
> <+35>: movb   0x8(%rbx), %cl  ; getKind()
> <+38>: testq  %rbx, %rbx  ; XXX - nullptr check after deref is pointless
> <+41>: je 0x1377df6   ; <+54>
> <+43>: cmpb   $0x12, %cl  ; isa()
> <+46>: jne0x1377df6   ; <+54>
> <+48>: addq   $0x10, %rbx ; (void*)this + offsetof(ClassType, TheDecl)
> <+52>: jmp0x1377e06   ; <+70>
> <+54>: xorl   %eax, %eax  ; the default return value (nullptr)
> <+56>: testq  %rbx, %rbx  ; XXX - another pointless nullptr check?
> <+59>: je 0x1377e09   ; <+73>
> <+61>: cmpb   $0x29, %cl  ; isa()
> <+64>: jne0x1377e09   ; <+73>
> <+66>: addq   $0x18, %rbx ; (void*)this + offsetof(BoundGenericClassType, 
> TheDecl)
> <+70>: movq   (%rbx), %rax; load the decl pointer
> <+73>: popq   %rbx
> <+74>: retq   
> 
> I’ve tried adding different “nonnull” spellings in various parts of both 
> Swift and LLVM’s casting machinery, but with no luck. The only thing that 
> seems to work is to create a free function that takes a non-null “const 
> TypeBase *” parameter and then have CanType::getClassOrBoundGenericClass() 
> call that.
> 
> FWIW – I *suspect* this is because LLVM’s casting machinery internally 
> converts traditional pointers into C++ references before ultimately calling 
> classof().
> 
> Before I file a bug against clang/llvm, might I be missing something? Can 
> anybody think of a good workaround?
> 
> Dave
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

Felix, on a completely different note. Do you think you could contribute these 
workloads to the swift benchmark suite. Then we can track it over time.

Michael

> On Dec 30, 2017, at 10:03 PM, Michael Gottesman via swift-dev 
> <swift-dev@swift.org> wrote:
> 
> 
> 
>> On Dec 30, 2017, at 9:41 PM, Félix Cloutier <felixclout...@icloud.com> wrote:
>> 
>> 
>>> Le 30 déc. 2017 à 14:22, Michael Gottesman <mgottes...@apple.com> a écrit :
>>>> That sounds fun. I'll have to check with my manager after the holidays.
>>> 
>>> Nerd snipe success? = p
>> 
>> I guess so? 邏 I'm an easy target.
>> 
>>>> It sounds like having flexible parameter ownership rules doesn't have too 
>>>> much overhead if it can be user-specified (in some future). Would it be 
>>>> feasible to use escape analysis to decide if a parameter should be +0 or 
>>>> +1?
>>> 
>>> No. A parameter's convention is ABI. You don't want to change ABI related 
>>> things like that via escape analysis since it means that as a function 
>>> changes, due to the optimizer, the ABI can change =><=.
>> 
>> That makes sense, I hadn't thought about it.
>> 
>>> Cases like this are due to the optimizer seeing some use that it can not 
>>> understand. The optimizer must be conservative so sometimes things that the 
>>> user thinks the optimizer should see through/understand, it can not. The 
>>> way to see that is to look at the SIL level and see what is stopping the 
>>> code motion. There are ways that you can get debug output from the 
>>> optimizer. This additionally may be a case where an opt-remark like system 
>>> could help guide the user on why code motion has stopped.
>> 
>> My limited testing was basically checking this program:
>> 
>>> final class Foo {
>>> var bar = 4
>>> }
>>> 
>>> let instance = Foo()
>>> 
>>> @inline(never)
>>> func print(_ x: Int) {
>>> Swift.print(x)
>>> }
>>> 
>>> func main() {
>>> let foo = instance
>>> print(foo.bar)
>>> print(0)
>>> }
>> 
>> 
>> On my first pass I noticed that foo is released at the end of the function 
>> (hence the rest of my message), but upon closer inspection I see that it is, 
>> in fact, retained after `foo.bar` is accessed:
>> 
>>> sil hidden @_T04test4mainyyF : $@convention(thin) () -> () {
>>> bb0:
>>> %0 = global_addr @_T04test8instanceAA3FooCv : $*Foo // user: %1
>>> %1 = load %0 : $*Foo// users: %11, %6, %4, %2
>>> debug_value %1 : $Foo, let, name "foo"  // id: %2
>>> // function_ref print(_:)
>>> %3 = function_ref @_T04test5printySiF : $@convention(thin) (Int) -> () // 
>>> users: %10, %7
>>> %4 = ref_element_addr %1 : $Foo, #Foo.bar   // user: %5
>>> %5 = load %4 : $*Int// user: %7
>>> strong_retain %1 : $Foo // id: %6
>>> %7 = apply %3(%5) : $@convention(thin) (Int) -> ()
>>> %8 = integer_literal $Builtin.Int64, 0  // user: %9
>>> %9 = struct $Int (%8 : $Builtin.Int64)  // user: %10
>>> %10 = apply %3(%9) : $@convention(thin) (Int) -> ()
>>> strong_release %1 : $Foo// id: %11
>>> %12 = tuple ()  // user: %13
>>> return %12 : $()// id: %13
>>> } // end sil function '_T04test4mainyyF'
>> 
>> 
>> So while I thought earlier that I didn't know why it wasn't released, I 
>> guess that the better question is why it's retained at all!
>> 
>>>> I guess that the question is: what does Swift gain by keeping objects 
>>>> around for longer than they need to? Is it all about matching C++ or is 
>>>> there something else?
>>> 
>>> Again, I think you are extrapolating a bit. Swift is not attempting to keep 
>>> objects around for longer than they need to be at all. Such situations are 
>>> more likely due to optimizer inadequacies or unimplemented optimizations 
>>> [again, nerd snipe alert, patches welcome ; )]. All of these things take 
>>> engineering time to do and engineering time is something that must be 
>>> prioritized with respect to the overall needs of the project.
>> 
>> Of course. I think that I was being a bit aggressive with "what is the 
>> benefit of this"; I knew there was a fair chance that it was "we had other 
>> things to do".
> 
> I do not believe in bad questions and I think that you asked said questions 
> in an appropriate/respectful way. So don't worry about it: everything's good. 
> = ).
> 
> That being said, if you want to take me up on the "nerd snipe", just send an 
> email to swift-dev and +CC me. I may be the appropriate person to help you, 
> but if not I will hook you up with the right person to help you.
> 
> Michael
> 
>> 
>> Félix
>> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

> On Dec 30, 2017, at 9:41 PM, Félix Cloutier  wrote:
> 
> 
>> Le 30 déc. 2017 à 14:22, Michael Gottesman  a écrit :
>>> That sounds fun. I'll have to check with my manager after the holidays.
>> 
>> Nerd snipe success? = p
> 
> I guess so? 邏 I'm an easy target.
> 
>>> It sounds like having flexible parameter ownership rules doesn't have too 
>>> much overhead if it can be user-specified (in some future). Would it be 
>>> feasible to use escape analysis to decide if a parameter should be +0 or +1?
>> 
>> No. A parameter's convention is ABI. You don't want to change ABI related 
>> things like that via escape analysis since it means that as a function 
>> changes, due to the optimizer, the ABI can change =><=.
> 
> That makes sense, I hadn't thought about it.
> 
>> Cases like this are due to the optimizer seeing some use that it can not 
>> understand. The optimizer must be conservative so sometimes things that the 
>> user thinks the optimizer should see through/understand, it can not. The way 
>> to see that is to look at the SIL level and see what is stopping the code 
>> motion. There are ways that you can get debug output from the optimizer. 
>> This additionally may be a case where an opt-remark like system could help 
>> guide the user on why code motion has stopped.
> 
> My limited testing was basically checking this program:
> 
>> final class Foo {
>>  var bar = 4
>> }
>> 
>> let instance = Foo()
>> 
>> @inline(never)
>> func print(_ x: Int) {
>>  Swift.print(x)
>> }
>> 
>> func main() {
>>  let foo = instance
>>  print(foo.bar)
>>  print(0)
>> }
> 
> 
> On my first pass I noticed that foo is released at the end of the function 
> (hence the rest of my message), but upon closer inspection I see that it is, 
> in fact, retained after `foo.bar` is accessed:
> 
>> sil hidden @_T04test4mainyyF : $@convention(thin) () -> () {
>> bb0:
>>  %0 = global_addr @_T04test8instanceAA3FooCv : $*Foo // user: %1
>>  %1 = load %0 : $*Foo// users: %11, %6, %4, %2
>>  debug_value %1 : $Foo, let, name "foo"  // id: %2
>>  // function_ref print(_:)
>>  %3 = function_ref @_T04test5printySiF : $@convention(thin) (Int) -> () // 
>> users: %10, %7
>>  %4 = ref_element_addr %1 : $Foo, #Foo.bar   // user: %5
>>  %5 = load %4 : $*Int// user: %7
>>  strong_retain %1 : $Foo // id: %6
>>  %7 = apply %3(%5) : $@convention(thin) (Int) -> ()
>>  %8 = integer_literal $Builtin.Int64, 0  // user: %9
>>  %9 = struct $Int (%8 : $Builtin.Int64)  // user: %10
>>  %10 = apply %3(%9) : $@convention(thin) (Int) -> ()
>>  strong_release %1 : $Foo// id: %11
>>  %12 = tuple ()  // user: %13
>>  return %12 : $()// id: %13
>> } // end sil function '_T04test4mainyyF'
> 
> 
> So while I thought earlier that I didn't know why it wasn't released, I guess 
> that the better question is why it's retained at all!
> 
>>> I guess that the question is: what does Swift gain by keeping objects 
>>> around for longer than they need to? Is it all about matching C++ or is 
>>> there something else?
>> 
>> Again, I think you are extrapolating a bit. Swift is not attempting to keep 
>> objects around for longer than they need to be at all. Such situations are 
>> more likely due to optimizer inadequacies or unimplemented optimizations 
>> [again, nerd snipe alert, patches welcome ; )]. All of these things take 
>> engineering time to do and engineering time is something that must be 
>> prioritized with respect to the overall needs of the project.
> 
> Of course. I think that I was being a bit aggressive with "what is the 
> benefit of this"; I knew there was a fair chance that it was "we had other 
> things to do".

I do not believe in bad questions and I think that you asked said questions in 
an appropriate/respectful way. So don't worry about it: everything's good. = ).

That being said, if you want to take me up on the "nerd snipe", just send an 
email to swift-dev and +CC me. I may be the appropriate person to help you, but 
if not I will hook you up with the right person to help you.

Michael

> 
> Félix
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

> On Dec 30, 2017, at 1:21 PM, Félix Cloutier  wrote:
> 
> 
>> Le 29 déc. 2017 à 20:50, Michael Gottesman  a écrit :
>> 
>> No worries. Happy to help = ). If you are interested, I would be happy to 
>> help guide you in implementing one of these optimizations.
> 
> That sounds fun. I'll have to check with my manager after the holidays.

Nerd snipe success? = p. In general, we haven't had as many people from oss 
work on the optimizer. This is unfortunate, so I am more than happy to help.

> 
>> The main downside is that in some cases you do want to have +1 parameters, 
>> namely places where you are forwarding values into a function for storage. 
>> An example of such a case is a value passed to a setter or an initializer. I 
>> would think of it more as an engineering trade-off.
>> 
>> I am currently experimenting with changing normal function arguments to +0 
>> for this purpose, leaving initializers and setters as taking values at +1. 
>> Regardless of the default, we can always provide an attribute for the 
>> purpose of allowing the user to specify such behavior.
> 
> It sounds like having flexible parameter ownership rules doesn't have too 
> much overhead if it can be user-specified (in some future). Would it be 
> feasible to use escape analysis to decide if a parameter should be +0 or +1?

No. A parameter's convention is ABI. You don't want to change ABI related 
things like that via escape analysis since it means that as a function changes, 
due to the optimizer, the ABI can change =><=. *BUT* recently Adam Nemet has 
added support for LLVM's opt remark infrastructure to Swift. Something like 
that /could/ provide suggestions when it is run that this might be a profitable 
parameter to change from +0 to +1 or vis-a-versa. Then the user could make that 
change via annotation. Keep in mind that this would most likely be an ABI 
breaking change.

> 
> More ARC questions! I remember that some time ago, someone else (John 
> McCall?) said that references aren't tied to a scope. This was in the context 
> of deterministic deallocation, and the point was that contrary to C++, an 
> object could potentially be released immediately after its last use in the 
> function rather than at the end of its scope. However, that's not really what 
> you said ("When the assignment to xhat occurs we retain x and at the end of 
> the function [...], we release xhat")

You are extrapolating too far from my example. That being said, I was unclear. 
What I was trying to show was the difference in between a case where xhat is 
passed off to another function, e.g.:

func g(x) {
  let xhat = x
  h(xhat)
}

and a situation where xhat is not consumed and is destroyed within g.

> , and it's not how Swift 4 works from what I can tell (release is not called 
> "as soon as possible" from my limited testing).

Cases like this are due to the optimizer seeing some use that it can not 
understand. The optimizer must be conservative so sometimes things that the 
user thinks the optimizer should see through/understand, it can not. The way to 
see that is to look at the SIL level and see what is stopping the code motion. 
There are ways that you can get debug output from the optimizer. This 
additionally may be a case where an opt-remark like system could help guide the 
user on why code motion has stopped.

> 
> It seems to me that this penalizes functional-style programming in at least 
> two ways:
> 
>   • This kills tail call optimizations, because often the compiler will 
> put a release call in the tail position
>   • Especially in recursion-heavy algorithms, objects can be kept around 
> for longer than they need to
> 
> Here's an example where both hit:
> 
>> class Foo {
>>  var value: [Int]
>>  
>>  init() {
>>  value = Array(repeating: 0, count: 1)
>>  }
>>  
>>  init(from foo: Foo) {
>>  value = foo.value
>>  value[0] += 1
>>  }
>> }
>> 
>> func silly(loop: Int, foo: Foo) -> Foo {
>>  guard loop != 0 else { return foo }
>>  let copy = Foo(from: foo)
>>  return silly(loop: loop - 1, foo: copy)
>> }
>> 
>> print(silly(loop: 1, foo: Foo()).value[0])
> 
> I wouldn't call that "expert Swift code" (indeed, if I come to my senses and 
> use a regular loop, it does just fine), but in this form, it does need about 
> 800MB of memory, and it can't use tail calls.
> 
> The function has the opposite problem from the pattern matching case. It is 
> specialized such that `foo` is passed at +0: it is retained at `return foo` 
> and released (in the caller) after the call to `silly`. However, the optimal 
> implementation would pass it at +1, do nothing for `return foo`, and release 
> it (in the callee) after the call to `Foo(from: foo)`. (Or, even better, it 
> would release it after `value = foo.value` in the init function.)
> 
> I'll note that escape analysis would correctly find that +1 is the 

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

> On Dec 29, 2017, at 4:58 PM, Félix Cloutier  wrote:
> 
> Thanks Michael! Other people might have different opinions, but knowing that 
> a "definitive" fix is on the way is plenty for me and I'm not looking to make 
> a fuss.

No worries. Happy to help = ). If you are interested, I would be happy to help 
guide you in implementing one of these optimizations.

> 
> If you still have some time, I have another question or two. You said that 
> only self is passed at +0. Does this mean, for instance, that custom 
> operators would still get +1 Expr values?

Yes. But again assuming a non-resilient operator, the optimizer can convert the 
+1 in many cases to +0 via the owned->guaranteed optimization.

> It seems to me that if you receive any boxed value at +0, then it's always 
> safe to also pass it further at +0 since it is immutable.
> That would have a huge impact on this program. Are there downsides to more 
> pervasive +0 parameters?

The main downside is that in some cases you do want to have +1 parameters, 
namely places where you are forwarding values into a function for storage. An 
example of such a case is a value passed to a setter or an initializer. I would 
think of it more as an engineering trade-off.

I am currently experimenting with changing normal function arguments to +0 for 
this purpose, leaving initializers and setters as taking values at +1. 
Regardless of the default, we can always provide an attribute for the purpose 
of allowing the user to specify such behavior.

Michael

> 
> Félix
> 
>> Le 29 déc. 2017 à 15:42, Michael Gottesman  a écrit :
>> 
>> 
>> 
>>> On Dec 28, 2017, at 4:27 PM, Chris Lattner via swift-dev 
>>>  wrote:
>>> 
>>> 
>>> 
>>> This is a great question, I’m not sure what the answer is: maybe it is a 
>>> simple case missed by the arc optimizer?
>> 
>> It is a bit more complicated than that. There are codegen aspects and 
>> optimization aspects. In both cases there are "structural issues" that I am 
>> currently in the process of fixing as part of the "semantic sil" effort. 
>> There are some retain/release pairs that we could eliminate by teaching the 
>> optimizer about enums/indirect boxes. I go over all of this below.
>> 
>> TLDR: There are structural issues here that make this difficult/unsafe in 
>> the general case. Semantic SIL eliminates all of the structural issues with 
>> ARC and makes this problem (among other problems) "trivial", so we have been 
>> focusing our engineering time on implementing that. That being said, we 
>> could use the effects analysis to handle this trivial case. But it would 
>> have to be very conservative. Chris, feel like doing some ARC Optimization? 
>> = p.
>> 
>> # Codegen Aspects
>> 
>> ## Background Info: Local Retain/Release Pairs
>> 
>> If one does not know about local vs non-local retain/release pairs, see the 
>> appendix for an explanation.
>> 
>> ## +0 Self
>> 
>> In Swift today all parameters except for self are passed at +1. Self is 
>> passed at +0. Even though this is true, often times SILGen will be 
>> conservative around self and will emit a copy/destroy around the call site. 
>> This is actually better since it eliminates a non-local retain/release pair 
>> in the callee/caller favor of a local retain/release pair in the caller that 
>> can be eliminated by the optimizer  without any form of inter-procedural 
>> optimization. That is why there is a retain/release at the count call. I 
>> will go into why we are not optimizing it below in the optimization section.
>> 
>> ## Pattern Matching at +1
>> 
>> All pattern matching in Swift today is done at +1. That means that in the 
>> following Swift:
>> 
>> 
>> enum Expr: CustomStringConvertible {
>>  case Int(n: Int)
>>  indirect case Var(x: String)
>>  indirect case Add(f: Expr, g: Expr)
>>  indirect case Mul(f: Expr, g: Expr)
>>  indirect case Pow(f: Expr, g: Expr)
>>  indirect case Ln(f: Expr)
>> 
>>  ...
>> 
>>  var count: Int {
>>  switch self {
>>  case .Int, .Var: return 1
>>  case let .Ln(f): return f.count
>>  case let .Add(f, g), let .Mul(f, g), let .Pow(f, g):
>>  return f.count + g.count
>>  }
>>  }
>> }
>> 
>> 
>> even though self is passed at +0 to count, SILGen will emit a retain before 
>> the switch. This is unfortunate and adds unnecessary ARC traffic in read 
>> only methods on enums. There is no reason not to emit switches that way, so 
>> as part of the work to make SILGenPattern able to pass the ownership 
>> verifier, I plan on changing all pattern emission to be done at +0. As an 
>> added benefit once that work is complete, refactorings in SILGenPattern will 
>> be exposed that will enable us to significantly simplify SILGenPattern's 
>> handling of cleanups. This complex code has lead to obscure bugs in the past 
>> and is IMO overly complex.
>> 

Re: [swift-dev] "available externally" vs build time

[Michael Gottesman via swift-dev]

> On Dec 28, 2017, at 7:32 PM, Chris Lattner via swift-dev 
>  wrote:
> 
> Folks working on the SIL optimizer, particularly those interested in faster 
> builds:
> 
> If I understand the SIL optimizer correctly, it seems that when the current 
> program references an external symbol declared as @_inlinable, that 
> SILModule::linkFunction eagerly deserializes the @_inlinable body and splat 
> it into the current module.  That SIL function exists in the current module, 
> gets optimized, inlined, etc along with existing functions, then gets dropped 
> on the floor at IRGen time if it still exists.
> 
> If this is true, this seems like an incredibly wasteful approach, 
> particularly given how many @_inlinable functions exist in the standard 
> library, and particularly for programs that have lots of small files.

In the past, I had talked about making all linking in of functions be lazy. The 
implementation was still there. There was a large backlash from other 
performance people since it adds complexity every place that one attempts to 
analyze a function. I didn't push too hard on this (gotta save your political 
capital when possible), but I still think that such an issue is an API problem, 
not an optimizer design problem. With the proper API design around looking up 
functions and function related instructions, such problems would go away.

>  Try this:
> 
> $ cat u.swift 
> func f() {
>   print("hello")
> }
> 
> $ swiftc u.swift -emit-sil -o - | wc -l
> 7191
> 
> That is a *TON* of SIL, most having to do with array internals, string 
> internals, and other stuff.  This eats memory and costs a ton of compile time 
> to deserialize and slog this around, which gets promptly dropped on the floor 
> by IRGen.  It also makes the -emit-sil output more difficult to work with...
> 
> 
> Optimized builds are also bad:
> $ swiftc u.swift -emit-sil -o - -O | wc -l
>  861
> 
> If you look at it, only about 70 lines of that is the actual program being 
> compiled, the rest is dropped on the floor by IRGen. This costs a ton of 
> memory and compile time to deserialize and represent this, then even more is 
> wasted running the optimizer on code which was presumably optimized when the 
> stdlib was built.
> 
> I imagine that this approach was inspired by LLVM’s available_externally 
> linkage, which does things the same way.  This is a simple way to make sure 
> that interprocedural optimizations can see the bodies of external functions 
> to inline them, etc.   However, LLVM doesn’t have the benefit of a module 
> system like Swift’s, so it has no choice.
> 
> 
> So here are the questions:  :-)
> 
> 1. It looks like the MandatoryInliner is the biggest culprit at -O0 here: it 
> deserializes the referenced function (MandatoryInlining.cpp:384) and *then* 
> checks to see if the callee is @_transparent.  Would it make sense to change 
> this to check for @_transparent first (which might require a SIL change?), 
> and only deserialize if so?

I think the reason why this happened is that a transparent function IIRC must 
have a body and the verifier asserts upon this. I imagine you could add an 
intermediate step. IMO this is worth the change not only for -Onone compile 
time, but also b/c SourceKit relies on mandatory inlining for the purposes of 
diagnostics, so it /could/ speed up the editor experience as well.

> 
> 2. The performance inliner will have the same issue after this, and 
> deserializing the bodies of all inlinable referenced functions is unavoidable 
> for it.  However, we don’t have to copy the SIL into the current module and 
> burn compile time by subjecting it to all of the standard optimizations 
> again.  Would it make sense to put deserialized function bodies into a 
> separate SIL module, and teach the (few) IPA/IPO optimizations about this 
> fact?  This should be very straight-forward to do for all of the 
> optimizations I’m aware of.

I haven't thought about this completely, but this could potentially cause 
problems. In general the SIL optimizer assumes that there is one SILModule. I 
would be careful. Couldn't you just turn off optimizations on 
available_external functions? Also, one could argue that there are potential 
cases where optimizing the available_external functions by themselves could 
save compile time since you are optimizing in one place instead of in multiple 
places after inlining. That being said, off the top of my head I can't think of 
any situation where optimizing in an imported module would result in more 
optimization opportunities than in the original module beyond cases where there 
are circular references (e.g. a function in the imported module refers to a 
function in my module, so I can devirtualize/optimize further in my module and 
do that once before inlining). But IIRC circular references are no-beuano in 
Swift, so it is not clear to me if that is a /real/ case. Jordan would know 
more about this. +CC Jordan.

> 
> I haven’t 

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

> On Dec 28, 2017, at 4:27 PM, Chris Lattner via swift-dev 
>  wrote:
> 
> 
> 
> This is a great question, I’m not sure what the answer is: maybe it is a 
> simple case missed by the arc optimizer?

It is a bit more complicated than that. There are codegen aspects and 
optimization aspects. In both cases there are "structural issues" that I am 
currently in the process of fixing as part of the "semantic sil" effort. There 
are some retain/release pairs that we could eliminate by teaching the optimizer 
about enums/indirect boxes. I go over all of this below.

TLDR: There are structural issues here that make this difficult/unsafe in the 
general case. Semantic SIL eliminates all of the structural issues with ARC and 
makes this problem (among other problems) "trivial", so we have been focusing 
our engineering time on implementing that. That being said, we could use the 
effects analysis to handle this trivial case. But it would have to be very 
conservative. Chris, feel like doing some ARC Optimization? = p.

# Codegen Aspects

## Background Info: Local Retain/Release Pairs

If one does not know about local vs non-local retain/release pairs, see the 
appendix for an explanation.

## +0 Self

In Swift today all parameters except for self are passed at +1. Self is passed 
at +0. Even though this is true, often times SILGen will be conservative around 
self and will emit a copy/destroy around the call site. This is actually better 
since it eliminates a non-local retain/release pair in the callee/caller favor 
of a local retain/release pair in the caller that can be eliminated by the 
optimizer  without any form of inter-procedural optimization. That is why there 
is a retain/release at the count call. I will go into why we are not optimizing 
it below in the optimization section.

## Pattern Matching at +1

All pattern matching in Swift today is done at +1. That means that in the 
following Swift:


enum Expr: CustomStringConvertible {
case Int(n: Int)
indirect case Var(x: String)
indirect case Add(f: Expr, g: Expr)
indirect case Mul(f: Expr, g: Expr)
indirect case Pow(f: Expr, g: Expr)
indirect case Ln(f: Expr)

   ...

var count: Int {
switch self {
case .Int, .Var: return 1
case let .Ln(f): return f.count
case let .Add(f, g), let .Mul(f, g), let .Pow(f, g):
return f.count + g.count
}
}
}


even though self is passed at +0 to count, SILGen will emit a retain before the 
switch. This is unfortunate and adds unnecessary ARC traffic in read only 
methods on enums. There is no reason not to emit switches that way, so as part 
of the work to make SILGenPattern able to pass the ownership verifier, I plan 
on changing all pattern emission to be done at +0. As an added benefit once 
that work is complete, refactorings in SILGenPattern will be exposed that will 
enable us to significantly simplify SILGenPattern's handling of cleanups. This 
complex code has lead to obscure bugs in the past and is IMO overly complex.

# Optimizer Aspects

Lets start by looking at count.getter. Here is the SIL in CFG form:



count.getter.pdf
Description: Adobe PDF document


Lets analyze bb4 (I think bb5 suffers from similar issues).

The retain, release pair on %0 here suffers from the turning off of the 
guaranteed optimization. The guaranteed optimization says that any 
retain/release pairs on a guaranteed value can be eliminated if they are 
"semantic pairings". We had to turn off this optimization since without 
semantic-sil we were guessing what retain/release pairings were "semantic 
pairings". In certain cases this would result in mispairings so we had to turn 
it off in the general case. Specifically, consider the following straw man:


strong_retain %x
...
strong_release obfuscated(%x)
...
strong_retain obfuscated'(%x)
...
strong_release %x


in this case we would pair/remove the outer retain, release pair yielding:


strong_release obfuscated(%x)
...
strong_retain obfuscated'(%x)


While the reference count on %x is still balanced, since the retain count 
balancing is in inverse order this can result in pre-mature releases and 
use-after-frees. Once semantic-sil is complete, all pairings are explicit in 
the IR so such a bug can not happen. To work around that problem, today we are 
conservative and require any retain/release pair around a guaranteed parameter 
to be +1 before and have a use afterwards.

The second retain, release pair (i.e. %13) is slightly different since this 
suffers from the optimizer needing to make a connection in between the box in 
the enum and the value loaded from the box. Today we have enough knowledge in 
the optimizer to make the first connection without issue via the RCIdentity 
analysis. We do not make the connection though in between the box being 
guaranteed and the value 

Re: [swift-dev] [arc optimization] Why doesn't enum destructuring use guaranteed references?

[Michael Gottesman via swift-dev]

(FYI I am already writing a longer response, but got interrupted by visiting 
with my mother ; )). Give me a bit.

> On Dec 29, 2017, at 2:03 PM, Robert Widmann  wrote:
> 
> Ran into this over the summer.  My understanding is that Michael Gottesman 
> (CC’d) has been looking into pattern matching at +0.
> 
> The code in SILGenPattern needs to be reworked, but I found this problem 
> intersects with the really old dynamic casting entry points in SILGen too 
> which would seem to complicate a “simple fix”.  It’d be good to hear what 
> plans, if any, there are to address this. It would be a real win to clean all 
> of this up.
> 
> ~Robert Widmann 
> 
> 2017/12/28 14:27、Chris Lattner via swift-dev のメール:
> 
>> 
>> 
>> This is a great question, I’m not sure what the answer is: maybe it is a 
>> simple case missed by the arc optimizer?
>> 
>> -Chris
>> 
>> 
>> 
>>> On Dec 27, 2017, at 9:19 PM, Félix Cloutier via swift-evolution 
>>>  wrote:
>>> 
>>> Running this on my MBP with 10 as the parameter: 
>>> https://gist.github.com/zneak/ae33bb970a08632cfb2925e2049f9e7a 
>>> 
>>> I get a runtime of about 10 seconds, 45% of which is spent in 
>>> retain/release calls according to Instruments (!!), and at least half of 
>>> that from Expr.count. Looking at the IR, Swift generously sprinkles 
>>> retain/release calls through the outlined copy method:
>>> 
>>> • `self` is retained at the beginning of `count`
>>> • The values that you get out of destructuring are retained
>>> • Of course, when you get `count` on these, they are retained 
>>> again
>>> 
>>> Of course, Expr.count cannot modify itself or its subexpressions because 
>>> the functions are not mutating; in fact, no function in that program can 
>>> mutate an enum case. Why, then, is Swift retaining/releasing `self` and the 
>>> values obtained from destructured patterns? They can't go away. Shouldn't 
>>> we be getting guaranteed references instead of owning references?
>>> 
>>> That seems to hit pattern-matching-heavy programs with indirect cases 
>>> pretty hard, and it's pretty frustrating because that seems to be about the 
>>> nicest way to write this program, and there's no workaround from the 
>>> developer's perspective. I don't think that this is a fatal flaw of 
>>> refcounting, but unless I'm missing something, that's sub-par behavior.
>>> 
>>> Félix
>>> ___
>>> swift-evolution mailing list
>>> swift-evolut...@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Protocol Devirtualizer Pass

[Michael Gottesman via swift-dev]

Is it possible to merge them?

> On Dec 21, 2017, at 11:07 AM, Raj Barik via swift-dev  
> wrote:
> 
> Hi,
> 
> Thanks.
> 
> 
> Are you implementing it as a separate pass, or is it part of function 
> signature specialization?
> 
> 
> I am currently implementing this as a separate pass. There is some code 
> overlap between the two (FunctionSignatureOpt and ProtocolDevirtualizerOpt) 
> in terms of checking which functions can be optimized. Barring that, the code 
> is quite different even though they follow the same pattern (thunk and a 
> separate function).
> 
> --Raj
> 
>  
> 
> Slava
> 
>> 
>> @inline(never) internal func wrap_inc_optional(a:SumProtocol?, val:Int) -> 
>> Int?{
>>  return a?.increment(i:val)
>> }
>> 
>> The generated SIL looks something like this:
>> 
>> sil hidden [noinline] 
>> @_T04main21wrap_inc_optionalSiSgAA11SumProtocol_pSg1a_Si3valtF : 
>> $@convention(thin) (@owned Optional, Int) -> Optional {
>> // %0 // users: %11, %4, %7, %2
>> // %1 // users: %10, %3
>> bb0(%0 : $Optional, %1 : $Int):
>>   debug_value %0 : $Optional, let, name "a", argno 1 // id: %2
>>   debug_value %1 : $Int, let, name "val", argno 2 // id: %3
>>   switch_enum %0 : $Optional, case #Optional.some!enumelt.1: 
>> bb2, case #Optional.none!enumelt: bb1 // id: %4
>> 
>> bb1:  // Preds: bb0
>>   %5 = enum $Optional, #Optional.none!enumelt // user: %6
>>   br bb3(%5 : $Optional) // id: %6
>> 
>> bb2:  // Preds: bb0
>>   %7 = unchecked_enum_data %0 : $Optional, 
>> #Optional.some!enumelt.1 // user: %8
>>   %8 = open_existential_ref %7 : $SumProtocol to 
>> $@opened("F0395A0A-E5DE-11E7-A06A-420039484801") SumProtocol // users: %10, 
>> %10, %9
>>   %9 = witness_method $@opened("F0395A0A-E5DE-11E7-A06A-420039484801") 
>> SumProtocol, #SumProtocol.increment!1 :  
>> (Self) -> (Int) -> Int, %8 : 
>> $@opened("F0395A0A-E5DE-11E7-A06A-420039484801") SumProtocol : 
>> $@convention(witness_method) <τ_0_0 where τ_0_0 : SumProtocol> (Int, 
>> @guaranteed τ_0_0) -> Int // type-defs: %8; user: %10
>>   %10 = apply %9<@opened("F0395A0A-E5DE-11E7-A06A-420039484801") 
>> SumProtocol>(%1, %8) : $@convention(witness_method) <τ_0_0 where τ_0_0 : 
>> SumProtocol> (Int, @guaranteed τ_0_0) -> Int // type-defs: %8; user: %12
>>   release_value %0 : $Optional   // id: %11
>>   %12 = enum $Optional, #Optional.some!enumelt.1, %10 : $Int // user: 
>> %13
>>   br bb3(%12 : $Optional)// id: %13
>> 
>> // %14// user: %15
>> bb3(%14 : $Optional):// Preds: bb1 bb2
>>   return %14 : $Optional // id: %15
>> 
>> 
>> The above branching code (in red) in the SIL makes it non-trivial to 
>> abstract out the non-nil path to a generic outlined method while keeping the 
>> branching code in the thunk and also its not clear if the SILCombiner 
>> peephole optimizer will actually come into affect for this scenario (because 
>> of the branching code getting inlined in the caller).  It also gets more 
>> complicated if there are more than one optional types as parameter to 
>> wrap_inc_optional. Any clue on how one can handle optional types for 
>> devirtualization or if there are any existing transformations in Swift 
>> compiler that can help implement this easily? Thanks.
>> 
>> -R
>> 
>> 
>> 
>> On Wed, Dec 13, 2017 at 3:28 PM, Arnold Schwaighofer 
>>  wrote:
>> You don’t need a second open_existential_ref in the _wrap_inc> SumProtocol> function. It should look something like this:
>> 
>> sil @_wrap_inc : $@convention(thin)  (@owned T, 
>> Int) -> Int {
>> bb0(%0 : $T, %1 : $Int):
>>   %5 = witness_method $T, #SumProtocol.inc!1 : > SumProtocol> (Self) -> (Int) -> Int : $@convention(witness_method: 
>> SumProtocol) <τ_0_0 where τ_0_0 : SumProtocol> (Int, @guaranteed τ_0_0) -> 
>> Int
>>   %6 = apply %5(%1, %0) : $@convention(witness_method: SumProtocol) 
>> <τ_0_0 where τ_0_0 : SumProtocol> (Int, @guaranteed τ_0_0) -> Int
>>   destroy_value %0 : $T
>>   return %6 : $Int
>> }
>> 
>> In the other function it looks like you need to apply the proper 
>> substitution list to the apply instruction:
>> 
>> sil hidden [thunk] [always_inline] 
>> @_T04main8wrap_incSiAA11SumProtocol_p1a_Si3valtF : $@convention(thin) 
>> (@owned SumProtocol, Int) -> Int {
>> bb0(%0 : $SumProtocol, %1 : $Int):
>>   // function_ref specialized wrap_inc(a:val:)
>>   %2 = function_ref @_T04main8wrap_incSiAA11SumProtocol_p1a_Si3valtFTf4nn_n
>>   %3 = open_existential_ref %0 : $SumProtocol to 
>> $@opened("E6196082-DF72-11E7-8C84-420039484801") SumProtocol
>>   %4 = apply %2<τ_0_0>(%3, %1) : $@convention(thin) <τ_0_0 where τ_0_0 : 
>> SumProtocol> (@owned τ_0_0, Int) -> Int // user: %5
>> 
>> τ_0_0 should have been substituted by the opened 

Re: [swift-dev] Request for review on Windows CMake changes

[Michael Gottesman via swift-dev]

I'll take a look sometime later in the week. If I forget, ping me next Friday.

> On Dec 8, 2017, at 5:36 PM, Thomas Roughton via swift-dev 
>  wrote:
> 
> Hi all,
> 
> I've had a PR (https://github.com/apple/swift/pull/13140) that's been open 
> for about 10 days with a few CMake changes to fix the Windows cross-compile 
> build. I wasn't sure who best to ask to review it, but was suggested to post 
> here with it.
> 
> Could someone familiar with that area maybe add themselves as a reviewer and 
> then later take a look when they get a chance (even if that's not for a few 
> weeks)? There's no particular urgency, but it'd be ideal if possible to have 
> the changes in before it needs to be rebased too many times.
> 
> Thanks,
> Thomas
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [Swift CI] Build Failure: 1. OSS - Swift ASAN - OS X (master) #710

[Michael Gottesman via swift-dev]

I wonder if we could emit this during the cmake configure phase. Then this will 
never happen.

> On Dec 1, 2017, at 3:27 AM, Arnold via swift-dev  wrote:
> 
> 
> Oh Rintaro is tryin to fix this issue:
> 
> https://github.com/apple/swift/pull/13203
> 
> Sent from my iPhone
> 
> On Dec 1, 2017, at 3:22 AM, Arnold via swift-dev  wrote:
> 
>> 
>> This is a “spurious error” (missing build dependency?) that c
>> keeps coming and going.
>> 
>> There is a radar assigned to Xi.
>> 
>> Sent from my iPhone
>> 
>> On Dec 1, 2017, at 12:46 AM, Pavel Yaskevich  wrote:
>> 
>>> /Users/buildnode/jenkins/workspace/oss-swift-incremental-ASAN-RA-osx/swift/include/swift/Syntax/RawSyntax.h:33:10:
>>>  fatal error: 'swift/Syntax/SyntaxKind.h' file not found
>>> 
>>> 
>>> 
 On Dec 1, 2017, at 12:43 AM, no-re...@swift.org wrote:
 
 [FAILURE] oss-swift-incremental-ASAN-RA-osx [#710]
 
 Build URL: https://ci.swift.org/job/oss-swift-incremental-ASAN-RA-osx/710/
 Project:   oss-swift-incremental-ASAN-RA-osx
 Date of build: Fri, 01 Dec 2017 01:18:26 -0600
 Build duration:1 hr 24 min
 
 Changes
 
• Commit 272fbb4609502cc9de319203535924231106ed43 by pyaskevich:
 [TypeChecker] NFC: Re-enable test-case for rdar://problem/18360240 in
 
• edit: 
 validation-test/Sema/type_checker_perf/fast/rdar18360240.swift.gyb
 
• Commit 0bc964801bb423e32e6f5236fac82917e4bc19cf by gottesmm:
 [+0-all-args] Accept guaranteed arguments to unsafe guaranteed.
 
• edit: lib/SIL/ValueOwnershipKindClassifier.cpp
• edit: test/SIL/ownership-verifier/use_verifier.sil
• edit: lib/SIL/SILOwnershipVerifier.cpp
>>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-dev
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Building a swift release

[Michael Gottesman via swift-dev]

> On Nov 16, 2017, at 7:09 AM, Alex Blewitt via swift-dev  
> wrote:
> 
>> On 16 Nov 2017, at 14:30, Geordie J via swift-dev > > wrote:
>> 
>> Hello,
>> 
>> I recently asked about building a particular Swift release and couldn’t find 
>> the correct tag. It turns out there are two issues with this:
>> 
>> 1. Somehow "git checkout swift-4.0.2-RELEASE” doesn’t work. In fact, the 
>> commit that tag references 
>> (https://github.com/apple/swift/commit/efb12f4d7a6aa7575333c13fbcdb7782426b130a
>>  
>> )
>>  doesn’t seem to exist locally even after running “git fetch”, so I can’t 
>> even check out that commit’s parent and cherry-pick the release commit.
>> 
>> This is spooky enough to make me think I’m doing something wrong: can 
>> someone confirm or deny that this release tag really exists outside of 
>> GitHub?
> 
> You almost certainly haven't fetched the tag from GitHub successfully. When 
> you run 'git fetch', you'll pull down the configured options as in the 
> .git/config file. For example, I have: 

Also, I think a PR landed in the past 2-3 months that changed update-checkout 
to use git fetch to always use --tags.

> 
> [remote "origin"]
>   url = https://github.com/apple/swift 
>   pushurl = no_push
>   fetch = +refs/heads/*:refs/remotes/origin/*
> 
> As a result, it won't explicitly fetch the tag for me. You can do fetch 
> --tags, in which case, it will pull all of them, or you can configure your 
> repository to fetch that specific tag as well (or specify it on the command 
> line, like git fetch origin 
> refs/tags/swift-4.0.2-RELEASE:refs/remotes/origin/swift-4.0.2-RELEASE).
> 
>> 2. The “stable” branch of llvm (which is automatically referenced when I run 
>> utils/update-checkout) isn’t compatible with that tag (its parent commit at 
>> least). Frustratingly, checking out "swift-4.0-branch” in both apple/swift 
>> and apple/swift-llvm does not build on linux either. This appears to be due 
>> to a rename/move of Dwarf.h somewhere along the line.
> 
> 
> If you are missing some of the required downstream projects, you can run:
> 
> ./swift/utils/update-checkout --clone
> 
> You can explicitly ask to check out a branch (aka 'scheme'):
> 
> ./swift/utils/update-checkout --scheme swift-4.0-branch
> 
> and you can checkout a specific tag:
> 
> ./swift/utils/update-checkout --tag swift-4.0.2-RELEASE
> 
> You can do a clean checkout by using --reset-to-remote, which might help 
> clean your repository state up a little.
> 
> I managed to use this to check out the repository, from a base clone of Swift:
> 
>  ./utils/update-checkout --clone --tag swift-4.0.2-RELEASE
> 
> What version are you at for the LLVM repository after using the above? Mine 
> resolves to:
> 
> 2dedb62a0bcb69354e15a54be89fb5dfa63275d2
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Building a swift release

[Michael Gottesman via swift-dev]

> On Nov 16, 2017, at 7:09 AM, Alex Blewitt via swift-dev  
> wrote:
> 
>> On 16 Nov 2017, at 14:30, Geordie J via swift-dev > > wrote:
>> 
>> Hello,
>> 
>> I recently asked about building a particular Swift release and couldn’t find 
>> the correct tag. It turns out there are two issues with this:
>> 
>> 1. Somehow "git checkout swift-4.0.2-RELEASE” doesn’t work. In fact, the 
>> commit that tag references 
>> (https://github.com/apple/swift/commit/efb12f4d7a6aa7575333c13fbcdb7782426b130a
>>  
>> )
>>  doesn’t seem to exist locally even after running “git fetch”, so I can’t 
>> even check out that commit’s parent and cherry-pick the release commit.
>> 
>> This is spooky enough to make me think I’m doing something wrong: can 
>> someone confirm or deny that this release tag really exists outside of 
>> GitHub?
> 
> You almost certainly haven't fetched the tag from GitHub successfully. When 
> you run 'git fetch', you'll pull down the configured options as in the 
> .git/config file. For example, I have: 
> 
> [remote "origin"]
>   url = https://github.com/apple/swift 
>   pushurl = no_push
>   fetch = +refs/heads/*:refs/remotes/origin/*
> 
> As a result, it won't explicitly fetch the tag for me. You can do fetch 
> --tags, in which case, it will pull all of them, or you can configure your 
> repository to fetch that specific tag as well (or specify it on the command 
> line, like git fetch origin 
> refs/tags/swift-4.0.2-RELEASE:refs/remotes/origin/swift-4.0.2-RELEASE).
> 
>> 2. The “stable” branch of llvm (which is automatically referenced when I run 
>> utils/update-checkout) isn’t compatible with that tag (its parent commit at 
>> least). Frustratingly, checking out "swift-4.0-branch” in both apple/swift 
>> and apple/swift-llvm does not build on linux either. This appears to be due 
>> to a rename/move of Dwarf.h somewhere along the line.
> 
> 
> If you are missing some of the required downstream projects, you can run:
> 
> ./swift/utils/update-checkout --clone
> 
> You can explicitly ask to check out a branch (aka 'scheme'):
> 
> ./swift/utils/update-checkout --scheme swift-4.0-branch

A scheme is not a branch. A scheme is a list of (repo, branch). This allows for 
the underlying branch to vary per repo.

> 
> and you can checkout a specific tag:
> 
> ./swift/utils/update-checkout --tag swift-4.0.2-RELEASE
> 
> You can do a clean checkout by using --reset-to-remote, which might help 
> clean your repository state up a little.
> 
> I managed to use this to check out the repository, from a base clone of Swift:
> 
>  ./utils/update-checkout --clone --tag swift-4.0.2-RELEASE
> 
> What version are you at for the LLVM repository after using the above? Mine 
> resolves to:
> 
> 2dedb62a0bcb69354e15a54be89fb5dfa63275d2
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Rationalizing FloatingPoint conformance to Equatable

[Michael Gottesman via swift-dev]

Before you guys decide anything, I would wait for Steve Canon to look at this. 
+CC Steve

Michael

> On Oct 20, 2017, at 10:21 AM, Xiaodi Wu via swift-dev  
> wrote:
> 
> 
> On Fri, Oct 20, 2017 at 10:10 Matthew Johnson  > wrote:
>> On Oct 20, 2017, at 9:36 AM, Xiaodi Wu via swift-dev > > wrote:
>> 
>> 
>> On Fri, Oct 20, 2017 at 07:21 David Zarzycki > > wrote:
>> 
>> 
>>> On Oct 20, 2017, at 07:51, Xiaodi Wu via swift-dev >> > wrote:
>>> 
>>> On Fri, Oct 20, 2017 at 1:22 AM, Jonathan Hull >> > wrote:
>>> +1 for trapping unless using &==.  In the case of ‘Float?’ we could also 
>>> map to nil.
>>> 
>>> This is probably a more appropriate discussion for evolution though...
>>> 
>>> 
 On Oct 19, 2017, at 9:48 PM, Brent Royal-Gordon via swift-dev 
 > wrote:
 
> On Oct 19, 2017, at 4:29 PM, Xiaodi Wu via swift-dev  > wrote:
> 
> D) Must floating-point IEEE-compliant equivalence be spelled `==`?
> 
> In my view, this is something open for debate. I see no reason why it 
> cannot be migrated to `&==` if it were felt that `==` *must* be a full 
> equivalence relation. I believe this is controversial, however.
 
 I actually got partway through writing up a pitch on this yesterday, but 
 my opinion is that NaNs are so exceptional, and so prone to misuse, that 
 we ought to treat them like integer arithmetic overflows: trap when 
 they're detected, unless you use an `&` variant operator which indicates 
 you know what you're doing.
 
 I strongly suspect that, in practice, most float-manipulating code is not 
 prepared to handle NaN and will not do anything sensible in its presence. 
 For example, Apple platforms use floating-point types for geometry, color 
 components, GPS locations, etc. Very little of this code will do anything 
 sensible in the presence of a NaN. Arguably, it'd be better to exclude 
 them through the type system, but I don't think that's a realistic 
 possibility—we would need to have done that in a more 
 source-break-friendly era. But that doesn't have to mean we're completely 
 stuck.
>>> 
>>> 
>>> Built-in floating point operators, as well as libc/libm math functions, are 
>>> designed to propagate NaN correctly. This is not meant to be a thread about 
>>> NaN, and we need to be cautious to define the scope of the problem to be 
>>> solved from the outset. The tendency of having ever-expanding discussion 
>>> where issues such as method names turn into discussions about the entire 
>>> standard library go nowhere.
>>> 
>>> The question here is about `==` specifically and how to accommodate partial 
>>> equivalence relations. For sanity, we start with the premise that NaN will 
>>> forever be as it is.
>> 
>> I support Jonathan’s argument. If Swift wants to trap on NaN to improve 
>> self-consistency and simplicity, then the tradeoff might be worth it. The 
>> alternative, teaching the Equality protocol about NaNs, feels like “the tail 
>> wagging the dog".
>> 
>> In short: what IEEE requires of floating-point hardware is separable from 
>> IEEE’s opinions about language/library design.
>> 
>> IEEE 754 requires certain behaviors of conforming implementations and is 
>> meant to allow for portable use of floating point. Swift’s floating point 
>> facilities are conformant to that standard, and breaking IEEE 754 
>> conformance has been repeatedly stated to be a non-starter.
>> 
>> The essential idea behind quiet NaN is that it can be used as input in 
>> operations that take floating point values without raising errors. Since 
>> breaking IEEE 754 conformance is a non-starter, trapping on NaN is outside 
>> the realm of available solutions.
>> 
>> It has been pointed out, however, that IEEE does not require a specific 
>> syntax for floating point equivalence, hence the question of whether it can 
>> be spelled differently. However, trapping on NaN is emphatically not an 
>> option.
>> 
>> But I really don’t want to discuss this at all here. The topic of this 
>> thread is about the semantics of Equatable.
> 
> On the other hand, changing the spelling of IEEE `==` was option in your 
> opening post represented by demand 4 and question  D.
> 
>> 4) IEEE-compliant floating-point equivalence must be spelled `==` and not 
>> some alternative such as `&==`.
> 
>> D) Must floating-point IEEE-compliant equivalence be spelled `==`?
> 
>> 
>> 
>> In my view, this is something open for debate. I see no reason why it cannot 
>> be migrated to `&==` if it were felt that `==` *must* be a full equivalence 
>> relation. I believe this is 

Re: [swift-dev] [Swift CI] Build Still Failing: 0. OSS - Swift Incremental RA - Ubuntu 14.04 (master) #1174

[Michael Gottesman via swift-dev]

Is this still an issue?

This Foundation test failure is blocking PR testing! This needs to be fixed 
ASAP or XFAILED.

Can you guys fix this now please.

Michael

> On Oct 17, 2017, at 2:24 PM, Ankit Aggarwal via swift-dev 
>  wrote:
> 
> Test Case 'TestNSNumberBridging.testNSNumberBridgeFromFloat' started at 
> 2017-10-17 21:14:51.197
>  <>TestFoundation/TestNSNumberBridging.swift:72: error: 
> TestNSNumberBridging.testNSNumberBridgeFromFloat : failed - 
> TestFoundation/TestNSNumberBridging.swift:497 nil != Optional(nan) Double
>  <>TestFoundation/TestNSNumberBridging.swift:72: error: 
> TestNSNumberBridging.testNSNumberBridgeFromFloat : failed - 
> TestFoundation/TestNSNumberBridging.swift:497 nil != Optional(nan) Double
> Test Case 'TestNSNumberBridging.testNSNumberBridgeFromFloat' failed (0.001 
> seconds)
> 
>> On 17-Oct-2017, at 2:15 PM, no-re...@swift.org  
>> wrote:
>> 
>> New issue found!
>> 
>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-14_04 [#1174]
>> 
>> Build URL:   
>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-14_04/1174/ 
>> 
>> Project: oss-swift-incremental-RA-linux-ubuntu-14_04
>> Date of build:   Tue, 17 Oct 2017 15:52:12 -0500
>> Build duration:  24 min
>> Identified problems:
>> 
>> Compile Error: This build failed because of a compile error. Below is a list 
>> of all errors in the build log:
>> Indication 1 
>> 
>> Changes
>> 
>> Commit 85b98c6784d2c5cfa0c25c2e08f23df192ebaa33 by ankit_aggarwal:
>> [BuildPlan] Move -X* args to the end of the constructed args
>> 
>> edit: Sources/Build/BuildPlan.swift
> 
> 
> --
> Ankit
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Reducing array abstraction

[Michael Gottesman via swift-dev]

> On Oct 8, 2017, at 11:57 AM, Michael Gottesman via swift-dev 
> <swift-dev@swift.org> wrote:
> 
>> 
>> On Oct 6, 2017, at 11:06 PM, Chris Lattner via swift-dev 
>> <swift-dev@swift.org> wrote:
>> 
>> This question is somewhere between swift-dev and swift-users, not sure where 
>> best to post this.  
>> 
>> I’m working on a project that wants to get very low-abstraction penalty 
>> array operations, particularly with varargs.  Given the currently language 
>> limitations (no fixed size arrays), the best I’ve been able to come up with 
>> is something like this, where “lowlevelAPI” contains the heavy lifting (and 
>> is assumed to be opaque), and the “safeAPI” wrappers exist merely to provide 
>> a convenient safe wrapper for clients:
>> 
>> 
>> 
>> Given whole module optimization of the program, we should in principle, be 
>> able to optimize this down to the equivalent of an LLVM array alloca in the 
>> clients, a few stores to it, then passing the pointers to the LLVM alloca 
>> into lowlevelAPI.  However, Swift is not doing this, not even with:
>> 
>> $ swiftc array_abstraction.swift -emit-sil -o - -O 
>> 
>> In this trivial case (with constant initializers) it does do the “array 
>> outlining” optimization,
> 
> What do you mean by the array outlining optimization specifically?
> 
> We definitely already have a heap->stack for classes in the guise of the 
> StackPromotion optimization is that what you are talking about with the 
> "array outlining" optimization? (outlining to me is referring to specifically 
> code outlining). IIRC Erik (+CC) do special work to make it work for fixed 
> size array. I would ask why that optimization is not kicking in for varargs. 
> Perhaps, we could add a special recognition that the given array will not 
> escape through a varargs? Or provide some way of saying, trust me this 
> doesn't escape.

*We definitely already have a heap->stack for classes in the guise of the 
StackPromotion optimization. Is that the sort of thing that you are talking 
about when you say the "array outlining" optimization? (outlining to me is 
referring to specifically code outlining). IIRC Erik (+CC) did special work to 
make heap->stack work for fixed size Swift.Array that do not escape. But my 
memory may be incorrect. If my memory is correct, I would ask why that 
optimization is not kicking in for varargs. I imagine it is an issue around the 
optimization assuming that the array escapes. Perhaps, we could add a special 
recognition that the given array will not escape through a varargs? Or provide 
some way of saying, trust me this doesn't escape? I imagine that a special case 
would work here since IIRC we can know in the optimizer via the AST that we are 
dealing with a varargs parameter. Things sort of follow from there.

(Sorry for the half written reply. Needed Coffee).

> 
> In terms of what Slava was talking about with copy-on-escape. That can be 
> implemented (assuming a sane runtime ; p) by initializing any COW data 
> structure with a count of 2. Then you are guaranteed to know that any write 
> use or escape from the COW data structure will cause a copy. Once we have 
> guaranteed, this comes for free since any guaranteed parameter must be copied 
> before a mutable use.
> 
> I do think that you will run into issues with escaping around C APIs though.
> 
>> but this is no where near the abstraction level it could be.
>> 
>> Is there a better way to achieve this, and if not, is there any planned work 
>> (perhaps aligned with the ABI stability efforts) to improve vararg array 
>> performance to be able to avoid heap abstractions?  Any individual call to a 
>> vararg array is a known length, so it is a perfect candidate for a stack 
>> allocation.
>> 
>> -Chris
>> 
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org <mailto:swift-dev@swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-dev 
>> <https://lists.swift.org/mailman/listinfo/swift-dev>
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org <mailto:swift-dev@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev 
> <https://lists.swift.org/mailman/listinfo/swift-dev>
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Reducing array abstraction

[Michael Gottesman via swift-dev]

> On Oct 6, 2017, at 11:06 PM, Chris Lattner via swift-dev 
>  wrote:
> 
> This question is somewhere between swift-dev and swift-users, not sure where 
> best to post this.  
> 
> I’m working on a project that wants to get very low-abstraction penalty array 
> operations, particularly with varargs.  Given the currently language 
> limitations (no fixed size arrays), the best I’ve been able to come up with 
> is something like this, where “lowlevelAPI” contains the heavy lifting (and 
> is assumed to be opaque), and the “safeAPI” wrappers exist merely to provide 
> a convenient safe wrapper for clients:
> 
> 
> 
> Given whole module optimization of the program, we should in principle, be 
> able to optimize this down to the equivalent of an LLVM array alloca in the 
> clients, a few stores to it, then passing the pointers to the LLVM alloca 
> into lowlevelAPI.  However, Swift is not doing this, not even with:
> 
> $ swiftc array_abstraction.swift -emit-sil -o - -O 
> 
> In this trivial case (with constant initializers) it does do the “array 
> outlining” optimization,

What do you mean by the array outlining optimization specifically?

We definitely already have a heap->stack for classes in the guise of the 
StackPromotion optimization is that what you are talking about with the "array 
outlining" optimization? (outlining to me is referring to specifically code 
outlining). IIRC Erik (+CC) do special work to make it work for fixed size 
array. I would ask why that optimization is not kicking in for varargs. 
Perhaps, we could add a special recognition that the given array will not 
escape through a varargs? Or provide some way of saying, trust me this doesn't 
escape.

In terms of what Slava was talking about with copy-on-escape. That can be 
implemented (assuming a sane runtime ; p) by initializing any COW data 
structure with a count of 2. Then you are guaranteed to know that any write use 
or escape from the COW data structure will cause a copy. Once we have 
guaranteed, this comes for free since any guaranteed parameter must be copied 
before a mutable use.

I do think that you will run into issues with escaping around C APIs though.

> but this is no where near the abstraction level it could be.
> 
> Is there a better way to achieve this, and if not, is there any planned work 
> (perhaps aligned with the ABI stability efforts) to improve vararg array 
> performance to be able to avoid heap abstractions?  Any individual call to a 
> vararg array is a known length, so it is a perfect candidate for a stack 
> allocation.
> 
> -Chris
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Exclusivity checker hacking?

[Michael Gottesman via swift-dev]

> On Oct 5, 2017, at 1:42 PM, David Zarzycki via swift-dev 
>  wrote:
> 
> Hello,
> 
> As an experiment, I’d like to force the exclusivity checking logic to always 
> error at compile time, rather than a mix of compile time and run time. Near 
> as I can tell, there is no built in debugging logic to do this (not even to 
> warn when dynamic checks are added). Am I missing something? Where would be 
> the best place in the code to make the dynamic checker error/warning at 
> compile time? Would a warning be useful to others? Or should I just keep this 
> on a throwaway branch?

Dave, what exactly are you trying to do?

At some point, I plan on adding static checking for exclusivity like the 
ownership model. I imagine we could just error whenever we can not eliminate a 
runtime check. Would that solve your problem?

> 
> Thanks,
> Dave
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] New to building Swift toolchain

[Michael Gottesman via swift-dev]

Ok. My side machine info:

```
$ sw_vers
ProductName:Mac OS X
ProductVersion: 10.12.6
BuildVersion:   16G26
$ xcodebuild -version
Xcode 9.0
Build version 9M214v
$ cmake --version
cmake version 3.8.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
$
```

So I think I should be able to reproduce since we have the same xcode/os 
version. I am starting a build now.

> On Sep 26, 2017, at 9:44 AM, Michael Gottesman <mgottes...@apple.com> wrote:
> 
> Ok. I am looking at this now on my side machine.
> 
> Sorry for the delay, I had to land code and then I had vacation = (.
> 
> Michael
> 
>> On Sep 20, 2017, at 10:05 PM, Michael Gottesman via swift-dev 
>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>> 
>> FYI, I haven't forgotten you! I just need to land some code tonight ; ). I 
>> want to try to reproduce exactly what you are doing.
>> 
>> Michael
>> 
>>> On Sep 20, 2017, at 6:08 AM, Maksym Grebenets <mgreben...@gmail.com 
>>> <mailto:mgreben...@gmail.com>> wrote:
>>> 
>>> I tried few more options.
>>> 
>>> I tried switching my shell to bash and giving it a go.
>>> This time I got some unusual configuration error.
>>> The error came from 
>>> /usr/local/Cellar/cmake/3.9.2/share/cmake/Modules/FindPythonInterp.cmake.
>>> Unfortunately the error message was in stderr, while I only captured 
>>> stdout: https://gist.github.com/mgrebenets/bf3a54cb90141c17c715f5ddfddf6c04 
>>> <https://gist.github.com/mgrebenets/bf3a54cb90141c17c715f5ddfddf6c04>
>>> 
>>> So then I added CMake.app bin to the path
>>> export PATH=/Applications/CMake.app/Contents/bin:PATH
>>> and ran the build again.
>>> This time it didn't fail with configuration error, but failed with "too 
>>> many files open" and somehow still using cmake 3.9.2 from Homebrew 
>>> installation:
>>> https://gist.github.com/mgrebenets/100cf995fbb9c5415a26e0968aa7ed58 
>>> <https://gist.github.com/mgrebenets/100cf995fbb9c5415a26e0968aa7ed58>
>>> 
>>> So finally, I decided to use CMake.app only:
>>> brew unlink cmake
>>> Run again, and same 3 tests fail on same i386 simulator: 
>>> https://gist.github.com/mgrebenets/cf722b3e19894e7cbeeb3c4fa6d2511e 
>>> <https://gist.github.com/mgrebenets/cf722b3e19894e7cbeeb3c4fa6d2511e>
>>> 
>>> Regards,
>>> Max
>>> 
>>> 
>>> 
>>> On Wed, Sep 20, 2017 at 2:59 PM, Maksym Grebenets <mgreben...@gmail.com 
>>> <mailto:mgreben...@gmail.com>> wrote:
>>> I see.
>>> 
>>> I gave it another go: 
>>> https://gist.github.com/mgrebenets/e2c981951586910c679df17d377c9e69 
>>> <https://gist.github.com/mgrebenets/e2c981951586910c679df17d377c9e69>
>>> 
>>> Looking at the build logs, I can definitely see llvm source code being 
>>> built.
>>> I've looked up a bunch of llvm .cpp files in the build log, e.g. 
>>> LLVMTargetMachine.cpp.
>>> Unless I'm not looking in the right place.
>>> 
>>> Still same 3 failures in the end.
>>> 
>>> I have noticed that in certain places Jenkings builds are using 
>>> /Applications/CMake.app, while on my machine it is 
>>> /usr/local/Cellar/cmake/3.9.1/bin/cmake.
>>> So I've tried to upgrade to 3.9.2 and run same test command manually - same 
>>> failures. 
>>> Same for using CMake.app downloaded from cmake.org <http://cmake.org/> 
>>> (3.9.2) version.
>>> 
>>> I've double checked python version that I have, it's 2.7.10, same as on 
>>> Jenkins instances.
>>> 
>>> I do have python3 installed in /usr/local/bin/python3, not sure if this 
>>> could be related.
>>> 
>>> Looking at the failing tests, it seems like all of them are related to 
>>> encoding.
>>> The first one is related to Unicode 9 graphemes.
>>> For a string "️‍" the expected count is 4, while on my machine it's 
>>> 5.
>>> Other failures are in CodableTests.swift, but all of them report an error 
>>> like:
>>> Decoded URLComponents http://0.0.0.0/>> not equal to original 
>>> http://0.0.0.0/>>
>>> I can't really spot the difference, unless there's some invisible escape 
>>> character...
>>> 
>>> Last failing test is for NSValue bridging:
>>> stdout>>> check failed at 
>>> /Users/grebenma/Projec

Re: [swift-dev] New to building Swift toolchain

[Michael Gottesman via swift-dev]

Ok. I am looking at this now on my side machine.

Sorry for the delay, I had to land code and then I had vacation = (.

Michael

> On Sep 20, 2017, at 10:05 PM, Michael Gottesman via swift-dev 
> <swift-dev@swift.org> wrote:
> 
> FYI, I haven't forgotten you! I just need to land some code tonight ; ). I 
> want to try to reproduce exactly what you are doing.
> 
> Michael
> 
>> On Sep 20, 2017, at 6:08 AM, Maksym Grebenets <mgreben...@gmail.com 
>> <mailto:mgreben...@gmail.com>> wrote:
>> 
>> I tried few more options.
>> 
>> I tried switching my shell to bash and giving it a go.
>> This time I got some unusual configuration error.
>> The error came from 
>> /usr/local/Cellar/cmake/3.9.2/share/cmake/Modules/FindPythonInterp.cmake.
>> Unfortunately the error message was in stderr, while I only captured stdout: 
>> https://gist.github.com/mgrebenets/bf3a54cb90141c17c715f5ddfddf6c04 
>> <https://gist.github.com/mgrebenets/bf3a54cb90141c17c715f5ddfddf6c04>
>> 
>> So then I added CMake.app bin to the path
>> export PATH=/Applications/CMake.app/Contents/bin:PATH
>> and ran the build again.
>> This time it didn't fail with configuration error, but failed with "too many 
>> files open" and somehow still using cmake 3.9.2 from Homebrew installation:
>> https://gist.github.com/mgrebenets/100cf995fbb9c5415a26e0968aa7ed58 
>> <https://gist.github.com/mgrebenets/100cf995fbb9c5415a26e0968aa7ed58>
>> 
>> So finally, I decided to use CMake.app only:
>> brew unlink cmake
>> Run again, and same 3 tests fail on same i386 simulator: 
>> https://gist.github.com/mgrebenets/cf722b3e19894e7cbeeb3c4fa6d2511e 
>> <https://gist.github.com/mgrebenets/cf722b3e19894e7cbeeb3c4fa6d2511e>
>> 
>> Regards,
>> Max
>> 
>> 
>> 
>> On Wed, Sep 20, 2017 at 2:59 PM, Maksym Grebenets <mgreben...@gmail.com 
>> <mailto:mgreben...@gmail.com>> wrote:
>> I see.
>> 
>> I gave it another go: 
>> https://gist.github.com/mgrebenets/e2c981951586910c679df17d377c9e69 
>> <https://gist.github.com/mgrebenets/e2c981951586910c679df17d377c9e69>
>> 
>> Looking at the build logs, I can definitely see llvm source code being built.
>> I've looked up a bunch of llvm .cpp files in the build log, e.g. 
>> LLVMTargetMachine.cpp.
>> Unless I'm not looking in the right place.
>> 
>> Still same 3 failures in the end.
>> 
>> I have noticed that in certain places Jenkings builds are using 
>> /Applications/CMake.app, while on my machine it is 
>> /usr/local/Cellar/cmake/3.9.1/bin/cmake.
>> So I've tried to upgrade to 3.9.2 and run same test command manually - same 
>> failures. 
>> Same for using CMake.app downloaded from cmake.org <http://cmake.org/> 
>> (3.9.2) version.
>> 
>> I've double checked python version that I have, it's 2.7.10, same as on 
>> Jenkins instances.
>> 
>> I do have python3 installed in /usr/local/bin/python3, not sure if this 
>> could be related.
>> 
>> Looking at the failing tests, it seems like all of them are related to 
>> encoding.
>> The first one is related to Unicode 9 graphemes.
>> For a string "️‍" the expected count is 4, while on my machine it's 
>> 5.
>> Other failures are in CodableTests.swift, but all of them report an error 
>> like:
>> Decoded URLComponents http://0.0.0.0/>> not equal to original 
>> http://0.0.0.0/>>
>> I can't really spot the difference, unless there's some invisible escape 
>> character...
>> 
>> Last failing test is for NSValue bridging:
>> stdout>>> check failed at 
>> /Users/grebenma/Projects/oss/swift/swift/stdlib/private/StdlibUnittestFoundationExtras/StdlibUnittestFoundationExtras.swift,
>>  line 130
>> stdout>>> expected: < 3140  4340> (of type 
>> NSConcreteValue)
>> stdout>>> actual: < 3140  4340> (of type 
>> NSConcreteValue)
>> 
>> It appears that all failures are for iPhone Simulator i386.
>> So it must be related to 32-bit platforms only.
>> 
>> I don't have much ideas on what it could be.
>> Is it something set in my bash (zsh) profile?
>> E.g. something in the path, or some 3rd party tool installed.
>> Are there any hints in the log that may point me in the right direction?
>> 
>> Thanks.
>> 
>> 
>> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] New to building Swift toolchain

[Michael Gottesman via swift-dev]

FYI, I haven't forgotten you! I just need to land some code tonight ; ). I want 
to try to reproduce exactly what you are doing.

Michael

> On Sep 20, 2017, at 6:08 AM, Maksym Grebenets  wrote:
> 
> I tried few more options.
> 
> I tried switching my shell to bash and giving it a go.
> This time I got some unusual configuration error.
> The error came from 
> /usr/local/Cellar/cmake/3.9.2/share/cmake/Modules/FindPythonInterp.cmake.
> Unfortunately the error message was in stderr, while I only captured stdout: 
> https://gist.github.com/mgrebenets/bf3a54cb90141c17c715f5ddfddf6c04 
> 
> 
> So then I added CMake.app bin to the path
> export PATH=/Applications/CMake.app/Contents/bin:PATH
> and ran the build again.
> This time it didn't fail with configuration error, but failed with "too many 
> files open" and somehow still using cmake 3.9.2 from Homebrew installation:
> https://gist.github.com/mgrebenets/100cf995fbb9c5415a26e0968aa7ed58 
> 
> 
> So finally, I decided to use CMake.app only:
> brew unlink cmake
> Run again, and same 3 tests fail on same i386 simulator: 
> https://gist.github.com/mgrebenets/cf722b3e19894e7cbeeb3c4fa6d2511e 
> 
> 
> Regards,
> Max
> 
> 
> 
> On Wed, Sep 20, 2017 at 2:59 PM, Maksym Grebenets  > wrote:
> I see.
> 
> I gave it another go: 
> https://gist.github.com/mgrebenets/e2c981951586910c679df17d377c9e69 
> 
> 
> Looking at the build logs, I can definitely see llvm source code being built.
> I've looked up a bunch of llvm .cpp files in the build log, e.g. 
> LLVMTargetMachine.cpp.
> Unless I'm not looking in the right place.
> 
> Still same 3 failures in the end.
> 
> I have noticed that in certain places Jenkings builds are using 
> /Applications/CMake.app, while on my machine it is 
> /usr/local/Cellar/cmake/3.9.1/bin/cmake.
> So I've tried to upgrade to 3.9.2 and run same test command manually - same 
> failures. 
> Same for using CMake.app downloaded from cmake.org  
> (3.9.2) version.
> 
> I've double checked python version that I have, it's 2.7.10, same as on 
> Jenkins instances.
> 
> I do have python3 installed in /usr/local/bin/python3, not sure if this could 
> be related.
> 
> Looking at the failing tests, it seems like all of them are related to 
> encoding.
> The first one is related to Unicode 9 graphemes.
> For a string "️‍" the expected count is 4, while on my machine it's 5.
> Other failures are in CodableTests.swift, but all of them report an error 
> like:
> Decoded URLComponents http://0.0.0.0/>> not equal to original 
> http://0.0.0.0/>>
> I can't really spot the difference, unless there's some invisible escape 
> character...
> 
> Last failing test is for NSValue bridging:
> stdout>>> check failed at 
> /Users/grebenma/Projects/oss/swift/swift/stdlib/private/StdlibUnittestFoundationExtras/StdlibUnittestFoundationExtras.swift,
>  line 130
> stdout>>> expected: < 3140  4340> (of type 
> NSConcreteValue)
> stdout>>> actual: < 3140  4340> (of type 
> NSConcreteValue)
> 
> It appears that all failures are for iPhone Simulator i386.
> So it must be related to 32-bit platforms only.
> 
> I don't have much ideas on what it could be.
> Is it something set in my bash (zsh) profile?
> E.g. something in the path, or some 3rd party tool installed.
> Are there any hints in the log that may point me in the right direction?
> 
> Thanks.
> 
> 
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] New to building Swift toolchain

[Michael Gottesman via swift-dev]

The reason why I know it wasn't clean is that you didn't rebuild LLVM.

> On Sep 19, 2017, at 5:11 PM, Maksym Grebenets  wrote:
> 
> Hm. But this one was pretty much a clean build. I’ve made sure I removed all 
> build directories before running it. 
> 
> Is there a flag to pass to build-toolchain to ensure clean build? Would 
> —clean work?
> On Wed, 20 Sep 2017 at 9:48 am, Michael Gottesman  > wrote:
> Can you do a new clean build and post the log?
> 
>> On Sep 19, 2017, at 4:11 PM, Maksym Grebenets > > wrote:
>> 
>> Sure, I've put it in a gist here: 
>> https://gist.github.com/mgrebenets/9f50bc07615002cfe6ebb3a9f74cb3de 
>> 
>> 
>> Cheers.
>> 
>> On Wed, Sep 20, 2017 at 5:46 AM, Michael Gottesman > > wrote:
>> Hey Maksym. Your email was too big b/c of the attachment. Can you put the 
>> attachment into a pastebin or something like that?
>> 
>>> On Sep 19, 2017, at 12:35 AM, Maksym Grebenets >> > wrote:
>>> 
>>> Hi Michael
>>> 
>>> Thanks for the feedback!
>>> 
>>> I gave it another go and this time made sure I have a proper setup.
>>> 
>>> ./swift/utils/update-checkout --clone
>>> ./swift/utils/update-checkout --tag swift-DEVELOPMENT-SNAPSHOT-2017-09-18-a
>>> 
>>> I've double checked all the submodules directories to make sure the tag was 
>>> properly checked out.
>>> 
>>> Next I've configured Xcode with this command:
>>> 
>>> 
>>> sudo xcode-select -s /Applications/Xcode-beta.app/Contents/Developer
>>> 
>>> The xcodebuild -version output is:
>>> 
>>> Xcode 9.0
>>> Build version 9M214v
>>> 
>>> And the Host OS is 10.12.6, which matches requirements on CI: 
>>> https://ci.swift.org/ .
>>> 
>>> With all that configured I finally kick off the build:
>>> 
>>> 
>>> ./swift/utils/build-toolchain swift.local | tee build.log
>>> 
>>> After about 2+ hours it ends up with same 3 failures:
>>> 
>>> Failing Tests (3):
>>> 
>>> Swift(iphonesimulator-i386) :: stdlib/Character.swift
>>> Swift(iphonesimulator-i386) :: stdlib/CodableTests.swift
>>> Swift(iphonesimulator-i386) :: stdlib/MapKit.swift
>>> 
>>>   Expected Passes: 9957
>>>   Expected Failures  : 27
>>>   Unsupported Tests  : 446
>>>   Unexpected Failures: 3
>>> 
>>> Here's log for one of the failing tests:
>>> 
>>> 
>>> 
>>> --
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Testing: 0
>>> 
>>> FAIL: Swift(iphonesimulator-i386) :: stdlib/MapKit.swift (566 of 10433)
>>> 
>>>  TEST 'Swift(iphonesimulator-i386) :: 
>>> stdlib/MapKit.swift' FAILED 
>>> 
>>> Script:
>>> 
>>> --
>>> 
>>> rm -rf 
>>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp
>>>  && mkdir -p 
>>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp
>>>  && xcrun --toolchain default --sdk 
>>> /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk
>>>  
>>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/bin/swiftc
>>>  -target i386-apple-ios7.0  -module-cache-path 
>>> '/var/folders/vs/50_hfsmx4m1cpwbn7j3x6r6cgp/T/swift-testsuite-clang-module-cachefYcCzs'
>>>  -F 
>>> /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk/../../../Developer/Library/Frameworks
>>>   -swift-version 3  -module-cache-path 
>>> '/var/folders/vs/50_hfsmx4m1cpwbn7j3x6r6cgp/T/swift-testsuite-clang-module-cachefYcCzs'
>>>  /Users/grebenma/Projects/oss/swift/swift/test/stdlib/MapKit.swift -o 
>>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out
>>>  -module-name main && xcrun --toolchain default --sdk 
>>> /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk
>>>  simctl spawn 'iPhone 5'  
>>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out
>>> 
>>> --
>>> 
>>> Exit Code: 134
>>> 
>>> 
>>> 
>>> Command Output (stdout):
>>> 
>>> --
>>> 
>>> [ RUN  ] MapKit.NSValue bridging
>>> 
>>> stdout>>> check failed at 
>>> /Users/grebenma/Projects/oss/swift/swift/stdlib/private/StdlibUnittestFoundationExtras/StdlibUnittestFoundationExtras.swift,
>>>  line 130
>>> 
>>> stdout>>> expected: < 3140  4340> (of type 
>>> NSConcreteValue)
>>> 
>>> stdout>>> actual: < 3140  4340> (of type 
>>> NSConcreteValue)
>>> 
>>> [ FAIL ] MapKit.NSValue 

Re: [swift-dev] New to building Swift toolchain

[Michael Gottesman via swift-dev]

Can you do a new clean build and post the log?

> On Sep 19, 2017, at 4:11 PM, Maksym Grebenets  wrote:
> 
> Sure, I've put it in a gist here: 
> https://gist.github.com/mgrebenets/9f50bc07615002cfe6ebb3a9f74cb3de 
> 
> 
> Cheers.
> 
> On Wed, Sep 20, 2017 at 5:46 AM, Michael Gottesman  > wrote:
> Hey Maksym. Your email was too big b/c of the attachment. Can you put the 
> attachment into a pastebin or something like that?
> 
>> On Sep 19, 2017, at 12:35 AM, Maksym Grebenets > > wrote:
>> 
>> Hi Michael
>> 
>> Thanks for the feedback!
>> 
>> I gave it another go and this time made sure I have a proper setup.
>> 
>> ./swift/utils/update-checkout --clone
>> ./swift/utils/update-checkout --tag swift-DEVELOPMENT-SNAPSHOT-2017-09-18-a
>> 
>> I've double checked all the submodules directories to make sure the tag was 
>> properly checked out.
>> 
>> Next I've configured Xcode with this command:
>> 
>> 
>> sudo xcode-select -s /Applications/Xcode-beta.app/Contents/Developer
>> 
>> The xcodebuild -version output is:
>> 
>> Xcode 9.0
>> Build version 9M214v
>> 
>> And the Host OS is 10.12.6, which matches requirements on CI: 
>> https://ci.swift.org/ .
>> 
>> With all that configured I finally kick off the build:
>> 
>> 
>> ./swift/utils/build-toolchain swift.local | tee build.log
>> 
>> After about 2+ hours it ends up with same 3 failures:
>> 
>> Failing Tests (3):
>> 
>> Swift(iphonesimulator-i386) :: stdlib/Character.swift
>> Swift(iphonesimulator-i386) :: stdlib/CodableTests.swift
>> Swift(iphonesimulator-i386) :: stdlib/MapKit.swift
>> 
>>   Expected Passes: 9957
>>   Expected Failures  : 27
>>   Unsupported Tests  : 446
>>   Unexpected Failures: 3
>> 
>> Here's log for one of the failing tests:
>> 
>> 
>> 
>> --
>> 
>> 
>> 
>> 
>> 
>> Testing: 0
>> 
>> FAIL: Swift(iphonesimulator-i386) :: stdlib/MapKit.swift (566 of 10433)
>> 
>>  TEST 'Swift(iphonesimulator-i386) :: 
>> stdlib/MapKit.swift' FAILED 
>> 
>> Script:
>> 
>> --
>> 
>> rm -rf 
>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp
>>  && mkdir -p 
>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp
>>  && xcrun --toolchain default --sdk 
>> /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk
>>  
>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/bin/swiftc
>>  -target i386-apple-ios7.0  -module-cache-path 
>> '/var/folders/vs/50_hfsmx4m1cpwbn7j3x6r6cgp/T/swift-testsuite-clang-module-cachefYcCzs'
>>  -F 
>> /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk/../../../Developer/Library/Frameworks
>>   -swift-version 3  -module-cache-path 
>> '/var/folders/vs/50_hfsmx4m1cpwbn7j3x6r6cgp/T/swift-testsuite-clang-module-cachefYcCzs'
>>  /Users/grebenma/Projects/oss/swift/swift/test/stdlib/MapKit.swift -o 
>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out
>>  -module-name main && xcrun --toolchain default --sdk 
>> /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk
>>  simctl spawn 'iPhone 5'  
>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out
>> 
>> --
>> 
>> Exit Code: 134
>> 
>> 
>> 
>> Command Output (stdout):
>> 
>> --
>> 
>> [ RUN  ] MapKit.NSValue bridging
>> 
>> stdout>>> check failed at 
>> /Users/grebenma/Projects/oss/swift/swift/stdlib/private/StdlibUnittestFoundationExtras/StdlibUnittestFoundationExtras.swift,
>>  line 130
>> 
>> stdout>>> expected: < 3140  4340> (of type 
>> NSConcreteValue)
>> 
>> stdout>>> actual: < 3140  4340> (of type 
>> NSConcreteValue)
>> 
>> [ FAIL ] MapKit.NSValue bridging
>> 
>> MapKit: Some tests failed, aborting
>> 
>> UXPASS: []
>> 
>> FAIL: ["NSValue bridging"]
>> 
>> SKIP: []
>> 
>> To debug, run:
>> 
>> $ 
>> /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out
>>  --stdlib-unittest-in-process --stdlib-unittest-filter "NSValue bridging"
>> 
>> 
>> 
>> --
>> 
>> Command Output (stderr):
>> 
>> --
>> 
>> Child process terminated with signal 6: Abort trap
>> 
>> 
>> 
>> --
>> 
>> The logs don't make much sense to me, especially since actual and expected 
>> values look the same :shrugs:.
>> 
>> I'm not sure 

Re: [swift-dev] libdispatch switched to CMake

[Michael Gottesman via swift-dev]

> On Sep 18, 2017, at 3:28 PM, Saleem Abdulrasool  wrote:
> 
> Hello swift-dev,
> 
> This change should be transparent to everyone, but, it still is a pretty big 
> milestone.
> 
> As of this afternoon, the non-Darwin builds use CMake to build libdispatch.  
> This work has been underway for quite some time, and we have finally switched 
> to that as the default on most targets.  This should be more or less 
> transparent to everyone developing with build-script or even invoking CMake 
> for swift directly.
> 
> This simplifies some of the logic for integrating the projects and avoid 
> unnecessary commands from being run.
> 
> Thanks particularly to Michael Gottesman, Chris Bieneman, Daniel Steffen, 
> David Grove, and Pierre Habouzit.  Sorry if I accidentally left someone off 
> the list!

+1!

For anyone listening in, if you run into any issues, you may need a clean build 
or a reconfigure.

Also in case anyone is not familiar with the bigger picture here, this down the 
line will enable us to eliminate autoconf as a dependency for building swift 
since libdispatch is the only part of swift now that depends on autoconf.

Michael

> 
> Saleem
> 
> -- 
> Saleem Abdulrasool
> compnerd (at) compnerd (dot) org

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] New to building Swift toolchain

[Michael Gottesman via swift-dev]

> On Sep 18, 2017, at 8:54 AM, Maksym Grebenets via swift-dev 
>  wrote:
> 
> Hi all
> 
> I guess like many other iOS devs with c/c++ background I really wanted to get 
> into Swift development one day, at least a tiny bit, at least to understand 
> how things work.
> 
> So new refactoring tools and starter tasks finally pulled me in.
> 
> With a lot of struggle and a lot of help from community I was able to 
> understand how lit works, how to walk the AST and write changes back to the 
> editor. I've got code and tests for new refactoring kind and new refactoring 
> action, so I was like "yay, time to build the keychain and just try it in 
> Xcode!!!".
> 
> That's where I was thrown back to the "what do I do?" state again...
> 
> I pulled latest master branch and ran build-toolchain and all of that just to 
> be told after almost an hour or so that there are 3 failing tests...

What were the failing tests?

> 
> So I thought:
> 
> cd swift
> git checkout swift-DEVELOPMENT-SNAPSHOT-2017-09-15-a
> cd ..
> ./swift/utils/update-checkout --tag swift-DEVELOPMENT-SNAPSHOT-2017-09-15-a
> 
> then build and fail again :(...

So you are saying that you took the last "known good snapshot" and that also 
failed. How did it fail? On the same 3 tests?

> 
> OK then, so I just disable the tests and give it another go:
> 
> # build-presets.ini
> #test
> #validation-test
> #long-test
> 
> Yay! I got a toolchain!
> I copy it over to /Library/Developer/Toolchains/ 
> Select it in Xcode (latest Xcode 9 GM) and... no luck
> The source code highlighting disappears, 
> I get "An internal error occurred. Source editor functionality is limited..." 
> error.
> All of the refactoring actions are disabled.
> 
> I guess those 3 failing tests did matter in the end...
> 
> 
> I may be frustrated, because I don't understand what's going on, but I don't 
> feel like giving up.
> 
> I'd appreciate any help or advice I can get!
> 
> Should I choose an older tag from before to get all tests passing?

Do you mean before the last known good snapshot? If a snapshot was published on 
swift.org , it /should/ pass all of the tests.

> Should I build using Xcode beta rather then GM?

You should be using the Xcode beta. This can be seen either via the README.md 
under System Requirements for macOS [and don't worry I run into this issue 
sometimes as well = )]. It is also documented on the CI: https://ci.swift.org 
 with a nice link.

> Is there any way to build/re-build the OS X toolchain any faster? 
> Why does it take so long even if I just re-run same command again?

When you rerun is it not incremental? (I forgot TBH). In terms of the amount of 
work performed, build-toolchain command is intended to be used for producing 
snapshots and thus it has to be complete and include code for all 
platforms/etc. We could potentially have a preset that builds a toolchain only 
for 1 platform. I am not 100% sure if such a toolchain would work, but it 
should take significantly less time to build. Would that help?

> Where did I get it wrong so the toolchain doesn't really work with Xcode?
> Can I connect lldb to process that runs my lit tests?
> 
> I'm sorry for having so many questions, but you can imagine there isn't a lot 
> of information for newbies like me trying to get into Swift compiler 
> development. Well, let's just say I couldn't find much :) I've just realized 
> that this mail list is better option that StackOverflow about an hour ago :) 
> Though, of course, I have read Brian Gesiak's posts and the swift.org 
>  blog post on refactoring.

There are no bad questions or too many questions. Welcome to the dev-list = ).

> 
> Just to sum up all of the above, how to I get to build local Xcode toolchain 
> which I can plug in to Xcode and test?

Answer my questions above... I'll help you get across the finish line.

On another note, I noticed you mention starter tasks: keep in mind there are 
more starter tasks than have been documented. If you are interested in a 
specific area of the compiler and can not find starter tasks in this area, 
please feel free to send a nice email to this list. I am sure we can find 
/something/ ; ).

Michael

> 
> Thank you!
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [Swift CI] Build Still Failing: OSS - Swift Package - OS X (master) #178

[Michael Gottesman via swift-dev]

I don't think it is that. What is actually happening here is that I enabled the 
ownership verifier on a bunch of SILGen tests. This causes some syntactic 
changes (i.e. SILArugments must now specify an ownership convention textually. 
Looks like I missed a spot. I am on it!

Michael

> On Aug 31, 2017, at 1:46 PM, Michael Ilseman  wrote:
> 
> +CC Michael G.
> 
>> On Aug 31, 2017, at 1:45 PM, Michael Ilseman via swift-dev 
>> > wrote:
>> 
>> None of the changes look relevant to this error, unfortunately. This looks 
>> like it’s a @owned calling convention change.
>> 
>> 
>> /Users/buildnode/jenkins/workspace/oss-swift-package-osx/swift/test/SILGen/objc_bridging.swift:139:22:
>>  error: expected string not found in input
>> 15:23:59 // CHECK-ios-x86_64: bb0([[SELF:%.*]] : $Foo):
>> 15:23:59  ^
>> 15:23:59 :225:79: note: scanning from here
>> 15:23:59 sil hidden @_T013objc_bridging6getZimSbSo3FooCF : 
>> $@convention(thin) (@owned Foo) -> Bool {
>> 15:23:59 
>>   ^
>> 15:23:59 :227:4: note: possible intended match here
>> 15:23:59 bb0(%0 : @owned $Foo):
>> 15:23:59^
>> 15:23:59 
>> /Users/buildnode/jenkins/workspace/oss-swift-package-osx/swift/test/SILGen/objc_bridging.swift:183:22:
>>  error: expected string not found in input
>> 15:23:59 // CHECK-ios-x86_64: bb0([[ARG0:%.*]] : $Foo, [[ARG1:%.*]] : $Bool):
>> 15:23:59  ^
>> 15:23:59 :238:84: note: scanning from here
>> 15:23:59 sil hidden @_T013objc_bridging6setZimySo3FooC_Sb1btF : 
>> $@convention(thin) (@owned Foo, Bool) -> () {
>> 15:23:59 
>>^
>> 15:23:59 :241:17: note: possible intended match here
>> 15:23:59 bb0(%0 : @owned $Foo, %1 : @trivial $Bool):
>> 15:23:59 ^
>> 15:23:59 
>> 15:23:59 --
>> 15:23:59 
>> 15:23:59 
>> 
>> 
>>> On Aug 31, 2017, at 1:39 PM, no-re...@swift.org  
>>> wrote:
>>> 
>>> New issue found!
>>> 
>>> [FAILURE] oss-swift-package-osx [#178]
>>> 
>>> Build URL:  https://ci.swift.org/job/oss-swift-package-osx/178/ 
>>> 
>>> Project:oss-swift-package-osx
>>> Date of build:  Thu, 31 Aug 2017 14:01:42 -0500
>>> Build duration: 1 hr 38 min
>>> Identified problems:
>>> 
>>> Regression test failed: This build failed because a regression test in the 
>>> test suite FAILed. Below is a list of all errors:
>>> Indication 1 
>>> 
>>> Compile Error: This build failed because of a compile error. Below is a 
>>> list of all errors in the build log:
>>> Indication 1 
>>> 
>>> Changes
>>> 
>>> Commit 3c8f5fa04ace3292b052322fb3934f5ffd8cb8a7 by lance_parker:
>>> Add COW test for Array
>>> 
>>> edit: validation-test/stdlib/Dictionary.swift
>>> edit: validation-test/stdlib/Arrays.swift.gyb
>>> 
>>> Commit 0ebb2fecca8f1c468babe7d9633949c67ee70042 by lance_parker:
>>> Add COW test for String
>>> 
>>> edit: validation-test/stdlib/String.swift
>>> 
>>> Commit c623881c789cf7d18703602ed75401e65263e5e3 by lance_parker:
>>> Fix Linux tests
>>> 
>>> edit: validation-test/stdlib/String.swift
>>> 
>>> Commit 3f60f1c80fd08496614aa2c454538f51fd416abd by lance_parker:
>>> Remove commented out code
>>> 
>>> edit: validation-test/stdlib/String.swift
>>> 
>>> Commit 19d43c24d18831f4393de228872710ce2c72b9c4 by lance_parker:
>>> Some more test cleanup
>>> 
>>> edit: validation-test/stdlib/String.swift
>>> 
>>> Commit b14d5865797fb4ae36c121712d0b1daf82301ab1 by github:
>>> Strip obsolete constraints
>>> 
>>> edit: test/Prototypes/PatternMatching.swift
>>> 
>>> Commit fcb2797d6023b6ad7434dac11c0539fa9088757c by github:
>>> Strip obsolete workaround
>>> 
>>> edit: test/Prototypes/PatternMatching.swift
>>> 
>>> Commit 77b0f39cc297588d1b97dd1e0166c85e5bbd86e2 by lance_parker:
>>> Max's feedback
>>> 
>>> edit: validation-test/stdlib/String.swift
>>> edit: validation-test/stdlib/Arrays.swift.gyb
>>> 
>>> Commit d302f8caca057fed648254c562f205bcea7ba304 by mark.lacey:
>>> Add --invert-result option to scale-test.
>>> 
>>> edit: utils/scale-test
>>> 
>>> Commit c049b14bface46c43b28d6cf0e5ba74a0fc93941 by lance_parker:
>>> Removed useless parts of Array tests
>>> 
>>> edit: validation-test/stdlib/Arrays.swift.gyb
>>> 
>>> Commit 6e7b7f5f18617bcd15beaf2820196d8c928bd400 by mark.lacey:
>>> Update type checker performance tests to use --invert-result.
>>> 
>>> edit: validation-test/Sema/type_checker_perf/slow/rdar20233198.swift.gyb
>>> edit: validation-test/Sema/type_checker_perf/slow/rdar30596744_2.swift.gyb
>>> edit: 

Re: [swift-dev] Foundation Data Behavior Different (read: crashes) in Swift 3.1 and Swift 3.2

[Michael Gottesman via swift-dev]

> On Aug 24, 2017, at 10:47 AM, Ryan Lovelett via swift-dev 
>  wrote:
> 
> I've found what I believe is a bug. Though I'm unclear if the bug is in
> Swift 3.1 or Swift 3.2/4.0. All I can say for sure is the behavior is
> quite drastically different between the two.
> 
> For the code below (and attached):
> 
>import Cocoa
> 
>var data = Data(bytes: [0x50, 0x4B, 0x01, 0x02, 0x41, 0x61])
>data.removeFirst(4)
>let base64 = data.base64EncodedString()
>let str = String(bytes: data[0..<2], encoding: .utf8)!
>print("Base64: \(base64), String: \(str)")
> 
> If I compile and run that with the Swift included in Xcode 8.3.3 (e.g.,
> swift ./data-bug.swift) it outputs: Base64: QWE=, String: Aa. Which is
> what I expect.
> 
> With the Swift that is included with Xcode 9.0 beta 6 (9M214v) (e.g.,
> swift -swift-version 3 ./data-bug.swift). It performs an illegal
> hardware instruction and crashes. It also does this if I use use the
> version 4 of the compiler.
> 
> Is this a bug? If so where is the bug? Was this always meant to not work
> and Swift 3.1 just happened to work or is there now an issue in the
> Swift 3.2 implementation?

I have not engaged my brain with the particulars of the rest of the email, but 
high level question: does this happen without optimization? Or does it happen 
only with optimization?

Michael

> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Is swift.org/contributing/ stale?

[Michael Gottesman via swift-dev]

> On Aug 14, 2017, at 10:32 AM, David Zarzycki via swift-dev 
>  wrote:
> 
> Hi Slava,
> 
> Ah. So “GitHub collaborator” status doesn’t imply commit access then?

I think that only means that you have a commit in the repository, nothing more. 
But my memory might be wrong.

> 
> Dave
> 
>> On Aug 14, 2017, at 13:22, Slava Pestov  wrote:
>> 
>> I think it’s still up to date. You need commit access to trigger CI, and to 
>> merge PRs. Otherwise you’ll have to ask someone with commit access to do 
>> that for you.
>> 
>> Slava
>> 
>>> On Aug 14, 2017, at 1:14 PM, David Zarzycki via swift-dev 
>>>  wrote:
>>> 
>>> On the swift.org/contributing/ web page, under “Commit Access”, the section 
>>> talks about commit access in a way that suggests that it was written before 
>>> mandatory continuous integration testing was added. Is this true? Can 
>>> somebody update the page? If CI is not required, then is there something I 
>>> need to do before pushing a commit?
>>> 
>>> Thanks!
>>> Dave
>>> ___
>>> swift-dev mailing list
>>> swift-dev@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-dev
>> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Debugging on Xcode

[Michael Gottesman via swift-dev]

Add to debugging the compiler?

Sent from my iPhone

> On Aug 11, 2017, at 11:38 AM, Robert Widmann via swift-dev 
>  wrote:
> 
> Full disclaimer: I haven’t looked into this bug, so I’m just going to 
> describe higher-level workflow tips.
> 
>> On Aug 11, 2017, at 5:02 AM, Mike Choi via swift-dev  
>> wrote:
>> 
>> Hello Swift community,
>> 
>> I’m currently trying to debug SR-5556 
>> https://bugs.swift.org/plugins/servlet/mobile#issue/SR-5556. 
>> TL;Dr This bug has to do with Xcode not providing a fixit when it should. 
>> 
> 
> The way to do what you’re asking for is to build a toolchain, which is far 
> too time-consuming for the debugging workflow you want here.  This diagnostic 
> is coming from somewhere in Semantic Analysis, so you just need to take your 
> minimally-reproducing test case and only run the compiler up to that point.  
> To do this in Xcode, since you already have the project in hand, you can do 
> the following:
> 
> - Select the ‘swift’ scheme
> - Pull up the scheme editor (⌘+⇧+<)
> - Select the ‘Arguments’ tab and click the ‘+’
> - (For this particular case) add ‘-frontend -typecheck 
> /Path/To/This/Test/Case.swift’
> - Close the scheme editor
> - Build and run
> 
> ~Robert Widmann
> 
>> To proceed, I built Swift and generated an Xcode project but have no idea 
>> how to debug the problem. Is there a way for me to input Swift code into the 
>> built Swift compiler via Xcode and debug in Xcode??
>> 
>> If not, what is the usual way to debug something like this?
>> 
>> Thank you for your time!
>> 
>> Best,
>> Mike JS. Choi
>> 
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Debugging performance regressions?

[Michael Gottesman via swift-dev]

> On Aug 9, 2017, at 11:34 AM, Arnold Schwaighofer via swift-dev 
>  wrote:
> 
> Mark do you have suggestions for Dave?

Also, whatever comes from this, can we document in updating the compiler.

> 
>> On Aug 9, 2017, at 11:22 AM, David Zarzycki via swift-dev 
>> > wrote:
>> 
>> Any tips on how to debug performance regressions?
>> 
>> And as aside, if this test is critical enough to cause a revert, why isn’t 
>> it in the public repo?
>> 
> 
> This is part of a larger internal test.
> 
>> 
>>> On Aug 9, 2017, at 14:04, Arnold Schwaighofer >> > wrote:
>>> 
>>> Hi Dave,
>>> I have reverted this commit because it broke internal bots.
>>> 
>>> import Foundation
>>> class P {
>>> var x : Int = 0
>>> var y : Int = 1
>>> }
>>> 
>>> let dist : (P, P) -> Double = {
>>>   (p : P, s : P)  -> Double in
>>> sqrt(Double((p.x-s.x)*(p.x-s.x) + (p.y-s.y)*(p.y-s.y)))
>>> }
>>> fails now with expression to complex.
>>> 
>>> —
>>> You are receiving this because you authored the thread.
>>> Reply to this email directly, view it on GitHub 
>>> , or mute 
>>> the thread 
>>> .
>>> 
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-dev 
> 
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] 107 lines of "dead" code in ConstraintSystem::performMemberLookup?

[Michael Gottesman via swift-dev]

I think there are docs on this in the main README or in a CONTRIBUTING file.

Mishal (+CC) knows for sure.

Michael

> On Aug 9, 2017, at 5:41 AM, David Zarzycki via swift-dev 
>  wrote:
> 
> Hi Slava,
> 
> Thanks! I appreciate the suggestion to run tests, but what I’ve tried in the 
> past, @swift-ci ignored me. Is there a whitelist for who can request that 
> tests be run? That would make sense from a security perspective.
> 
> Dave
> 
> 
> 
>> On Aug 8, 2017, at 23:48, Slava Pestov > > wrote:
>> 
>> I see you’re two steps ahead of me already: 
>> https://github.com/apple/swift/pull/11397 
>> 
>> 
>> This looks good, nice catch!
>> 
>> Slava
>> 
>>> On Aug 8, 2017, at 8:45 PM, Slava Pestov >> > wrote:
>>> 
>>> 
 On Aug 8, 2017, at 3:34 PM, David Zarzycki via swift-dev 
 > wrote:
 
 In ConstraintSystem::performMemberLookup(), constructors with “simple” 
 names have a dedicated lookup path. In contrast, constructors with 
 compound names are handled by the normal lookup. If I delete this code and 
 let the normal lookup path handle both simple and compound named 
 constructors, I find that all 10,214 validation tests pass on my machine 
 (albeit with slightly different error messages in three test files).
>>> 
>>> I would suggest running the source compatibility test suite also (see “pull 
>>> request testing” in https://swift.org/source-compatibility/ 
>>> ), but it is quite possible the 
>>> code is indeed unnecessary.
>>> 
 
 Is the test suite missing a test for this code path and if so, what? Or 
 should it be scheduled for deletion after identical error messages can be 
 generated by the normal lookup path?
>>> 
>>> Are the new error messages worse or just different? If the latter there’s 
>>> really no requirement to keep them identical.
>>> 
>>> Slava
>>> 
 
 Dave
 ___
 swift-dev mailing list
 swift-dev@swift.org 
 https://lists.swift.org/mailman/listinfo/swift-dev 
 
>>> 
>> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Faster debug builds?

[Michael Gottesman via swift-dev]

> On Aug 6, 2017, at 11:11 AM, David Zarzycki via swift-dev 
>  wrote:
> 
> Hello,
> 
> Unless I’m missing a build-script flag, it seems to me that compiling the 
> Swift stdlib with the unoptimized debug swift compiler takes about 15 minutes 
> on a fast machine.

I am assuming that you mean a debug swift compiler building an optimized stdlib?

> Other than forcing the type checker to be optimized, what if any tricks can I 
> use to building the stdlib faster with the debug compiler? Is there a way to 
> tell Clang to enable the inliner and only the inliner during -O0 builds? I 
> have an anecdotal experiment[1] that suggests that this would yield 
> appreciably faster Swift stdlib builds with the debug compiler (and selfishly 
> speaking, I can tolerate the minor impact on debugging that inlining does to 
> otherwise unoptimized code).

Are building LLVM in release + Swift in debug? I.e.:

--release-debuginfo --debug-swift --force-optimized-typechecker

> 
> Thanks!
> 
> Dave
> 
> [1] – If one force inlines LLVM’s casting logic and associated callbacks 
> (like classof() and getKind()), then the Swift stdlib builds 18% faster on my 
> machine with the debug Swift compiler. One can imagine how much faster the 
> whole stdlib would compile if all trivial functions were inlined 
> automatically.
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] Enums with Many Cases/Covered Switches

[Michael Gottesman via swift-dev]

I was reading https://github.com/apple/swift/pull/11098 and it brought my mind 
back to a strawman discussion that Roman and I had some time ago. Namely, in 
some cases, we want the functionality of a covered switch, but it would be 
wasteful to write out all of the cases and fatal error due to the size of the 
enum (consider ValueKind which I am pretty sure has > 100+ case). So we don't 
write the covered switch, do use a default case, and, of course when the switch 
needs to be updated, forgot to update the switch yielding bugs.

Keep in mind I am not 100% sure what the right solution to this is or even if 
we /should/ solve this solution. But since it is easy for bugs to result from 
this, a solution to this problem would improve the quality of the compiler. The 
key thing to notice here is that we actually do not intrinsically care about 
all of the cases being covered by a switch but rather that the user is notified 
whenever an enum is changed that a specific switch needs to be re-visited and 
or updated.

This made me wonder I wonder if we could create some sort of tooling with such 
an affect. My simple strawman would be to mark a switch as being associated 
with a specific enum/its cases at a specific revision.

Your thoughts?
Michael
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

+1!

> On Jul 26, 2017, at 8:29 AM, Erik Eckstein via swift-dev 
>  wrote:
> 
> As John said, this is out of scope for swift 4.
> I think it’s find to just have it in master.
> 
> Thanks again for your work!
> 
>> On Jul 25, 2017, at 10:00 AM, John McCall via swift-dev 
>>  wrote:
>> 
>> 
>>> On Jul 25, 2017, at 12:55 PM, Johannes Weiß via swift-dev 
>>>  wrote:
>>> 
>>> Thanks very much Eric & Michael for your help! And thanks for merging it. 
>>> Will this automatically be part of Swift 4 or do I need to make another PR 
>>> against a swift 4  branch?
>> 
>> It's very hard to imagine we would take any optimizer work this late for 
>> Swift 4.
>> 
>> John.
>> 
>>> 
>>> Thanks,
>>> Johannes
>>> 
 On 18 Jul 2017, at 9:21 pm, Johannes Weiß via swift-dev 
  wrote:
 
 great, thank you. I think I have worked all the suggestions in, let's do 
 the rest in this PR: https://github.com/apple/swift/pull/11040
 
 It's getting late here so I'll probably to the test tomorrow (but marked 
 the patch as 'do not merge' & added that a test is still missing).
 
> On 18 Jul 2017, at 7:29 pm, Erik Eckstein  wrote:
> 
>> 
>> On Jul 18, 2017, at 10:40 AM, Johannes Weiß  
>> wrote:
>> 
>> Thanks, both suggestions look great. Will work them in tomorrow and will 
>> also try to add a test for the whole thing.
>> 
>>> On 18 Jul 2017, at 5:53 pm, Michael Gottesman  
>>> wrote:
>>> 
 
 On Jul 18, 2017, at 8:39 AM, Johannes Weiß  
 wrote:
 
 Hi Erik,
 
> On 17 Jul 2017, at 10:26 pm, Erik Eckstein  
> wrote:
> 
> Hi Johannes,
> 
> great that you want to work on this!
 
 Thanks for your help, without Michael's and your help I wouldn't have 
 been able to do anything here really!
 
 
> Some ideas:
> SideEffectAnalysis currently does not have a notion of “this argument 
> is not modified by the callee” if the callee is unknown or does 
> anything non-trivial.
> Therefore I think it’s best to put the in_guarantee check directly 
> into MemoryBehaviorVisitor::visitApplyInst():
> If the parameter convention is in_guaranteed and the parameter is V, 
> then the behavior can be MemBehavior::MayRead
 
 
 Thanks, that actually makes a lot of sense. Here's my diff right now
 
 diff --git a/lib/SILOptimizer/Analysis/MemoryBehavior.cpp 
 b/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
 index b1fe7fa665..c44cc64f94 100644
 --- a/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
 +++ b/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
 @@ -245,6 +245,23 @@ MemBehavior 
 MemoryBehaviorVisitor::visitApplyInst(ApplyInst *AI) {
 (InspectionMode == RetainObserveKind::ObserveRetains &&
  ApplyEffects.mayAllocObjects())) {
 Behavior = MemBehavior::MayHaveSideEffects;
> 
> You should move your new code out of the if (ApplyEffects.mayReadRC() ...
> Otherwise it will not be executed if the called function does not read a 
> reference count.
> 
> Beside that it looks good to me.
> 
 +
 +unsigned Idx = 0;
 +bool AllReadOnly = false;
 +for (Operand  : AI->getArgumentOperands()) {
 +if (operand.get() == V && 
 AI->getOrigCalleeType()->getParameters()[Idx].isIndirectInGuaranteed())
  {
 +AllReadOnly = true;
 +} else {
 +AllReadOnly = false;
 +break;
 +}
 +
 +Idx++;
 +}
 +
 +if (AllReadOnly) {
 +Behavior = MemBehavior::MayRead;
 +}
>>> 
>>> Suggestion:
>>> 
>>> if (all_of(enumerate(AI->getArgumentOperands()),
>>> [&](std::pair pair) -> bool {
>>>return pair.second.get() == V && 
>>> AI->getOrigCalleeType()->getParameters()[Idx].isIndirectInGuaranteed()
>>> })) {
>>> Behavior = MemBehavior::MayRead;
>>> }
>>> 
>>> I may have gotten the order of the pair templates wrong. But something 
>>> like this is a little cleaner.
>>> 
>>> Michael
>>> 
 } else {
 auto  = ApplyEffects.getGlobalEffects();
 Behavior = GlobalEffects.getMemBehavior(InspectionMode);
 
 which indeed turns
 
 --- SNIP ---
 sil @bar : $@convention(thin) (@in Int) -> () {
 bb0(%0 : $*Int):
 %value_raw = integer_literal $Builtin.Int64, 42
 %value = struct $Int 

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

> On Jul 18, 2017, at 9:53 AM, Michael Gottesman via swift-dev 
> <swift-dev@swift.org> wrote:
> 
>> 
>> On Jul 18, 2017, at 8:39 AM, Johannes Weiß <johanneswe...@apple.com> wrote:
>> 
>> Hi Erik,
>> 
>>> On 17 Jul 2017, at 10:26 pm, Erik Eckstein <eeckst...@apple.com> wrote:
>>> 
>>> Hi Johannes,
>>> 
>>> great that you want to work on this!
>> 
>> Thanks for your help, without Michael's and your help I wouldn't have been 
>> able to do anything here really!
>> 
>> 
>>> Some ideas:
>>> SideEffectAnalysis currently does not have a notion of “this argument is 
>>> not modified by the callee” if the callee is unknown or does anything 
>>> non-trivial.
>>> Therefore I think it’s best to put the in_guarantee check directly into 
>>> MemoryBehaviorVisitor::visitApplyInst():
>>> If the parameter convention is in_guaranteed and the parameter is V, then 
>>> the behavior can be MemBehavior::MayRead
>> 
>> 
>> Thanks, that actually makes a lot of sense. Here's my diff right now
>> 
>> diff --git a/lib/SILOptimizer/Analysis/MemoryBehavior.cpp 
>> b/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
>> index b1fe7fa665..c44cc64f94 100644
>> --- a/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
>> +++ b/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
>> @@ -245,6 +245,23 @@ MemBehavior 
>> MemoryBehaviorVisitor::visitApplyInst(ApplyInst *AI) {
>>  (InspectionMode == RetainObserveKind::ObserveRetains &&
>>   ApplyEffects.mayAllocObjects())) {
>>Behavior = MemBehavior::MayHaveSideEffects;
>> +
>> +unsigned Idx = 0;
>> +bool AllReadOnly = false;
>> +for (Operand  : AI->getArgumentOperands()) {
>> +if (operand.get() == V && 
>> AI->getOrigCalleeType()->getParameters()[Idx].isIndirectInGuaranteed()) {
>> +AllReadOnly = true;
>> +} else {
>> +AllReadOnly = false;
>> +break;
>> +}
>> +
>> +Idx++;
>> +}
>> +
>> +if (AllReadOnly) {
>> +Behavior = MemBehavior::MayRead;
>> +}
> 
> Suggestion:
> 
> if (all_of(enumerate(AI->getArgumentOperands()),
>  [&](std::pair<unsigned, SILValue> pair) -> bool {
> return pair.second.get() == V && 
> AI->getOrigCalleeType()->getParameters()[Idx].isIndirectInGuaranteed()
>  })) {
>   Behavior = MemBehavior::MayRead;
> }
> 
> I may have gotten the order of the pair templates wrong. But something like 
> this is a little cleaner.

Also, I think you want to use the SubstCalleeType, not the original callee 
type. The difference is that orig callee type can refer to the base 
unspecialized type of a specialized function. You always want to use the 
specialized function if you have it (which is the subst callee type.

> 
> Michael
> 
>>  } else {
>>auto  = ApplyEffects.getGlobalEffects();
>>Behavior = GlobalEffects.getMemBehavior(InspectionMode);
>> 
>> which indeed turns
>> 
>> --- SNIP ---
>> sil @bar : $@convention(thin) (@in Int) -> () {
>> bb0(%0 : $*Int):
>> %value_raw = integer_literal $Builtin.Int64, 42
>> %value = struct $Int (%value_raw : $Builtin.Int64)
>> store %value to %0 : $*Int
>> 
>> %f_buz = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> ()
>> %r1 = apply %f_buz(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
>> 
>> %value_again = load %0 : $*Int
>> %f_test = function_ref @test : $@convention(thin) (Int) -> ()
>> %r2 = apply %f_test(%value_again) : $@convention(thin) (Int) -> ()
>> 
>> /*
>> %f_bad = function_ref @bad : $@convention(thin) (@in_guaranteed Int) -> ()
>> %r3 = apply %f_bad(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
>> 
>> %value_again2 = load %0 : $*Int
>> %r4 = apply %f_test(%value_again2) : $@convention(thin) (Int) -> ()
>> */
>> 
>> % = tuple()
>> return % : $()
>> }
>> --- SNAP ---
>> 
>> into
>> 
>> --- SNIP ---
>> bb0(%0 : $*Int):
>> %1 = integer_literal $Builtin.Int64, 42 // user: %2
>> %2 = struct $Int (%1 : $Builtin.Int64)  // users: %7, %3
>> store %2 to %0 : $*Int  // id: %3
>> // function_ref buz
>> %4 = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> () // 
>> user: %5
>> %5 = apply %4(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
>

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

> On Jul 18, 2017, at 8:39 AM, Johannes Weiß  wrote:
> 
> Hi Erik,
> 
>> On 17 Jul 2017, at 10:26 pm, Erik Eckstein  wrote:
>> 
>> Hi Johannes,
>> 
>> great that you want to work on this!
> 
> Thanks for your help, without Michael's and your help I wouldn't have been 
> able to do anything here really!
> 
> 
>> Some ideas:
>> SideEffectAnalysis currently does not have a notion of “this argument is not 
>> modified by the callee” if the callee is unknown or does anything 
>> non-trivial.
>> Therefore I think it’s best to put the in_guarantee check directly into 
>> MemoryBehaviorVisitor::visitApplyInst():
>> If the parameter convention is in_guaranteed and the parameter is V, then 
>> the behavior can be MemBehavior::MayRead
> 
> 
> Thanks, that actually makes a lot of sense. Here's my diff right now
> 
> diff --git a/lib/SILOptimizer/Analysis/MemoryBehavior.cpp 
> b/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
> index b1fe7fa665..c44cc64f94 100644
> --- a/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
> +++ b/lib/SILOptimizer/Analysis/MemoryBehavior.cpp
> @@ -245,6 +245,23 @@ MemBehavior 
> MemoryBehaviorVisitor::visitApplyInst(ApplyInst *AI) {
>   (InspectionMode == RetainObserveKind::ObserveRetains &&
>ApplyEffects.mayAllocObjects())) {
> Behavior = MemBehavior::MayHaveSideEffects;
> +
> +unsigned Idx = 0;
> +bool AllReadOnly = false;
> +for (Operand  : AI->getArgumentOperands()) {
> +if (operand.get() == V && 
> AI->getOrigCalleeType()->getParameters()[Idx].isIndirectInGuaranteed()) {
> +AllReadOnly = true;
> +} else {
> +AllReadOnly = false;
> +break;
> +}
> +
> +Idx++;
> +}
> +
> +if (AllReadOnly) {
> +Behavior = MemBehavior::MayRead;
> +}

Suggestion:

if (all_of(enumerate(AI->getArgumentOperands()),
  [&](std::pair pair) -> bool {
 return pair.second.get() == V && 
AI->getOrigCalleeType()->getParameters()[Idx].isIndirectInGuaranteed()
  })) {
   Behavior = MemBehavior::MayRead;
}

I may have gotten the order of the pair templates wrong. But something like 
this is a little cleaner.

Michael

>   } else {
> auto  = ApplyEffects.getGlobalEffects();
> Behavior = GlobalEffects.getMemBehavior(InspectionMode);
> 
> which indeed turns
> 
> --- SNIP ---
> sil @bar : $@convention(thin) (@in Int) -> () {
> bb0(%0 : $*Int):
>  %value_raw = integer_literal $Builtin.Int64, 42
>  %value = struct $Int (%value_raw : $Builtin.Int64)
>  store %value to %0 : $*Int
> 
>  %f_buz = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> ()
>  %r1 = apply %f_buz(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
> 
>  %value_again = load %0 : $*Int
>  %f_test = function_ref @test : $@convention(thin) (Int) -> ()
>  %r2 = apply %f_test(%value_again) : $@convention(thin) (Int) -> ()
> 
>  /*
>  %f_bad = function_ref @bad : $@convention(thin) (@in_guaranteed Int) -> ()
>  %r3 = apply %f_bad(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
> 
>  %value_again2 = load %0 : $*Int
>  %r4 = apply %f_test(%value_again2) : $@convention(thin) (Int) -> ()
>  */
> 
>  % = tuple()
>  return % : $()
> }
> --- SNAP ---
> 
> into
> 
> --- SNIP ---
> bb0(%0 : $*Int):
>  %1 = integer_literal $Builtin.Int64, 42 // user: %2
>  %2 = struct $Int (%1 : $Builtin.Int64)  // users: %7, %3
>  store %2 to %0 : $*Int  // id: %3
>  // function_ref buz
>  %4 = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> () // 
> user: %5
>  %5 = apply %4(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
>  // function_ref test
>  %6 = function_ref @test : $@convention(thin) (Int) -> () // user: %7
>  %7 = apply %6(%2) : $@convention(thin) (Int) -> ()
>  %8 = tuple ()   // user: %9
>  return %8 : $() // id: %9
> } // end sil function 'bar'
> --- SNAP ---
> 
> so the load has successfully been eliminated. Also taking my initial repro 
> [1], the retain, the load, and the release are now gone .
> 
> Is that roughly what you were thinking of?
> 
> Many thanks,
>  Johannes
> 
> [1]: https://bugs.swift.org/secure/attachment/12378/test.swift
> 
>> 
>> Erik
>> 
>>> On Jul 17, 2017, at 9:23 AM, Johannes Weiß  wrote:
>>> 
>>> Thanks Michael!
>>> 
>>> Erik, I hope what I wrote makes some sense. Any questions or suggestions, 
>>> please let me know.
>>> 
 On 14 Jul 2017, at 7:30 pm, Michael Gottesman  wrote:
 
> 
> On Jul 12, 2017, at 9:53 AM, Johannes Weiß  
> wrote:
> 
> Hi Michael,
> 
> 
>> [...]
>>> Long story short, I think the RLE actually works for the test case I 
>>> created. It's even clever enough to see through my invalid function 
>>> bad() which modified the storage despite its 

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

> On Jul 12, 2017, at 9:53 AM, Johannes Weiß  wrote:
> 
> Hi Michael,
> 
> 
>> [...]
>>> Long story short, I think the RLE actually works for the test case I 
>>> created. It's even clever enough to see through my invalid function bad() 
>>> which modified the storage despite its claim that it doesn't. I might also 
>>> be misunderstanding something.
>> 
>> When something is marked as in_guaranteed, it should be immutable. If the 
>> callee violates that, then the SIL is malformed. Perhaps, we can add some 
>> sort of verification check.
> 
> Right, yes I was aware that that'd be illegal but I added it as a way to 
> check whether this optimisation is 'looking into' the called function.
> 
> 
>> That being said, I have a good feeling that there is some sort of analysis 
>> occuring here since you provided enough information to the optimizer. The 
>> optimization here is regardless of whether or not we can see the body of a 
>> function, we know that it is safe to optimize this just based off the 
>> @in_guaranteed. This implies using a declaration, not a definition of the 
>> bad function.
> 
> makes sense, didn't think about just only declaring it...
> 
> 
>> When I said write the SIL by hand, what I meant was writing the whole 
>> program by hand. In general, we prefer SIL programs that do not have 
>> extraneous stuff that is added by the compiler (for instance debug_value). 
>> Additionally, it is important for SIL files to not have dependencies on the 
>> stdlib unless absolutely necessary (i.e. your usage of Int). This prevents 
>> the stdlib maintainers from having to update these tests given chances to 
>> the stdlib. Below is a cleaned up version that shows the problem. Look at 
>> how small it is and how it tests /exactly/ what we are trying to test and 
>> via the use of declarations and the like we are able to exclude other 
>> optimizations:
> 
> That makes a lot of sense, thank you. I wasn't yet that familiar with SIL so 
> I thought I start from a program generated by the compiler and then replace 
> the guts with handwritten SIL. But your version is clearly a lot easier to 
> understand and shows what precisely we want to see!
> 
> Today, I looked into why this is happening more precisely.
> 
> So we don't get the RLE because in this code
> 
>  if (isComputeAvailValue(Kind) || isPerformingRLE(Kind)) {
>for (unsigned i = 0; i < Locs.size(); ++i) {
>  if (isTrackingLocation(ForwardSetIn, Ctx.getLocationBit(Locs[i])))
>continue;
>  updateForwardSetAndValForRead(Ctx, Ctx.getLocationBit(Locs[i]),
>Ctx.getValueBit(Vals[i]));
>  // We can not perform the forwarding as we are at least missing
>  // some pieces of the read location.
>  CanForward = false;
> 
> (https://github.com/apple/swift/blob/86620aaa7ebd32d33f4cdf61add5c63a72d3f02a/lib/SILOptimizer/Transforms/RedundantLoadElimination.cpp#L917)
> 
> we're not taking the `continue` for the call to `buz()`. The reason why is 
> that here
> 
>if (!AA->mayWriteToMemory(I, R.getBase()))
>  continue;
>// MayAlias.
>stopTrackingLocation(ForwardSetIn, i);
>stopTrackingValue(ForwardValIn, i);
> 
> (https://github.com/apple/swift/blob/86620aaa7ebd32d33f4cdf61add5c63a72d3f02a/lib/SILOptimizer/Transforms/RedundantLoadElimination.cpp#L972)
> 
> we're not taking the `continue`, ie. `AA->mayWriteToMemory(I, R.getBase())` 
> is true. The reason for that is that the `SILFunction` for `buz` has
> 
>   EffectsKindAttr = Unspecified
> 
> which equates to `MayHaveSideEffects`, that's also what 
> `-debug-only=sil-redundant-load-elim,sil-membehavior` outputs:
> 
> GET MEMORY BEHAVIOR FOR:
>  %5 = apply %4(%0) : $@convention(thin) (@in_guaranteed Int) -> ()
>  %0 = argument of bb0 : $*Int// users: %6, %5, %3
>  Found apply, returning MayHaveSideEffects
> 
> 
> So where I'm stuck today is that I'm not sure how `EffectsKindAttr` is 
> actually defined. Sure, `$@convention(thin) (@in_guaranteed Int) -> ()` 
> doesn't actually write to the `@in_guaranteed Int` (as that'd be illegal) but 
> it may have other side effects. So I'm not sure if we can just create the 
> function differently if we find only "read-only" kind of parameters. That'd 
> be I think in
> 
> 
>  auto *fn = SILMod.createFunction(SILLinkage::Private, Name.str(), Ty,
>   nullptr, loc, IsNotBare,
>   IsNotTransparent, IsNotSerialized);
> 
> (https://github.com/apple/swift/blob/ec6fc4d54db95f78ae72dab29734533f709ea2d7/lib/Parse/ParseSIL.cpp#L508
>  -> 
> https://github.com/apple/swift/blob/157db57506b813837481b574a9d38e806bf954b6/lib/SIL/SILModule.cpp#L249)
> 
> which doesn't specify any EffectsAttrKind and therefore it defaults to 
> `Unspecified`.
> 
> 
> Just as a test, I did put a `[readonly]` in `sil @buz : $@convention(thin) 
> (@in_guaranteed Int) -> ()` and as expected everything propagates 

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

> On Jul 11, 2017, at 4:21 PM, Michael Gottesman via swift-dev 
> <swift-dev@swift.org> wrote:
> 
>> 
>> On Jul 11, 2017, at 10:16 AM, Johannes Weiß <johanneswe...@apple.com 
>> <mailto:johanneswe...@apple.com>> wrote:
>> 
>> Hi Michael,
>> 
>>> [...]
>>>> Now you advise to run the '-debug-only=sil-redundant-load-elim' so I tried
>>>> 
>>>> sil-opt [...] -debug-only=sil-redundant-load-elim
>>>> 
>>>> but it doesn't seem happy with that. Did I misunderstand how to pass this 
>>>> option?
>>> 
>>> What do you mean by it doesn't seem happy?
>> 
>> I was using a too old/wrong version of sil-opt (the one from Xcode 9 beta N) 
>> which didn't have that option. Resolved by using the one from my own swift 
>> build.
> 
> Ok.
> 
>> 
>> 
>>> [...]
>>>> is this roughly what you had in mind?
>>> 
>>> Nope. I was suggesting that you write the SIL by hand. It will be much 
>>> easier.
>> 
>> Ha, that's a much better idea, thanks for your help, much appreciated! No 
>> idea why I didn't think of that. This is today's update:
>> 
>> --- SNIP (whole file attached as test-load-forwarding.sil) ---
>> // bar()
>> sil hidden @bar : $@convention(thin) () -> () {
>> bb0:
>>  alloc_global @MyIntStorage
>>  %int_storage = global_addr @MyIntStorage : $*Int
>>  %value_raw = integer_literal $Builtin.Int64, 42
>>  %value = struct $Int (%value_raw : $Builtin.Int64)
>>  store %value to %int_storage : $*Int
>> 
>>  %f_buz = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> ()
>>  %r1 = apply %f_buz(%int_storage) : $@convention(thin) (@in_guaranteed Int) 
>> -> ()
>> 
>>  %value_again = load %int_storage : $*Int
>>  %f_test = function_ref @testNumber12345 : $@convention(thin) (Int) -> ()
>>  %r2 = apply %f_test(%value_again) : $@convention(thin) (Int) -> ()
>> 
>>  // just to test an illegal function
>>  %f_bad = function_ref @bad : $@convention(thin) (@in_guaranteed Int) -> ()
>>  %r3 = apply %f_bad(%int_storage) : $@convention(thin) (@in_guaranteed Int) 
>> -> ()
>> 
>>  %value_again2 = load %int_storage : $*Int
>>  %r4 = apply %f_test(%value_again2) : $@convention(thin) (Int) -> ()
>> 
>>  return %r2 : $()
>> } // end sil function 'bar'
>> --- SNAP ---
>> 
>> So I make a global Int storage, store the number 42 to it, invoke buz() 
>> which takes an @in_guaranteed Int, then load it again (this is redundant as 
>> @in_guaranteed isn't allowed to modify it). I also pass the loaded number to 
>> the function testNumber12345() which just tests that it's the number
>> 12345 (which it isn't but I wanted to be sure it's actually consumed)
>> 
>> Then (because I believe the RLE actually works) I also introduced a function 
>> `bad` which is a bit like `buz` but it does actually write to the *Int which 
>> I believe is illegal (right?).
>> 
>> Now let's see what we get with sil-opt
>> 
>> I run the following command:
>> 
>>sil-opt -verify -assume-parsing-unqualified-ownership-sil 
>> -redundant-load-elim -debug-only=sil-redundant-load-elim -debug 
>> test-load-forwarding.sil
>> 
>> which gives me the following output (my annotations marked as [JW: ... ])
>> 
>> 
>> --- SNIP (whole file attached as test-load-forwarding.sil-opt) ---
>> *** RLE on function: testNumber12345 ***
>> *** RLE on function: buz ***
>> *** RLE on function: bad ***
>> *** RLE on function: bar ***
>> LSLocation #0  %1 = global_addr @MyIntStorage : $*Int  // users: 
>> %12, %11, %7, %6, %4
>> Projection Path [$*Int
>>  Field: var _value: Int64 of: $*Builtin.Int64]
>> PROCESS bb0 for RLE.
>> WRITE   alloc_global @MyIntStorage  // id: %0
>> WRITE   %6 = apply %5(%1) : $@convention(thin) (@in_guaranteed Int) -> ()
>> FORWARD   %3 = struct $Int (%2 : $Builtin.Int64)  // user: %4
>>  to  %7 = load %1 : $*Int// user: %9
>> WRITE   %9 = apply %8(%7) : $@convention(thin) (Int) -> () // user: %14
>> WRITE   %11 = apply %10(%1) : $@convention(thin) (@in_guaranteed Int) -> ()
>> WRITE   %13 = apply %8(%12) : $@convention(thin) (Int) -> ()
>> Replacing%7 = load %1 : $*Int// user: %9
>> With   %3 = struct $Int (%2 : $Builtin.Int64)  // user: %4
>> [JW:  this looks pretty promising to me, don't understand all the output 
>> 

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

> On Jul 11, 2017, at 10:16 AM, Johannes Weiß  wrote:
> 
> Hi Michael,
> 
>> [...]
>>> Now you advise to run the '-debug-only=sil-redundant-load-elim' so I tried
>>> 
>>> sil-opt [...] -debug-only=sil-redundant-load-elim
>>> 
>>> but it doesn't seem happy with that. Did I misunderstand how to pass this 
>>> option?
>> 
>> What do you mean by it doesn't seem happy?
> 
> I was using a too old/wrong version of sil-opt (the one from Xcode 9 beta N) 
> which didn't have that option. Resolved by using the one from my own swift 
> build.

Ok.

> 
> 
>> [...]
>>> is this roughly what you had in mind?
>> 
>> Nope. I was suggesting that you write the SIL by hand. It will be much 
>> easier.
> 
> Ha, that's a much better idea, thanks for your help, much appreciated! No 
> idea why I didn't think of that. This is today's update:
> 
> --- SNIP (whole file attached as test-load-forwarding.sil) ---
> // bar()
> sil hidden @bar : $@convention(thin) () -> () {
> bb0:
>  alloc_global @MyIntStorage
>  %int_storage = global_addr @MyIntStorage : $*Int
>  %value_raw = integer_literal $Builtin.Int64, 42
>  %value = struct $Int (%value_raw : $Builtin.Int64)
>  store %value to %int_storage : $*Int
> 
>  %f_buz = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> ()
>  %r1 = apply %f_buz(%int_storage) : $@convention(thin) (@in_guaranteed Int) 
> -> ()
> 
>  %value_again = load %int_storage : $*Int
>  %f_test = function_ref @testNumber12345 : $@convention(thin) (Int) -> ()
>  %r2 = apply %f_test(%value_again) : $@convention(thin) (Int) -> ()
> 
>  // just to test an illegal function
>  %f_bad = function_ref @bad : $@convention(thin) (@in_guaranteed Int) -> ()
>  %r3 = apply %f_bad(%int_storage) : $@convention(thin) (@in_guaranteed Int) 
> -> ()
> 
>  %value_again2 = load %int_storage : $*Int
>  %r4 = apply %f_test(%value_again2) : $@convention(thin) (Int) -> ()
> 
>  return %r2 : $()
> } // end sil function 'bar'
> --- SNAP ---
> 
> So I make a global Int storage, store the number 42 to it, invoke buz() which 
> takes an @in_guaranteed Int, then load it again (this is redundant as 
> @in_guaranteed isn't allowed to modify it). I also pass the loaded number to 
> the function testNumber12345() which just tests that it's the number 
> 12345 (which it isn't but I wanted to be sure it's actually consumed)
> 
> Then (because I believe the RLE actually works) I also introduced a function 
> `bad` which is a bit like `buz` but it does actually write to the *Int which 
> I believe is illegal (right?).
> 
> Now let's see what we get with sil-opt
> 
> I run the following command:
> 
>sil-opt -verify -assume-parsing-unqualified-ownership-sil 
> -redundant-load-elim -debug-only=sil-redundant-load-elim -debug 
> test-load-forwarding.sil
> 
> which gives me the following output (my annotations marked as [JW: ... ])
> 
> 
> --- SNIP (whole file attached as test-load-forwarding.sil-opt) ---
> *** RLE on function: testNumber12345 ***
> *** RLE on function: buz ***
> *** RLE on function: bad ***
> *** RLE on function: bar ***
> LSLocation #0  %1 = global_addr @MyIntStorage : $*Int  // users: %12, 
> %11, %7, %6, %4
> Projection Path [$*Int
>  Field: var _value: Int64 of: $*Builtin.Int64]
> PROCESS bb0 for RLE.
> WRITE   alloc_global @MyIntStorage  // id: %0
> WRITE   %6 = apply %5(%1) : $@convention(thin) (@in_guaranteed Int) -> ()
> FORWARD   %3 = struct $Int (%2 : $Builtin.Int64)  // user: %4
>  to  %7 = load %1 : $*Int// user: %9
> WRITE   %9 = apply %8(%7) : $@convention(thin) (Int) -> () // user: %14
> WRITE   %11 = apply %10(%1) : $@convention(thin) (@in_guaranteed Int) -> ()
> WRITE   %13 = apply %8(%12) : $@convention(thin) (Int) -> ()
> Replacing%7 = load %1 : $*Int// user: %9
> With   %3 = struct $Int (%2 : $Builtin.Int64)  // user: %4
> [JW:  this looks pretty promising to me, don't understand all the output 
> but looks like it's replacing a load :) ]
> *** RLE on function: main ***
> sil_stage canonical
> 
> [...]
> 
> // bar
> sil hidden @bar : $@convention(thin) () -> () {
> bb0:
>  alloc_global @MyIntStorage  // id: %0
>  %1 = global_addr @MyIntStorage : $*Int  // users: %11, %10, %6, %4
>  %2 = integer_literal $Builtin.Int64, 42 // user: %3
>  %3 = struct $Int (%2 : $Builtin.Int64)  // users: %8, %4
>  store %3 to %1 : $*Int  // id: %4
>  // function_ref buz
>  %5 = function_ref @buz : $@convention(thin) (@in_guaranteed Int) -> () // 
> user: %6
>  %6 = apply %5(%1) : $@convention(thin) (@in_guaranteed Int) -> ()
>  // function_ref testNumber12345
>  %7 = function_ref @testNumber12345 : $@convention(thin) (Int) -> () // 
> users: %12, %8
>  [JW: Indeed, it seems to have optimised out the redundant load. So that 
> works! ]
>  %8 = apply %7(%3) : $@convention(thin) (Int) -> () // user: %13
>  // function_ref bad

[swift-dev] Enabling the ownership verifier on parts of the stdlib

[Michael Gottesman via swift-dev]

Hey everyone.

Over the weekend I got stdlibCore (and some other parts of the stdlib as well) 
working with the ownership verifier. I am going to be enabling the verifier by 
default on stdlibCore sometime this afternoon. If you get hit by any 
verification issues as you are working, I am adding a semantic tag in 6859de0 
that will enable the ownership verifier to be disabled on specific functions. 
The specific tag is: "verify.ownership.sil.never". The intended workflow is 
place the semantics tag on the function failing verification:

@_semantics("verify.ownership.sil.never")
func myFunctionThatForSomeReasonFailsOwnershipVerification() {
...
}

and then file a bug report so I can see if we are having a real ownership 
violation or if we have a false positive.

Michael
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] SR-5403 / Memory Optimization Opportunity (Load/Store forwarding)

[Michael Gottesman via swift-dev]

> On Jul 10, 2017, at 10:56 AM, Johannes Weiß  wrote:
> 
> Hi Michael,
> 
> Firstly: if swift-dev is not the right place for me to ask silly questions, 
> please let me know :).
> 
> Thanks very much for all your pointers on the JIRA [1] ticket. I was just 
> trying to get started and already have a few questions. Apologies, they're 
> very basic as I'm just getting started with the Swift compiler source and 
> tooling.
> 
> The first question is just about the tooling: I got sil-opt to run with an 
> invocation like this:
> 
>   swiftc -emit-module -O test.swift
>   sil-opt -sdk $(xcrun --show-sdk-path) test.swiftmodule
> 
> Now you advise to run the '-debug-only=sil-redundant-load-elim' so I tried
> 
>   sil-opt [...] -debug-only=sil-redundant-load-elim
> 
> but it doesn't seem happy with that. Did I misunderstand how to pass this 
> option?

What do you mean by it doesn't seem happy?

> 
> 
> My second question actually relates to your first suggestion: 'creating a 
> simple test that performs a store, then passes the address to an 
> in_guaranteed function and then reloads the value'
> 
> I read up about @in_guaranteed and I managed to make a @in_guaranteed 
> function this way:
> 
> protocol Foo {
>func foo()
> }
> extension Foo {
>func foo() {}
> }
> 
> The `foo()` function will now be an @in_guaranteed function taking `self`, 
> correct? (any other ways to create an `@in_guaranteed` function easily?) This 
> is probably me being a bit slow but what exactly do you mean with performing 
> a store, passing the address to an @in_guaranteed function and then reloading 
> the value? Very naively I thought about
> 
> class C: Foo {}
> func something() {
>let c = C()
>var b: Foo = c   /* store */
>b.foo()  /* pass the address to an `@in_guaranteed` func */
>c.foo()  /* load again */
> }
> 
> is this roughly what you had in mind?

Nope. I was suggesting that you write the SIL by hand. It will be much easier.

> 
> Thanks,
>  Johannes
> 
> [1]: https://bugs.swift.org/browse/SR-5403

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] State of Semantic ARC?

[Michael Gottesman via swift-dev]

> On Jul 8, 2017, at 10:36 AM, Johannes Weiß via swift-dev 
>  wrote:
> 
> Hi swift-dev,
> 
> I haven't heard anything about semantic ARC and couldn't really find much 
> documentation besides 
> https://gottesmm.github.io/proposals/high-level-arc-memory-operations.html . 
> So I wanted to ask how much of it is implemented and what the plan for Swift 
> 4/5 regarding Semantic ARC is.

Many of Semantic ARC's changes were in Swift 4, but we did not turn on the 
ownership verifier for Swift 4. This is because getting the stdlib to pass the 
verifier (a constraint for turning it on for non-stdlib code) would have been 
too disruptive at that time to land. The overall plan is to continue working on 
Semantic ARC (If you look at the commits list, I have been landing code for it 
recently). In fact, I just got stdlibCore (*note*, not the entire stdlib) to 
pass the ownership verifier, so progress is being made.

Did I answer your question?

Michael

> 
> -- Johannes
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] External pass pipeline YAML format

[Michael Gottesman via swift-dev]

> On Jun 29, 2017, at 12:50 PM, Vasileios Kalintiris  wrote:
> 
> I plan on writing a patch for this. Based on which branch should I create the 
> pull request? Is the swift-3.1-branch frozen?

master.

> 
> On Thu, Jun 29, 2017 at 7:27 PM, Michael Gottesman  > wrote:
> 
>> On Jun 28, 2017, at 10:15 AM, Vasileios Kalintiris via swift-dev 
>> > wrote:
>> 
>> > Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, it 
>> > looks like it might give some hints?
>> 
>> I've tried generating a pass pipeline with 
>> utils/pass-pipeline/scripts/pipelines_generator.py and use it with the 
>> -external-pass-pipeline-filename. However, AFAICT the generated pipeline is 
>> not in the correct format that the compiler expects:
>> 
>> [
>> [
>> "HighLevel",
>> "run_n_times",
>> 2,
>> "SimplifyCFG",
>> ...
>> "GlobalARCOpts"
>> ],
>> [
>> "EarlyLoopOpt",
>> "run_n_times",
>> 1,
>> "LowerAggregateInstrs",
>> ...
>> "SwiftArrayOpts"
>> ],
>> ...
>> ]
>> 
>> Each generated pass pipeline contains the "run_n_times"|"run_to_fixed_point" 
>> field, followed by the number of iterations, which is not what the compiler 
>> expects.
>> 
>> I had no luck even when I tried to re-format the file containing the 
>> pipelines to something that I believe the compiler would expect based on the 
>> source code of SILPassPipelinePlan::getPassPipelineFromFile():
>> 
>> [
>> [
>> "HihLevel",
>> "SimplifyCFG",
>> ...
>> "GlobalARCOpts"
>> ],
>> ...
>> ]
>> 
>> or even:
>> 
>> [
>> [
>>"HighLevel",
>>[ "SimplifyCFG" ],
>>...
>>[ "GlobalARCOpts" ]
>> ],
>> ...
>> ]
>> 
>> I suspect that we don't use the pass pipeline python scripts in our 
>> buildbots anymore and the relevant bits, ie. the code in 
>> SILPassPipelinePlan::getPassPipelineFromFile and/or the python scripts, have 
>> not been kept up-to-date. 
> 
> Yes. I think that is true. Here is what I would suggest:
> 
> 1. It would be really trivial to change this to use yamltraits.
> 2. If you make this change, make sure that a test is added (maybe to sil-opt) 
> that makes sure that we can roundtrip from the dumper.
> 
> MIchael
> 
>> 
>> 
>> On Wed, Jun 28, 2017 at 6:18 PM, Daniel Dunbar > > wrote:
>> Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, it 
>> looks like it might give some hints?
>> --
>> $ sgit grep external-pass-pipeline-filename
>> include/swift/Option/FrontendOptions.td:def external_pass_pipeline_filename 
>> : Separate<["-"], "external-pass-pipeline-filename">,
>> utils/pass-pipeline/scripts/pipelines_build_script.py:
>> '-external-pass-pipeline-filename\;-Xfrontend\;%s' % data_file]
>> --
>> 
>>  - Daniel
>> 
>> > On Jun 28, 2017, at 5:27 AM, Vasileios Kalintiris via swift-dev 
>> > > wrote:
>> >
>> > Hi all,
>> >
>> > Please, let me know if I should post this to another list.
>> >
>> > I'm trying to figure out what is the expected YAML format of the 
>> > -external-pass-pipeline-filename option.
>> >
>> > Dumping the pass pipeline under -O and feeding it back to the compiler 
>> > with this option doesn't work (from swift-3.1-branch).
>> >
>> > I thought to ask here because I'm not entirely sure that the relevant YAML 
>> > parsing code from SILPassPipelinePlan::getPassPipelineFromFile() accepts 
>> > valid YAML input.
>> >
>> > Thanks,
>> > Vasileios
>> > ___
>> > swift-dev mailing list
>> > swift-dev@swift.org 
>> > https://lists.swift.org/mailman/listinfo/swift-dev 
>> > 
>> 
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev 
>> 
> 
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] External pass pipeline YAML format

[Michael Gottesman via swift-dev]

> On Jun 28, 2017, at 10:15 AM, Vasileios Kalintiris via swift-dev 
>  wrote:
> 
> > Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, it 
> > looks like it might give some hints?
> 
> I've tried generating a pass pipeline with 
> utils/pass-pipeline/scripts/pipelines_generator.py and use it with the 
> -external-pass-pipeline-filename. However, AFAICT the generated pipeline is 
> not in the correct format that the compiler expects:
> 
> [
> [
> "HighLevel",
> "run_n_times",
> 2,
> "SimplifyCFG",
> ...
> "GlobalARCOpts"
> ],
> [
> "EarlyLoopOpt",
> "run_n_times",
> 1,
> "LowerAggregateInstrs",
> ...
> "SwiftArrayOpts"
> ],
> ...
> ]
> 
> Each generated pass pipeline contains the "run_n_times"|"run_to_fixed_point" 
> field, followed by the number of iterations, which is not what the compiler 
> expects.
> 
> I had no luck even when I tried to re-format the file containing the 
> pipelines to something that I believe the compiler would expect based on the 
> source code of SILPassPipelinePlan::getPassPipelineFromFile():
> 
> [
> [
> "HihLevel",
> "SimplifyCFG",
> ...
> "GlobalARCOpts"
> ],
> ...
> ]
> 
> or even:
> 
> [
> [
>"HighLevel",
>[ "SimplifyCFG" ],
>...
>[ "GlobalARCOpts" ]
> ],
> ...
> ]
> 
> I suspect that we don't use the pass pipeline python scripts in our buildbots 
> anymore and the relevant bits, ie. the code in 
> SILPassPipelinePlan::getPassPipelineFromFile and/or the python scripts, have 
> not been kept up-to-date. 

Yes. I think that is true. Here is what I would suggest:

1. It would be really trivial to change this to use yamltraits.
2. If you make this change, make sure that a test is added (maybe to sil-opt) 
that makes sure that we can roundtrip from the dumper.

MIchael

> 
> 
> On Wed, Jun 28, 2017 at 6:18 PM, Daniel Dunbar  > wrote:
> Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, it 
> looks like it might give some hints?
> --
> $ sgit grep external-pass-pipeline-filename
> include/swift/Option/FrontendOptions.td:def external_pass_pipeline_filename : 
> Separate<["-"], "external-pass-pipeline-filename">,
> utils/pass-pipeline/scripts/pipelines_build_script.py:
> '-external-pass-pipeline-filename\;-Xfrontend\;%s' % data_file]
> --
> 
>  - Daniel
> 
> > On Jun 28, 2017, at 5:27 AM, Vasileios Kalintiris via swift-dev 
> > > wrote:
> >
> > Hi all,
> >
> > Please, let me know if I should post this to another list.
> >
> > I'm trying to figure out what is the expected YAML format of the 
> > -external-pass-pipeline-filename option.
> >
> > Dumping the pass pipeline under -O and feeding it back to the compiler with 
> > this option doesn't work (from swift-3.1-branch).
> >
> > I thought to ask here because I'm not entirely sure that the relevant YAML 
> > parsing code from SILPassPipelinePlan::getPassPipelineFromFile() accepts 
> > valid YAML input.
> >
> > Thanks,
> > Vasileios
> > ___
> > swift-dev mailing list
> > swift-dev@swift.org 
> > https://lists.swift.org/mailman/listinfo/swift-dev 
> > 
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Pathway to becoming an effective contributor

[Michael Gottesman via swift-dev]

> On Jun 22, 2017, at 12:32 AM, Jacob Bandes-Storch via swift-dev 
>  wrote:
> 
> Hi,
> I’ve thought about this issue in the past too, and would be happy to help 
> write some content or at least proofread / contribute tidbits to a guide for 
> new developers.
> 
> As pointed out in this thread, a lot of Swift infrastructure and paradigms 
> comes from the LLVM community, but spelling that out in a “Getting Started” 
> guide would be nice.

Patch?

> It’d also be worth gathering some tips & tricks for developing & debugging 
> issues, using Xcode or otherwise.

There is already a file like this if you want to extend it. It is 
./docs/DebuggingTheCompiler.rst.

> 
> Would a Markdown document in the apple/swift repo be the best place for such 
> a thing?

See above ; ).

I imagine if you wanted to convert it to markdown, I don't think anyone would 
object. But I could be wrong.

> 
> 
> On Tue, Jun 20, 2017 at 12:37 PM Halen Wooten via swift-dev 
> > wrote:
> Hi,
> 
> I would like to start a discussion about getting started with the
> Swift Compiler. The end goal is to see if there are ways we can help
> newcomers to the compiler get started and more quickly grow into a
> more effective contributor. The motivation is that I've had a
> surprising amount of trouble with my starter bug. Swift is amazing and
> I would love to contribute more regularly, but I also don't want to
> burden the core team.
> 
> I know that documentation on a huge project like this is a
> non-starter, but I wonder if we could have better information on the
> contribution process, which likely wouldn't change frequently. For
> example, I learned through a conference talk that Swift uses LLVM's
> lit testing. I couldn't find that in any of Swift's documentation. The
> docs explain how to run tests, but not how to write them. I would be
> happy to help out with documentation if we can decide on changes that
> would be useful.
> 
> Also, after I'm able to get my starter bug merged, I have no idea
> where to go from there. I don't want to take another starter bug and
> deprive someone else of the opportunity to contribute (although maybe
> that's an unnecessary restriction I'm placing on myself), but I'm also
> not in a place where I could take on something much larger.
> 
> The learning curve for a compiler is always going to be high, but does
> anyone have ideas on how we could assist newcomers with their first
> starter bug and then transitioning into something larger?
> 
> Thanks,
> 
> Halen
> ___
> swift-dev mailing list
> swift-dev@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-dev 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Pathway to becoming an effective contributor

[Michael Gottesman via swift-dev]

> On Jun 20, 2017, at 12:37 PM, Halen Wooten via swift-dev 
>  wrote:
> 
> Hi,
> 
> I would like to start a discussion about getting started with the
> Swift Compiler. The end goal is to see if there are ways we can help
> newcomers to the compiler get started and more quickly grow into a
> more effective contributor. The motivation is that I've had a
> surprising amount of trouble with my starter bug. Swift is amazing and
> I would love to contribute more regularly, but I also don't want to
> burden the core team.
> 
> I know that documentation on a huge project like this is a
> non-starter, but I wonder if we could have better information on the
> contribution process, which likely wouldn't change frequently. For
> example, I learned through a conference talk that Swift uses LLVM's
> lit testing. I couldn't find that in any of Swift's documentation. The
> docs explain how to run tests, but not how to write them. I would be
> happy to help out with documentation if we can decide on changes that
> would be useful.
> 
> Also, after I'm able to get my starter bug merged, I have no idea
> where to go from there. I don't want to take another starter bug and
> deprive someone else of the opportunity to contribute (although maybe
> that's an unnecessary restriction I'm placing on myself), but I'm also
> not in a place where I could take on something much larger.

I would like to echo what Adrian/Greg have said. There is always more work to 
be done, so I wouldn't worry about this. But I just wanted to add, if there is 
an area of the compiler that interests you and there are not open starter bugs 
in that area, it may just be because no one has taken the time to file one. In 
such a case, I would just send an email to this list and ask what other things 
need to be fixed in this area of the compiler. We will be more than happy to 
guide you [that is how open source /should/ work, after all = )].

Michael

> 
> The learning curve for a compiler is always going to be high, but does
> anyone have ideas on how we could assist newcomers with their first
> starter bug and then transitioning into something larger?
> 
> Thanks,
> 
> Halen
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] statically initialized arrays

[Michael Gottesman via swift-dev]

> On Jun 14, 2017, at 11:24 AM, Erik Eckstein via swift-dev 
>  wrote:
> 
> Hi,
> 
> I’m about implementing statically initialized arrays. It’s about allocating 
> storage for arrays in the data section rather than on the heap.
> 
> Info: the array storage is a heap object. So in the following I’m using the 
> general term “object” but the optimization will (probably) only handle array 
> buffers.
> 
> This optimization can be done for array literals containing only other 
> literals as elements.
> Example:
> 
> func createArray() -> [Int] {
>   return [1, 2, 3]
> }
> 
> The compiler can allocate the whole array buffer as a statically initialized 
> global llvm-variable with a reference count of 2 to make it immortal.

I was thinking about this a little bit. IMO, we probably actually want a bit in 
the header. The reason why is that even though setting the ref count to be 
unbalanced makes an object immortal, retain/release will still modify the 
reference count meaning that the statically initialized constant can not be in 
read only memory. On the other hand, if we had a bit in the header, we could 
use read only memory.

Michael

> It avoids heap allocations for array literals in cases stack-promotion can’t 
> kick in. It also saves code size.
> 
> What’s needed for this optimization?
> 
> 1) An optimization pass (GlobalOpt) which detects such array literal 
> initialization patterns and “outlines” those into a statically initialized 
> global variable
> 2) A representation of statically initialized global variables in SIL
> 3) IRGen to create statically initialized objects as global llvm-variables
> 
> ad 2) Changes in SIL:
> 
> Currently a static initialized sil_global is represented by having a 
> reference to a globalinit function which has to match a very specific pattern 
> (e.g. must contain a single store to the global).
> This is somehow quirky and would get even more complicated for statically 
> initialized objects.
> 
> I’d like to change that so that the sil_global itself contains the 
> initialization value.
> This part is not yet related to statically initialized objects. It just 
> improves the representation of statically initialized global in general.
> 
> @@ -1210,7 +1210,9 @@ Global Variables
>  ::
>  
>decl ::= sil-global-variable
> +  static-initializer ::= '{' sil-instruction-def* '}'
>sil-global-variable ::= 'sil_global' sil-linkage identifier ':' sil-type
> + (static-initializer)?
>  
>  SIL representation of a global variable.
>  
> @@ -1221,6 +1223,19 @@ SIL instructions. Prior to performing any access on 
> the global, the
>  Once a global's storage has been initialized, ``global_addr`` is used to
>  project the value.
>  
> +A global can also have a static initializer if it's initial value can be
> +composed of literals. The static initializer is represented as a list of
> +literal and aggregate instructions where the last instruction is the 
> top-level
> +value of the static initializer::
> +
> +  sil_global hidden @_T04test3varSiv : $Int {
> +%0 = integer_literal $Builtin.Int64, 27
> +%1 = struct $Int (%0 : $Builtin.Int64)
> +  }
> +
> +In case a global has a static initializer, no ``alloc_global`` is needed 
> before
> +it can be accessed.
> +
> 
> Now to represent a statically initialized object, we need a new instruction. 
> Note that this “instruction" can only appear in the initializer of a 
> sil_global.
> 
> +object
> +``
> +::
> +
> +  sil-instruction ::= 'object' sil-type '(' (sil-operand (',' 
> sil-operand)*)? ')'
> +
> +  object $T (%a : $A, %b : $B, ...)
> +  // $T must be a non-generic or bound generic reference type
> +  // The first operands must match the stored properties of T
> +  // Optionally there may be more elements, which are tail-allocated to T
> +
> +Constructs a statically initialized object. This instruction can only appear
> +as final instruction in a global variable static initializer list.
> 
> Finally we need an instruction to use such a statically initialized global 
> object.
> 
> +global_object
> +`
> +::
> +
> +  sil-instruction ::= 'global_object' sil-global-name ':' sil-type
> +
> +  %1 = global_object @v : $T
> +  // @v must be a global variable with a static initialized object
> +  // $T must be a reference type
> +
> +Creates a reference to the address of a global variable which has a static
> +initializer which is an object, i.e. the last instruction of the global's
> +static initializer list is an ``object`` instruction.
> 
> 
> ad 3) IRGen support
> 
> Generating statically initialized globals is already done today for structs 
> and tuples.
> What’s needed is the handling of objects.
> In addition to creating the global itself, we also need a runtime call to 
> initialize the object header. In other words: the object is statically 
> initialized, except the header.
> 
> HeapObject *swift::swift_initImmortalObject(HeapMetadata const *metadata, 
> 

Re: [swift-dev] Measuring MEAN Performance (was: Questions about Swift-CI)

[Michael Gottesman via swift-dev]

So I did a bit more research. Check out how LNT does this:

https://github.com/llvm-mirror/lnt 


I talked with Chris Matthews (+CC) about how LNT uses Mann-Whitney. In the 
following let n be the number of samples taken. From what he told me this is 
what LNT does:

1. If n is < 5, then some sort of computation around confidence intervals is 
used.
2. If the number of samples is > 5, then Mann-Whitney U is done.

I am not 100% sure what 1 is, but I think it has to do with some sort of 
quartile measurements. I.e. Find the median of the new data and make sure it is 
within +- median absolute deviation (basically mean + std-dev but more robust 
to errors). I believe the code is in LNT so we can find it for sure.

Thus in my mind the natural experiment here in terms of Mann-Whitney U.

1. This seems to suggest that for small numbers we do some sort of simple 
comparison that we do today and if a regression is "identified", we grab more 
samples of before/after and run mann-whitney u.
2. Try out different versions of n. I am not 100% sure if 5 is the right or 
wrong answer or if it should be dependent on the test.

Chris, did I get it right?

Michael

> On Jun 13, 2017, at 7:11 AM, Pavol Vaskovic via swift-dev 
>  wrote:
> 
> On Tue, Jun 13, 2017 at 8:51 AM, Andrew Trick  > wrote:
> I’m confused though because I thought we agreed that all samples need to run 
> with exactly the same number of iterations. So, there would be one short run 
> to find the desired `num_iters` for each benchmark, then each subsequent 
> invocation of the benchmark harness would be handed `num_iters` as input.
> 
> That was agreed on in the discussion about measuring memory consumption (PR 
> 8793) . 
> MAX_RSS was variable between runs, due to dynamic `num_iters` adjustment 
> inside `DriverUtils` to fit the ~1s budget.
> 
> This could work for keeping the num_iters same during comparison between the 
> [master] and [branch], give we logged the num_iters from [master] and used 
> them to drive [branch] MAX_RSS memory. I don't know how to extend this to 
> make memory consumption comparable between different measurement runs (over 
> time...), tough.
> 
> --Pavol
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Measuring MEAN Performance (was: Questions about Swift-CI)

[Michael Gottesman via swift-dev]

> On Jun 12, 2017, at 4:45 PM, Pavol Vaskovic via swift-dev 
>  wrote:
> 
> Hi Andrew,
> 
> On Mon, Jun 12, 2017 at 11:55 PM, Andrew Trick  > wrote:
>> To partially address this issue (I'm guessing) the last SPEEDUP column 
>> sometimes features mysterious question mark in brackets. Its emitted when 
>> the new MIN falls inside the (MIN..MAX) range of the OLD baseline. It is not 
>> checked the other way around.
> 
> That bug must have been introduced during one of the rewrites. Is that in the 
> driver or compare script? Why not fix that bug?
> 
> That is in the compare script. It looks like the else branch got lost during 
> a rewrite 
> 
>  (search for "(?)" in that diff). I could certainly fix that too, but I'm not 
> sure that would be enough to fix all our problems.
>  
> We clearly don’t want to see any false changes. The ‘?’ is a signal to me to 
> avoid reporting those results. They should either be ignored as flaky 
> benchmarks or rerun. I thought rerunning them was the fix you were working on.
> 
> If you have some other proposal for fixing this then please, in a separate 
> proposal, explain your new approach, why your new approach works, and 
> demonstrate it’s effectiveness with results that you’ve gathered over time on 
> the side. Please don’t change how the driver computes performance changes on 
> a whim while introducing other features.
> ... 
> I honestly don’t know what MEAN/SD has to do with the problem you’re pointing 
> to above. The benchmark harness is already setup to compute the average 
> iteration time, and our benchmarks are not currently designed to measure 
> cache effects or any other phenomenon that would have a statistically 
> meaningful sample distribution. Statistical methods might be interesting if 
> you’re analyzing benchmark results over a long period of time or system noise 
> levels across benchmarks.
> 
> The primary purpose of the benchmark suite is identifying performance 
> bugs/regressions at the point they occur. It should be no more complicated 
> than necessary to do that. The current approach is simple: run a 
> microbenchmark long enough in a loop to factor out benchmark startup time, 
> cache/cpu warmup effects, and timer resolution, then compute the average 
> iteration time. Throw away any run that was apparently impacted by system 
> noise.
> 
> We really have two problems:
> 1. spurious results 
> 2. the turnaround time for the entire benchmark suite
> 
> 
> I don't think we can get more consistent test results just from re-running 
> tests that were detected as changes in the first pass, as described in 
> SR-4669 , because that improves 
> accuracy only for one side of the comparison - the branch. When the 
> measurement error is with the baseline from the master, re-running the branch 
> would not help.

When we are benchmarking, we can always have access to the baseline compiler by 
stashing the build directory. So we can always take more samples (in fact when 
I was talking about re-running I always assumed we would).

> 
> I have sketched an algorithm for getting more consistent test results, so far 
> its in Numbers. I have ran the whole test suite for 100 samples and observed 
> the varying distribution of test results. The first result is quite often an 
> outlier, with subsequent results being quicker. Depending on the "weather" on 
> the test machine, you sometimes measure anomalies. So I'm tracking the 
> coefficient of variance from the sample population and purging anomalous 
> results when it exceeds 5%. This results in solid sample population where 
> standard deviation is a meaningful value, that can be use in judging the 
> significance of change between master and branch.
> 
> This week I'm working on transferring this algorithm to Python and putting it 
> probably somewhere around `Benchmark_Driver`. It is possible this would 
> ultimately land in Swift (DriverUtil.swift), but to demonstrate the soundness 
> of this approach to you all, I wanted to do the Python implementation first.
> 
> Depending on how this behaves, my hunch is we could speed up the benchmark 
> suite, by not running test samples for 1 second and taking many samples, but 
> to adaptively sample each benchmark until we get a stable sample population. 
> In worst case this would degrade to current (1s/sample)*num_samples. This 
> could be further improved on by running multiple passes through the test 
> suite, to eliminate anomalies caused by other background processes. That is 
> the core idea from --rerun (SR-4669). 
> 
> --Pavol
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___

Re: [swift-dev] Measuring MEAN Performance (was: Questions about Swift-CI)

[Michael Gottesman via swift-dev]

> On Jun 12, 2017, at 4:54 PM, Pavol Vaskovic  wrote:
> 
> 
> 
> On Mon, Jun 12, 2017 at 11:55 PM, Michael Gottesman  > wrote:
> 
> The current design assumes that in such cases, the workload will be increased 
> so that is not an issue.
> 
> I understand. But clearly some part of our process is failing, because there 
> are multiple benchmarks in 10ms range in the tree for months without fixing 
> this.

I think that is just inertia and being busy. Patch? I'll review = ).

>  
> The reason why we use the min is that statistically we are not interesting in 
> estimated the "mean" or "center" of the distribution. Rather, we are actually 
> interested in the "speed of light" of the computation implying that we are 
> looking for the min.
> 
> I understand that. But all measurements have a certain degree of error 
> associated with them. Our issue is two-fold: we need to differentiate between 
> normal variation between measured samples under "perfect" conditions and 
> samples that are worse because of interference from other background 
> processes.

I disagree. CPUs are inherently messy but disruptions tend to be due to 
temporary spikes most of the time once you have quieted down your system by 
unloading a few processes.

>  
> What do you mean by anomalous results?
> 
> I mean results that significantly stand out from the measured sample 
> population.

What that could mean is that we need to run a couple of extra iterations to 
warm up the cpu/cache/etc before we start gathering samples.

> 
>> Currently I'm working on improved sample filtering algorithm. Stay tuned for 
>> demonstration in Benchmark_Driver (Python), if it pans out, it might be time 
>> to change adaptive sampling in DriverUtil.swift.
> 
> Have you looked at using the Mann-Whitney U algorithm? (I am not sure if we 
> are using it or not)
> 
> I don't know what that is.

Check it out: https://en.wikipedia.org/wiki/Mann–Whitney_U_test 
. It is a 
non-parametric test that two sets of samples are from the same distribution. As 
a bonus, it does not assume that our data is from a normal distribution (a 
problem with using mean/standard deviation which assumes a normal distribution).

We have been using Mann-Whitney internally for a while successfully to reduce 
the noise.

> Here's what I've been doing:
> 
> Depending on the "weather" on the test machine, you sometimes measure 
> anomalies. So I'm tracking the coefficient of variance from the sample 
> population and purging anomalous results (1 sigma from max) when it exceeds 
> 5%. This results in quite solid sample population where standard deviation is 
> a meaningful value, that can be use in judging the significance of change 
> between master and branch.
> 
> --Pavol

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Measuring MEAN Performance (was: Questions about Swift-CI)

[Michael Gottesman via swift-dev]

> On Jun 12, 2017, at 1:45 PM, Pavol Vaskovic via swift-dev 
>  wrote:
> 
> On Tue, May 16, 2017 at 9:10 PM, Dave Abrahams via swift-dev 
> > wrote:
> 
> on Thu May 11 2017, Pavol Vaskovic  wrote:
> 
> I have run Benchmark_O with --num-iters=100 on my machine for the the
> whole performance test suite, to get a feeling for the distribution of
> benchmark samples, because I also want to move the Benchmark_Driver to
> use MEAN instead of MIN in the analysis.
> 
> I'm concerned about that, especially for microbenchmarks; it seems to me
> as though MIN is the right measurement.  Can you explain why MEAN is
> better?
> 
> 
> On Wed, May 17, 2017 at 1:26 AM, Andrew Trick  > wrote:
> Using MEAN wasn’t part of the aforementioned SR-4669. The purpose of that 
> task is to reduce the time CI takes to get useful results (e.g. by using 3 
> runs as a baseline). MEAN isn’t useful if you’re only gathering 3 data points.
> 
> 
> Current approach to detecting performance changes is fragile for tests that 
> have very low absolute runtime, as they are easily over the 5% 
> improvement/regression threshold when the test machine gets a little bit 
> noisy. For example in benchmark on PR #9806 
> :
> 
> BitCount  12  14  +16.7%  0.86x
> SuffixCountableRange  10  11  +10.0%  0.91x
> MapReduce 303 331 +9.2%   0.92x
> These are all false changes (and there are quite a few more there).

The current design assumes that in such cases, the workload will be increased 
so that is not an issue.

The reason why we use the min is that statistically we are not interesting in 
estimated the "mean" or "center" of the distribution. Rather, we are actually 
interested in the "speed of light" of the computation implying that we are 
looking for the min.

> 
> To partially address this issue (I'm guessing) the last SPEEDUP column 
> sometimes features mysterious question mark in brackets. Its emitted when the 
> new MIN falls inside the (MIN..MAX) range of the OLD baseline. It is not 
> checked the other way around.
> 
> I'm suggesting to use MEAN value in combination with SD (standard-deviation) 
> to detect the changes (improvements/regressions). At the moment, this is hard 
> to do, because the aggregate test results reported by Benchmark_O (and co.) 
> can include anomalous results in the sample population that messes up the 
> MEAN and SD, too. Currently it is only visible in the high sample range - the 
> difference between reported MIN and MAX. But it is not clear how many results 
> are anomalous.

What do you mean by anomalous results?

> 
> Currently I'm working on improved sample filtering algorithm. Stay tuned for 
> demonstration in Benchmark_Driver (Python), if it pans out, it might be time 
> to change adaptive sampling in DriverUtil.swift.

Have you looked at using the Mann-Whitney U algorithm? (I am not sure if we are 
using it or not)

> 
> Best regards
> Pavol Vaskovic
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Renaming SILSuccessor -> SILCFGEdge

[Michael Gottesman via swift-dev]

> On Apr 26, 2017, at 1:44 PM, John McCall <rjmcc...@apple.com> wrote:
> 
>> On Apr 26, 2017, at 4:24 PM, Michael Gottesman via swift-dev 
>> <swift-dev@swift.org> wrote:
>> Hey everyone.
>> 
>> I am currently doing some small fixes to SILSuccessor (adding some comments 
>> and fixing some issues exposed by LLVM upstream). As I read the code it 
>> became pretty apparent that the name is a misnomer... SILSuccessor is not 
>> just representing a successor, rather it is representing a whole CFG edge. 
>> This can be seen in how SILSuccessor is used to iterate over the 
>> predecessors of the block.
>> 
>> With that in mind, I would like to rename SILSuccessor to SILCFGEdge. It 
>> will make it much clearer without knowing any context what this data 
>> structure is used for.
>> 
>> Any objections, disagreements, flames, etc?
> 
> It seems a little unnecessary to me.  The successor relationship is an edge, 
> and all the edges of the local CFG are successor relationships.  I guess it 
> looks a little funny that the edges into a block are represented by 
> "successors", but I think when you think about it it makes sense.

IMO this is more of an issue than something "looking funny". Using code named 
"successor" to look up the "predecessors" of a block is misleading and results 
in unnecessary cognitive overhead for the reader who has to "think about it" 
for it to make sense.

> 
> "SILCFGEdge" is also not a very attractive name because of the two 
> abbreviations.  If you had a nice alternative to "CFGEdge" that was less 
> biased to the beginning/end (like Successor/Predecessor are), I probably 
> wouldn't object.

Ok. Maybe SILControlFlowEdge?

> 
> John.

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] Renaming SILSuccessor -> SILCFGEdge

[Michael Gottesman via swift-dev]

Hey everyone.

I am currently doing some small fixes to SILSuccessor (adding some comments and 
fixing some issues exposed by LLVM upstream). As I read the code it became 
pretty apparent that the name is a misnomer... SILSuccessor is not just 
representing a successor, rather it is representing a whole CFG edge. This can 
be seen in how SILSuccessor is used to iterate over the predecessors of the 
block.

With that in mind, I would like to rename SILSuccessor to SILCFGEdge. It will 
make it much clearer without knowing any context what this data structure is 
used for.

Any objections, disagreements, flames, etc?
Michael
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Emitted SIL does not parse

[Michael Gottesman via swift-dev]

> On Apr 6, 2017, at 5:07 PM, Ben Ng  wrote:
> 
> It doesn’t look like it:
> 
> swift (LLVM option parsing): Unknown command line argument 
> '-assume-parsing-unqualified-ownership-sil'.  Try: 'swift (LLVM option 
> parsing) -help'

I misunderstood what you said before (I thought you said that it didn't work as 
a frontend flag). My bad.

> 
> Ben Ng
> https://benng.me 
> 
> On Apr 4, 2017, 11:37 PM -0400, Michael Gottesman , 
> wrote:
>> I think it is an -Xllvm option.
>>> On Apr 4, 2017, at 7:41 PM, Ben Ng > 
>>> wrote:
>>> 
>>> Do you mean like:
>>> 
>>> $ swiftc -O moof.sil -o moof -Xfrontend 
>>> -assume-parsing-unqualified-ownership-sil
>>> 
>>> It seems like the `-assume-parsing-unqualified-ownership-sil` argument 
>>> isn’t understood without the -Xfrontend flag.
>>> 
>>> The above command results in a different error:
>>> 
>>> Cannot construct Inlined loc from the given location.
>>> UNREACHABLE executed at /swift/lib/SIL/SILLocation.cpp:209!
>>> 0  swift0x00010ee694d8 
>>> llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
>>> 1  swift0x00010ee69b86 SignalHandler(int) + 454
>>> 2  libsystem_platform.dylib 0x7fffa3c8eb3a _sigtramp + 26
>>> 3  libsystem_platform.dylib 0x00010001 _sigtramp + 1547113697
>>> 4  libsystem_c.dylib0x7fffa3b13420 abort + 129
>>> 5  swift0x00010ee06bd7 
>>> llvm::llvm_unreachable_internal(char const*, char const*, unsigned int) + 
>>> 471
>>> 6  swift0x00010c670d83 
>>> swift::InlinedLocation::getInlinedLocation(swift::SILLocation) + 387
>>> 7  swift0x00010c3a64af 
>>> swift::SILInliner::inlineFunction(swift::FullApplySite, 
>>> llvm::ArrayRef) + 255
>>> 8  swift0x00010c48f93e (anonymous 
>>> namespace)::SILPerformanceInlinerPass::run() + 1854
>>> 9  swift0x00010c3c9732 
>>> swift::SILPassManager::runPassOnFunction(swift::SILFunctionTransform*, 
>>> swift::SILFunction*) + 2258
>>> 10 swift0x00010c3ca5ba 
>>> swift::SILPassManager::runFunctionPasses(llvm::ArrayRef)
>>>  + 1082
>>> 11 swift0x00010c3cb8d4 
>>> swift::SILPassManager::runOneIteration() + 948
>>> 12 swift0x00010bca963b 
>>> swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan 
>>> const&) + 187
>>> 13 swift0x00010c3d4322 
>>> swift::runSILOptimizationPasses(swift::SILModule&) + 114
>>> 14 swift0x00010bb54fb7 
>>> swift::performFrontend(llvm::ArrayRef, char const*, void*, 
>>> swift::FrontendObserver*) + 12887
>>> 15 swift0x00010bb0f8f0 main + 3312
>>> 16 libdyld.dylib0x7fffa3a7f235 start + 1
>>> 17 libdyld.dylib0x000f start + 1549274587

This is a different issue. Can you post the SIL? I am also not the best person 
to look at that. A better person would be Adrian (+CC).

>>> 
>>> So, I tried again without optimizations, and got this instead:
>>> 
>>> $ swiftc moof.sil -o moof -Xfrontend 
>>> -assume-parsing-unqualified-ownership-sil
>>> 
>>> Undefined symbols for architecture x86_64:
>>>   "__T0s27_allocateUninitializedArraySayxG_BptBwlFyp_Tgq5", referenced from:
>>>   _main in .last_generated-41ab00.o
>>> ld: symbol(s) not found for architecture x86_64

Again, can you post the SIL.

Michael

>>> 
>>> Ben Ng
>>> https://benng.me 
>>> 
>>> On Apr 4, 2017, 2:02 AM -0400, Michael Gottesman >> >, wrote:
 
> On Apr 3, 2017, at 7:55 PM, Ben Ng via swift-dev  > wrote:
> 
> I see that SR-3774  already 
> exists, but that ticket was about declarations not appearing in the 
> emitted SIL.
> 
> I'm able to get declarations to appear with the -frontend flag, but the 
> emitted SIL still doesn't parse on master:
> 
> $  echo 'print("moof")' > test.swift
> $  swiftc -frontend -emit-sil test.swift > test.sil
> $  swiftc -parse-sil test.sil
> 
> Assertion failed: ((Qualifier != StoreOwnershipQualifier::Unqualified) || 
> F.hasUnqualifiedOwnership() && "Unqualified inst in qualified function"), 
> function createStore, file 
> /swift-dev/swift/include/swift/SIL/SILBuilder.h, line 503.
> 0  swift0x00011298d4d8 
> llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
> 1  swift0x00011298db86 SignalHandler(int) + 454
> 2  libsystem_platform.dylib 0x7fffa48ceb3a _sigtramp + 26
> 3  libsystem_platform.dylib 00 _sigtramp + 1534268640
> 4  libsystem_c.dylib0x7fffa4753420 abort + 129

Re: [swift-dev] Using gyb with benchmarks

[Michael Gottesman via swift-dev]

> On Apr 6, 2017, at 1:14 PM, Pavol Vaskovic  wrote:
> 
> On Thursday, 6 April 2017 at 19:44, Michael Gottesman wrote:
>>> On Apr 6, 2017, at 6:16 AM, Pavol Vaskovic via swift-dev 
>>>  wrote:
>>> 
>>> ...
>>> How do I make benchmarks work with gyb?
>> 
> 
> Upon further inspection, one needs to regenerate harness _after_ modifying 
> the gyb anyway, so manually invoking gyb works fine for now, given the 
> [FILE].swift.gyb is ignored by the benchmarking and compilation machinery.
> 
> Is that so bad to add .gyb templates alongside the .swift sources and 
> generate the boilerplate as a first step when generating harness?

I think in the short term this is fine. And even if we switch to use swiftpm, 
we could use this same gyb approach. I do have 1 request though. My concern 
overall with generating the harness how we are doing it today is that sometimes 
people do not know about it and do not regenerate the file when they need to.

Do you think you could add a test to the validation suite that locally 
generates the gyb/harness files and performs a diff? This will ensure that at 
least the bots will catch if someone forgets to run the update.

Specifically, if you look in ./validation-test/Python/, you will see a test 
called bug-reducer.test-sh.

I would create a separate folder called ./validation-test/benchmarks and in 
that folder I would create a file called "generate-harness.test-sh". This will 
just be a shell script along the lines of bug-reducer.test-sh that will 
generate the harness/gyb files in a temp directory and make sure that the diff 
to what is checked into tree is empty. Then at least we will know if someone 
forgets to regenerate the harness or gyb files.

Michael

> 
>> 
>> Please do not do this. We have been talking about switching the benchmarks 
>> to use swiftpm instead of our own custom cmake goop. swiftpm does not 
>> support using custom things like gyb.
>> 
>> Michael
> What’s the motivation here? I’m guessing GYB will not be removed from other 
> parts of project… The benchmark files for sequence operations I’ve been 
> looking at are ripe for templating, reducing possibility to make accidental 
> errors when adding new variations.  
> 
> I’m about to add coverage for a lot more of sequence operations, as their 
> current performance is horrible. These are almost identical, varying only by 
> the concrete type of sequence/collection tested, plus lazy variants. Being 
> able to automate this seems vital to me.
> 
> Best regards
> Pavol Vaskovic
> 
> 
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Using gyb with benchmarks

[Michael Gottesman via swift-dev]

> On Apr 6, 2017, at 6:16 AM, Pavol Vaskovic via swift-dev 
>  wrote:
> 
> Hi!  
> 
> I’ve been trying to use gyb to simplify maintenance of some tests in 
> benchmark/single-source/.
> 
> I have created DropLast.swift.gyb, but it appears this approach is currently 
> not supported by the build system. Doing a clean build I got the following 
> error:
> 
>> + /usr/local/bin/cmake --build 
>> /Users/mondo/Developer/swift-source/build/Ninja-ReleaseAssert/swift-macosx-x86_64
>>  -- -j2 all swift-test-stdlib-macosx-x86_64 swift-benchmark-macosx-x86_64
>> ninja: error: 
>> '/Users/mondo/Developer/swift-source/swift/benchmark/single-source/DropLast.swift',
>>  needed by 'benchmark/Onone-x86_64-apple-macosx10.9/DropLast.o', missing and 
>> no known rule to make it
>> ./swift/utils/build-script: fatal error: command terminated with a non-zero 
>> exit status 1, aborting
> 
> 
> 
> 
> How do I make benchmarks work with gyb?

Please do not do this. We have been talking about switching the benchmarks to 
use swiftpm instead of our own custom cmake goop. swiftpm does not support 
using custom things like gyb.

Michael

> 
> Best regards
> Pavol Vaskovic
> 
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Emitted SIL does not parse

[Michael Gottesman via swift-dev]

I think it is an -Xllvm option.
> On Apr 4, 2017, at 7:41 PM, Ben Ng  wrote:
> 
> Do you mean like:
> 
> $ swiftc -O moof.sil -o moof -Xfrontend 
> -assume-parsing-unqualified-ownership-sil
> 
> It seems like the `-assume-parsing-unqualified-ownership-sil` argument isn’t 
> understood without the -Xfrontend flag.
> 
> The above command results in a different error:
> 
> Cannot construct Inlined loc from the given location.
> UNREACHABLE executed at /swift/lib/SIL/SILLocation.cpp:209!
> 0  swift0x00010ee694d8 
> llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
> 1  swift0x00010ee69b86 SignalHandler(int) + 454
> 2  libsystem_platform.dylib 0x7fffa3c8eb3a _sigtramp + 26
> 3  libsystem_platform.dylib 0x00010001 _sigtramp + 1547113697
> 4  libsystem_c.dylib0x7fffa3b13420 abort + 129
> 5  swift0x00010ee06bd7 
> llvm::llvm_unreachable_internal(char const*, char const*, unsigned int) + 471
> 6  swift0x00010c670d83 
> swift::InlinedLocation::getInlinedLocation(swift::SILLocation) + 387
> 7  swift0x00010c3a64af 
> swift::SILInliner::inlineFunction(swift::FullApplySite, 
> llvm::ArrayRef) + 255
> 8  swift0x00010c48f93e (anonymous 
> namespace)::SILPerformanceInlinerPass::run() + 1854
> 9  swift0x00010c3c9732 
> swift::SILPassManager::runPassOnFunction(swift::SILFunctionTransform*, 
> swift::SILFunction*) + 2258
> 10 swift0x00010c3ca5ba 
> swift::SILPassManager::runFunctionPasses(llvm::ArrayRef)
>  + 1082
> 11 swift0x00010c3cb8d4 
> swift::SILPassManager::runOneIteration() + 948
> 12 swift0x00010bca963b 
> swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan 
> const&) + 187
> 13 swift0x00010c3d4322 
> swift::runSILOptimizationPasses(swift::SILModule&) + 114
> 14 swift0x00010bb54fb7 
> swift::performFrontend(llvm::ArrayRef, char const*, void*, 
> swift::FrontendObserver*) + 12887
> 15 swift0x00010bb0f8f0 main + 3312
> 16 libdyld.dylib0x7fffa3a7f235 start + 1
> 17 libdyld.dylib0x000f start + 1549274587
> 
> So, I tried again without optimizations, and got this instead:
> 
> $ swiftc moof.sil -o moof -Xfrontend -assume-parsing-unqualified-ownership-sil
> 
> Undefined symbols for architecture x86_64:
>   "__T0s27_allocateUninitializedArraySayxG_BptBwlFyp_Tgq5", referenced from:
>   _main in .last_generated-41ab00.o
> ld: symbol(s) not found for architecture x86_64
> 
> Ben Ng
> https://benng.me 
> 
> On Apr 4, 2017, 2:02 AM -0400, Michael Gottesman , 
> wrote:
>> 
>>> On Apr 3, 2017, at 7:55 PM, Ben Ng via swift-dev >> > wrote:
>>> 
>>> I see that SR-3774  already exists, 
>>> but that ticket was about declarations not appearing in the emitted SIL.
>>> 
>>> I'm able to get declarations to appear with the -frontend flag, but the 
>>> emitted SIL still doesn't parse on master:
>>> 
>>> $  echo 'print("moof")' > test.swift
>>> $  swiftc -frontend -emit-sil test.swift > test.sil
>>> $  swiftc -parse-sil test.sil
>>> 
>>> Assertion failed: ((Qualifier != StoreOwnershipQualifier::Unqualified) || 
>>> F.hasUnqualifiedOwnership() && "Unqualified inst in qualified function"), 
>>> function createStore, file /swift-dev/swift/include/swift/SIL/SILBuilder.h, 
>>> line 503.
>>> 0  swift0x00011298d4d8 
>>> llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
>>> 1  swift0x00011298db86 SignalHandler(int) + 454
>>> 2  libsystem_platform.dylib 0x7fffa48ceb3a _sigtramp + 26
>>> 3  libsystem_platform.dylib 00 _sigtramp + 1534268640
>>> 4  libsystem_c.dylib0x7fffa4753420 abort + 129
>>> 5  libsystem_c.dylib0x7fffa471a893 basename_r + 0
>>> 6  swift0x00011017b2fc 
>>> swift::SILBuilder::createStore(swift::SILLocation, swift::SILValue, 
>>> swift::SILValue, swift::StoreOwnershipQualifier) + 396
>>> 7  swift0x00011038ddec (anonymous 
>>> namespace)::SILParser::parseSILInstruction(swift::SILBasicBlock*, 
>>> swift::SILBuilder&) + 70412
>>> 8  swift0x000110373217 
>>> swift::Parser::parseDeclSIL() + 5111
>>> 9  swift0x00011033112d 
>>> swift::Parser::parseTopLevel() + 269
>>> 10 swift0x000110367edb 
>>> swift::parseIntoSourceFile(swift::SourceFile&, unsigned int, bool*, 
>>> swift::SILParserState*, swift::PersistentParserState*, 
>>> swift::DelayedParsingCallbacks*) + 283
>>> 11 swift0x00011011da0c 
>>> swift::CompilerInstance::performSema() + 

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 14.04 (master) #1708

[Michael Gottesman via swift-dev]

The swiftpm test succeeded the next time.

Michael

> On Mar 26, 2017, at 7:50 PM, Michael Gottesman via swift-dev 
> <swift-dev@swift.org> wrote:
> 
> I am taking a look at this.
> 
> Michael
> 
>> On Mar 26, 2017, at 4:00 PM, no-re...@swift.org <mailto:no-re...@swift.org> 
>> wrote:
>> 
>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-14_04 [#1708]
>> 
>> Build URL:   
>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-14_04/1708/ 
>> <https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-14_04/1708/>
>> Project: oss-swift-incremental-RA-linux-ubuntu-14_04
>> Date of build:   Sun, 26 Mar 2017 15:24:17 -0700
>> Build duration:  36 min
>> Identified problems:
>> 
>> Compile Error: This build failed because of a compile error. Below is a list 
>> of all errors in the build log:
>> Indication 1 
>> <https://ci.swift.org//job/oss-swift-incremental-RA-linux-ubuntu-14_04/1708/consoleFull#2041339067ee1a197b-acac-4b17-83cf-a53b95139a76>
>> Changes
>> 
>> Commit 8f2011b4fa99f304eab1d02c229935bcfdcb6c8d by Michael Gottesman:
>> Revert "Revert "[semantic-sil] Fix ownership forwarding in
>> 
>> edit: lib/SILGen/SILGenBuiltin.cpp
>> edit: test/SILGen/builtins.swift
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 14.04 (master) #1708

[Michael Gottesman via swift-dev]

I am taking a look at this.

Michael

> On Mar 26, 2017, at 4:00 PM, no-re...@swift.org wrote:
> 
> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-14_04 [#1708]
> 
> Build URL:
> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-14_04/1708/ 
> 
> Project:  oss-swift-incremental-RA-linux-ubuntu-14_04
> Date of build:Sun, 26 Mar 2017 15:24:17 -0700
> Build duration:   36 min
> Identified problems:
> 
> Compile Error: This build failed because of a compile error. Below is a list 
> of all errors in the build log:
> Indication 1 
> 
> Changes
> 
> Commit 8f2011b4fa99f304eab1d02c229935bcfdcb6c8d by Michael Gottesman:
> Revert "Revert "[semantic-sil] Fix ownership forwarding in
> 
> edit: lib/SILGen/SILGenBuiltin.cpp
> edit: test/SILGen/builtins.swift

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Running benchmark in release mode

[Michael Gottesman via swift-dev]

> On Mar 25, 2017, at 5:03 AM, Pavol Vaskovic via swift-dev 
>  wrote:
> 
> Hi,  
> 
> I’m trying to run the benchmarks for swift-source on my machine.
> 
>> ./swift/utils/build-script --benchmark
> 
> works, but takes ages. So I was trying to run the benchmark in release mode:
> 
>> ./swift/utils/build-script -R -B
> 
> this fails with the last command being:
> 
>> /Users/mondo/Developer/swift-source/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/Benchmark_Driver
>>  run -o O --output-dir 
>> /Users/mondo/Developer/swift-source/build/Ninja-ReleaseAssert/swift-macosx-x86_64/benchmark/logs
>>  --swift-repo /Users/mondo/Developer/swift-source/swift --iterations
>> [..].
>> Benchmark_Driver run: error: argument -i/--iterations: expected one argument
>> 
> 
> 
> I have tried this to no avail:
> 
>> ./swift/utils/build-script -R --benchmark-num-onone-iterations 3
> 
> Is what I’m trying to do not supported by build-script, or am I 
> misunderstanding how to use the benchmark suit?

I would copy what the benchmark bot does. Specifically it also specifies 
specifies benchmark-num-o-iterations.

That being said, I thought those parameters had default values.

Michael

> 
> In case this is a bug in build script, I’ve filed a Jira issue: 
> https://bugs.swift.org/browse/SR-4357
> 
> Best regards
> Pavol Vaskovic
> 
> 
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [Swift CI] Build Failure: 1. OSS - Swift ASAN - OS X (master) #212

[Michael Gottesman via swift-dev]

> On Mar 19, 2017, at 4:59 PM, Greg Parker  wrote:
> 
> Michael, one of your changes broke all three ASAN bots. Please fix or revert 
> ASAP.
> 
> (Could this be related to https://github.com/apple/swift/pull/8169 
>  ? I didn't confirm that the bot's 
> change lists are correct.)

It is more complicated than you would think. I looked at this earlier. The 
change itself looks benign and in fact it was happening without ASAN when I had 
written it in a previous manner. I thought it was a compiler bug so I rewrote 
it in a simpler manner and the problem went away. If you look at 
53eab13be8dd1202428af1b5c315f17c1ab02186, I am just moving some code into a 
method.

It is not related to the other PR which is due to more COW copies occuring. It 
appears that I fixed all of the COW issues with a release stdlib (when client 
is compiled in Debug), but there may be additional COW copies when the stdlib 
is compiled in Debug mode. = /.

Michael

> 
> 
>> On Mar 18, 2017, at 11:48 AM, no-re...@swift.org  
>> wrote:
>> 
>> [FAILURE] oss-swift-incremental-ASAN-RA-osx [#212]
>> 
>> Build URL:   https://ci.swift.org/job/oss-swift-incremental-ASAN-RA-osx/212/ 
>> 
>> Project: oss-swift-incremental-ASAN-RA-osx
>> Date of build:   Sat, 18 Mar 2017 10:26:14 -0700
>> Build duration:  1 hr 22 min
>> Identified problems:
>> 
>> Assertion failure: This build failed because of an assertion failure. Below 
>> is a list of all errors in the build log:
>> Indication 1 
>> 
>> Changes
>> 
>> Commit e52eea2199d8c23ceb82b6bbe85aeb341ccb0fa2 by Michael Gottesman:
>> [silgen] When a formal evaluation scope is in an inout conversion scope,
>> 
>> edit: lib/SILGen/FormalEvaluation.cpp
>> edit: lib/SILGen/FormalEvaluation.h
>> 
>> Commit 53eab13be8dd1202428af1b5c315f17c1ab02186 by Michael Gottesman:
>> [silgen] Move foreignErrorPreparation code in ResultPlanBuilder into a
>> 
>> edit: lib/SILGen/ResultPlan.cpp
>> edit: lib/SILGen/ResultPlan.h
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Profiling ARC

[Michael Gottesman via swift-dev]

> On Feb 20, 2017, at 6:31 PM, Jiho Choi  wrote:
> 
> I used the older versions (binary-trees #6 & binary-trees #7) which I 
> downloaded a couple of weeks ago.  It seems like they updated binary-trees 
> benchmarks since then.
> 
> I just profiled the one you linked and got a similar result.  The optimizer 
> removed about 30% of ARC operations, which is better than almost none in the 
> older versions.  However, compared to other benchmarks, where most of ARC 
> operations in the user code are removed, it is still pretty low.

Sure. I wasn't saying anything about the number of ARC operations in that 
benchmark. I just wanted to be clear which benchmark was being talked about 
that is all.

> 
> On Mon, Feb 20, 2017 at 5:20 PM Michael Gottesman  > wrote:
> Are you talking about this one (there are two)?
> 
> http://benchmarksgame.alioth.debian.org/u64q/program.php?test=binarytrees=swift=1
>  
> 
> 
> Michael
>> On Feb 20, 2017, at 2:24 PM, Jiho Choi via swift-dev > > wrote:
>> 
> 
>> You are right that regex has many ARC operations from libFoundation.  
>> Another outlier in terms of the number of ARC operations is binary-tree.  In 
>> this case, ARC operations are from the user code, and the optimizer couldn't 
>> make much difference.
>> 
>> Other than these two, the optimizer seems working pretty well in removing 
>> ARC operations.
>> 
>> On Fri, Feb 17, 2017 at 1:34 PM David P Grove > > wrote:
>> swift-dev-boun...@swift.org  wrote on 
>> 02/16/2017 09:48:28 PM:
>> > 
>> > I was curious about the overhead of ARC and started profiling some 
>> > benchmarks found in the Computer Language Benchmark Game (http://
>> > benchmarksgame.alioth.debian.org/u64q/measurements.php?lang=swift 
>> > ).
>> >  
>> > So far, it seems that ARC sequence optimization is surprisingly good
>> > and most benchmarks don't have to perform ARC operations as often as
>> > I expected.  I have some questions regarding this finding.
>> > 
>> > I compiled all benchmarks with "-O -wmo" flags and counted the 
>> > number of calls to ARC runtime (e.g., swift_rt_swift_retain) using Pin.
>> > 
>> > 1. Reference counting is considered to have high overhead due to 
>> > frequent counting operations which also have to be atomic.  At least
>> > for the benchmarks I tested, it is not the case and there is almost 
>> > no overhead.  Is it expected behavior?  Or is it because the 
>> > benchmarks are too simple (they are all single-file programs)?  How 
>> > do you estimate the overhead of ARC would be?
>> > 
>> 
>> 
>> hmm,  I wonder if your method of profiling is really finding all the ARC 
>> operations.  The Swift version of regex-dna is about 25x slower than the 
>> Java version (on Linux).  I looked at some prof profiles about a month ago 
>> and at the time roughly 80% of all execution samples were attributed to 
>> swift_retain/swift_release operations coming from CoreFoundation's regex 
>> implementation.  
>> 
>> --dave
>> 
>> (See attached file: regex-dna.svg)
> 
>> ___
> 
>> 
>> swift-dev mailing list
>> swift-dev@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev 
>> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Profiling ARC

[Michael Gottesman via swift-dev]

Are you talking about this one (there are two)?

http://benchmarksgame.alioth.debian.org/u64q/program.php?test=binarytrees=swift=1
 


Michael

> On Feb 20, 2017, at 2:24 PM, Jiho Choi via swift-dev  
> wrote:
> 
> You are right that regex has many ARC operations from libFoundation.  Another 
> outlier in terms of the number of ARC operations is binary-tree.  In this 
> case, ARC operations are from the user code, and the optimizer couldn't make 
> much difference.
> 
> Other than these two, the optimizer seems working pretty well in removing ARC 
> operations.
> 
> On Fri, Feb 17, 2017 at 1:34 PM David P Grove  > wrote:
> swift-dev-boun...@swift.org  wrote on 
> 02/16/2017 09:48:28 PM:
> > 
> > I was curious about the overhead of ARC and started profiling some 
> > benchmarks found in the Computer Language Benchmark Game (http://
> > benchmarksgame.alioth.debian.org/u64q/measurements.php?lang=swift 
> > ).
> >  
> > So far, it seems that ARC sequence optimization is surprisingly good
> > and most benchmarks don't have to perform ARC operations as often as
> > I expected.  I have some questions regarding this finding.
> > 
> > I compiled all benchmarks with "-O -wmo" flags and counted the 
> > number of calls to ARC runtime (e.g., swift_rt_swift_retain) using Pin.
> > 
> > 1. Reference counting is considered to have high overhead due to 
> > frequent counting operations which also have to be atomic.  At least
> > for the benchmarks I tested, it is not the case and there is almost 
> > no overhead.  Is it expected behavior?  Or is it because the 
> > benchmarks are too simple (they are all single-file programs)?  How 
> > do you estimate the overhead of ARC would be?
> > 
> 
> 
> hmm,  I wonder if your method of profiling is really finding all the ARC 
> operations.  The Swift version of regex-dna is about 25x slower than the Java 
> version (on Linux).  I looked at some prof profiles about a month ago and at 
> the time roughly 80% of all execution samples were attributed to 
> swift_retain/swift_release operations coming from CoreFoundation's regex 
> implementation.  
> 
> --dave
> 
> (See attached file: regex-dna.svg)
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Profiling ARC

[Michael Gottesman via swift-dev]

I am familiar with it, just not all of the benchmarks by name.


> On Feb 17, 2017, at 3:04 PM, David P Grove  wrote:
> 
> Sorry, I shouldn't have assumed that everyone know what the Computer Language 
> Benchmark Game was.
> 
> There's a set of 10 toy benchmarks written in 30 different languages that are 
> available here: http://benchmarksgame.alioth.debian.org/ 
>  The webpage also lets you see the 
> results of regular performance runs and see how the various programs stack up 
> against each other.
> 
> As usual with small benchmarks, there are lots of ways in which the programs 
> aren't realistic. 
> 
> After dispatch became available on Linux with Swift 3, I had a side project 
> of going through the Swift implementations and adding concurrency to a few 
> Swift programs that didn't have it already.
> 
> regex-dna is the main outlier for Swift (which is why I had profiled it). 
> There's enough ARC overhead that using dispatch actually made it slower...
> The sequential version for of regex-dna for Swift is: 
> http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexdna=swift=2
>  
> 
> My slower concurrent version is: 
> http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexdna=swift=3
>  
> 
> 
> I suspect that the main fix for improving the performance of this program is 
> actually doing something in CoreFoundation, but I got sidetracked and didn't 
> finish looking into it.
> 
> --dave
> 
> Michael Gottesman ---02/17/2017 05:32:03 PM---> On Feb 17, 2017, 
> at 11:30 AM, David P Grove via swift-dev  wrote: >
> 
> From: Michael Gottesman 
> To: David P Grove/Watson/IBM@IBMUS
> Cc: Jiho Choi , swift-dev 
> Date: 02/17/2017 05:32 PM
> Subject: Re: [swift-dev] Profiling ARC
> Sent by: mgottes...@apple.com
> 
> 
> 
> 
> On Feb 17, 2017, at 11:30 AM, David P Grove via swift-dev 
> > wrote:
> swift-dev-boun...@swift.org  wrote on 
> 02/16/2017 09:48:28 PM:
> > 
> > I was curious about the overhead of ARC and started profiling some 
> > benchmarks found in the Computer Language Benchmark Game (http://
> > benchmarksgame.alioth.debian.org/u64q/measurements.php?lang=swift 
> > ).
> >  
> > So far, it seems that ARC sequence optimization is surprisingly good
> > and most benchmarks don't have to perform ARC operations as often as
> > I expected.  I have some questions regarding this finding.
> > 
> > I compiled all benchmarks with "-O -wmo" flags and counted the 
> > number of calls to ARC runtime (e.g., swift_rt_swift_retain) using Pin.
> > 
> > 1. Reference counting is considered to have high overhead due to 
> > frequent counting operations which also have to be atomic.  At least
> > for the benchmarks I tested, it is not the case and there is almost 
> > no overhead.  Is it expected behavior?  Or is it because the 
> > benchmarks are too simple (they are all single-file programs)?  How 
> > do you estimate the overhead of ARC would be?
> > 
> 
> hmm,  I wonder if your method of profiling is really finding all the ARC 
> operations.  The Swift version of regex-dna is about 25x slower than the Java 
> version (on Linux).  I looked at some prof profiles about a month ago and at 
> the time roughly 80% of all execution samples were attributed to 
> swift_retain/swift_release operations coming from CoreFoundation's regex 
> implementation.  
> 
> Question. Where is this regex-dna benchmark, is it in the swift benchmark 
> suite?
> 
> --dave
> 
> (See attached file: regex-dna.svg)
> ___
> swift-dev mailing list
> swift-dev@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-dev 
> 
> 
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Profiling ARC

[Michael Gottesman via swift-dev]

> On Feb 17, 2017, at 11:30 AM, David P Grove via swift-dev 
>  wrote:
> 
> swift-dev-boun...@swift.org wrote on 02/16/2017 09:48:28 PM:
> > 
> > I was curious about the overhead of ARC and started profiling some 
> > benchmarks found in the Computer Language Benchmark Game (http://
> > benchmarksgame.alioth.debian.org/u64q/measurements.php?lang=swift). 
> > So far, it seems that ARC sequence optimization is surprisingly good
> > and most benchmarks don't have to perform ARC operations as often as
> > I expected.  I have some questions regarding this finding.
> > 
> > I compiled all benchmarks with "-O -wmo" flags and counted the 
> > number of calls to ARC runtime (e.g., swift_rt_swift_retain) using Pin.
> > 
> > 1. Reference counting is considered to have high overhead due to 
> > frequent counting operations which also have to be atomic.  At least
> > for the benchmarks I tested, it is not the case and there is almost 
> > no overhead.  Is it expected behavior?  Or is it because the 
> > benchmarks are too simple (they are all single-file programs)?  How 
> > do you estimate the overhead of ARC would be?
> > 
> 
> hmm,  I wonder if your method of profiling is really finding all the ARC 
> operations.  The Swift version of regex-dna is about 25x slower than the Java 
> version (on Linux).  I looked at some prof profiles about a month ago and at 
> the time roughly 80% of all execution samples were attributed to 
> swift_retain/swift_release operations coming from CoreFoundation's regex 
> implementation.  
> 
Question. Where is this regex-dna benchmark, is it in the swift benchmark suite?
> 
> --dave
> 
> (See attached file: regex-dna.svg)
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] master fails to build

[Michael Gottesman via swift-dev]

> On Feb 13, 2017, at 4:10 AM, Kevin Hendry <ke...@lugnut.io> wrote:
> 
> Heh. I’ll just retreat to my noob hovel and try that then. Thanks.

There is no such a thing as a noob hovel. ; ). We all are learning = ), and who 
knows this may be actually a documentation error (see below).

In case you are curious: the reason I guessed that it was an SDK issue is that 
it seemed that the overlays were expecting the SDK to provide something that 
either was not there or was different. In such cases, it is always a good thing 
to check.

As DaveA always says:
HTH

Michael

> The getting started section says you need the latest Xcode, but I didn’t 
> realize that meant betas as well. I’d assumed it was the latest release. 

Mishal, maybe we should update the README to spell this out explicitly. 
Thoughts?

> 
> On Feb 13, 2017, 1:24 AM -0500, mishal_shah <mishal_s...@apple.com>, wrote:
>> Hi, 
>> 
>> You will need Xcode 8.3 beta 2 (https://developer.apple.com/download/ 
>> <https://developer.apple.com/download/>).
>> 
>> Thanks, 
>> Mishal Shah
>>> On Feb 12, 2017, at 9:17 PM, Kevin Hendry via swift-dev 
>>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>>> 
>>> I'm not. Is this the part where you tell me I should be?
>>> 
>>> 
>>> On Feb 12, 2017, 11:53 PM -0500, Michael Gottesman via swift-dev 
>>> <swift-dev@swift.org <mailto:swift-dev@swift.org>>, wrote:
>>>> Just a stab in the dark, but are you using the latest beta SDK?
>>>> 
>>>> Michael
>>>> 
>>>>> On Feb 11, 2017, at 5:03 PM, Ben Ng via swift-dev <swift-dev@swift.org 
>>>>> <mailto:swift-dev@swift.org>> wrote:
>>>>> 
>>>>> Hi Greg,
>>>>> 
>>>>> Thanks for the tip. It fails in a different way now:
>>>>> 
>>>>> FAILED: stdlib/public/SDK/SafariServices/macosx/x86_64/SafariServices.o
>>>>> 
>>>>> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:18:68:
>>>>>  error: unknown type name 'SFSafariServicesVersion'
>>>>> static inline bool 
>>>>> _swift_SafariServices_isSafariServicesAvailable(SFSafariServicesVersion 
>>>>> version) {
>>>>>^
>>>>> :1:9: note: in file included from :1:
>>>>> #import "SafariServicesOverlayShims.h"
>>>>> ^
>>>>> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:19:18:
>>>>>  error: use of undeclared identifier 'SFSafariServicesVersion10_0'
>>>>>   if (version == SFSafariServicesVersion10_0) {
>>>>>  ^
>>>>> :1:9: note: in file included from :1:
>>>>> #import "SafariServicesOverlayShims.h"
>>>>> ^
>>>>> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:23:3:
>>>>>  error: use of undeclared identifier 'SFSafariServicesVersion'
>>>>>   SFSafariServicesVersion* latestVersion = &_SFSafariServicesVersion;
>>>>>   ^
>>>>> :1:9: note: in file included from :1:
>>>>> #import "SafariServicesOverlayShims.h"
>>>>> ^
>>>>> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:23:28:
>>>>>  error: use of undeclared identifier 'latestVersion'
>>>>>   SFSafariServicesVersion* latestVersion = &_SFSafariServicesVersion;
>>>>>^
>>>>> :1:9: note: in file included from :1:
>>>>> #import "SafariServicesOverlayShims.h"
>>>>> ^
>>>>> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:23:45:
>>>>>  error: use of undeclared identifier '_SFSafariServicesVersion'
>>>>>   SFSafariServicesVersion* latestVersion = &_SFSafariServicesVersion;
>>>>> ^
>>>>> :1:9: note: in file included from :1:
>>>>> #import "SafariServicesOverlayShims.h"
>>>>> ^
>>>>> /Users/ben/Documents/cod

Re: [swift-dev] master fails to build

[Michael Gottesman via swift-dev]

Just a stab in the dark, but are you using the latest beta SDK?

Michael

> On Feb 11, 2017, at 5:03 PM, Ben Ng via swift-dev  wrote:
> 
> Hi Greg,
> 
> Thanks for the tip. It fails in a different way now:
> 
> FAILED: stdlib/public/SDK/SafariServices/macosx/x86_64/SafariServices.o
> 
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:18:68:
>  error: unknown type name 'SFSafariServicesVersion'
> static inline bool 
> _swift_SafariServices_isSafariServicesAvailable(SFSafariServicesVersion 
> version) {
>^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:19:18:
>  error: use of undeclared identifier 'SFSafariServicesVersion10_0'
>   if (version == SFSafariServicesVersion10_0) {
>  ^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:23:3:
>  error: use of undeclared identifier 'SFSafariServicesVersion'
>   SFSafariServicesVersion* latestVersion = &_SFSafariServicesVersion;
>   ^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:23:28:
>  error: use of undeclared identifier 'latestVersion'
>   SFSafariServicesVersion* latestVersion = &_SFSafariServicesVersion;
>^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:23:45:
>  error: use of undeclared identifier '_SFSafariServicesVersion'
>   SFSafariServicesVersion* latestVersion = &_SFSafariServicesVersion;
> ^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:24:15:
>  error: use of undeclared identifier 'latestVersion'; did you mean 
> 'ICGetVersion'?
>   if (NULL == latestVersion) {
>   ^
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/ApplicationServices.framework/Frameworks/HIServices.framework/Headers/InternetConfig.h:711:1:
>  note: 'ICGetVersion' declared here
> ICGetVersion(
> ^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:24:15:
>  warning: comparison of function 'ICGetVersion' equal to a null pointer is 
> always false
>   if (NULL == latestVersion) {
>   ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:24:15:
>  note: prefix with the address-of operator to silence this warning
>   if (NULL == latestVersion) {
>   ^
> :1:9: note: in file included from :1:
> #import "SafariServicesOverlayShims.h"
> ^
> /Users/ben/Documents/code/swift-sources/build/Ninja-ReleaseAssert/swift-macosx-x86_64/./lib/swift/shims/SafariServicesOverlayShims.h:28:11:
>  error: use of undeclared identifier 'latestVersion'; did you mean 
> 'ICGetVersion'?
>   return *latestVersion >= version;
>   ^
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/ApplicationServices.framework/Frameworks/HIServices.framework/Headers/InternetConfig.h:711:1:
>  note: 'ICGetVersion' declared here
> ICGetVersion(
> ^
> /Users/ben/Documents/code/swift-sources/swift/stdlib/public/SDK/SafariServices/SafariServices.swift:14:8:
>  error: could not build Objective-C module '_SwiftSafariServicesOverlayShims'
> import _SwiftSafariServicesOverlayShims
>^
> 
> Ben Ng
> 
> On Feb 11, 2017, 3:59 PM -0500, Greg Parker , wrote:
>> 
>>> On Feb 11, 2017, at 10:46 AM, Ben Ng via swift-dev >> > wrote:
>>> 
>>> I can’t seem to build master right now:
>>> 
>>> git pull
>>> ./utils/update-checkout --clone-with-ssh
>>> ./utils/build-script --release --assertions
>>> 
>>> (…)
>>> 
>>> [17/558] Linking CXX shared library 
>>> lib/swift/macosx/x86_64/libswiftCore.dylib
>>> FAILED: lib/swift/macosx/x86_64/libswiftCore.dylib
>>> Undefined symbols for 

Re: [swift-dev] No return functions and program exit

[Michael Gottesman via swift-dev]

> On Feb 7, 2017, at 11:56 AM, Andrew Trick <atr...@apple.com> wrote:
> 
>> 
>> On Feb 6, 2017, at 12:19 PM, Michael Gottesman via swift-dev 
>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>> 
>> 
>>> On Feb 6, 2017, at 11:44 AM, Jordan Rose <jordan_r...@apple.com 
>>> <mailto:jordan_r...@apple.com>> wrote:
>>> 
>>> 
>>>> On Feb 6, 2017, at 11:25, Joe Groff via swift-dev <swift-dev@swift.org 
>>>> <mailto:swift-dev@swift.org>> wrote:
>>>> 
>>>> 
>>>>> On Feb 6, 2017, at 11:22 AM, Michael Gottesman <mgottes...@apple.com 
>>>>> <mailto:mgottes...@apple.com>> wrote:
>>>>> 
>>>>> Here is my suggestion:
>>>>> 
>>>>> 1. We assume by default the leaking case.
>>>>> 2. We change noreturn functions from C to maybe have a special semantic 
>>>>> tag on them that says that cleanups should occur before them (i.e. 
>>>>> UIApplicationMain).
>>> 
>>> I'm not sure what you mean by this. Functions from C exist in both groups, 
>>> and I don't see why one assumption is better than the other.
>>> 
>>> 
>>>> 
>>>> I feel that "clean up before" is the safer ground case, and if we do any 
>>>> work to whitelist a group, it should be for the common "leakable" 
>>>> noreturns, like exit/_exit/abort/fatalError. That way, we momentarily burn 
>>>> some pointless cycles in the case we get it "wrong" rather than 
>>>> permanently leak memory.
>>> 
>>> I don't like this because of the reverse issue: under -Onone, you may want 
>>> to pop back up the stack in the debugger and see what values you had, and 
>>> they won't be available. It's almost always possible to get things released 
>>> sooner; usually more awkward to get them to stay alive.
>> 
>> On the other hand, this is safe to do in the short term. We can special case 
>> asserts. One thing to consider though is if this should be provided to 
>> users. If not, we can just use semantics. Otherwise, we would need to 
>> discuss how to surface this at the language level.
>> 
>> Michael
> 
> Sorry I didn't jump in yesterday. I'm afraid I don't follow most of the 
> reasoning expressed in the thread. I do completely understand Jordan's points.
> 
> 'noreturn' functions are called from may-return functions. Guaranteeing 
> cleanup would result in inconsistent behavior as a result of optimization.
> 
> The optimizer can always shorten lifetimes when it determines that the caller 
> can't access the object. But I don't see what that has to do with 'noreturn'.
> 
> I agree that we *could* add a special "cleanup before" semantic tag for some 
> C functions, but I'm not aware of a need to do that and there are definite 
> drawbacks.

No worries. (TBH I was just trying to get a decision and was posting straw men 
proposals). Basically after further talk off list, we got in agreement to go 
with the scope splitting approach (i.e. no cleanups).

What I really just wanted was something definitive that I could explicitly 
document. I added a small blurb to ./docs/ARCOptimization.*. I need to add an 
example and make it slightly clearer.

Michael

> 
> -Andy

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] No return functions and program exit

[Michael Gottesman via swift-dev]

> On Feb 6, 2017, at 3:41 PM, Erik Eckstein <eeckst...@apple.com> wrote:
> 
> I’m not sure if I understood.
> What if there is a call to a function and that conditionally calls a noreturn 
> function:
> 
> func foo() {
>   let x = Myclass()
>   bar(true)
>   // release x here?
> }
> 
> func bar(_ dontReturn: Bool) {
>   if (dontReturn) {
> noreturn_func()
>   }
> }
> 
> Is it even possible to “clean up before” in such a case?

The no return function in a certain sense causes the scope for x to never end. 
So no cleanup is needed. The cleanup for x should be after bar always.

> 
> Erik
> 
>> On Feb 6, 2017, at 12:19 PM, Michael Gottesman via swift-dev 
>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>> 
>>> 
>>> On Feb 6, 2017, at 11:44 AM, Jordan Rose <jordan_r...@apple.com 
>>> <mailto:jordan_r...@apple.com>> wrote:
>>> 
>>> 
>>>> On Feb 6, 2017, at 11:25, Joe Groff via swift-dev <swift-dev@swift.org 
>>>> <mailto:swift-dev@swift.org>> wrote:
>>>> 
>>>> 
>>>>> On Feb 6, 2017, at 11:22 AM, Michael Gottesman <mgottes...@apple.com 
>>>>> <mailto:mgottes...@apple.com>> wrote:
>>>>> 
>>>>> Here is my suggestion:
>>>>> 
>>>>> 1. We assume by default the leaking case.
>>>>> 2. We change noreturn functions from C to maybe have a special semantic 
>>>>> tag on them that says that cleanups should occur before them (i.e. 
>>>>> UIApplicationMain).
>>> 
>>> I'm not sure what you mean by this. Functions from C exist in both groups, 
>>> and I don't see why one assumption is better than the other.
>>> 
>>> 
>>>> 
>>>> I feel that "clean up before" is the safer ground case, and if we do any 
>>>> work to whitelist a group, it should be for the common "leakable" 
>>>> noreturns, like exit/_exit/abort/fatalError. That way, we momentarily burn 
>>>> some pointless cycles in the case we get it "wrong" rather than 
>>>> permanently leak memory.
>>> 
>>> I don't like this because of the reverse issue: under -Onone, you may want 
>>> to pop back up the stack in the debugger and see what values you had, and 
>>> they won't be available. It's almost always possible to get things released 
>>> sooner; usually more awkward to get them to stay alive.
>> 
>> On the other hand, this is safe to do in the short term. We can special case 
>> asserts. One thing to consider though is if this should be provided to 
>> users. If not, we can just use semantics. Otherwise, we would need to 
>> discuss how to surface this at the language level.
>> 
>> Michael
>> 
>>> 
>>> Jordan
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org <mailto:swift-dev@swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-dev 
>> <https://lists.swift.org/mailman/listinfo/swift-dev>

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] No return functions and program exit

[Michael Gottesman via swift-dev]

> On Feb 6, 2017, at 11:44 AM, Jordan Rose  wrote:
> 
> 
>> On Feb 6, 2017, at 11:25, Joe Groff via swift-dev > > wrote:
>> 
>> 
>>> On Feb 6, 2017, at 11:22 AM, Michael Gottesman >> > wrote:
>>> 
>>> Here is my suggestion:
>>> 
>>> 1. We assume by default the leaking case.
>>> 2. We change noreturn functions from C to maybe have a special semantic tag 
>>> on them that says that cleanups should occur before them (i.e. 
>>> UIApplicationMain).
> 
> I'm not sure what you mean by this. Functions from C exist in both groups, 
> and I don't see why one assumption is better than the other.
> 
> 
>> 
>> I feel that "clean up before" is the safer ground case, and if we do any 
>> work to whitelist a group, it should be for the common "leakable" noreturns, 
>> like exit/_exit/abort/fatalError. That way, we momentarily burn some 
>> pointless cycles in the case we get it "wrong" rather than permanently leak 
>> memory.
> 
> I don't like this because of the reverse issue: under -Onone, you may want to 
> pop back up the stack in the debugger and see what values you had, and they 
> won't be available. It's almost always possible to get things released 
> sooner; usually more awkward to get them to stay alive.

On the other hand, this is safe to do in the short term. We can special case 
asserts. One thing to consider though is if this should be provided to users. 
If not, we can just use semantics. Otherwise, we would need to discuss how to 
surface this at the language level.

Michael

> 
> Jordan

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] No return functions and program exit

[Michael Gottesman via swift-dev]

> On Feb 6, 2017, at 10:04 AM, Joe Groff <jgr...@apple.com> wrote:
> 
> 
>> On Feb 6, 2017, at 9:48 AM, Michael Gottesman via swift-dev 
>> <swift-dev@swift.org> wrote:
>> 
>> One thing that is an issue that has come up with ownership is that at the 
>> SIL level we do not distinguish in between exceptional noreturn functions 
>> and exceptional return functions.
>> 
>> This is important since in the non-exceptional case, we would like to clean 
>> up all of the values used in the current function before calling the 
>> no-return function. An example of such a function is dispatch_main from 
>> libdispatch. In the exceptional case though, we are ok with leaking since 
>> the program will be exiting. Beyond reducing code size (I guess?), the 
>> argument I have heard for this is that this will allow for people to examine 
>> values in the debugger since we will not have cleaned things up before the 
>> abort is called.
>> 
>> From what I can tell, if we are going to distinguish in between these cases, 
>> then we need a distinction in between the two baked into the compiler. 
>> Thoughts? I have code written that will enable either case to be handled as 
>> long as I can distinguish in between them at the SIL level.
> 
> The interesting distinction here to me strikes me as being the temporal 
> rather than exceptional nature of the exit. _exit(2) isn't an "exceptional" 
> noreturn per se, but you'd still want to leak cleanups before it since the 
> program's termination comes immediately after the call. If it's a profitable 
> distinction to be made, I think there are few enough immediate-exit 
> primitives like exit, abort, fatalError, etc. that we could probably 
> whitelist them with a @_semantics attribute for now, and maybe use an early 
> SIL pass to propagate the attribute in obvious cases where a function is a 
> simple wrapper around one of those functions.

I am fine with either approach as long as we make a decision.

But just to bring it up in the discussion, Alexis brought up in chat an 
interesting point, namely what about higher level functions? I guess we don't 
care about that case?

Michael

> 
> -Joe

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] No return functions and program exit

[Michael Gottesman via swift-dev]

> On Feb 6, 2017, at 10:05 AM, Joe Groff  wrote:
> 
> 
>> On Feb 6, 2017, at 10:02 AM, Jordan Rose via swift-dev  
>> wrote:
>> 
>> I’ve said this Michael in person, but I’ll put it on the thread for others 
>> to see:
>> 
>> I don’t think modeling this is worth the increase in complexity. There’s a 
>> partial workaround for anyone who notices this being a problem, which is to 
>> put any expensive work in a ‘do’ block. And actually emitting cleanups 
>> before a call to, e.g., dispatch_main would be a change in semantics, which 
>> could both break existing programs and makes the language a bit harder to 
>> reason about. The current rule that “defers happen at the close brace, 
>> releases may happen sooner” is simple. (We certainly would not want to make 
>> ‘defer’s happen before calling dispatch_main, though perhaps we should warn 
>> that they will never be executed.)
> 
> And we should keep that rule. The optimizer can't move `defer`s no matter 
> what, since they're semantically fixed to happen at end of scope, so they 
> would always "leak" in a case like `defer { doSomething() }; abort()`. It's 
> only value lifetimes that could be shortened, and shortening lifetimes before 
> a noreturn call wouldn't be a change to the existing model.

As long as we are clear with users that we are making this assumption that 
no-returns and (semantically at the SIL level) unreachables are leaking, I am 
happy. I can deal with that no problem. We should probably document that though.



> 
> -Joe

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] No return functions and program exit

[Michael Gottesman via swift-dev]

One thing that is an issue that has come up with ownership is that at the SIL 
level we do not distinguish in between exceptional noreturn functions and 
exceptional return functions.

This is important since in the non-exceptional case, we would like to clean up 
all of the values used in the current function before calling the no-return 
function. An example of such a function is dispatch_main from libdispatch. In 
the exceptional case though, we are ok with leaking since the program will be 
exiting. Beyond reducing code size (I guess?), the argument I have heard for 
this is that this will allow for people to examine values in the debugger since 
we will not have cleaned things up before the abort is called.

From what I can tell, if we are going to distinguish in between these cases, 
then we need a distinction in between the two baked into the compiler. 
Thoughts? I have code written that will enable either case to be handled as 
long as I can distinguish in between them at the SIL level.

Michael
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Performance issues in automatic reference counting (ARC)?

[Michael Gottesman via swift-dev]

> On Jan 31, 2017, at 12:07 AM, Mikio Takeuchi <mikio.takeu...@gmail.com> wrote:
> 
> Hi Michael,
> 
> > If you are interested in the perf difference with ARC atomics, Roman 
> > recently added a mode to the compiler called -assume-single-threaded that 
> > uses non-atomic reference counts anywhere.
> 
> I think that is not exactly true.   As of now, -assume-single-threaded option 
> can eliminate atomic reference counts only for reference types. 
> 
> I tried -assume-single-threaded for compiling applications as well as swift 
> runtime, and found that atomic reference counts were still used for value 
> types and improvements were limited because of them.
> 
> SIL Instructions on value types (such as CopyValue) are not subtype of 
> RefCountingInst, therefore they don't have a mechanism to represent 
> atomicity. 

This is not an issue since currently copy value is lowered right after SILGen 
to instructions that /do/ have atomicity. My understanding is that the assume 
single threaded option runs a pass late to set these all to non-atomic.

> COW requires reference counts and, because of the lack of information, atomic 
> reference counts are assumed at many places in their codegen and runtime.

IMO again, this is not an issue due to the late pass.

> 
> I made a prototype which returns the right atomicity based on the compiler 
> option in order to eliminate atomic reference counts from generated code. I 
> also modified value witness functions to eliminate atomic reference counts 
> from them.  With these changes, atomic reference counts almost disappeared. 

The value witness functions I think are the larger potential issue.

> 
> If it makes sense, I am happy to contribute my changes to the community.  
> 
> I understand there are two problems with my prototype. 
> 1) We may need to introduce a mechanism to represent atomicity for value 
> types as well.  It will open an opportunity for compiler to use non-atomic 
> reference counts for thread-local values. 

Again, I do not think that is an issue since we lower these away today. This 
may require changes at a later time though once these value operations go 
further back into the compiler.

> 2) We need to either extend value witness table to add non-atomic version of 
> functions, or pass atomicity information to the functions as an extra 
> parameter.

Since your option makes an assumption that the whole program is single 
threaded, why couldn't we just emit the value witness functions such that they 
use non-atomic reference counts?

> 
> Since they are not trivial changes, I would like your endorsement before 
> starting serious work.

Send a PR and lets talk about it. [i.e. what Roman said ; )]

> 
> -- Mikio
> 
> 
> 2016-12-18 5:49 GMT+09:00 Michael Gottesman via swift-dev 
> <swift-dev@swift.org <mailto:swift-dev@swift.org>>:
> 
>> On Dec 17, 2016, at 11:13 AM, Brian Gesiak via swift-dev 
>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>> 
>> Hello all!
>> 
>> I really enjoyed Chris Lattner's slides from his talk at IBM 
>> <http://researcher.watson.ibm.com/researcher/files/us-lmandel/lattner.pdf 
>> <http://researcher.watson.ibm.com/researcher/files/us-lmandel/lattner.pdf>>. 
>> 
>> The speaker notes mention ARC:
>> 
>> "There are two principle downsides to ARC that people cite: one is the need 
>> for atomic increment/decrements, which can be slow." [...] "The performance 
>> problems it can cause are real in some important cases"
>> 
>> Can someone point me to a good resource that explains these problems? I 
>> guess atomic reference count changes create overhead in multithreaded 
>> applications? Are there more detailed explorations into this topic?
> 
> With a proper concurrency model, I believe you can make most reference 
> counting operations local (my opinion). I have done some explorations in this 
> area in the past using what I call thread local vs global reference counts 
> and using marked concurrency boundaries to mediate transitions in between 
> them (moving from thread local -> atomic of course if one escapes in an 
> undefined way).
> 
> If you are interested in the perf difference with ARC atomics, Roman recently 
> added a mode to the compiler called -assume-single-threaded that uses 
> non-atomic reference counts anywhere.
> 
> There are some interesting optimizations in this area as well, specifically 
> even today, COW gives a nice guarantee of thread localness allowing you to 
> eliminate atomic reference counts once you have a uniqued cow data structure.
> 
> Michael
> 
>> 
>> Thanks!
>> 
>> - Brian Gesiak
>&g

Re: [swift-dev] Using git-clang-format in the Swift compiler code base.

[Michael Gottesman via swift-dev]

Can I make a more radical suggestion. Maybe the thing to do here is to enforce 
git-clang-format so that everyone uses it. It would be really simple to do. You 
would require people to put it in a pre commit hook. Then swift-ci would have a 
job that ran git-clang-format and would require it to come through clean.

Michael

> On Jan 26, 2017, at 2:07 AM, Andrew Trick via swift-dev  
> wrote:
> 
> Before I pull in a large PR that "accidentally" reformats a bunch of code, I 
> want to get feedback on how Swift compiler devs plan to use `clang-format`. 
> (BTW, here's the PR https://github.com/apple/swift/pull/6922).
> 
> During the code review, I ran `git clang-format` as part of being a good 
> citizen. There's only one problem with the tool. It rewraps long boolean 
> expressions, hiding those unsightly operators at the end of lines (after all 
> who wants to see those?).
> 
>if (some_expression->with_calls() ||
>another_expression->with(nested_calls()) &&
>an_even_longer_expression->making_your_eyes->glaze_over())
> 
> I don't get involved in style wars, but this is not a style change, it's an 
> objective code quality degradation. It's a demonstrably bug-prone thing to 
> do. It's hurt me too many times in the past, and I'm not happy using a 
> formatting tool that injects future bugs and harms code comprehension.
> 
> When the LLVM coding style was originally ratified, this aspect was left up 
> to individual preference and didn't get any discussion AFAIK. I think
> clang-format then ended up with a `BreakBeforeBinaryOperators: None` style 
> out of sheer inertia. 
> 
> So, how should I use clang-format on Swift compiler code? Vote please.
> 
> ** Option 1: Add a simple configuration option to swift/.clang-format:
> 
> 1a. BreakBeforeBinaryOperators: All
> 
> 1b. BreakBeforeBinaryOperators: NonAssignment
> 
> I have absolutely no preference between 1a and 1b. It's purely style.
> 
> 1a:
> SomeLongTypeName someLongVariableName =
>  someLongExpression();
> 
> 1b:
> SomeLongTypeName someLongVariableName
>  = someLongExpression();
> 
> ** Option 2: Contributors each tweak clang-format according to their (in my 
> case strong) bias:
> 
> My own command line:
> 2a. " --style "{BasedOnStyle: LLVM, BreakBeforeBinaryOperators: All}"
> 2b. " --style "{BasedOnStyle: LLVM, BreakBeforeBinaryOperators: 
> NonAssignment}"
> 
> ** Option 3: Don't bother using clang-format.
> 
> Let emacs do its indentation thing. Embrace idiosyncrasies in the code base.
> 
> -Andy
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Including the requirement, but not the SILFunction in default witness tables

[Michael Gottesman via swift-dev]

Awesome!

Michael

> On Dec 18, 2016, at 12:26 AM, Slava Pestov  wrote:
> 
> That’s a good idea. Default witness tables are still not 100% done and need 
> some work. I’ll be sure to revisit the printing part too.
> 
> Slava
> 
>> On Dec 17, 2016, at 9:52 PM, Michael Gottesman  wrote:
>> 
>> Hey Slava.
>> 
>> I just noticed that when we print out the SIL for default witness tables:
>> 
>> sil_default_witness_table _CVarArgAligned {
>> no_default
>> no_default
>> }
>> 
>> This really does not provide any information. Is it possible to include the 
>> requirement, but not the SILFunction. Then we could do this:
>> 
>> sil_default_witness_table _CVarArgAligned {
>> method #_CVarArgAligned.foo: no_default
>> method #_CVarArgAligned.bar: no_default
>> }
>> 
>> That is at least easy to read and states what the requirement is.
>> 
>> Michael
>> 
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] Including the requirement, but not the SILFunction in default witness tables

[Michael Gottesman via swift-dev]

Hey Slava.

I just noticed that when we print out the SIL for default witness tables:

sil_default_witness_table _CVarArgAligned {
  no_default
  no_default
}

This really does not provide any information. Is it possible to include the 
requirement, but not the SILFunction. Then we could do this:

sil_default_witness_table _CVarArgAligned {
  method #_CVarArgAligned.foo: no_default
  method #_CVarArgAligned.bar: no_default
}

That is at least easy to read and states what the requirement is.

Michael

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Performance issues in automatic reference counting (ARC)?

[Michael Gottesman via swift-dev]

> On Dec 17, 2016, at 11:13 AM, Brian Gesiak via swift-dev 
>  wrote:
> 
> Hello all!
> 
> I really enjoyed Chris Lattner's slides from his talk at IBM 
>  >. 
> 
> The speaker notes mention ARC:
> 
> "There are two principle downsides to ARC that people cite: one is the need 
> for atomic increment/decrements, which can be slow." [...] "The performance 
> problems it can cause are real in some important cases"
> 
> Can someone point me to a good resource that explains these problems? I guess 
> atomic reference count changes create overhead in multithreaded applications? 
> Are there more detailed explorations into this topic?

With a proper concurrency model, I believe you can make most reference counting 
operations local (my opinion). I have done some explorations in this area in 
the past using what I call thread local vs global reference counts and using 
marked concurrency boundaries to mediate transitions in between them (moving 
from thread local -> atomic of course if one escapes in an undefined way).

If you are interested in the perf difference with ARC atomics, Roman recently 
added a mode to the compiler called -assume-single-threaded that uses 
non-atomic reference counts anywhere.

There are some interesting optimizations in this area as well, specifically 
even today, COW gives a nice guarantee of thread localness allowing you to 
eliminate atomic reference counts once you have a uniqued cow data structure.

Michael

> 
> Thanks!
> 
> - Brian Gesiak
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Namespacing Compiler-Stdlib Interfaces

[Michael Gottesman via swift-dev]

One thing to be careful with here is that sometimes SIL test cases say that are 
the stdlib but only implement the functionality that they will use (taking 
advantage of how I believe currently these are checked lazily). If I am correct 
(which my memory may not be), we will need a flag to be less strict perhaps or 
something like that?

Keep in mind, I would only suggest implementing this idea if the patch that 
moves this change making the intrinsics explicit triggers this issue.

Michael

> On Dec 16, 2016, at 10:46 AM, Joe Groff via swift-dev  
> wrote:
> 
> 
>> On Dec 15, 2016, at 3:46 PM, Alexis via swift-dev  
>> wrote:
>> 
>> Triaging old radars, I came across rdar://27268632 from Dmitri. The gist of 
>> it is that he was proposing grouping up all the random global functions that 
>> are implicitly called by the compiler under a “namespace” by making them 
>> static methods of an enum called _Intrinsic. (This name is kind’ve 
>> confusing, because it seems to be the opposite of the traditional notion of 
>> an intrinsic. _RuntimeHook?)
>> 
>> Since this affects how the compiler interfaces with the standard library, I 
>> figured I should check with everyone what they think about doing this. This 
>> is an ABI stability issue, because future compilers will emit these symbols.
>> 
>> I don’t really have a strong opinion on the matter. Avoiding global 
>> pollution and clarifying interfaces seems nice?
> 
> I agree. It'd be general goodness to enumerate the set of things the compiler 
> expects the standard library to provide, and to see if there's any cleaning 
> up or simplification of these interfaces we can do before ABI lockdown.
> 
> -Joe
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Proposal: SILValue SSA Instructions

[Michael Gottesman via swift-dev]

Are there any more concerns here? The thread has seemed to die down and I would 
like to begin upstreaming this code if possible.

Michael

> On Dec 6, 2016, at 2:23 PM, John McCall <rjmcc...@apple.com> wrote:
> 
>> On Dec 6, 2016, at 11:35 AM, Joe Groff <jgr...@apple.com 
>> <mailto:jgr...@apple.com>> wrote:
>>> On Dec 6, 2016, at 11:29 AM, John McCall <rjmcc...@apple.com 
>>> <mailto:rjmcc...@apple.com>> wrote:
>>> 
>>>> On Dec 6, 2016, at 10:17 AM, Joe Groff via swift-dev <swift-dev@swift.org 
>>>> <mailto:swift-dev@swift.org>> wrote:
>>>>> On Dec 5, 2016, at 4:24 PM, Michael Gottesman via swift-dev 
>>>>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>>>>> 
>>>>> Hello everyone!
>>>>> 
>>>>> This is a proposal for 2 instructions needed to express borrowing via SSA 
>>>>> at the SIL level. The need for these were discovered while I was 
>>>>> prototyping a SIL ownership verifier.
>>>>> 
>>>>> A html version of the proposal:
>>>>> 
>>>>> https://gottesmm.github.io/proposals/sil-ownership-value-ssa-operations.html
>>>>>  
>>>>> <https://gottesmm.github.io/proposals/sil-ownership-value-ssa-operations.html>
>>>>> 
>>>>> And inline:
>>>>> 
>>>>> 
>>>>> 
>>>>> # Summary
>>>>> 
>>>>> This document proposes the addition of the following new SIL instructions:
>>>>> 
>>>>> 1. `store_borrow`
>>>>> 2. `begin_borrow`
>>>>> 
>>>>> These enable the expression of the following operations in Semantic SIL:
>>>>> 
>>>>> 1. Passing an `@guaranteed` value to an `@in_guaranteed` argument without
>>>>> performing a copy. (`store_borrow`)
>>>>> 2. Copying a field from an `@owned` aggregate without consuming or 
>>>>> copying the entire
>>>>> aggregate. (`begin_borrow`)
>>>>> 3. Passing an `@owned` value as an `@guaranteed` argument parameter.
>>>>> 
>>>>> # Definitions
>>>>> 
>>>>> ## store_borrow
>>>>> 
>>>>> Define `store_borrow` as:
>>>>> 
>>>>>  store_borrow %x to %y : $*T
>>>>>  ...
>>>>>  end_borrow %y from %x : $*T, $T
>>>>> 
>>>>>=>
>>>>> 
>>>>>  store %x to %y
>>>>> 
>>>>> `store_borrow` is needed to convert `@guaranteed` values to 
>>>>> `@in_guaranteed`
>>>>> arguments. Without a `store_borrow`, this can only be expressed via an
>>>>> inefficient `copy_value` + `store` + `load` + `destroy_value` sequence:
>>>>> 
>>>>>  sil @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>> 
>>>>>  sil @f : $@convention(thin) (@guaranteed Foo) -> () {
>>>>>  bb0(%0 : $Foo):
>>>>>%1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>%2 = alloc_stack $Foo
>>>>>%3 = copy_value %0 : $Foo
>>>>>store %3 to [init] %2 : $Foo
>>>>>apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>%4 = load [take] %2 : $*Foo
>>>>>destroy_value %4 : $Foo
>>>>>dealloc_stack %2 : $Foo
>>>>>...
>>>>>  }
>>>>> 
>>>>> `store_borrow` allows us to express this in a more efficient and 
>>>>> expressive SIL:
>>>>> 
>>>>>  sil @f : $@convention(thin) (@guaranteed Foo) -> () {
>>>>>  bb0(%0 : $Foo):
>>>>>%1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>%2 = alloc_stack $Foo
>>>>>store_borrow %0 to %2 : $*T
>>>>>apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>end_borrow %2 from %0 : $*T, $T
>>>>>dealloc_stack %2 : $Foo
>>>>>...
>>>>>  }
>>>>> 
>>>>> **NOTE** Once `@in_guaranteed` arguments become passed as values, 
>>>>> `store_borrow`
>>>>> will no longer be necessary.
>>>>> 
>>>>> ## begin_borrow
>>>>> 
>>>>> Define a `b

Re: [swift-dev] proposed change for master-next merges

[Michael Gottesman via swift-dev]

> On Dec 8, 2016, at 7:43 AM, Saleem Abdulrasool via swift-dev 
>  wrote:
> 
> Having been involved in the update process for the next branches, I'm really 
> excited to see this type of change.
> 
> I think that the "simple" approach is both better to work and collaborate in 
> as well as closer to the llvm development model which makes it easier to 
> cross pollinate.
> 
> The one thing that I think could be more strongly called out is that normal 
> PR CI shouldn't be gated on master next passing.
> 
> Beyond that, I think that this proposal should ease collaboration and 
> maintenance pains in the current process.

I agree with everything said here. The collaboration issue is the high order 
bit that needs to be fixed. Any other issues that come up can be fixed 
incrementally if necessary based on experience (for instance using the "back 
up" alternative). This is even more true since it is clear that that 
infrastructure would take time to develop and an incremental solution now that 
improves the collaboration will not make it more difficult to develop such a 
solution.

My only strong feeling here is that I think we need documentation of this 
process in ./docs in addition to this proposal.

Michael

> 
> On Wed, Dec 7, 2016 at 7:30 PM Bob Wilson via swift-dev  > wrote:
> I would like to make a change in the way we handle the master-next branch.
> 
> Summary: I’d like to switch to a model where we continuously test against the 
> latest upstream LLVM changes. The goal is to simplify the process and make it 
> easier to collaborate on maintaining master-next.
> 
> Background: We develop Swift against “stable” branches of LLVM (which I am 
> using here to refer to the llvm, clang, and compiler-rt repositories) that 
> are typically rebranched from trunk once for each release, with other commits 
> individually cherry-picked for specific bug fixes and other changes. This 
> insulates Swift development from the churn of changes in LLVM. At the same 
> time, we maintain the “master-next” branches of Swift repos to keep up to 
> date with trunk LLVM. For Swift, our “trunk” comes from the 
> “upstream-with-swift” branches in our GitHub LLVM repos. We have existing 
> automation to continuously merge changes from llvm.org  
> into those upstream-with-swift branches.
> 
> We currently use a manual process to update master-next. Someone on the Swift 
> team is designated as the "merge czar" and is responsible for this. This 
> merge typically happens once every few weeks. Michael Gottesman developed 
> some internal tools to help automate the process, but someone still needs to 
> drive those tools manually. The process involves merging “master” to 
> “master-next” for all the Swift repos and updating the “stable-next” branches 
> of the GitHub LLVM repos for Swift. The “stable-next” branches are basically 
> snapshots of the LLVM upstream-with-swift branches at the point where 
> master-next was most recently merged.
> 
> Swift CI includes a set of Jenkins bots to test master-next building with the 
> stable-next branches of LLVM (https://ci.swift.org/view/swift-master-next 
> ). The merge czar can use these 
> bots to confirm that everything is working after a merge.
> 
> Reasons to change: The current process has the advantage that the merge czar 
> can choose when to do a merge and can schedule that around other work, but it 
> has some significant problems.
> 
> - It is difficult for multiple people to collaborate on updating master-next. 
> The changes involved are often rev-locked between Swift and the LLVM repos, 
> so there is no good way for someone to fix a problem without doing the whole 
> merge process.
> 
> - The current system is hard to understand. I’ve been serving as the merge 
> czar for the last few months, and it took me a while to figure out how to do 
> it well.
> 
> - It requires extra “stable-next” branches in our GitHub LLVM repos, further 
> adding to the complexity.
> 
> - The tools we have to help automate the process are currently internal to 
> Apple and require ongoing maintenance. They could be cleaned up to release 
> publicly but that would take more work.
> 
> Proposal: We already have Jenkins bots testing master-next. I would like to 
> add a job to continuously merge master to master-next and change the existing 
> bots to build against the “upstream-with-swift” branches in our GitHub LLVM 
> repos. The bots would then detect any new problems soon after they are 
> introduced. Anyone could fix those problems, whether they are merge 
> conflicts, build failures, or test issues. A partial fix could be applied 
> directly without needing to resolve all of the outstanding issues.
> 
> This would avoid the need for our current internal merging tools. We already 
> have automatic merging bots, so adding another one would not be difficult.
> 

Re: [swift-dev] Proposal: SILValue SSA Instructions

[Michael Gottesman via swift-dev]

> On Dec 7, 2016, at 2:52 PM, Andrew Trick via swift-dev <swift-dev@swift.org> 
> wrote:
> 
> 
>> On Dec 6, 2016, at 2:23 PM, John McCall via swift-dev <swift-dev@swift.org 
>> <mailto:swift-dev@swift.org>> wrote:
>> 
>>> On Dec 6, 2016, at 11:35 AM, Joe Groff <jgr...@apple.com 
>>> <mailto:jgr...@apple.com>> wrote:
>>>> On Dec 6, 2016, at 11:29 AM, John McCall <rjmcc...@apple.com 
>>>> <mailto:rjmcc...@apple.com>> wrote:
>>>> 
>>>>> On Dec 6, 2016, at 10:17 AM, Joe Groff via swift-dev <swift-dev@swift.org 
>>>>> <mailto:swift-dev@swift.org>> wrote:
>>>>>> On Dec 5, 2016, at 4:24 PM, Michael Gottesman via swift-dev 
>>>>>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>>>>>> 
>>>>>> Hello everyone!
>>>>>> 
>>>>>> This is a proposal for 2 instructions needed to express borrowing via 
>>>>>> SSA at the SIL level. The need for these were discovered while I was 
>>>>>> prototyping a SIL ownership verifier.
>>>>>> 
>>>>>> A html version of the proposal:
>>>>>> 
>>>>>> https://gottesmm.github.io/proposals/sil-ownership-value-ssa-operations.html
>>>>>>  
>>>>>> <https://gottesmm.github.io/proposals/sil-ownership-value-ssa-operations.html>
>>>>>> 
>>>>>> And inline:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> # Summary
>>>>>> 
>>>>>> This document proposes the addition of the following new SIL 
>>>>>> instructions:
>>>>>> 
>>>>>> 1. `store_borrow`
>>>>>> 2. `begin_borrow`
>>>>>> 
>>>>>> These enable the expression of the following operations in Semantic SIL:
>>>>>> 
>>>>>> 1. Passing an `@guaranteed` value to an `@in_guaranteed` argument without
>>>>>> performing a copy. (`store_borrow`)
>>>>>> 2. Copying a field from an `@owned` aggregate without consuming or 
>>>>>> copying the entire
>>>>>> aggregate. (`begin_borrow`)
>>>>>> 3. Passing an `@owned` value as an `@guaranteed` argument parameter.
>>>>>> 
>>>>>> # Definitions
>>>>>> 
>>>>>> ## store_borrow
>>>>>> 
>>>>>> Define `store_borrow` as:
>>>>>> 
>>>>>>  store_borrow %x to %y : $*T
>>>>>>  ...
>>>>>>  end_borrow %y from %x : $*T, $T
>>>>>> 
>>>>>>=>
>>>>>> 
>>>>>>  store %x to %y
>>>>>> 
>>>>>> `store_borrow` is needed to convert `@guaranteed` values to 
>>>>>> `@in_guaranteed`
>>>>>> arguments. Without a `store_borrow`, this can only be expressed via an
>>>>>> inefficient `copy_value` + `store` + `load` + `destroy_value` sequence:
>>>>>> 
>>>>>>  sil @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>> 
>>>>>>  sil @f : $@convention(thin) (@guaranteed Foo) -> () {
>>>>>>  bb0(%0 : $Foo):
>>>>>>%1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>>%2 = alloc_stack $Foo
>>>>>>%3 = copy_value %0 : $Foo
>>>>>>store %3 to [init] %2 : $Foo
>>>>>>apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>>%4 = load [take] %2 : $*Foo
>>>>>>destroy_value %4 : $Foo
>>>>>>dealloc_stack %2 : $Foo
>>>>>>...
>>>>>>  }
>>>>>> 
>>>>>> `store_borrow` allows us to express this in a more efficient and 
>>>>>> expressive SIL:
>>>>>> 
>>>>>>  sil @f : $@convention(thin) (@guaranteed Foo) -> () {
>>>>>>  bb0(%0 : $Foo):
>>>>>>%1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>>%2 = alloc_stack $Foo
>>>>>>store_borrow %0 to %2 : $*T
>>>>>>apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>>>>end_borrow %2 from %0 : $*T, $T
>&

[swift-dev] Proposal: SIL Ownership Model + Verifier

[Michael Gottesman via swift-dev]

This is a proposal for a new SIL Ownership Model and verifier. An online HTML 
version of the document is available here:

https://gottesmm.github.io/proposals/sil-ownership-model.html 


and inline below.

Michael



# Summary

This document defines a SIL ownership model and a compile time static verifier
for the model. This will allow for static compile time verification that a SIL
program satisfies all ownership model constraints.

The proposed SIL ownership model embeds ownership into SIL's SSA def-use
edges. This is accomplished by:

1. Formalizing into SIL API's the distinction in between defs
   (e.g. `SILInstruction` and `SILArgument`) and the values produced by defs
   (e.g. `SILValue`). This is needed to model values as having ownership and
   defs as not having ownership. The main implication here is that the two can
   no longer be implicitly convertible and the API must enforce this.
2. Specifying a set of ownership kinds and specifying a method for mapping a
   `SILValue` to an ownership kind.
3. Specifying ownership constraints on all `SILInstruction`s and `SILArgument`s
   that constrain what ownership kinds their operands and incoming values
   respectively can possess.
4. Implementing a verifier to ensure that all `SILInstruction` and `SILArgument`
   are compatible with the ownership kind propagated by their operand
   `SILValue`s and that pseudo-linear dataflow constraints are maintained.

# Cleanly separating Value and Def APIs in SIL

All values in SIL are defined via an assignment statement of the form: ` = 
`.
In English, we say `foo` is a value that is defined by the def
`bar`. Originally, these two concepts were distinctly represented by the classes
`SILValue` and `ValueBase`. All `ValueBase` defined a list of `SILValue` that
were related, but not equivalent to the defining `ValueBase`. With the decision
to represent multiple return values as instruction projections instead of as a
list of `SILValue`, this distinction in between a def and the values was lost,
resulting in `SILValue` being used interchangeably with `ValueBase` throughout
the swift codebase. This exposes a modeling issue when one attempts to add
ownership to SIL, namely that values have ownership, while the defs that define
the values do not. This implies that defs and values *should not* be
interchangeable.

In order to model that values, not defs, have ownership, we separate the
`SILValue` and `ValueBase` APIs. This is done by:

1. Renaming `ValueBase` to `ValueDef`. This makes it clear from a naming
perspective that a `ValueDef` is not a value, but may define values.
2. Eliminate all operator methods on `SILValue` that allow one to work with the
`ValueDef` API via a `SILValue` in favor of an explicit method for getting the
internal `ValueDef` of a `SILValue`, i.e.:

class SILValue {
  ...
  ValueDef *operator->() const; // deleted
  ValueDef *() const; // deleted
  operator ValueDef *() const; // deleted

  bool operator==(ValueDef *RHS) const; // deleted
  bool operator!=(ValueDef *RHS) const; // deleted
  ...
  ValueDef *SILValue::getDef() const; // new
  ...
};

3. Use private access control and friend classes to only allow for `SILValue`s
   to be constructed and vended by their defining `ValueDef` via a new method on
   `ValueDef`:

class SILValue {
  friend class SILUndef; // new
  friend class SILArgument; // new
  friend class SILInstruction; // new
public:
  SILValue(ValueDef *); // deleted
private:
  SILValue(ValueDef *); // new
};

class ValueDef {
  ...
  SILValue getValue() const; // new
  ...
};

To see how specific common code patterns change in light of these changes,
please see the [appendix](#changes-to-silvalue-api-for-sil-ownership).

# Values and ValueOwnershipKinds

Define `ValueOwnershipKind` as the enum with the following cases:

enum class ValueOwnershipKind {
  Trivial,
  Unowned,
  Owned,
  Guaranteed,
}

Our approach to mapping a `SILValue` to a `ValueOwnershipKind` is to use a
`SILVisitor` called `ValueOwnershipKindVisitor`. This works well since if one
holds `ValueKind` constant, `SILValue` have well defined ownership
constraints. Thus we can handle each case individually via the visitor. We use
SILNodes.def to ensure that all `ValueKind` have a defined visitor method. This
will ensure that when a new `ValueKind` is added, the compiler will emit a
warning that the visitor must be updated, ensuring correctness.

The visitor will be hidden in a *.cpp file and will expose its output via a new
API on `SILValue` :

ValueOwnershipKind SILValue::getOwnershipKind() const;

Since the implementation of `SILValue::getOwnershipKind()` will be out of line,
none of the visitor code will be exposed to the 

Re: [swift-dev] Proposal: SILValue SSA Instructions

[Michael Gottesman via swift-dev]

> On Dec 6, 2016, at 11:29 AM, John McCall <rjmcc...@apple.com> wrote:
> 
>> On Dec 6, 2016, at 10:17 AM, Joe Groff via swift-dev <swift-dev@swift.org> 
>> wrote:
>>> On Dec 5, 2016, at 4:24 PM, Michael Gottesman via swift-dev 
>>> <swift-dev@swift.org> wrote:
>>> 
>>> Hello everyone!
>>> 
>>> This is a proposal for 2 instructions needed to express borrowing via SSA 
>>> at the SIL level. The need for these were discovered while I was 
>>> prototyping a SIL ownership verifier.
>>> 
>>> A html version of the proposal:
>>> 
>>> https://gottesmm.github.io/proposals/sil-ownership-value-ssa-operations.html
>>> 
>>> And inline:
>>> 
>>> 
>>> 
>>> # Summary
>>> 
>>> This document proposes the addition of the following new SIL instructions:
>>> 
>>> 1. `store_borrow`
>>> 2. `begin_borrow`
>>> 
>>> These enable the expression of the following operations in Semantic SIL:
>>> 
>>> 1. Passing an `@guaranteed` value to an `@in_guaranteed` argument without
>>> performing a copy. (`store_borrow`)
>>> 2. Copying a field from an `@owned` aggregate without consuming or copying 
>>> the entire
>>> aggregate. (`begin_borrow`)
>>> 3. Passing an `@owned` value as an `@guaranteed` argument parameter.
>>> 
>>> # Definitions
>>> 
>>> ## store_borrow
>>> 
>>> Define `store_borrow` as:
>>> 
>>>  store_borrow %x to %y : $*T
>>>  ...
>>>  end_borrow %y from %x : $*T, $T
>>> 
>>>=>
>>> 
>>>  store %x to %y
>>> 
>>> `store_borrow` is needed to convert `@guaranteed` values to `@in_guaranteed`
>>> arguments. Without a `store_borrow`, this can only be expressed via an
>>> inefficient `copy_value` + `store` + `load` + `destroy_value` sequence:
>>> 
>>>  sil @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>> 
>>>  sil @f : $@convention(thin) (@guaranteed Foo) -> () {
>>>  bb0(%0 : $Foo):
>>>%1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>%2 = alloc_stack $Foo
>>>%3 = copy_value %0 : $Foo
>>>store %3 to [init] %2 : $Foo
>>>apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>%4 = load [take] %2 : $*Foo
>>>destroy_value %4 : $Foo
>>>dealloc_stack %2 : $Foo
>>>...
>>>  }
>>> 
>>> `store_borrow` allows us to express this in a more efficient and expressive 
>>> SIL:
>>> 
>>>  sil @f : $@convention(thin) (@guaranteed Foo) -> () {
>>>  bb0(%0 : $Foo):
>>>%1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>%2 = alloc_stack $Foo
>>>store_borrow %0 to %2 : $*T
>>>apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
>>>end_borrow %2 from %0 : $*T, $T
>>>dealloc_stack %2 : $Foo
>>>...
>>>  }
>>> 
>>> **NOTE** Once `@in_guaranteed` arguments become passed as values, 
>>> `store_borrow`
>>> will no longer be necessary.
>>> 
>>> ## begin_borrow
>>> 
>>> Define a `begin_borrow` instruction as:
>>> 
>>>  %borrowed_x = begin_borrow %x : $T
>>>  %borrow_x_field = struct_extract %borrowed_x : $T, #T.field
>>>  apply %f(%borrowed_x) : $@convention(thin) (@guaranteed T) -> ()
>>>  end_borrow %borrowed_x from %x : $T, $T
>>> 
>>>=>
>>> 
>>>  %x_field = struct_extract %x : $T, #T.field
>>>  apply %f(%x_field) : $@convention(thin) (@guaranteed T) -> ()
>>> 
>>> A `begin_borrow` instruction explicitly converts an `@owned` value to a
>>> `@guaranteed` value. The result of the `begin_borrow` is paired with an
>>> `end_borrow` instruction that explicitly represents the end scope of the
>>> `begin_borrow`.
>>> 
>>> `begin_borrow` also allows for the explicit borrowing of an `@owned` value 
>>> for
>>> the purpose of passing the value off to an `@guaranteed` parameter.
>>> 
>>> *NOTE* Alternatively, we could make it so that *_extract operations started
>>> borrow scopes, but this would make SIL less explicit from an ownership
>>> perspective since one wouldn't be able to visually identify the first
>>> `struct_extract` in a chain of `struct_extract`. In the case of 
>>> `begin_borrow`,
>>> there is no question and it is completely explicit.
>> 
>> begin_borrow SGTM. Does end_borrow need to be explicit, or could we leave it 
>> implicit and rely on dataflow diagnostics to ensure the borrowed value's 
>> lifetime is dominated by the owner's? It seems to me like, even if 
>> end_borrow is explicit, we'd want a lifetime-shortening pass to shrinkwrap 
>> end_borrows to the precise lifetime of the borrowed value's uses.
> 
> I definitely think it should be explicit, as Michael has it.
> 
> Michael, does store_borrow go away if/when we eliminate the need for 
> different code patterns for direct vs. indirect arguments?

Yes.

> 
> John.

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] Proposal: SILValue SSA Instructions

[Michael Gottesman via swift-dev]

Hello everyone!

This is a proposal for 2 instructions needed to express borrowing via SSA at 
the SIL level. The need for these were discovered while I was prototyping a SIL 
ownership verifier.

A html version of the proposal:

https://gottesmm.github.io/proposals/sil-ownership-value-ssa-operations.html

And inline:



# Summary

This document proposes the addition of the following new SIL instructions:

1. `store_borrow`
2. `begin_borrow`

These enable the expression of the following operations in Semantic SIL:

1. Passing an `@guaranteed` value to an `@in_guaranteed` argument without
   performing a copy. (`store_borrow`)
2. Copying a field from an `@owned` aggregate without consuming or copying the 
entire
   aggregate. (`begin_borrow`)
3. Passing an `@owned` value as an `@guaranteed` argument parameter.

# Definitions

## store_borrow

Define `store_borrow` as:

store_borrow %x to %y : $*T
...
end_borrow %y from %x : $*T, $T

  =>

store %x to %y

`store_borrow` is needed to convert `@guaranteed` values to `@in_guaranteed`
arguments. Without a `store_borrow`, this can only be expressed via an
inefficient `copy_value` + `store` + `load` + `destroy_value` sequence:

sil @g : $@convention(thin) (@in_guaranteed Foo) -> ()

sil @f : $@convention(thin) (@guaranteed Foo) -> () {
bb0(%0 : $Foo):
  %1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
  %2 = alloc_stack $Foo
  %3 = copy_value %0 : $Foo
  store %3 to [init] %2 : $Foo
  apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
  %4 = load [take] %2 : $*Foo
  destroy_value %4 : $Foo
  dealloc_stack %2 : $Foo
  ...
}

`store_borrow` allows us to express this in a more efficient and expressive SIL:

sil @f : $@convention(thin) (@guaranteed Foo) -> () {
bb0(%0 : $Foo):
  %1 = function_ref @g : $@convention(thin) (@in_guaranteed Foo) -> ()
  %2 = alloc_stack $Foo
  store_borrow %0 to %2 : $*T
  apply %1(%2) : $@convention(thin) (@in_guaranteed Foo) -> ()
  end_borrow %2 from %0 : $*T, $T
  dealloc_stack %2 : $Foo
  ...
}

**NOTE** Once `@in_guaranteed` arguments become passed as values, `store_borrow`
will no longer be necessary.

## begin_borrow

Define a `begin_borrow` instruction as:

%borrowed_x = begin_borrow %x : $T
%borrow_x_field = struct_extract %borrowed_x : $T, #T.field
apply %f(%borrowed_x) : $@convention(thin) (@guaranteed T) -> ()
end_borrow %borrowed_x from %x : $T, $T

  =>

%x_field = struct_extract %x : $T, #T.field
apply %f(%x_field) : $@convention(thin) (@guaranteed T) -> ()

A `begin_borrow` instruction explicitly converts an `@owned` value to a
`@guaranteed` value. The result of the `begin_borrow` is paired with an
`end_borrow` instruction that explicitly represents the end scope of the
`begin_borrow`.

`begin_borrow` also allows for the explicit borrowing of an `@owned` value for
the purpose of passing the value off to an `@guaranteed` parameter.

*NOTE* Alternatively, we could make it so that *_extract operations started
borrow scopes, but this would make SIL less explicit from an ownership
perspective since one wouldn't be able to visually identify the first
`struct_extract` in a chain of `struct_extract`. In the case of `begin_borrow`,
there is no question and it is completely explicit.


___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Questions about ARC

[Michael Gottesman via swift-dev]

> On Nov 30, 2016, at 6:25 PM, Jiho Choi via swift-dev  > wrote:
> 
> Thanks for clarifications.  I have a couple of follow-up questions.
> 
> 1. Could you please provide more information (e.g. source code location) 
> about the optimization applying non-atomic reference counting?  What's the 
> scope of the optimization?  Is it method-based?
> 
> 2. Looking at the source code, I assume Swift implements immediate reference 
> counting (i.e. immediate reclamation of dead objects)  without requiring 
> explicit garbage collection phase for techniques, such as deferred reference 
> counting or coalescing multiple updates.  Is it right?  If so, is there any 
> plan to implement such techniques?

We do coalesce multiple updates although late and at the LLVM level and in a 
very limited way. The current work in ARC that is being done is creating the 
ability to represent ARC pairings in SIL. This will allow us to perform ARC 
optimizations in a much simpler manner with a much simpler algorithm. This is 
the Semantic SIL work.

Michael

> 
> On Wed, Nov 30, 2016 at 11:41 AM John McCall  > wrote:
>> On Nov 30, 2016, at 8:33 AM, Jiho Choi via swift-dev > > wrote:
>> Hi,
>> 
>> I am new to Swift, and I have several questions about how ARC works in Swift.
>> 
>> 1. I read from one of the previous discussions in the swift-evolution list 
>> (https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160208/009422.html
>>  
>> )
>>  that ARC operations are currently not atomic as Swift has no memory model 
>> and concurrency model.  Does it mean that the compiler generates non-atomic 
>> instructions for updating reference counts (e.g. using incrementNonAtomic() 
>> instead of increment() in RefCount.h)?
> 
> No.  We have the ability to do non-atomic reference counting as an 
> optimization, but we only trigger it when we can prove that an object hasn't 
> escaped yet.  Therefore, at the user level, retain counts are atomic.
> 
> Swift ARC is non-atomic in the sense that a read/write or write/write race on 
> an individual property/variable/whatever has undefined behavior and can lead 
> to crashes or leaks.  This differs from Objective-C ARC only in that a 
> (synthesized) atomic strong or weak property in Objective-C does promise 
> correctness even in the face of race conditions.  But this guarantee is not 
> worth much in practice because a failure to adequately synchronize accesses 
> to a class's instance variables is likely to have all sorts of other 
> unpleasant effects, and it is quite expensive, so we decided not to make it 
> in Swift.
> 
>> 2. If not, when does it use non-atomic ARC operations? Is there an 
>> optimization pass to recognize local objects?
>> 
>> 3. Without the concurrency model in the language, if not using GCD (e.g. all 
>> Swift benchmark applications), I assume Swift applications are 
>> single-threaded.  Then, I think we can safely use non-atomic ARC operations. 
>>  Am I right?
> 
> When we say that we don't have a concurrency model, we mean that (1) we 
> aren't providing a more complete language solution than the options available 
> to C programmers and (2) like C pre-C11/C++11, we have not yet formalized a 
> memory model for concurrency that provides formal guarantees about what 
> accesses are guaranteed to not conflict if they do race.  (For example, we 
> are unlikely to guarantee that accesses to different properties of a struct 
> can occur in parallel, but we may choose to make that guarantee for different 
> properties of a class.)
> 
>> 4. Lastly, is there a way to measure the overhead of ARC (e.g. a compiler 
>> flag to disable ARC)?
> 
> No, because ARC is generally necessary for correctness.
> 
> John.
> ___
> swift-dev mailing list
> swift-dev@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] High-level SIL Optimization: How do I get a FuncRef from the stdlib?

[Michael Gottesman via swift-dev]

+CC Slava.

He has been messing around with this area in the past bit since many of us have 
looked at this. He is the person you want.

Michael

> On Nov 25, 2016, at 8:42 PM, Ben Ng  wrote:
> 
> Hi everyone,
> 
> I’ve made good progress with the information in this thread but I can’t 
> figure out how to create the proper set of substitutions for the method that 
> I’m calling.
> 
> The error I’m getting, as expected, is "SIL verification failed: callee of 
> apply without substitutions must not be polymorphic: !fnTy->isPolymorphic()"
> 
> I was hoping that there would be a way to delay specialization of the 
> function that I’m replacing so that I can simply reuse those substitutions, 
> but it doesn’t seem like that’s possible.
> 
> At a high level I assumed that I’d simply be able to substitute a type like 
> `Int` for `τ_0_0`, but it looks like I have to use a ProtocolConformanceRef, 
> which I don’t understand.
> 
> I looked into iterating through 
> `getLoweredFunctionType()->getGenericSignature()->getGenericParams()`, but I 
> don’t see how I can turn the information there into ProtocolConformanceRef.
> 
> Thanks for the help as always,
> 
> Ben
> 
>> On Nov 16, 2016, at 10:47 PM, Ben Ng  wrote:
>> 
>>> On Nov 16, 2016, at 7:11 PM, Arnold Schwaighofer  
>>> wrote:
>>> 
>>> 
 On Nov 16, 2016, at 2:58 PM, Ben Ng  wrote:
 
 Correct, that is what I am trying to do.
 
> On Nov 16, 2016, at 12:22 PM, Arnold Schwaighofer 
>  wrote:
> 
> // Really, by the time you look at this in array value prop
> // this call should have been inline and you would see a call
> // to:
> // a.append(contentsOf: [1])
 
 I do not understand this comment; I thought that inlining of stdlib 
 functions happened after high-level SIL optimizations are run. Is my 
 understanding incorrect?
>>> 
>>> 
>>> Inlining of functions with @_semantics is delayed until after high-level 
>>> SIL optimizations are run. Other functions are inlined.
>>> 
>>> https://github.com/apple/swift/blob/master/lib/SILOptimizer/PassManager/Passes.cpp#L221
>>> https://github.com/apple/swift/blob/master/lib/SILOptimizer/Transforms/PerformanceInliner.cpp#L722
>>> 
>>> 
>>> I recommend looking at the SIL function dump in 
>>> ArrayElementValuePropagation of an example function after adding 
>>> @semantics(“array.mutate_unknown”) to “append(contentsOf:)”.
>>> 
>>> 
>>> $ git diff HEAD~
>>> diff --git a/lib/SILOptimizer/Transforms/ArrayElementValuePropagation.cpp 
>>> b/lib/SILOptimizer/Transforms/ArrayElementValuePropagation.cpp
>>> index 76328a6..cb976f7 100644
>>> --- a/lib/SILOptimizer/Transforms/ArrayElementValuePropagation.cpp
>>> +++ b/lib/SILOptimizer/Transforms/ArrayElementValuePropagation.cpp
>>> @@ -259,6 +259,8 @@ public:
>>> void run() override {
>>>   auto  = *getFunction();
>>> 
>>> +Fn.dump();
>>> +
>>>   bool Changed = false;
>>> 
>>>   // Propagate the elements an of array value to its users.
>>> diff --git a/stdlib/public/core/Arrays.swift.gyb 
>>> b/stdlib/public/core/Arrays.swift.gyb
>>> index f00cc23..2acfd06 100644
>>> --- a/stdlib/public/core/Arrays.swift.gyb
>>> +++ b/stdlib/public/core/Arrays.swift.gyb
>>> @@ -1344,6 +1344,7 @@ extension ${Self} : RangeReplaceableCollection, 
>>> _ArrayProtocol {
>>> /// - Parameter newElements: The elements to append to the array.
>>> ///
>>> /// - Complexity: O(*n*), where *n* is the length of the resulting array.
>>> +  @_semantics("array.mutate_unknown")
>>> public mutating func append(contentsOf newElements: C)
>>>   where C.Iterator.Element == Element {
>>> 
>>> 
>>> # Rebuild the compiler and stdlib (without stdlib assertions).
>>> $ swift/utils/build-script -r  --assertions --no-swift-stdlib-assertions
>>> 
>>> 
>>> $ cat TestArray.swift 
>>> @inline(never)
>>> public func test() {
>>> var a = [1, 2, 3]
>>> a += [1]
>>> print(a)
>>> }
>>> 
>>> $ bin/swiftc -O 2>&1 | less
>>> ...
>>> sil shared [_semantics "array.mutate_unknown"] 
>>> @_TTSg5Si_GSaSi_GSaSi_s10Collections___TFSa6appenduRd__s10CollectionxzWd__8Iterator7Element_rfT10contentsOfqd___T_
>>>  : $@convention(method) (@owned Array>> nt>, @inout Array) -> () {
>>> 
>>> …
>>> // testArray() -> ()
>>> sil [noinline] @_TF9TestArray9testArrayFT_T_ : $@convention(thin) () -> () {
>>> bb0:
>>> %0 = alloc_stack $Array, var, name "a", loc "TestArray.swift":3:7, 
>>> scope 2 // users: %54, %32, %60, %23, %43
>>> %1 = integer_literal $Builtin.Word, 3, loc "TestArray.swift":3:12, scope 2 
>>> // user: %4
>>> %2 = integer_literal $Builtin.Int64, 3, scope 5 // user: %3
>>> %3 = struct $Int (%2 : $Builtin.Int64), scope 5 // users: %22, %7
>>> %4 = alloc_ref [tail_elems $Int * %1 : $Builtin.Word] 
>>> $_ContiguousArrayStorage, scope 5 // user: %7
>>> %5 = metatype $@thin Array.Type, scope 5   // users: %25, %7
>>> // function_ref specialized static 
>>> Array._adoptStorage(_ContiguousArrayStorage, 

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 16.10 (master) #519

[Michael Gottesman via swift-dev]

SGTM!

Thanks!

> On Nov 16, 2016, at 12:02 PM, ankit_aggar...@apple.com wrote:
> 
> I figured this was a 16.10 issue because PR bot (16.04) passed, however I 
> wasn't able to reproduce this locally after installing 16.10, I pushed a 
> commit to pin point the failure line but that integration went fine. I will 
> keep an eye if this comes again. 
> 
> Sent from my iPhone
> 
> On 17-Nov-2016, at 12:57 AM, Michael Gottesman  > wrote:
> 
>> Question. How did this get through PR testing?
>> 
>> Sent from my iPhone
>> 
>> On Nov 16, 2016, at 6:01 AM, Ankit Aggarwal via swift-dev 
>> > wrote:
>> 
>>> This is mine, checking.
>>> 
 On 16-Nov-2016, at 6:12 PM, no-re...@swift.org  
 wrote:
 
 [FAILURE] oss-swift-incremental-RA-linux-ubuntu-16_10 [#519]
 
 Build URL: 
 https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_10/519/ 
 
 Project:   oss-swift-incremental-RA-linux-ubuntu-16_10
 Date of build: Wed, 16 Nov 2016 04:32:08 -0800
 Build duration:10 min
 Identified problems:
 
 Compile Error: This build failed because of a compile error. Below is a 
 list of all errors in the build log:
 Indication 1 
 
 Tests:
 
 Name: Swift(linux-x86_64)
 Failed: 0 test(s), Passed: 8564 test(s), Total: 8564 test(s)
 Name: Swift-Unit
 Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
 
 Changes
 
 Commit 17a732b4d351d74f7bbb392d8cf6d1097b8bd10e by practicalswift:
 [swiftc (134 vs. 5198)] Add crasher in ?
 
 add: 
 validation-test/compiler_crashers/28523-child-source-range-not-contained-within-its-parent-sequence-expr-type-null.swift
 
 Commit 8dac299dc3109a70ae1ce12c8cd9c201c562f9e6 by ankit_aggarwal:
 [PackageTool] Add pinning CLI
 
 edit: Tests/CommandsTests/PackageToolTests.swift
 edit: Sources/Commands/Workspace.swift
 edit: Sources/Commands/SwiftPackageTool.swift
 edit: Sources/Commands/Error.swift
>>> 
>>> ___
>>> swift-dev mailing list
>>> swift-dev@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-dev 
>>> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 16.10 (master) #519

[Michael Gottesman via swift-dev]

Question. How did this get through PR testing?

Sent from my iPhone

> On Nov 16, 2016, at 6:01 AM, Ankit Aggarwal via swift-dev 
>  wrote:
> 
> This is mine, checking.
> 
>> On 16-Nov-2016, at 6:12 PM, no-re...@swift.org wrote:
>> 
>> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-16_10 [#519]
>> 
>> Build URL:   
>> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_10/519/
>> Project: oss-swift-incremental-RA-linux-ubuntu-16_10
>> Date of build:   Wed, 16 Nov 2016 04:32:08 -0800
>> Build duration:  10 min
>> Identified problems:
>> 
>> Compile Error: This build failed because of a compile error. Below is a list 
>> of all errors in the build log:
>> Indication 1
>> Tests:
>> 
>> Name: Swift(linux-x86_64)
>> Failed: 0 test(s), Passed: 8564 test(s), Total: 8564 test(s)
>> Name: Swift-Unit
>> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
>> 
>> Changes
>> 
>> Commit 17a732b4d351d74f7bbb392d8cf6d1097b8bd10e by practicalswift:
>> [swiftc (134 vs. 5198)] Add crasher in ?
>> 
>> add: 
>> validation-test/compiler_crashers/28523-child-source-range-not-contained-within-its-parent-sequence-expr-type-null.swift
>> 
>> Commit 8dac299dc3109a70ae1ce12c8cd9c201c562f9e6 by ankit_aggarwal:
>> [PackageTool] Add pinning CLI
>> 
>> edit: Tests/CommandsTests/PackageToolTests.swift
>> edit: Sources/Commands/Workspace.swift
>> edit: Sources/Commands/SwiftPackageTool.swift
>> edit: Sources/Commands/Error.swift
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] High-level SIL Optimization: How do I get a FuncRef from the stdlib?

[Michael Gottesman via swift-dev]

> On Nov 15, 2016, at 9:02 PM, Ben Ng  wrote:
> 
> Hi Michael,
> 
> No, I did not. I don't see swift-dev in the cc line of your email... this 
> might be an issue with the Inbox for iOS client that I'm using.

Ok! This happens sometimes (hit reply instead of reply-all or the like). +CC 
swift-dev.

> On Wed, Nov 16, 2016 at 12:01 AM Michael Gottesman  > wrote:
>> On Nov 15, 2016, at 8:54 PM, Ben Ng > 
>> wrote:
>> 
>> Hi Michael,
>> 
>> It can probably be stuck in the Array Value Propagation pass since its 
>> pretty similar. Yes, it uses Array semantics. I isolated it in a separate 
>> file because it's my first time working on the Swift compiler and I wanted 
>> to keep the moving parts to a minimum.
> 
> Ok. I was just saying this transformation is small enough it probably makes 
> sense not to have a pass just for it. That is all.
> 
>> 
>> Am I confusing the nouns? i.e Is a pass the same thing as a transform?
> 
> No you are not confusing anything. A pass is a transform. 
> 
> Also, did you remove swift-dev for a reason? Can I re-add?
> 
> Michael
> 
>> 
>> On Tue, Nov 15, 2016 at 11:49 PM Michael Gottesman > > wrote:
>> Question, is there any reason why you want to create a new pass? We already 
>> have array semantic ops. Maybe stick this there?
>> 
>> +CC Arnold who is more familiar with this area.
>> 
>> Sent from my iPhone
>> 
>> > On Nov 15, 2016, at 5:17 PM, Ben Ng via swift-dev > > > wrote:
>> >
>> > Good evening, folks.
>> >
>> > I’ve been working on a high-level SIL optimization pass that replaces an 
>> > application of the `+=` function with an application of `Array.concat(A: 
>> > Element)`. I’m blocked because I can’t figure out how to get a FuncRef to 
>> > `Array.append` from the SILTransform.
>> >
>> > I’ve tried getting the stdlib module from the ASTContext, and looking 
>> > through the visible decls, but the function that I want isn’t there. The 
>> > next thing I was going to try was recursively looking through the imports, 
>> > but I thought I’d stop to do a sanity check with the mailing list.
>> >
>> > It seems like even if I did succeed in getting a FuncRef to the generic 
>> > `Array.append`, I’d still need to figure out how to get the specialized 
>> > version, and add the appropriate declaration to the SIL. It also feels 
>> > wrong that I’m depending on stuff in the AST from a SILTransform.
>> >
>> > Thanks,
>> >
>> > Ben
>> > ___
>> > swift-dev mailing list
>> > swift-dev@swift.org 
>> > https://lists.swift.org/mailman/listinfo/swift-dev 
>> > 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Casting shadow protocols

[Michael Gottesman via swift-dev]

> On Nov 7, 2016, at 11:23 AM, Alexis via swift-dev  wrote:
> 
> 
>> On Nov 4, 2016, at 11:55 PM, Dave Abrahams via swift-dev 
>> > wrote:
>> 
>> 
>> on Fri Nov 04 2016, Slava Pestov > > wrote:
>> 
>>> If the casts are always in one direction, can you make one protocol
>>> refine another?
>> 
>> Yeah, I am shocked if they don't do that already.
> 
> They do; _NSFoo: _NSFooCore
> 
> But the problem is that we have _NSFooCores and want _NSFoos.

I am confused. Then why can't you just do an unconditional checked down cast to 
_NSFoo from _NSFooCore? That can definitely be expressed at the SIL level?

> 
>> 
>>> Also note that @objc protocols are self-conforming as long as they
>>> don’t contain initializers or static methods, but I’m not sure if that
>>> helps.
>> 
>> Doesn't; we're not using these in a generic context; they're just
>> existentials.
>> 
>>> 
>>> 
 On Nov 4, 2016, at 4:29 PM, Alexis via swift-dev > wrote:
 
 The swift standard library has this nasty little pattern/problem in it:
 
 The types in the core library want to know about several types
 defined in foundation: NSString, NSArray, NSDictionary, etc. But
 core is imported by Foundation, so it can’t (circular references
 between modules). Thankfully, everything in ObjC is pretty opaquely
 defined and uniform, and Swift knows how to hook into that uniform
 layout. So the core library defines Shadow Protocols which provide
 whatever subset of that type’s API is considered interesting. These
 protocols are then used in place of the ObjC types. There’s also
 some magic compiler hooks so core lib types can subclass those
 foundation types.
 
 However there’s sometimes two Shadow Protocols: one that defines the
 APIs the stdlib should provide (_NSFooCore), and one that extends
 that with extra APIs the stdlib wants to consume (_NSFoo). This
 leads to an awkward situation: as far as the runtime is concerned,
 the stdlib’s _NSFooCores don’t conform to _NSFoo! We know they do
 because it’s all just a big lie to hook into ObjC message passing
 with a bit of type safety, but the runtime doesn’t. So if you try to
 do a safe type cast, it will fail. This leads to a situation where
 we sprinkle code with unsafeBitCast’s to _NSFoo which is a massive
 refactoring hazard.
 
 For instance, there was a struct-containing-a-class that was being
 cast to _NSFoo in HashedCollections. This happened to work (but was
 probably still a violation of strict aliasing?) because the struct’s
 only field was the class. However the struct was later changed to a
 class, which silently made the cast completely incorrect, banishing
 the real _NSFoo to the shadow (protocol) realm.
 
 Can we do anything better here? Note that there’s a few places where
 we also cast an AnyObject into an _NSFoo, but there’s some chance
 this is all legacy junk that can be updated to at least use
 _NSFooCore, if not _NSFoo itself.
 ___
 swift-dev mailing list
 swift-dev@swift.org 
 https://lists.swift.org/mailman/listinfo/swift-dev
>>> 
>>> ___
>>> swift-dev mailing list
>>> swift-dev@swift.org 
>>> https://lists.swift.org/mailman/listinfo/swift-dev 
>>> 
>>> 
>> 
>> -- 
>> -Dave
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev 
>> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Performance degradation with 3.0.1(String comparison)

[Michael Gottesman via swift-dev]

> On Nov 3, 2016, at 11:07 PM, Pushkar N Kulkarni via swift-dev 
>  wrote:
> 
> Hello,
> 
> A colleague of mine who's doing performance testing with some server-side 
> Swift frameworks like Kitura, Perfect and Zewo found a 15-20% degradation 
> with a simple "Hello World" benchmark, with the 3.0.1 previews on Linux. 
> Analysis revealed it is related to the disabling of an optimisation around 
> String comparison for Swift runtimes - this commit - apple/swift@ef974af 
> 
> 
> Do we have a plan to reimplement this optimisation for Swift runtimes in the 
> future? Any more information about this would be highly appreciated. 

Can you please provide a benchmark in a pull request that shows the performance 
degradation?

Michael

> 
> Thank you!
> 
> Pushkar N Kulkarni,
> IBM Runtimes
> 
> Simplicity is prerequisite for reliability - Edsger W. Dijkstra
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Stack dump of the compiler?

[Michael Gottesman via swift-dev]

Are you talking about the prettyprint stack dump?

I think that may have gone away after the recent update to the newer 
llvm/clang. I think there is a bug to look into what happened here, but no one 
has had the time to look into it.

Michael

> On Oct 26, 2016, at 8:53 PM, rintaro ishizaki via swift-dev 
>  wrote:
> 
> > set_target_properties(${swift_binaries} properties ENABLE_EXPORTS 1)  ?
> > I don't know if it's this simple though.
> 
> Yeah, that should work as well.
> But, I'm not sure which tools should be `ENABLE_EXPORT`ed.
> 
> 
> 2016-10-27 3:04 GMT+09:00 Flamedoge  >:
> > Do not add flags to export symbols from executables without the 
> > ENABLE_EXPORTS 
> > 
> >  target property.
> 
> set_target_properties(${swift_binaries} properties ENABLE_EXPORTS 1)  ?
> I don't know if it's this simple though.
> 
> On Tue, Oct 25, 2016 at 8:51 PM, rintaro ishizaki via swift-dev 
> > wrote:
> Hi all,
> I've noticed recent build of swift compiler doesn't show the stack dump on 
> crash.
> 
> Here is the result with October 25, 2016 snapshot.
> https://gist.github.com/rintaro/cb694898821fbbe8b02734862eb69534 
> 
> 
> On macOS, the dump completely doesn't show up.
> On Linux, the dump shows up, but without symbols.
> 
> As for Linux, I figured out that it's because of recent change in 
> CMakeList.txt
> cmake_minimum_required(VERSION 3.4.3)
> Specifically, the policy change of CMP0065
> https://cmake.org/cmake/help/v3.4/policy/CMP0065.html 
> 
> backtrace_symbols_fd() needs symbols exported.
> 
> As for macOS, I'm not sure why.
> 
> I don't know what is the right way to fix this.
> cmake_policy(SET CMP0065 OLD)
> would fix the Linux build but ...
> 
> Any thought?
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-dev 
> 
> 
> 
> 
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Purpose of validation-test/Reflection/reflect_*.swift

[Michael Gottesman via swift-dev]

> On Oct 26, 2016, at 5:29 PM, Alexis via swift-dev  wrote:
> 
> I’m not clear on what the reflection tests are attempting to actually verify. 
> Just that we don’t change the internal layout of these types accidentally? 
> We’re churning up the layouts of a lot of the collections to get things all 
> set up for ABI stability, which means mechanically updating these tests to 
> expect “whatever output we now happen to output”.
> 
> In Dave’s initial eager bridging stuff he left a comment indicating that 
> these are incorrectly relying on implementation details. Without any context, 
> I’m inclined to agree. The fact that somewhere deep in the guts of String 
> there lives an enum doesn’t seem important to verify. (how big it is, and how 
> many extra inhabitants it has, does seem worth verifying longterm).

I think this was originally done by Dave Farler.

Michael

> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] PRs not triggering with @swift-ci

[Michael Gottesman via swift-dev]

I imagine it is dns caching. (Went to their status page and they specifically 
mention that issue).

So sounds like you are correct.

Michael

> On Oct 21, 2016, at 10:24 AM, Ron Olson <tachokni...@gmail.com> wrote:
> 
> Weird that it's working for you; I can't get to any github.com page on any 
> machine (and I was using it just this morning).
> 
> On 21 Oct 2016, at 12:22, Michael Gottesman wrote:
> 
>> 
>>> On Oct 21, 2016, at 10:14 AM, Ron Olson via swift-dev <swift-dev@swift.org> 
>>> wrote:
>>> 
>>> Looks like GitHub is down due to that DDOS on the Dyn DNS servers that is 
>>> in the news at the moment.
>> 
>> I am not having that issue. But maybe it is b/c I have github in my dns 
>> cache and the bots do not (even though I find that to be unlikely). Also 
>> when I look online, I see that the news reports are saying that the issue 
>> has been resolved.
>> 
>> Mishal/Doug, do you have any insight into this?
>> 
>>> 
>>> On 21 Oct 2016, at 12:12, Michael Gottesman via swift-dev wrote:
>>> 
>>>> I have been trying to trigger pull request testing for a little bit and 
>>>> nothing seems to be happening. Is anyone else running into this? This is 
>>>> the PR:
>>>> 
>>>> https://github.com/apple/swift/pull/5396
>>>> ___
>>>> swift-dev mailing list
>>>> swift-dev@swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-dev
>>> ___
>>> swift-dev mailing list
>>> swift-dev@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-dev
>> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev