Re: [swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

Sure thing! I’ll see if I can put small reproducers together tomorrow.

Ben

On Thu, Dec 1, 2016 at 7:08 PM, Mark Lacey  wrote:

>
> On Dec 1, 2016, at 5:48 PM, Ben Asher via swift-dev 
> wrote:
>
> The build failed compiling 3 files in our project. Our app is crashing
> that snapshot; here is some output from the failures:
>
>
> It would be awesome if you could come up with small reproducers for these
> and open bugs for them. Alternately, opening a radar with the full project
> attached will give us an opportunity to get these fixed ASAP!
>
> Mark
>
>
> - Assertion failed: (newType == type || (isa(newType) &&
> cast(newType)->getNumElements() == 1 &&
> cast(newType).getElementType(0) == type)), function
> rewriteType, file /Users/buildn
> ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h,
> line 207.
>
> - SIL verification failed: method's Self parameter should be constrained
> by protocol: selfRequirement.getKind() == RequirementKind::Conformance &&
> selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
> equirement.getSecondType()->getAs() ->getDecl() == protocol
>
> We're not getting all the way to linking, but compiling everything for
> that snapshot took ~11min45s. Based on that, it appears that about a minute
> is saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1
> (App Store).
>
> With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If
> Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build,
> the extra time here (compared to the snapshot) could be explained by
> linking, signing, and a few other custom builds scripts that run after
> compiling.
>
> Thanks everyone for your help!
>
> Ben
>
>
> On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher  wrote:
>
>> I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going,
>> so I don't have any time results yet. But, I did notice something new (or
>> maybe existed before but didn't have this warning to expose it). For every
>> Swift file that's compiled (only using -Onone here), it outputs the same 2
>> warnings (easy to fix, but not related to any of this) from the same method
>> in the same Obj-C header referring to the arguments and the return types in
>> that method:
>>
>> - "array parameter is missing a nullability type specifier (_Nonnull,
>> _Nullable, or _Null_unspecified)"
>> - "inferring '_Nonnull' for pointer type within array is deprecated"
>>
>> Here's an example of what this method looks like:
>>
>> + (NSSet *)setWithSELs:(SEL[])sels count:(NSUInteger)count;
>>
>> Could this point to more duplicated work?
>>
>> Ben
>>
>> On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher  wrote:
>>
>>> Sure! Thanks for reminding me. I'll follow up on that, test, and get
>>> back to you.
>>>
>>> Ben
>>>
>>> On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey  wrote:
>>>

 On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev <
 swift-dev@swift.org> wrote:

 Just running a quick trial before and after I made this change in our
 project, we were previously seeing builds of our main target that took just
 under 13min. With this hack, a clean debug build takes about 4.5min.


 You may find that recent snapshot builds from swift.org help with your
 build times even without enabling -Owholemodule. The redundant type
 checking of synthesized accessors which we talked about a month or two
 should now be fixed on master, and it would be great to verify that’s the
 case with your code and to get an idea of how much it improves your build
 times.

 Mark


 Ben

 On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher  wrote:

> Okay I think that worked! And just to clarify, you meant set
> SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?
>
> I'll file a radar this afternoon with some details and DM you the
> number.
>
> Thanks again!
>
> Ben
>
> On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  w
> rote:
>
>> Xcode needs to know that you're building in WMO mode, so rather than
>> putting -whole-module-optimization in your "Other Swift Flags", put 
>> -Onone
>> there. It's an ugly hack but it should work in the near term.
>>
>> We do want to work to make this drastic speed difference go away, so
>> if you're able we (at Apple) would love to have a source drop of your 
>> Swift
>> 3 project, for additional data on where the problems are. Mind filing a
>> Radar?
>>
>> Best,
>> Jordan
>>
>>
>> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <
>> swift-dev@swift.org> wrote:
>> >
>> > Hello! Someone recently tipped me off to using
>> -whole-module-optimization flag with -Onone for use during debug builds 
>> to
>> speed up compile times. In our project, the speedup feels quite dramatic,
>> but when it gets to the linking step (after compiling both Swift an

Re: [swift-dev] -whole-module-optimization with -Onone

[Mark Lacey via swift-dev]

> On Dec 1, 2016, at 5:48 PM, Ben Asher via swift-dev  
> wrote:
> 
> The build failed compiling 3 files in our project. Our app is crashing that 
> snapshot; here is some output from the failures:

It would be awesome if you could come up with small reproducers for these and 
open bugs for them. Alternately, opening a radar with the full project attached 
will give us an opportunity to get these fixed ASAP!

Mark

> 
> - Assertion failed: (newType == type || (isa(newType) && 
> cast(newType)->getNumElements() == 1 && 
> cast(newType).getElementType(0) == type)), function rewriteType, 
> file /Users/buildn
> ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h, line 
> 207.
> 
> - SIL verification failed: method's Self parameter should be constrained by 
> protocol: selfRequirement.getKind() == RequirementKind::Conformance && 
> selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
> equirement.getSecondType()->getAs() ->getDecl() == protocol
> 
> We're not getting all the way to linking, but compiling everything for that 
> snapshot took ~11min45s. Based on that, it appears that about a minute is 
> saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1 (App 
> Store).
> 
> With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If 
> Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build, 
> the extra time here (compared to the snapshot) could be explained by linking, 
> signing, and a few other custom builds scripts that run after compiling.
> 
> Thanks everyone for your help!
> 
> Ben
> 
> 
> On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher  > wrote:
> I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going, so I 
> don't have any time results yet. But, I did notice something new (or maybe 
> existed before but didn't have this warning to expose it). For every Swift 
> file that's compiled (only using -Onone here), it outputs the same 2 warnings 
> (easy to fix, but not related to any of this) from the same method in the 
> same Obj-C header referring to the arguments and the return types in that 
> method:
> 
> - "array parameter is missing a nullability type specifier (_Nonnull, 
> _Nullable, or _Null_unspecified)"
> - "inferring '_Nonnull' for pointer type within array is deprecated"
> 
> Here's an example of what this method looks like:
> 
> + (NSSet *)setWithSELs:(SEL[])sels count:(NSUInteger)count;
> 
> Could this point to more duplicated work?
> 
> Ben
> 
> On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher  > wrote:
> Sure! Thanks for reminding me. I'll follow up on that, test, and get back to 
> you.
> 
> Ben
> 
> On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey  > wrote:
> 
>> On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev > > wrote:
>> 
>> Just running a quick trial before and after I made this change in our 
>> project, we were previously seeing builds of our main target that took just 
>> under 13min. With this hack, a clean debug build takes about 4.5min.
> 
> You may find that recent snapshot builds from swift.org  
> help with your build times even without enabling -Owholemodule. The redundant 
> type checking of synthesized accessors which we talked about a month or two 
> should now be fixed on master, and it would be great to verify that’s the 
> case with your code and to get an idea of how much it improves your build 
> times.
> 
> Mark
> 
>> 
>> Ben
>> 
>> On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher > > wrote:
>> Okay I think that worked! And just to clarify, you meant set 
>> SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?
>> 
>> I'll file a radar this afternoon with some details and DM you the number.
>> 
>> Thanks again!
>> 
>> Ben
>> 
>> On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose > > wrote:
>> Xcode needs to know that you're building in WMO mode, so rather than putting 
>> -whole-module-optimization in your "Other Swift Flags", put -Onone there. 
>> It's an ugly hack but it should work in the near term.
>> 
>> We do want to work to make this drastic speed difference go away, so if 
>> you're able we (at Apple) would love to have a source drop of your Swift 3 
>> project, for additional data on where the problems are. Mind filing a Radar?
>> 
>> Best,
>> Jordan
>> 
>> 
>> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev > > > wrote:
>> >
>> > Hello! Someone recently tipped me off to using -whole-module-optimization 
>> > flag with -Onone for use during debug builds to speed up compile times. In 
>> > our project, the speedup feels quite dramatic, but when it gets to the 
>> > linking step (after compiling both Swift and Obj-C in the project) it 
>> > fails because ld can't find the individual object files that normally get 
>> > emitted during the debug

Re: [swift-dev] Questions about ARC

[Roman Levenstein via swift-dev]

> On Dec 1, 2016, at 4:55 PM, Jiho Choi  wrote:
> 
> Thanks for the explanation.
> One last question is why non-atomic ARC operations still use atomic load and 
> store.  Wouldn't regular memory operations be enough?

They use atomic operations with __ATOMIC_RELAXED, which basically means that 
they should load/store the counts as one single entity (i.e. a single 64bit 
word) and not as multiple parts. Aside from that, it does not involve any 
further overheads known from CAS/fetch-and-add type of atomic operations. 

You can actually see the assembly for this functions to see which machine 
instructions are generated.

> 
> On Thu, Dec 1, 2016 at 11:46 AM Roman Levenstein  > wrote:
>> On Nov 30, 2016, at 9:40 PM, Jiho Choi > > wrote:
>> 
>> Thanks for providing the pointer.
>> Do you have any preliminary result or goal (e.g. the replacement ratio) of 
>> the optimization? 
>> Is it going to replace all ARC operations with non-atomic ones for 
>> single-threaded applications?
> 
> In the ideal world, it would be nice to replace all ARC operations with 
> non-atomic ones for single-threaded applications. 
> 
> But in reality, it is way more difficult as it may seem at the first glance. 
> 
> If this needs to happen without any hints from the developer, just by means 
> of a static analysis of a program, then it is rather difficult. The main 
> problem is that the compiler needs to reason whether a given reference may 
> escape to another thread. For references created inside a function, we have 
> rather good chances to figure out if a reference escapes the thread. But if 
> the origin (i.e. how it was created or if it has escaped before) of a given 
> reference is unknown, which is a typical case with function parameters or 
> references inside class instances, then the compiler has to assume that any 
> such reference has escaped its original thread and thus it needs to use 
> atomic ARC-operations. Some sort of a global, whole-module/whole-program 
> analysis may help here somewhat. But even if we would introduce such kind of 
> analysis, it is likely to remain a problem for dynamic libraries and 
> frameworks, because they don’t know and cannot reason which parameters 
> required by their exposed APIs escaped in the user-code.
> 
> Alternatively , a developer could provide a hint and assure that compiler 
> that the app is single-threaded. One simple possibility could be to have a 
> special -single-threaded compiler option, which would basically claim that 
> the app being developed is single threaded and thus there is no need for 
> performing the atomic ARC operations. In this case, all ARC operations would 
> be marked non-atomic by default in the code emitted from the user-code. The 
> problem with this option could be that if a user app starts multiple threads 
> directly or indirectly (e.g. it calls a library API, which starts a new 
> thread), even though the option claimed the app would not do it,  and some 
> references will be shared between threads, then the execution of such an app 
> may become unpredictable and end up with hard to find crashes. Mixing object 
> files and libraries where a subset is compiled with this option and another 
> part without is another receipt for a disaster. So, one would need to be 
> extremely cautious when using this option. 
> 
> There could be also something in between, where one would use special 
> attributes indicating something related to thread-safety of a given 
> reference/type/function/etc. These hints could help a compiler to reason 
> about references and check if they may escape to a different thread.
> 
> -Roman
> 
>> 
>> On Wed, Nov 30, 2016 at 8:50 PM Roman Levenstein > > wrote:
>>> On Nov 30, 2016, at 6:25 PM, Jiho Choi via swift-dev >> > wrote:
>>> 
>>> Thanks for clarifications.  I have a couple of follow-up questions.
>>> 
>>> 1. Could you please provide more information (e.g. source code location) 
>>> about the optimization applying non-atomic reference counting?  What's the 
>>> scope of the optimization?  Is it method-based?
>> 
>> The optimization itself is not merged yet. But all the required machinery, 
>> e.g. non-atomic versions of the ARC operations, special non-atomic flag on 
>> SIL instructions, etc is in place already.
>> 
>> As for the prototype implementation, you can find it here, on my local 
>> branch:
>> https://github.com/swiftix/swift/blob/30409865ff49a4268363cd359f82f29c9a90cce8/lib/SILOptimizer/Transforms/NonAtomicRC.cpp
>>  
>> 
>> 
>>> 
>>> 2. Looking at the source code, I assume Swift implements immediate 
>>> reference counting (i.e. immediate reclamation of dead objects)  without 
>>> requiring explicit garbage collection phase for techniques, such as 
>>> deferred reference 

Re: [swift-dev] Questions about ARC

[Jiho Choi via swift-dev]

Thanks for the explanation.
One last question is why non-atomic ARC operations still use atomic load
and store.  Wouldn't regular memory operations be enough?

On Thu, Dec 1, 2016 at 11:46 AM Roman Levenstein 
wrote:

> On Nov 30, 2016, at 9:40 PM, Jiho Choi  wrote:
>
> Thanks for providing the pointer.
> Do you have any preliminary result or goal (e.g. the replacement ratio) of
> the optimization?
>
> Is it going to replace all ARC operations with non-atomic ones for
> single-threaded applications?
>
>
> In the ideal world, it would be nice to replace all ARC operations with
> non-atomic ones for single-threaded applications.
>
> But in reality, it is way more difficult as it may seem at the first
> glance.
>
> If this needs to happen without any hints from the developer, just by
> means of a static analysis of a program, then it is rather difficult. The
> main problem is that the compiler needs to reason whether a given reference
> may escape to another thread. For references created inside a function, we
> have rather good chances to figure out if a reference escapes the thread.
> But if the origin (i.e. how it was created or if it has escaped before) of
> a given reference is unknown, which is a typical case with function
> parameters or references inside class instances, then the compiler has to
> assume that any such reference has escaped its original thread and thus it
> needs to use atomic ARC-operations. Some sort of a global,
> whole-module/whole-program analysis may help here somewhat. But even if we
> would introduce such kind of analysis, it is likely to remain a problem for
> dynamic libraries and frameworks, because they don’t know and cannot reason
> which parameters required by their exposed APIs escaped in the user-code.
>
> Alternatively , a developer could provide a hint and assure that compiler
> that the app is single-threaded. One simple possibility could be to have a
> special -single-threaded compiler option, which would basically claim that
> the app being developed is single threaded and thus there is no need for
> performing the atomic ARC operations. In this case, all ARC operations
> would be marked non-atomic by default in the code emitted from the
> user-code. The problem with this option could be that if a user app starts
> multiple threads directly or indirectly (e.g. it calls a library API, which
> starts a new thread), even though the option claimed the app would not do
> it,  and some references will be shared between threads, then the execution
> of such an app may become unpredictable and end up with hard to find
> crashes. Mixing object files and libraries where a subset is compiled with
> this option and another part without is another receipt for a disaster. So,
> one would need to be extremely cautious when using this option.
>
> There could be also something in between, where one would use special
> attributes indicating something related to thread-safety of a given
> reference/type/function/etc. These hints could help a compiler to reason
> about references and check if they may escape to a different thread.
>
> -Roman
>
>
> On Wed, Nov 30, 2016 at 8:50 PM Roman Levenstein 
> wrote:
>
> On Nov 30, 2016, at 6:25 PM, Jiho Choi via swift-dev 
> wrote:
>
> Thanks for clarifications.  I have a couple of follow-up questions.
>
> 1. Could you please provide more information (e.g. source code location)
> about the optimization applying non-atomic reference counting?  What's the
> scope of the optimization?  Is it method-based?
>
>
> The optimization itself is not merged yet. But all the required machinery,
> e.g. non-atomic versions of the ARC operations, special non-atomic flag on
> SIL instructions, etc is in place already.
>
> As for the prototype implementation, you can find it here, on my local
> branch:
>
> https://github.com/swiftix/swift/blob/30409865ff49a4268363cd359f82f29c9a90cce8/lib/SILOptimizer/Transforms/NonAtomicRC.cpp
>
>
> 2. Looking at the source code, I assume Swift implements immediate
> reference counting (i.e. immediate reclamation of dead objects)  without
> requiring explicit garbage collection phase for techniques, such as
> deferred reference counting or coalescing multiple updates.  Is it right?
> If so, is there any plan to implement such techniques?
>
>
> Yes. It is a correct understanding.
> Different extensions like deferred reference counting were discussed, but
> there are no short-term plans to implement it anytime soon.
>
> -Roman
>
>
> On Wed, Nov 30, 2016 at 11:41 AM John McCall  wrote:
>
> On Nov 30, 2016, at 8:33 AM, Jiho Choi via swift-dev 
> wrote:
> Hi,
>
> I am new to Swift, and I have several questions about how ARC works in
> Swift.
>
> 1. I read from one of the previous discussions in the swift-evolution list
> (
> https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160208/009422.html)
> that ARC operations are currently not atomic as Swift has no memory model
> and concurrency model.  Does it mean that the compi

[swift-dev] [Swift CI] Build Failure: 0. OSS - LLDB Incremental - OS X (master) #4921

[no-reply--- via swift-dev]

Title: Report


  
  
 
 

 [FAILURE] oss-lldb-incremental-osx [#4921] 


  Build URL:https://ci.swift.org/job/oss-lldb-incremental-osx/4921/
  Project:oss-lldb-incremental-osx
  Date of build:Thu, 01 Dec 2016 15:23:45 -0800
  Build duration:1 hr 29 min


Identified problems:Assertion failure: This build failed because of an assertion failure. Below is a list of all errors in the build log:Indication 1Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:Indication 1




  





  Changes
  

  Commit b33d4fc84c5ebbf0d52f00f3868319cea2ad3146 by bob.wilson: Revert "Disable validation-test/stdlib/UnicodeLongTest.swift for


  edit: validation-test/stdlib/UnicodeLongTest.swift


  
 

  Commit bb9382e994a4c029ef1ba332f3964d65d36b13ac by mgottesman: Re-enable/update two tests that have been disabled for a long time.


  edit: test/Frontend/OptimizationOptions-without-stdlib-checks.swift

  edit: test/Frontend/OptimizationOptions-with-stdlib-checks.swift


  
 

  Commit 7b73a242ef6172de251136469dc189c13f64587c by hughbellars: Fix MSVC warning compiling IRGen


  edit: lib/IRGen/GenType.cpp


  
 

  Commit 698d25cb3d62ca7dd0a33b13ab19b27f96829773 by github: [overlay] Fixing the incorrect type of fanSpeedPercentage (#5992)


  edit: stdlib/public/SDK/Intents/INSetClimateSettingsInCarIntent.swift


  
 

  Commit 6f6465834697f17e86d059cdb28c447f778c by dgregor: [AST] Stash an archetype builder in an "incomplete" generic environment.


  edit: include/swift/AST/ASTContext.h

  edit: include/swift/AST/GenericEnvironment.h

  edit: lib/AST/Builtins.cpp

  edit: lib/AST/ASTContext.cpp

  edit: lib/AST/GenericSignature.cpp

  edit: lib/AST/ArchetypeBuilder.cpp

  edit: lib/AST/GenericEnvironment.cpp

  edit: include/swift/AST/GenericSignature.h

  edit: lib/Serialization/Deserialization.cpp


  
 

  Commit a3a943a6837d6a5f1341c2b9570a2a91a5c2d401 by dgregor: [Archetype builder] Move the forced creation of archetypes slightly


  edit: lib/AST/ArchetypeBuilder.cpp


  
 

  Commit 9e465fa03de2c079fe10f4cb5dc6eeebb1052f39 by dgregor: [AST] Add a form of Type::subst() that takes an arbitrary substitution


  edit: include/swift/AST/Type.h

  edit: include/swift/AST/GenericEnvironment.h

  edit: lib/AST/Type.cpp

  edit: lib/AST/GenericEnvironment.cpp

  edit: lib/AST/GenericSignature.cpp

  edit: include/swift/AST/GenericSignature.h


  
 

  Commit f150d239188d36fdae02711c9b36cbe2cd47da8d by dgregor: [AST] Tail-allocate context types in GenericEnvironment.


  edit: include/swift/AST/GenericEnvironment.h

  edit: lib/AST/GenericEnvironment.cpp

  edit: lib/AST/ASTContext.cpp


  
 

  Commit bf2a75e7847cd91fea5f7a042372b4a948eaec6a by dgregor: [Archetype builder] Lazily form context types for generic parameters.


  edit: lib/AST/ArchetypeBuilder.cpp

  edit: test/Generics/superclass_constraint.swift

  edit: lib/AST/GenericEnvironment.cpp


  
 

  Commit 2df99313d8d2191232659c8442cba883fee522e5 by dgregor: [Archetype builder] Replace a custom Type::transform() with a


  edit: lib/AST/ArchetypeBuilder.cpp

  edit: include/swift/AST/Types.h


  
 

  Commit 51c2cee4d831687111aed91e9c5b15ac60ce826d by dgregor: [Immediate] Remove a redundant, pessimizing std::move().


  edit: lib/Immediate/Immediate.cpp


  
 

  Commit 188d218886a9d9bfa727c42d7c24c2cb04dc by dgregor: [Archetype builder] Remove some dead code. NFC


  edit: lib/AST/ArchetypeBuilder.cpp


  
 

  Commit 7a58f00b6d473ec68e8ce0d460d68799d4e3 by xi_ge: [utils] Modify writeEdit() to take multiple edits instead of a single


  edit: lib/Basic/Edit.cpp

  edit: lib/FrontendTool/FrontendTool.cpp

  edit: include/swift/Basic/Edit.h


  
 

  Commit dfb035289aa5039fa56bb0e23ba490de9b08d835 by spestov: IDE: Remove some redundant calls to getDesugaredType()


  edit: lib/IDE/TypeReconstruction.cpp


  
 

  Commit c71395a4c3a3b3a829e6e4d50fc2fca09754d021 by spestov: Sema: Small cleanup in CSDiag


  edit: lib/Sema/CSDiag.cpp

  edit: test/stdlib/RangeDiagnostics.swift


  
 

  Commit cb0126a30638118b58e2dd06f2b2e652e8a824f1 by spestov: IRGen: Remove some dead code


  edit: lib/IRGen/GenObjC.cpp


  
 

  Commit 8bdbe774e0a941cc66ee6003372f

Re: [swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

The build failed compiling 3 files in our project. Our app is crashing that
snapshot; here is some output from the failures:

- Assertion failed: (newType == type || (isa(newType) &&
cast(newType)->getNumElements() == 1 &&
cast(newType).getElementType(0) == type)), function rewriteType,
file /Users/buildn
ode/jenkins/workspace/oss-swift-package-osx/swift/lib/SILGen/RValue.h, line
207.

- SIL verification failed: method's Self parameter should be constrained by
protocol: selfRequirement.getKind() == RequirementKind::Conformance &&
selfRequirement.getFirstType()->isEqual(selfGenericParam) && selfR
equirement.getSecondType()->getAs() ->getDecl() == protocol

We're not getting all the way to linking, but compiling everything for that
snapshot took ~11min45s. Based on that, it appears that about a minute is
saved with the new fixes since the Swift 3.0 that shipped with Xcode 8.1
(App Store).

With Xcode 8.2 Beta 2, the app builds fine, and it takes about 12m15s. If
Xcode 8.2 Beta 2 also has those fixes that we expect to speed up the build,
the extra time here (compared to the snapshot) could be explained by
linking, signing, and a few other custom builds scripts that run after
compiling.

Thanks everyone for your help!

Ben


On Thu, Dec 1, 2016 at 4:16 PM, Ben Asher  wrote:

> I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going,
> so I don't have any time results yet. But, I did notice something new (or
> maybe existed before but didn't have this warning to expose it). For every
> Swift file that's compiled (only using -Onone here), it outputs the same 2
> warnings (easy to fix, but not related to any of this) from the same method
> in the same Obj-C header referring to the arguments and the return types in
> that method:
>
> - "array parameter is missing a nullability type specifier (_Nonnull,
> _Nullable, or _Null_unspecified)"
> - "inferring '_Nonnull' for pointer type within array is deprecated"
>
> Here's an example of what this method looks like:
>
> + (NSSet *)setWithSELs:(SEL[])sels count:(NSUInteger)count;
>
> Could this point to more duplicated work?
>
> Ben
>
> On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher  wrote:
>
>> Sure! Thanks for reminding me. I'll follow up on that, test, and get back
>> to you.
>>
>> Ben
>>
>> On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey  wrote:
>>
>>>
>>> On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev 
>>> wrote:
>>>
>>> Just running a quick trial before and after I made this change in our
>>> project, we were previously seeing builds of our main target that took just
>>> under 13min. With this hack, a clean debug build takes about 4.5min.
>>>
>>>
>>> You may find that recent snapshot builds from swift.org help with your
>>> build times even without enabling -Owholemodule. The redundant type
>>> checking of synthesized accessors which we talked about a month or two
>>> should now be fixed on master, and it would be great to verify that’s the
>>> case with your code and to get an idea of how much it improves your build
>>> times.
>>>
>>> Mark
>>>
>>>
>>> Ben
>>>
>>> On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher  wrote:
>>>
 Okay I think that worked! And just to clarify, you meant set
 SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

 I'll file a radar this afternoon with some details and DM you the
 number.

 Thanks again!

 Ben

 On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  w
 rote:

> Xcode needs to know that you're building in WMO mode, so rather than
> putting -whole-module-optimization in your "Other Swift Flags", put -Onone
> there. It's an ugly hack but it should work in the near term.
>
> We do want to work to make this drastic speed difference go away, so
> if you're able we (at Apple) would love to have a source drop of your 
> Swift
> 3 project, for additional data on where the problems are. Mind filing a
> Radar?
>
> Best,
> Jordan
>
>
> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <
> swift-dev@swift.org> wrote:
> >
> > Hello! Someone recently tipped me off to using
> -whole-module-optimization flag with -Onone for use during debug builds to
> speed up compile times. In our project, the speedup feels quite dramatic,
> but when it gets to the linking step (after compiling both Swift and Obj-C
> in the project) it fails because ld can't find the individual object files
> that normally get emitted during the debug-type build presumably because
> -whole-module-optimization only emits one (and this isn't a normal
> "-Owholemodule"-type build which works fine).
> >
> > I can't seem to reproduce this outside of Xcode, but I was curious
> if anyone has tried this and knows of a workaround to get
> -whole-module-optimization to work with -Onone in Xcode?
> >
> > I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
> Sierra.
> >

Re: [swift-dev] No commit emails

[Greg Parker via swift-dev]

FWIW most commits go through GitHub pull requests nowadays, so setting up 
GitHub's notifications for the apple/swift repository will keep you informed of 
most changes.


> On Dec 1, 2016, at 11:53 AM, Kenny Leung via swift-dev  
> wrote:
> 
> Hi All.
> 
> Just a heads up.
> 
> We got a big bolus on November 18, and then nothing after that.
> 
> Thanks.
> 
> -Kenny
> 
> 
>> On Nov 18, 2016, at 4:18 PM, Jordan Rose > > wrote:
>> 
>> Thanks for the heads-up. I think our old notifier system is finally invalid 
>> after a network reconfiguration. We've been meaning to move it to be just 
>> another job on ci.swift.org  but haven't gotten around 
>> to it yet.
>> 
>> Most of Apple is on holiday for American Thanksgiving all next week, but 
>> we'll look into it after that. I know that's a long gap; we just didn't get 
>> the chance to look into it before the end of the week. Sorry for the 
>> inconvenience (and the missed commits).
>> 
>> Jordan
>> 
>>> On Nov 17, 2016, at 9:23, Kenny Leung via swift-dev >> > wrote:
>>> 
>>> Hi There.
>>> 
>>> It looks like the conduit for commit emails has stopped working since Nov. 
>>> 13. Can someone take a look please?
>>> 
>>> Thanks.
>>> 
>>> -Kenny



___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

I'm trying out DEVELOPMENT-SNAPSHOT-2016-11-29-a now. It's still going, so
I don't have any time results yet. But, I did notice something new (or
maybe existed before but didn't have this warning to expose it). For every
Swift file that's compiled (only using -Onone here), it outputs the same 2
warnings (easy to fix, but not related to any of this) from the same method
in the same Obj-C header referring to the arguments and the return types in
that method:

- "array parameter is missing a nullability type specifier (_Nonnull,
_Nullable, or _Null_unspecified)"
- "inferring '_Nonnull' for pointer type within array is deprecated"

Here's an example of what this method looks like:

+ (NSSet *)setWithSELs:(SEL[])sels count:(NSUInteger)count;

Could this point to more duplicated work?

Ben

On Thu, Dec 1, 2016 at 2:50 PM, Ben Asher  wrote:

> Sure! Thanks for reminding me. I'll follow up on that, test, and get back
> to you.
>
> Ben
>
> On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey  wrote:
>
>>
>> On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev 
>> wrote:
>>
>> Just running a quick trial before and after I made this change in our
>> project, we were previously seeing builds of our main target that took just
>> under 13min. With this hack, a clean debug build takes about 4.5min.
>>
>>
>> You may find that recent snapshot builds from swift.org help with your
>> build times even without enabling -Owholemodule. The redundant type
>> checking of synthesized accessors which we talked about a month or two
>> should now be fixed on master, and it would be great to verify that’s the
>> case with your code and to get an idea of how much it improves your build
>> times.
>>
>> Mark
>>
>>
>> Ben
>>
>> On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher  wrote:
>>
>>> Okay I think that worked! And just to clarify, you meant set
>>> SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?
>>>
>>> I'll file a radar this afternoon with some details and DM you the number.
>>>
>>> Thanks again!
>>>
>>> Ben
>>>
>>> On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  w
>>> rote:
>>>
 Xcode needs to know that you're building in WMO mode, so rather than
 putting -whole-module-optimization in your "Other Swift Flags", put -Onone
 there. It's an ugly hack but it should work in the near term.

 We do want to work to make this drastic speed difference go away, so if
 you're able we (at Apple) would love to have a source drop of your Swift 3
 project, for additional data on where the problems are. Mind filing a 
 Radar?

 Best,
 Jordan


 > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev <
 swift-dev@swift.org> wrote:
 >
 > Hello! Someone recently tipped me off to using
 -whole-module-optimization flag with -Onone for use during debug builds to
 speed up compile times. In our project, the speedup feels quite dramatic,
 but when it gets to the linking step (after compiling both Swift and Obj-C
 in the project) it fails because ld can't find the individual object files
 that normally get emitted during the debug-type build presumably because
 -whole-module-optimization only emits one (and this isn't a normal
 "-Owholemodule"-type build which works fine).
 >
 > I can't seem to reproduce this outside of Xcode, but I was curious if
 anyone has tried this and knows of a workaround to get
 -whole-module-optimization to work with -Onone in Xcode?
 >
 > I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
 Sierra.
 >
 > Thanks!
 >
 > Ben
 > ___
 > swift-dev mailing list
 > swift-dev@swift.org
 > https://lists.swift.org/mailman/listinfo/swift-dev


>>>
>>>
>>> --
>>> Ben
>>>
>>
>>
>>
>> --
>> Ben
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-dev
>>
>>
>>
>
>
> --
> Ben
>



-- 
Ben
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

Sure! Thanks for reminding me. I'll follow up on that, test, and get back
to you.

Ben

On Thu, Dec 1, 2016 at 2:48 PM, Mark Lacey  wrote:

>
> On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev 
> wrote:
>
> Just running a quick trial before and after I made this change in our
> project, we were previously seeing builds of our main target that took just
> under 13min. With this hack, a clean debug build takes about 4.5min.
>
>
> You may find that recent snapshot builds from swift.org help with your
> build times even without enabling -Owholemodule. The redundant type
> checking of synthesized accessors which we talked about a month or two
> should now be fixed on master, and it would be great to verify that’s the
> case with your code and to get an idea of how much it improves your build
> times.
>
> Mark
>
>
> Ben
>
> On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher  wrote:
>
>> Okay I think that worked! And just to clarify, you meant set
>> SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?
>>
>> I'll file a radar this afternoon with some details and DM you the number.
>>
>> Thanks again!
>>
>> Ben
>>
>> On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  w
>> rote:
>>
>>> Xcode needs to know that you're building in WMO mode, so rather than
>>> putting -whole-module-optimization in your "Other Swift Flags", put -Onone
>>> there. It's an ugly hack but it should work in the near term.
>>>
>>> We do want to work to make this drastic speed difference go away, so if
>>> you're able we (at Apple) would love to have a source drop of your Swift 3
>>> project, for additional data on where the problems are. Mind filing a Radar?
>>>
>>> Best,
>>> Jordan
>>>
>>>
>>> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev 
>>> wrote:
>>> >
>>> > Hello! Someone recently tipped me off to using
>>> -whole-module-optimization flag with -Onone for use during debug builds to
>>> speed up compile times. In our project, the speedup feels quite dramatic,
>>> but when it gets to the linking step (after compiling both Swift and Obj-C
>>> in the project) it fails because ld can't find the individual object files
>>> that normally get emitted during the debug-type build presumably because
>>> -whole-module-optimization only emits one (and this isn't a normal
>>> "-Owholemodule"-type build which works fine).
>>> >
>>> > I can't seem to reproduce this outside of Xcode, but I was curious if
>>> anyone has tried this and knows of a workaround to get
>>> -whole-module-optimization to work with -Onone in Xcode?
>>> >
>>> > I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
>>> Sierra.
>>> >
>>> > Thanks!
>>> >
>>> > Ben
>>> > ___
>>> > swift-dev mailing list
>>> > swift-dev@swift.org
>>> > https://lists.swift.org/mailman/listinfo/swift-dev
>>>
>>>
>>
>>
>> --
>> Ben
>>
>
>
>
> --
> Ben
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev
>
>
>


-- 
Ben
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -whole-module-optimization with -Onone

[Mark Lacey via swift-dev]

> On Dec 1, 2016, at 3:13 PM, Ben Asher via swift-dev  
> wrote:
> 
> Just running a quick trial before and after I made this change in our 
> project, we were previously seeing builds of our main target that took just 
> under 13min. With this hack, a clean debug build takes about 4.5min.

You may find that recent snapshot builds from swift.org  
help with your build times even without enabling -Owholemodule. The redundant 
type checking of synthesized accessors which we talked about a month or two 
should now be fixed on master, and it would be great to verify that’s the case 
with your code and to get an idea of how much it improves your build times.

Mark

> 
> Ben
> 
> On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher  > wrote:
> Okay I think that worked! And just to clarify, you meant set 
> SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?
> 
> I'll file a radar this afternoon with some details and DM you the number.
> 
> Thanks again!
> 
> Ben
> 
> On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  > wrote:
> Xcode needs to know that you're building in WMO mode, so rather than putting 
> -whole-module-optimization in your "Other Swift Flags", put -Onone there. 
> It's an ugly hack but it should work in the near term.
> 
> We do want to work to make this drastic speed difference go away, so if 
> you're able we (at Apple) would love to have a source drop of your Swift 3 
> project, for additional data on where the problems are. Mind filing a Radar?
> 
> Best,
> Jordan
> 
> 
> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev  > > wrote:
> >
> > Hello! Someone recently tipped me off to using -whole-module-optimization 
> > flag with -Onone for use during debug builds to speed up compile times. In 
> > our project, the speedup feels quite dramatic, but when it gets to the 
> > linking step (after compiling both Swift and Obj-C in the project) it fails 
> > because ld can't find the individual object files that normally get emitted 
> > during the debug-type build presumably because -whole-module-optimization 
> > only emits one (and this isn't a normal "-Owholemodule"-type build which 
> > works fine).
> >
> > I can't seem to reproduce this outside of Xcode, but I was curious if 
> > anyone has tried this and knows of a workaround to get 
> > -whole-module-optimization to work with -Onone in Xcode?
> >
> > I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.
> >
> > Thanks!
> >
> > Ben
> > ___
> > swift-dev mailing list
> > swift-dev@swift.org 
> > https://lists.swift.org/mailman/listinfo/swift-dev 
> > 
> 
> 
> 
> 
> -- 
> Ben
> 
> 
> 
> -- 
> Ben
> ___
> swift-dev mailing list
> swift-dev@swift.org 
> https://lists.swift.org/mailman/listinfo/swift-dev 
> 
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] Details of what source compatibility with Swift 3 means

[ematejska--- via swift-dev]

Hi,

(I already sent this to swift-evolution yesterday but cross-posting.)

One of the Stage 1 goals of Swift 4 is source compatibility with Swift 3, but 
what does that precisely mean and how will we determine if the goal is 
achieved?  This is the current proposal of how this goal ties back to the Swift 
3.x and Swift 4 releases.  Is there anything else that’s missing or should be 
discussed?


The source compatibility goal for Swift 4 means that sources that previously 
compiled with the Swift 3 compiler will continue to compile with no source 
modifications with the Swift 3.x and Swift 4 compilers.  

Swift 3.x releases have the goal of source compatibility with Swift 3.  This is 
the default mode and there is no compatibility flag.  
This means that Swift 3 code should be able to build with the Swift 3.x 
compiler without modification as much as practically possible. The same applies 
to SwiftPM packages with Swift 3 source code.  
Swift 3.x code will not necessarily compile with the Swift 3 compiler.  For 
example, new language features could be added.

The Swift 4 compiler will include a language compatibility flag -swift-version 
that accepts 3 or 4 as arguments and controls the compatibility mode (SR- 2582 
).
The -swift-version 3 compatibility mode has the goal of source compatibility as 
much as practically possible with Swift 3.x (including 3, 3.0.x, 3.x). 
The -swift-version 4 compatibility allows approved source code breakages from 
Swift 3.x (including 3, 3.0.x, 3.x).  Proposed source breaking changes should 
be discussed on swift-evolution and highlighted in evolution proposals.
All the sources in a module (e.g,  SwiftPM packages, targets in Xcode) will be 
built in the same compatibility mode, but modules built with different 
compatibility modes can be linked into a single binary and will interoperate. 
Thus a module can be migrated from Swift 3 to 3.x to 4 independently of its 
dependencies.
The Swift Package Manager will allow package authors to specify a list of 
compatibility modes.
The Swift compiler already provides a mechanism for conditional compilation 
based on the compiler version with the #if swift syntax. SE-0020 
.
  Swift 4 will also provide mechanisms to conditionalize code based on the 
compatibility version. 
The @available(...) attribute has been expanded to support the swift language 
version with SE-0141 
.
 For example, @available(swift 3).

Maintaining source compatibility with Swift 3 in practice while continuing 
development is not always clear-cut.  It often comes down to tradeoffs.  Let’s 
review a few cases.
New features may become available if they do not take up syntactic space that 
might be commonly used by existing Swift code. 
If a bug fix breaks source in a fringe case as determined by a scan of existing 
source, the benefit might outweigh the cost and it could be allowed.
Changes to existing APIs have high bar to clear as they should provide a 
significant benefit to clients and have minimal impact on existing, real-life 
code.  These changes should be considered for inclusion in the -swift-version 4 
compatibility mode of the Swift 4 compiler instead.
New warnings are allowed because they do not require source code changes.
To summarize, any proposed source breakage from Swift 3 for the Swift 3.x 
compilers or the -swift-version 3 compatibility mode of the Swift 4 compiler, 
should be small in scope of code affected and weighed against the goals of 
source compatibility described above. It should be approved on swift-evolution. 

For Apple Platforms, source compatibility can also be affected by changes to 
the Apple SDKs.  Last year, Swift 2.3 required a migration from Swift 2.2 to 
account for changes in the SDKs even though no language changes were 
introduced.  This year, source compatibility is defined more strictly to 
require that the Apple SDKs only add source compatible changes when used with 
the Swift 3.x compilers or when used with the Swift 3 compatibility mode 
(-swift-version 3) in the Swift 4 compiler.  The  ‘API notes’ feature has been 
augmented and can now express how an API is imported for a specific Swift 
version.  This feature allows the view of the evolved Apple SDKs to stay the 
same as with Swift 3 when Swift 3 source compatibility is expected.


Thanks,
Ewa___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

Just running a quick trial before and after I made this change in our
project, we were previously seeing builds of our main target that took just
under 13min. With this hack, a clean debug build takes about 4.5min.

Ben

On Thu, Dec 1, 2016 at 1:33 PM, Ben Asher  wrote:

> Okay I think that worked! And just to clarify, you meant set
> SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?
>
> I'll file a radar this afternoon with some details and DM you the number.
>
> Thanks again!
>
> Ben
>
> On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  wrote:
>
>> Xcode needs to know that you're building in WMO mode, so rather than
>> putting -whole-module-optimization in your "Other Swift Flags", put -Onone
>> there. It's an ugly hack but it should work in the near term.
>>
>> We do want to work to make this drastic speed difference go away, so if
>> you're able we (at Apple) would love to have a source drop of your Swift 3
>> project, for additional data on where the problems are. Mind filing a Radar?
>>
>> Best,
>> Jordan
>>
>>
>> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev 
>> wrote:
>> >
>> > Hello! Someone recently tipped me off to using
>> -whole-module-optimization flag with -Onone for use during debug builds to
>> speed up compile times. In our project, the speedup feels quite dramatic,
>> but when it gets to the linking step (after compiling both Swift and Obj-C
>> in the project) it fails because ld can't find the individual object files
>> that normally get emitted during the debug-type build presumably because
>> -whole-module-optimization only emits one (and this isn't a normal
>> "-Owholemodule"-type build which works fine).
>> >
>> > I can't seem to reproduce this outside of Xcode, but I was curious if
>> anyone has tried this and knows of a workaround to get
>> -whole-module-optimization to work with -Onone in Xcode?
>> >
>> > I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
>> Sierra.
>> >
>> > Thanks!
>> >
>> > Ben
>> > ___
>> > swift-dev mailing list
>> > swift-dev@swift.org
>> > https://lists.swift.org/mailman/listinfo/swift-dev
>>
>>
>
>
> --
> Ben
>



-- 
Ben
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

Okay I think that worked! And just to clarify, you meant set
SWIFT_OPTIMIZATION_LEVEL = -Owholemodule and OTHER_SWIFT_FLAGS = -Onone ?

I'll file a radar this afternoon with some details and DM you the number.

Thanks again!

Ben

On Thu, Dec 1, 2016 at 1:10 PM, Jordan Rose  wrote:

> Xcode needs to know that you're building in WMO mode, so rather than
> putting -whole-module-optimization in your "Other Swift Flags", put -Onone
> there. It's an ugly hack but it should work in the near term.
>
> We do want to work to make this drastic speed difference go away, so if
> you're able we (at Apple) would love to have a source drop of your Swift 3
> project, for additional data on where the problems are. Mind filing a Radar?
>
> Best,
> Jordan
>
>
> > On Dec 1, 2016, at 11:51, Ben Asher via swift-dev 
> wrote:
> >
> > Hello! Someone recently tipped me off to using
> -whole-module-optimization flag with -Onone for use during debug builds to
> speed up compile times. In our project, the speedup feels quite dramatic,
> but when it gets to the linking step (after compiling both Swift and Obj-C
> in the project) it fails because ld can't find the individual object files
> that normally get emitted during the debug-type build presumably because
> -whole-module-optimization only emits one (and this isn't a normal
> "-Owholemodule"-type build which works fine).
> >
> > I can't seem to reproduce this outside of Xcode, but I was curious if
> anyone has tried this and knows of a workaround to get
> -whole-module-optimization to work with -Onone in Xcode?
> >
> > I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS
> Sierra.
> >
> > Thanks!
> >
> > Ben
> > ___
> > swift-dev mailing list
> > swift-dev@swift.org
> > https://lists.swift.org/mailman/listinfo/swift-dev
>
>


-- 
Ben
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -whole-module-optimization with -Onone

[Jordan Rose via swift-dev]

Xcode needs to know that you're building in WMO mode, so rather than putting 
-whole-module-optimization in your "Other Swift Flags", put -Onone there. It's 
an ugly hack but it should work in the near term.

We do want to work to make this drastic speed difference go away, so if you're 
able we (at Apple) would love to have a source drop of your Swift 3 project, 
for additional data on where the problems are. Mind filing a Radar?

Best,
Jordan


> On Dec 1, 2016, at 11:51, Ben Asher via swift-dev  wrote:
> 
> Hello! Someone recently tipped me off to using -whole-module-optimization 
> flag with -Onone for use during debug builds to speed up compile times. In 
> our project, the speedup feels quite dramatic, but when it gets to the 
> linking step (after compiling both Swift and Obj-C in the project) it fails 
> because ld can't find the individual object files that normally get emitted 
> during the debug-type build presumably because -whole-module-optimization 
> only emits one (and this isn't a normal "-Owholemodule"-type build which 
> works fine).
> 
> I can't seem to reproduce this outside of Xcode, but I was curious if anyone 
> has tried this and knows of a workaround to get -whole-module-optimization to 
> work with -Onone in Xcode?
> 
> I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.
> 
> Thanks!
> 
> Ben
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -Onone @inline(__always)?

[Joe Groff via swift-dev]

> On Dec 1, 2016, at 12:11 PM, Andrew Trick  wrote:
> 
> 
>> On Dec 1, 2016, at 11:55 AM, Joe Groff  wrote:
>> 
>> 
>>> On Dec 1, 2016, at 11:48 AM, Andrew Trick  wrote:
>>> 
>>> Responding on swift-dev to what turned into an interesting discussion…
>>> 
 On Dec 1, 2016, at 11:19 AM, Joe Groff  wrote:
 
> 
> On Dec 1, 2016, at 11:18 AM, Joe Groff  wrote:
> 
>> 
>> On Dec 1, 2016, at 11:16 AM, Andrew Trick  wrote:
>> 
>> 
>>> On Dec 1, 2016, at 10:34 AM, Jordan Rose  wrote:
>>> 
 
 On Nov 30, 2016, at 18:16, Joe Groff  wrote:
 
> 
> On Nov 30, 2016, at 6:10 PM, Andrew Trick  wrote:
> 
> We definitely want an inline-at-Onone, but I assumed we were treating 
> "@inline(__always)" as a normal heuristic because:
> - @transparent already does inline-at-Onone.
> - @inline(__always) without @transparent negatively affects the debug 
> experience.
> - @inline(__always) isn't useful if no optimizations are run after 
> inlining.
> 
> That said, if the common expectation is for @inline(__always) to kick 
> in independent of optimization, then we should honor that.
> 
> It's easy for me to add an inlining pass to runSILPassesForOnone. I'm 
> just not sure that will have the effect people are looking for. It's 
> unlikely to help performance since nothing is going to run after the 
> inlining (no capture promotion or alloc box-to-stack). The only 
> optimization we currently run at Onone after diagnostics is 
> prespecialization.
>>> 
>>> Actually, the inliner specializes as it goes, so inlining after mandatory 
>>> SIL passes would still be quite useful.
>>> 
>>> My concern is really that the user may expect inline(__always) to be as 
>>> effective a performance tool as @transparent, while preserving debug line 
>>> locations.
>>> If it runs late, it doesn’t expose capture promotion, box-to-stack, 
>>> guaranteed ARC optimization, etc.
>>> 
> If someone is using inline(__always) for -Onone performance reasons, 
> I think they really need to use @transparent.
> 
> An alternative is to have @inline(__always) be another way to spell 
> @transparent. People would need to implicitly know that this would 
> affect diagnostics of the inlined method.
> 
> Any thoughts?
 
 `@transparent` doesn't just mean "inline-at-Onone", it also allows 
 diagnostic passes to constant-fold through the implementation and 
 actively eliminates debug information. It was never intended to be 
 used for performance, but only for the limited set of things that need 
 it. I imagine that the specialization you get by inlining some things 
 is pretty important by itself. I suspect closure inlining would also 
 be helpful for a lot of low-level "with*" combinators; doesn't 
 transparent inlining do that, and could inline-always-inlining do the 
 same?
>>> 
>>> For reference: 
>>> https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst
>> 
>> The only remaining question: is it ok for a non-transparent function to 
>> be inlined prior to running some dataflow diagnostics?
> 
> Doing so changes the set of programs that are accepted, since it may 
> expose more code to diagnostic passes. We don't want diagnostics to 
> depend on optimizations.
 
 Perhaps we could avoid this by changing the diagnostic passes to 
 intentionally ignore instructions with inlined source locations, though 
 that seems brittle.
 
 -Joe
>>> 
>>> Well, inline(__always) would be mandatory, not an optimization. But I think 
>>> your point is that the user does not expect that attribute to affect 
>>> program legality. Having data-flow diagnostics “ignore” certain 
>>> instructions sounds horrible. Is there an obvious problem, or is this 
>>> hypothetical? i.e. is there some diagnostic that is currently enforced at 
>>> the function boundary but would be considered legal code after inlining?
>> 
>> An example of something that depends on dataflow diagnostics would be 
>> something that's constant-foldable causing static overflow diagnostics:
>> 
>> func double(_ x: UInt8) -> UInt8 {
>>   return x + x
>> }
>> 
>> _ = double(128)
>> 
>> Making `double` transparent causes the compiler to statically reject the 
>> call:
>> 
>> /Users/jgroff/bar.swift:6:5: error: arithmetic operation '128 + 128' (on 
>> unsigned 8-bit integer type) results in an overflow
>> _ = double(128)
>> ^
>> 
>> -Joe
> 
> Right. In fact, constant propagation appears to be the only diagnostic that 
> needs to run after mandatory inlining (I was misremembering SILDiagnostic 
> pipeline):

Unreachable code analysis is also affected. For instance:

import Swift

func die() {

Re: [swift-dev] -Onone @inline(__always)?

[Andrew Trick via swift-dev]

> On Dec 1, 2016, at 11:55 AM, Joe Groff  wrote:
> 
> 
>> On Dec 1, 2016, at 11:48 AM, Andrew Trick > > wrote:
>> 
>> Responding on swift-dev to what turned into an interesting discussion…
>> 
>>> On Dec 1, 2016, at 11:19 AM, Joe Groff >> > wrote:
>>> 
 
 On Dec 1, 2016, at 11:18 AM, Joe Groff  wrote:
 
> 
> On Dec 1, 2016, at 11:16 AM, Andrew Trick  wrote:
> 
> 
>> On Dec 1, 2016, at 10:34 AM, Jordan Rose  wrote:
>> 
>>> 
>>> On Nov 30, 2016, at 18:16, Joe Groff  wrote:
>>> 
 
 On Nov 30, 2016, at 6:10 PM, Andrew Trick  wrote:
 
 We definitely want an inline-at-Onone, but I assumed we were treating 
 "@inline(__always)" as a normal heuristic because:
 - @transparent already does inline-at-Onone.
 - @inline(__always) without @transparent negatively affects the debug 
 experience.
 - @inline(__always) isn't useful if no optimizations are run after 
 inlining.
 
 That said, if the common expectation is for @inline(__always) to kick 
 in independent of optimization, then we should honor that.
 
 It's easy for me to add an inlining pass to runSILPassesForOnone. I'm 
 just not sure that will have the effect people are looking for. It's 
 unlikely to help performance since nothing is going to run after the 
 inlining (no capture promotion or alloc box-to-stack). The only 
 optimization we currently run at Onone after diagnostics is 
 prespecialization.
>> 
>> Actually, the inliner specializes as it goes, so inlining after mandatory 
>> SIL passes would still be quite useful.
>> 
>> My concern is really that the user may expect inline(__always) to be as 
>> effective a performance tool as @transparent, while preserving debug line 
>> locations.
>> If it runs late, it doesn’t expose capture promotion, box-to-stack, 
>> guaranteed ARC optimization, etc.
>> 
 If someone is using inline(__always) for -Onone performance reasons, I 
 think they really need to use @transparent.
 
 An alternative is to have @inline(__always) be another way to spell 
 @transparent. People would need to implicitly know that this would 
 affect diagnostics of the inlined method.
 
 Any thoughts?
>>> 
>>> `@transparent` doesn't just mean "inline-at-Onone", it also allows 
>>> diagnostic passes to constant-fold through the implementation and 
>>> actively eliminates debug information. It was never intended to be used 
>>> for performance, but only for the limited set of things that need it. I 
>>> imagine that the specialization you get by inlining some things is 
>>> pretty important by itself. I suspect closure inlining would also be 
>>> helpful for a lot of low-level "with*" combinators; doesn't transparent 
>>> inlining do that, and could inline-always-inlining do the same?
>> 
>> For reference: 
>> https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst 
>> 
> 
> The only remaining question: is it ok for a non-transparent function to 
> be inlined prior to running some dataflow diagnostics?
 
 Doing so changes the set of programs that are accepted, since it may 
 expose more code to diagnostic passes. We don't want diagnostics to depend 
 on optimizations.
>>> 
>>> Perhaps we could avoid this by changing the diagnostic passes to 
>>> intentionally ignore instructions with inlined source locations, though 
>>> that seems brittle.
>>> 
>>> -Joe
>> 
>> Well, inline(__always) would be mandatory, not an optimization. But I think 
>> your point is that the user does not expect that attribute to affect program 
>> legality. Having data-flow diagnostics “ignore” certain instructions sounds 
>> horrible. Is there an obvious problem, or is this hypothetical? i.e. is 
>> there some diagnostic that is currently enforced at the function boundary 
>> but would be considered legal code after inlining?
> 
> An example of something that depends on dataflow diagnostics would be 
> something that's constant-foldable causing static overflow diagnostics:
> 
> func double(_ x: UInt8) -> UInt8 {
>   return x + x
> }
> 
> _ = double(128)
> 
> Making `double` transparent causes the compiler to statically reject the call:
> 
> /Users/jgroff/bar.swift:6:5: error: arithmetic operation '128 + 128' (on 
> unsigned 8-bit integer type) results in an overflow
> _ = double(128)
> ^
> 
> -Joe

Right. In fact, constant propagation appears to be the only diagnostic that 
needs to run after mandatory inlining (I was misremembering SILDiagnostic 
pipeline):

  PM.addCapturePromotion();
  PM.addAllocBoxToStack();
  PM.addNoReturnFolding();
  PM.addDefiniteInitialization();

  PM.addMandatoryInlini

Re: [swift-dev] -Onone @inline(__always)?

[Joe Groff via swift-dev]

> On Dec 1, 2016, at 11:48 AM, Andrew Trick  wrote:
> 
> Responding on swift-dev to what turned into an interesting discussion…
> 
>> On Dec 1, 2016, at 11:19 AM, Joe Groff  wrote:
>> 
>>> 
>>> On Dec 1, 2016, at 11:18 AM, Joe Groff  wrote:
>>> 
 
 On Dec 1, 2016, at 11:16 AM, Andrew Trick  wrote:
 
 
> On Dec 1, 2016, at 10:34 AM, Jordan Rose  wrote:
> 
>> 
>> On Nov 30, 2016, at 18:16, Joe Groff  wrote:
>> 
>>> 
>>> On Nov 30, 2016, at 6:10 PM, Andrew Trick  wrote:
>>> 
>>> We definitely want an inline-at-Onone, but I assumed we were treating 
>>> "@inline(__always)" as a normal heuristic because:
>>> - @transparent already does inline-at-Onone.
>>> - @inline(__always) without @transparent negatively affects the debug 
>>> experience.
>>> - @inline(__always) isn't useful if no optimizations are run after 
>>> inlining.
>>> 
>>> That said, if the common expectation is for @inline(__always) to kick 
>>> in independent of optimization, then we should honor that.
>>> 
>>> It's easy for me to add an inlining pass to runSILPassesForOnone. I'm 
>>> just not sure that will have the effect people are looking for. It's 
>>> unlikely to help performance since nothing is going to run after the 
>>> inlining (no capture promotion or alloc box-to-stack). The only 
>>> optimization we currently run at Onone after diagnostics is 
>>> prespecialization.
> 
> Actually, the inliner specializes as it goes, so inlining after mandatory SIL 
> passes would still be quite useful.
> 
> My concern is really that the user may expect inline(__always) to be as 
> effective a performance tool as @transparent, while preserving debug line 
> locations.
> If it runs late, it doesn’t expose capture promotion, box-to-stack, 
> guaranteed ARC optimization, etc.
> 
>>> If someone is using inline(__always) for -Onone performance reasons, I 
>>> think they really need to use @transparent.
>>> 
>>> An alternative is to have @inline(__always) be another way to spell 
>>> @transparent. People would need to implicitly know that this would 
>>> affect diagnostics of the inlined method.
>>> 
>>> Any thoughts?
>> 
>> `@transparent` doesn't just mean "inline-at-Onone", it also allows 
>> diagnostic passes to constant-fold through the implementation and 
>> actively eliminates debug information. It was never intended to be used 
>> for performance, but only for the limited set of things that need it. I 
>> imagine that the specialization you get by inlining some things is 
>> pretty important by itself. I suspect closure inlining would also be 
>> helpful for a lot of low-level "with*" combinators; doesn't transparent 
>> inlining do that, and could inline-always-inlining do the same?
> 
> For reference: 
> https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst
 
 The only remaining question: is it ok for a non-transparent function to be 
 inlined prior to running some dataflow diagnostics?
>>> 
>>> Doing so changes the set of programs that are accepted, since it may expose 
>>> more code to diagnostic passes. We don't want diagnostics to depend on 
>>> optimizations.
>> 
>> Perhaps we could avoid this by changing the diagnostic passes to 
>> intentionally ignore instructions with inlined source locations, though that 
>> seems brittle.
>> 
>> -Joe
> 
> Well, inline(__always) would be mandatory, not an optimization. But I think 
> your point is that the user does not expect that attribute to affect program 
> legality. Having data-flow diagnostics “ignore” certain instructions sounds 
> horrible. Is there an obvious problem, or is this hypothetical? i.e. is there 
> some diagnostic that is currently enforced at the function boundary but would 
> be considered legal code after inlining?

An example of something that depends on dataflow diagnostics would be something 
that's constant-foldable causing static overflow diagnostics:

func double(_ x: UInt8) -> UInt8 {
  return x + x
}

_ = double(128)

Making `double` transparent causes the compiler to statically reject the call:

/Users/jgroff/bar.swift:6:5: error: arithmetic operation '128 + 128' (on 
unsigned 8-bit integer type) results in an overflow
_ = double(128)
^

-Joe___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] No commit emails

[Kenny Leung via swift-dev]

Hi All.

Just a heads up.

We got a big bolus on November 18, and then nothing after that.

Thanks.

-Kenny


> On Nov 18, 2016, at 4:18 PM, Jordan Rose  wrote:
> 
> Thanks for the heads-up. I think our old notifier system is finally invalid 
> after a network reconfiguration. We've been meaning to move it to be just 
> another job on ci.swift.org  but haven't gotten around 
> to it yet.
> 
> Most of Apple is on holiday for American Thanksgiving all next week, but 
> we'll look into it after that. I know that's a long gap; we just didn't get 
> the chance to look into it before the end of the week. Sorry for the 
> inconvenience (and the missed commits).
> 
> Jordan
> 
>> On Nov 17, 2016, at 9:23, Kenny Leung via swift-dev > > wrote:
>> 
>> Hi There.
>> 
>> It looks like the conduit for commit emails has stopped working since Nov. 
>> 13. Can someone take a look please?
>> 
>> Thanks.
>> 
>> -Kenny
>> 
>> ___
>> swift-dev mailing list
>> swift-dev@swift.org 
>> https://lists.swift.org/mailman/listinfo/swift-dev
> 

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -Onone @inline(__always)?

[Andrew Trick via swift-dev]

> On Dec 1, 2016, at 11:48 AM, Andrew Trick  wrote:
> 
> Responding on swift-dev to what turned into an interesting discussion…
> 
>> On Dec 1, 2016, at 11:19 AM, Joe Groff > > wrote:
>> 
>>> 
>>> On Dec 1, 2016, at 11:18 AM, Joe Groff >> > wrote:
>>> 
 
 On Dec 1, 2016, at 11:16 AM, Andrew Trick >>> > wrote:
 
 
> On Dec 1, 2016, at 10:34 AM, Jordan Rose  > wrote:
> 
>> 
>> On Nov 30, 2016, at 18:16, Joe Groff > > wrote:
>> 
>>> 
>>> On Nov 30, 2016, at 6:10 PM, Andrew Trick >> > wrote:
>>> 
>>> We definitely want an inline-at-Onone, but I assumed we were treating 
>>> "@inline(__always)" as a normal heuristic because:
>>> - @transparent already does inline-at-Onone.
>>> - @inline(__always) without @transparent negatively affects the debug 
>>> experience.
>>> - @inline(__always) isn't useful if no optimizations are run after 
>>> inlining.
>>> 
>>> That said, if the common expectation is for @inline(__always) to kick 
>>> in independent of optimization, then we should honor that.
>>> 
>>> It's easy for me to add an inlining pass to runSILPassesForOnone. I'm 
>>> just not sure that will have the effect people are looking for. It's 
>>> unlikely to help performance since nothing is going to run after the 
>>> inlining (no capture promotion or alloc box-to-stack). The only 
>>> optimization we currently run at Onone after diagnostics is 
>>> prespecialization.
> 
> Actually, the inliner specializes as it goes, so inlining after mandatory SIL 
> passes would still be quite useful.
> 
> My concern is really that the user may expect inline(__always) to be as 
> effective a performance tool as @transparent, while preserving debug line 
> locations.
> If it runs late, it doesn’t expose capture promotion, box-to-stack, 
> guaranteed ARC optimization, etc.
> 
>>> If someone is using inline(__always) for -Onone performance reasons, I 
>>> think they really need to use @transparent.
>>> 
>>> An alternative is to have @inline(__always) be another way to spell 
>>> @transparent. People would need to implicitly know that this would 
>>> affect diagnostics of the inlined method.
>>> 
>>> Any thoughts?
>> 
>> `@transparent` doesn't just mean "inline-at-Onone", it also allows 
>> diagnostic passes to constant-fold through the implementation and 
>> actively eliminates debug information. It was never intended to be used 
>> for performance, but only for the limited set of things that need it. I 
>> imagine that the specialization you get by inlining some things is 
>> pretty important by itself. I suspect closure inlining would also be 
>> helpful for a lot of low-level "with*" combinators; doesn't transparent 
>> inlining do that, and could inline-always-inlining do the same?
> 
> For reference: 
> https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst 
> 
 
 The only remaining question: is it ok for a non-transparent function to be 
 inlined prior to running some dataflow diagnostics?
>>> 
>>> Doing so changes the set of programs that are accepted, since it may expose 
>>> more code to diagnostic passes. We don't want diagnostics to depend on 
>>> optimizations.
>> 
>> Perhaps we could avoid this by changing the diagnostic passes to 
>> intentionally ignore instructions with inlined source locations, though that 
>> seems brittle.
>> 
>> -Joe
> 
> Well, inline(__always) would be mandatory, not an optimization. But I think 
> your point is that the user does not expect that attribute to affect program 
> legality. Having data-flow diagnostics “ignore” certain instructions sounds 
> horrible. Is there an obvious problem, or is this hypothetical? i.e. is there 
> some diagnostic that is currently enforced at the function boundary but would 
> be considered legal code after inlining?
> 
> -Andy

I guess initialization of @inout arguments would be the obvious problem.
-Andy___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] -whole-module-optimization with -Onone

[Ben Asher via swift-dev]

Hello! Someone recently tipped me off to using -whole-module-optimization
flag with -Onone for use during debug builds to speed up compile times. In
our project, the speedup feels quite dramatic, but when it gets to the
linking step (after compiling both Swift and Obj-C in the project) it fails
because ld can't find the individual object files that normally get emitted
during the debug-type build presumably because -whole-module-optimization
only emits one (and this isn't a normal "-Owholemodule"-type build which
works fine).

I can't seem to reproduce this outside of Xcode, but I was curious if
anyone has tried this and knows of a workaround to get
-whole-module-optimization to work with -Onone in Xcode?

I'm currently using Xcode 8.1 (App Store build) and Swift 3 on macOS Sierra.

Thanks!

Ben
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] -Onone @inline(__always)?

[Andrew Trick via swift-dev]

Responding on swift-dev to what turned into an interesting discussion…

> On Dec 1, 2016, at 11:19 AM, Joe Groff  wrote:
> 
>> 
>> On Dec 1, 2016, at 11:18 AM, Joe Groff  wrote:
>> 
>>> 
>>> On Dec 1, 2016, at 11:16 AM, Andrew Trick  wrote:
>>> 
>>> 
 On Dec 1, 2016, at 10:34 AM, Jordan Rose  wrote:
 
> 
> On Nov 30, 2016, at 18:16, Joe Groff  wrote:
> 
>> 
>> On Nov 30, 2016, at 6:10 PM, Andrew Trick  wrote:
>> 
>> We definitely want an inline-at-Onone, but I assumed we were treating 
>> "@inline(__always)" as a normal heuristic because:
>> - @transparent already does inline-at-Onone.
>> - @inline(__always) without @transparent negatively affects the debug 
>> experience.
>> - @inline(__always) isn't useful if no optimizations are run after 
>> inlining.
>> 
>> That said, if the common expectation is for @inline(__always) to kick in 
>> independent of optimization, then we should honor that.
>> 
>> It's easy for me to add an inlining pass to runSILPassesForOnone. I'm 
>> just not sure that will have the effect people are looking for. It's 
>> unlikely to help performance since nothing is going to run after the 
>> inlining (no capture promotion or alloc box-to-stack). The only 
>> optimization we currently run at Onone after diagnostics is 
>> prespecialization.

Actually, the inliner specializes as it goes, so inlining after mandatory SIL 
passes would still be quite useful.

My concern is really that the user may expect inline(__always) to be as 
effective a performance tool as @transparent, while preserving debug line 
locations.
If it runs late, it doesn’t expose capture promotion, box-to-stack, guaranteed 
ARC optimization, etc.

>> If someone is using inline(__always) for -Onone performance reasons, I 
>> think they really need to use @transparent.
>> 
>> An alternative is to have @inline(__always) be another way to spell 
>> @transparent. People would need to implicitly know that this would 
>> affect diagnostics of the inlined method.
>> 
>> Any thoughts?
> 
> `@transparent` doesn't just mean "inline-at-Onone", it also allows 
> diagnostic passes to constant-fold through the implementation and 
> actively eliminates debug information. It was never intended to be used 
> for performance, but only for the limited set of things that need it. I 
> imagine that the specialization you get by inlining some things is pretty 
> important by itself. I suspect closure inlining would also be helpful for 
> a lot of low-level "with*" combinators; doesn't transparent inlining do 
> that, and could inline-always-inlining do the same?
 
 For reference: 
 https://github.com/apple/swift/blob/master/docs/TransparentAttr.rst
>>> 
>>> The only remaining question: is it ok for a non-transparent function to be 
>>> inlined prior to running some dataflow diagnostics?
>> 
>> Doing so changes the set of programs that are accepted, since it may expose 
>> more code to diagnostic passes. We don't want diagnostics to depend on 
>> optimizations.
> 
> Perhaps we could avoid this by changing the diagnostic passes to 
> intentionally ignore instructions with inlined source locations, though that 
> seems brittle.
> 
> -Joe

Well, inline(__always) would be mandatory, not an optimization. But I think 
your point is that the user does not expect that attribute to affect program 
legality. Having data-flow diagnostics “ignore” certain instructions sounds 
horrible. Is there an obvious problem, or is this hypothetical? i.e. is there 
some diagnostic that is currently enforced at the function boundary but would 
be considered legal code after inlining?

-Andy


___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Questions about ARC

[Roman Levenstein via swift-dev]

> On Nov 30, 2016, at 9:40 PM, Jiho Choi  wrote:
> 
> Thanks for providing the pointer.
> Do you have any preliminary result or goal (e.g. the replacement ratio) of 
> the optimization? 
> Is it going to replace all ARC operations with non-atomic ones for 
> single-threaded applications?

In the ideal world, it would be nice to replace all ARC operations with 
non-atomic ones for single-threaded applications. 

But in reality, it is way more difficult as it may seem at the first glance. 

If this needs to happen without any hints from the developer, just by means of 
a static analysis of a program, then it is rather difficult. The main problem 
is that the compiler needs to reason whether a given reference may escape to 
another thread. For references created inside a function, we have rather good 
chances to figure out if a reference escapes the thread. But if the origin 
(i.e. how it was created or if it has escaped before) of a given reference is 
unknown, which is a typical case with function parameters or references inside 
class instances, then the compiler has to assume that any such reference has 
escaped its original thread and thus it needs to use atomic ARC-operations. 
Some sort of a global, whole-module/whole-program analysis may help here 
somewhat. But even if we would introduce such kind of analysis, it is likely to 
remain a problem for dynamic libraries and frameworks, because they don’t know 
and cannot reason which parameters required by their exposed APIs escaped in 
the user-code.

Alternatively , a developer could provide a hint and assure that compiler that 
the app is single-threaded. One simple possibility could be to have a special 
-single-threaded compiler option, which would basically claim that the app 
being developed is single threaded and thus there is no need for performing the 
atomic ARC operations. In this case, all ARC operations would be marked 
non-atomic by default in the code emitted from the user-code. The problem with 
this option could be that if a user app starts multiple threads directly or 
indirectly (e.g. it calls a library API, which starts a new thread), even 
though the option claimed the app would not do it,  and some references will be 
shared between threads, then the execution of such an app may become 
unpredictable and end up with hard to find crashes. Mixing object files and 
libraries where a subset is compiled with this option and another part without 
is another receipt for a disaster. So, one would need to be extremely cautious 
when using this option. 

There could be also something in between, where one would use special 
attributes indicating something related to thread-safety of a given 
reference/type/function/etc. These hints could help a compiler to reason about 
references and check if they may escape to a different thread.

-Roman

> 
> On Wed, Nov 30, 2016 at 8:50 PM Roman Levenstein  > wrote:
>> On Nov 30, 2016, at 6:25 PM, Jiho Choi via swift-dev > > wrote:
>> 
>> Thanks for clarifications.  I have a couple of follow-up questions.
>> 
>> 1. Could you please provide more information (e.g. source code location) 
>> about the optimization applying non-atomic reference counting?  What's the 
>> scope of the optimization?  Is it method-based?
> 
> The optimization itself is not merged yet. But all the required machinery, 
> e.g. non-atomic versions of the ARC operations, special non-atomic flag on 
> SIL instructions, etc is in place already.
> 
> As for the prototype implementation, you can find it here, on my local branch:
> https://github.com/swiftix/swift/blob/30409865ff49a4268363cd359f82f29c9a90cce8/lib/SILOptimizer/Transforms/NonAtomicRC.cpp
>  
> 
> 
>> 
>> 2. Looking at the source code, I assume Swift implements immediate reference 
>> counting (i.e. immediate reclamation of dead objects)  without requiring 
>> explicit garbage collection phase for techniques, such as deferred reference 
>> counting or coalescing multiple updates.  Is it right?  If so, is there any 
>> plan to implement such techniques?
> 
> Yes. It is a correct understanding. 
> Different extensions like deferred reference counting were discussed, but 
> there are no short-term plans to implement it anytime soon.
> 
> -Roman
> 
> 
>> 
>> On Wed, Nov 30, 2016 at 11:41 AM John McCall > > wrote:
>>> On Nov 30, 2016, at 8:33 AM, Jiho Choi via swift-dev >> > wrote:
>>> Hi,
>>> 
>>> I am new to Swift, and I have several questions about how ARC works in 
>>> Swift.
>>> 
>>> 1. I read from one of the previous discussions in the swift-evolution list 
>>> (https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160208/009422.html
>>>  
>>> )
>>> 

Re: [swift-dev] COW for non-mutating methods

[Joe Groff via swift-dev]

> On Dec 1, 2016, at 3:29 AM, Patrick Pijnappel via swift-dev 
>  wrote:
> 
> I'm implementing a COW big int type but am running into problems with 
> non-mutating functions (e.g. the + operator). Simplified example code below 
> shows AFAIK the default way to implement COW, but the non-mutating method 
> doesn't identify the reference as unique (even with -O), resulting in a 
> needless copy.
> 
> I've tried everything I could think of, but only inout parameters seem to 
> work. How does the standard library do this, for e.g. String + String?

> 
> struct Foo {
>   var storage = Storage()
> 
>   class Storage { var x = 0 }
> 
>   init(_ x: Int) { storage.x = x }
> 
>   mutating func negate() {
> if !isKnownUniquelyReferenced(&storage) {
>   print("Copy")
> }
> storage.x = -storage.x
>   }
> 
>   func negated() -> Foo {
> var result = self // This counts as a second reference
> result.negate()
> return result
>   }
> }
> 
> func test() {
>   var a = Foo(5)
>   a.negate()
>   print(a.storage.x)
> 
>   let b = Foo(5)
>   let c = b.negated()
>   print(c.storage.x)
> }
> 
> test()

Unfortunately, the compiler currently always passes the 'self' parameter of a 
nonmutating method with a caller-release convention, meaning that any local 
mutable copy necessarily needs to retain a local copy and can't consume the 
incoming 'self' value, and there's currently no way to override this. 
Theoretically, defining `negated` as a free function should enable this 
optimization, since non-'self' parameters are passed callee-release and can be 
consumed:

func negate(foo: Foo) -> Foo {
  var result = foo
  result.negate()
  return result
}

though from what I've seen, the ARC optimizer doesn't always successfully 
shorten the lifetime of parameters enough to turn the "result = foo" into a 
move, which is a known bug.

-Joe
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] [Swift CI] Build Failure: 0. OSS - Swift Incremental RA - Ubuntu 16.10 (master) #695

[Matthias Braun via swift-dev]

That was me changing an llvm API, I added a compatibility cludge in 
209698dc097f5c1836445f778992f2e5f6cff404 that should unbreak the build (and not 
regress anything).

I'll create a pull request with a proper fix for swift.

- Matthias

> On Nov 30, 2016, at 6:37 PM, no-re...@swift.org wrote:
> 
> [FAILURE] oss-swift-incremental-RA-linux-ubuntu-16_10 [#695]
> 
> Build URL:
> https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_10/695/ 
> 
> Project:  oss-swift-incremental-RA-linux-ubuntu-16_10
> Date of build:Wed, 30 Nov 2016 18:35:50 -0800
> Build duration:   1 min 41 sec
> Identified problems:
> 
> Compile Error: This build failed because of a compile error. Below is a list 
> of all errors in the build log:
> Indication 1 
> 
> Tests:
> 
> Name: Swift(linux-x86_64)
> Failed: 0 test(s), Passed: 8605 test(s), Total: 8605 test(s)
> Name: Swift-Unit
> Failed: 0 test(s), Passed: 299 test(s), Total: 299 test(s)
> 
> Changes
> 
> Commit 251cd0a7ece66fa56743510078b792e81e8113fc by mbraun:
> Timer: Fix doxygen comments, use member initializer; NFC
> 
> edit: include/llvm/Support/Timer.h
> edit: lib/Support/Timer.cpp
> 
> Commit 391ecfaa831fd439f0e00dc76cd7b9ab99173093 by mbraun:
> Timer: Remove group-less NamedRegionTimer constructor.
> 
> edit: lib/Support/Timer.cpp
> edit: include/llvm/Support/Timer.h
> edit: lib/Target/Hexagon/HexagonGenInsert.cpp
> 
> Commit 22c50ae59d8e0c385f29be2d30d03fbe7af76c61 by mbraun:
> Timer: Track name and description.
> 
> edit: lib/CodeGen/AsmPrinter/AsmPrinter.cpp
> edit: lib/IR/LegacyPassManager.cpp
> edit: docs/ReleaseNotes.rst
> edit: lib/Target/Hexagon/RDFGraph.h
> edit: lib/CodeGen/RegAllocBase.h
> edit: utils/yaml-bench/YAMLBench.cpp
> edit: lib/IRReader/IRReader.cpp
> edit: lib/Support/Timer.cpp
> edit: lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
> edit: lib/Target/Hexagon/HexagonGenInsert.cpp
> edit: include/llvm/CodeGen/AsmPrinter.h
> edit: lib/Target/Hexagon/RDFGraph.cpp
> edit: unittests/Support/TimerTest.cpp
> edit: lib/CodeGen/AsmPrinter/DwarfDebug.cpp
> edit: lib/CodeGen/RegAllocGreedy.cpp
> edit: include/llvm/Support/Timer.h
> edit: lib/CodeGen/RegAllocBase.cpp
> 
> Commit 0fa6ea415350418c29a38b1970f7123824fb1aba by mbraun:
> Adapt to llvm NamedRegionTimer changes
> 
> edit: lib/CodeGen/CodeGenAction.cpp
> edit: lib/Frontend/CompilerInstance.cpp
> edit: lib/CodeGen/BackendUtil.cpp
> edit: lib/StaticAnalyzer/Frontend/AnalysisConsumer.cpp

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Re: [swift-dev] Automatically building SourceKit on Linux

[Alex Blewitt via swift-dev]

So the 5861/5862 have been merged now, which means that we can build SourceKit 
on Linux with:

swift/utils/build-script -R -T --libdispatch -- --reconfigure 
swift/utils/build-script -R -T --libdispatch 
--extra-cmake-options="-DSWIFT_BUILD_SOURCEKIT:BOOL=TRUE" -- --reconfigure 

I'd like to get SourceKit building by default on Linux in a single pass, so 
that things like PR tests will compile and run the SourceKit code on Linux and 
make it easier for downstream to depend on the SourceKit library.

I have updated 5903 which triggers an explicit 'make' call to build 
libdispatch. It's probably the wrong thing to do but it works. However the 
right approach would be to make the native libdispatch library (not the Swift 
part) an explicit component upon which the SourceKit could depend on Linux. Is 
the right approach to configure an external cmake dependency in this way, or is 
there a preferred alternative? If the solution in 5903 is the right way of 
doing it, what else needs to be passed out to the recursive make call?

Alex

> On 22 Nov 2016, at 18:05, Alex Blewitt via swift-dev  
> wrote:
> 
> I have created a few pull requests regarding SourceKit support on Linux:
> 
> https://github.com/apple/swift/pull/5861 
>  - Update documentation for 
> building on Linux
> https://github.com/apple/swift/pull/5862 
>  - Allow SourceKit to be built 
> without errors on Linux
> https://github.com/apple/swift/pull/5903 
>  - Allow SourceKit to be built by 
> default on Linux (contains 5862)
> 
> Because SourceKit isn't currently built on Linux as part of the build 
> process, it was missing dependencies on lto and coverage, both of which are 
> needed to build successfully.
> 
> SourceKit depends on the native libdispatch implementation, which on Darwin 
> is satisfied by the current platform. On Linux, the approach has been to 
> build a two-pass of swift; first without the SourceKit support, and then with 
> the SourceKit support. The second time around, the SourceKit build can pick 
> up the libdispatch library from the previous build.
> 
> The fix I've implemented in pull 5903 is to add a dummy dependency when 
> building SourceKit and on Linux that requires the libdispatch library exists, 
> and if it doesn't, to shell out to make it. It's ugly, but it works, and 
> SourceKit can be built in a single pass (though it still compiles libdispatch 
> twice). This would allow SourceKit to be used on Linux for other tools to 
> take advantage of.
> 
> More strategic solutions might look like:
> 
> * Refactoring the libdispatch build to use CMake - possibly unlikely, since 
> the libdispatch project has dependencies on libkqueue and libpwq, both of 
> which are external projects that use autoconf.
> * Splitting out SourceKit to its own repository/project, and then 
> interleaving the dependencies such that SourceKit could depend on the 
> libdispatch project on Linux and not on Darwin.
> * Changing the build script such that the libdispatch native library is built 
> first, prior to the main Swift project, but build the libdispatch Swift code 
> subsequently (potentially in a single repository, but may need two steps)
> 
> Since these alternatives require more work, and may not be applicable at the 
> current time, I've taken the approach of creating the simplest thing that 
> works (albeit unashamedly ugly) for pull 5903.
> 
> Comments welcome.
> 
> Alex
> ___
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

[swift-dev] COW for non-mutating methods

[Patrick Pijnappel via swift-dev]

I'm implementing a COW big int type but am running into problems with
non-mutating functions (e.g. the + operator). Simplified example code below
shows AFAIK the default way to implement COW, but the non-mutating method
doesn't identify the reference as unique (even with -O), resulting in a
needless copy.

I've tried everything I could think of, but only inout parameters seem to
work. How does the standard library do this, for e.g. String + String?


struct Foo {

  var storage = Storage()


  class Storage { var x = 0 }


  init(_ x: Int) { storage.x = x }


  mutating func negate() {

if !isKnownUniquelyReferenced(&storage) {

  print("Copy")

}

storage.x = -storage.x

  }


  func negated() -> Foo {

var result = self // This counts as a second reference

result.negate()

return result

  }

}


func test() {

  var a = Foo(5)

  a.negate()

  print(a.storage.x)


  let b = Foo(5)

  let c = b.negated()

  print(c.storage.x)

}


test()



*Output*

-5

Copy

-5
___
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev