Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Rick Ballard via swift-evolution]

+ swift-build-dev

To take a step back here, there are two problems I see in this space:

1) Building unnecessary test-only stuff.

Say you have a package A which depends on a package B. B might declare a 
library, a test module, and some auxiliary target that's only needed by the 
tests. Today, if you check out A and run `swift build`, it will build B's 
auxiliary target (among other things built). That's unnecessary, and it would 
be nice to avoid that.

The product definitions proposal will fix this. Once that's implemented, B 
would publish its library as its only product, and A would declare that it 
depends on that product. Then, when building A, we would know that we don't 
need the auxiliary target from B, and wouldn't build it. So I think the 
products proposal will solve this problem.

2) Cloning unnecessary test-only stuff.

Say you have a package A which depends on a package B, and B has a test module 
which depends on a package C. Once the products proposal has been implemented 
and you adopt it, running `swift build` on A will be smart enough not to build 
the test module from B or any of the targets from C; however, SwiftPM will 
still clone the C repository when doing dependency resolution, even though it 
never winds up building its products.

I don't personally see that as a major problem, but it is one that would be 
nice to address at some point. Ways we could address this include:

– We could add a "testDependencies" property on the package. These dependencies 
would only be resolved and cloned in situations where one or more test targets 
from that package need to be built. Some problems with this would be:

* Since these dependencies would only conditionally resolve, but 
otherwise would affect dependency resolution as normal, this could lead to some 
really unexpected emergent behavior. For example, if a package that's only 
included in the graph via testDependencies happens to depend on another package 
that's otherwise already in the graph, but with a more restrictive version 
specification, then whether or not you're building tests would force that other 
package to re-resolve with a different version.

* This is a non-generalized solution to a general problem. That is, 
once we have the products proposal implemented, you could wind up in this 
situation with no tests involved, but this mechanism wouldn't solve the problem 
in that case. To give a concrete example, if A depends on B, B vends two 
products, and only one of them depends on C, it's unnecessary to clone C unless 
the product that needs it is used by A. A properly general mechanism would let 
you make that dependency conditional on whether you need it for whatever 
reason, and not tie it to testing in particular. Another reason you might have 
this problem is if you have a dependency that only applies on one platform 
(e.g. linux but not macOS). (Conditional compilation blocks can be used for the 
platform case, but – once we have a better mechanism – I don't think that they 
should be used, for reasons that would be best discussed in its own thread).

* Adding additional properties to the Package should be done carefully. 
Every new property is new API surface area, and makes our documentation, 
initializer, etc more complicated. In my opinion, this new property doesn't 
justify that cost if it's just saving you from initial cloning of the extra 
packages.

– We could add in-line dependencies for targets, as Ankit suggested. This is a 
more generalizable mechanism than "testDependencies". That said, it could 
potentially require you to duplicate your dependency declaration, which is bad. 
(E.g. if two test targets in your package depend on the same auxiliary package, 
that dependency would need to be stated twice). It also adds unfortunate 
complexity.

– We could add a general mechanism for declaring dependencies as conditional, 
supporting a variety of possible conditions. Some of the SwiftPM developers are 
in the early discussion stages for a proposal for how to to make various parts 
of a manifest conditional, as part of a larger conversation on build settings; 
we'll bring our thoughts to this list once we have something coherent to 
propose. But if we come up with a general syntax for making things like 
settings conditional, the same consistent syntax could potentially be applied 
to dependencies as well.

- - -

Personally, I think that the products proposal solves the first problem nicely, 
and that we should revisit the second problem as part of an upcoming 
conversation about conditionals in the manifest.

- Rick

> On Jan 30, 2017, at 11:55 AM, Robert Widmann via swift-evolution 
>  wrote:
> 
> 
>> On Jan 30, 2017, at 2:38 PM, Ankit Agarwal > > wrote:
>> 
>> 
>> Not in practice (with respect to package manifests). In fact, it seems that, 
>> given there are separate commands (swift build and swift test), 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Robert Widmann via swift-evolution]

> On Jan 30, 2017, at 2:38 PM, Ankit Agarwal  wrote:
> 
> 
> Not in practice (with respect to package manifests). In fact, it seems that, 
> given there are separate commands (swift build and swift test), separate 
> directories (Sources and Tests), and separate products, that there's a hole 
> to be filled here by separate handling for test suites in package manifests.
> 
> FWIW, overriding conventions will potentially allow you control the layout of 
> the package (when we have that feature).

In that case, as long as there are plans for this kind of feature, I can hold 
off on submitting this proposal until it materializes.

>  
>  
> This is all not to discount the features you've brought up as well, but I'm 
> having trouble seeing why a distinction here is such a problem.
> 
> Could you elaborate on what kind of distinction are you proposing? Is it 
> separating targets and testTargets?

Yes.

> 
> -- 
> Ankit
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Ankit Agarwal via swift-evolution]

> Not in practice (with respect to package manifests). In fact, it seems
> that, given there are separate commands (swift build and swift test),
> separate directories (Sources and Tests), and separate products, that
> there's a hole to be filled here by separate handling for test suites in
> package manifests.
>

FWIW, overriding conventions will potentially allow you control the layout
of the package (when we have that feature).


>
>
This is all not to discount the features you've brought up as well, but I'm
> having trouble seeing why a distinction here is such a problem.
>

Could you elaborate on what kind of distinction are you proposing? Is it
separating targets and testTargets?

-- 
Ankit
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Robert Widmann via swift-evolution]

~Robert Widmann

2017/01/26 4:11、Ankit Agarwal  のメッセージ:

> 
> 
>> On Thu, Jan 26, 2017 at 1:55 PM, Robert Widmann via swift-evolution 
>>  wrote:
>> Three parts for three points:
>> 
>> 1) Thanks!
>> 
>> 2) I'm not sure generalizing this is particularly useful post-products.  The 
>> larger point is not to have non-exported/local target dependencies but to 
>> make clear the divide between a package and its test suite in the manifest 
>> as well as on disk.  Local dependencies seem like a separate proposal 
>> entirely.
> 
> We do have a clear divide between normal modules and test modules. 
> I am not sure adding a special case test dependencies makes sense right now. 
> We might end up having conditional dependencies later w.r.t platform and 
> other parameters.
> 

Not in practice (with respect to package manifests). In fact, it seems that, 
given there are separate commands (swift build and swift test), separate 
directories (Sources and Tests), and separate products, that there's a hole to 
be filled here by separate handling for test suites in package manifests.

This is all not to discount the features you've brought up as well, but I'm 
having trouble seeing why a distinction here is such a problem.

> Daniel, do you want to pitch in? 
> 
>> 3) Wow, I honestly should have noticed this: I took the time to peruse the 
>> source for the help text and still managed to miss it.  It has been struck 
>> from the gist of this proposal.
>> 
>> ~Robert Widmann
>> 
>> 2017/01/25 3:07、Ankit Aggarwal  のメッセージ:
>> 
>>> 
 On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution 
  wrote:
 
 Hello Swift Community,
 
 Harlan Haskins and I have been working on libraries to make interacting 
 with LLVM and Clang’s APIs more elegant with native Swift interfaces.  
 While writing up the packages we realized the package manager wouldn’t 
 allow us to specify testing targets and test-only dependencies.  To 
 rectify that, I have attached a draft proposal for adding test-only 
 targets and dependency fields to the Swift Package manager.  This proposal 
 can also be read in gist form.
 
 Cheers,
 
 ~Robert Widmann
 
>>> 
>>> Thanks for driving this! It is a very desirable feature which needs 
>>> proposal work. Comments inline.
>>> 
>>> 
>>> 
 Test-Only Package Dependencies and Targets
 Proposal: SE-
 Authors: Harlan Haskins, Robert Widmann
 Review Manager: TBD
 Status: Awaiting review
 Introduction
 
 This proposal reinstates Swift package manager’s ability to fetch 
 dependencies and build targets scoped exclusively to the testing module(s) 
 of a given package.
 
 Swift-evolution thread: Discussion thread topic for that proposal
 
 Motivation
 
 Soon after SE-0019 identified the need for richer test-only dependencies 
 and targets, a decision was made to remove the package manager’s fledgling 
 ability to treat certain dependencies as test-only. This has led to a 
 myriad of clever-but-needlessly-complex workarounds ([1], [2], [3]) on the 
 part of 3rd parties to recover the feature themselves. In addition, the 
 Swift community has come up with a number of their own frameworks to 
 augment functionality in XCTest but depending on these external testing 
 frameworks is brittle and difficult to get right.
 
 Proposed solution
 
 We propose the re-introduction of the testDependencies parameter in 
 Package Manifests to support external test-only dependencies. To support 
 local test-only targets we also propose the introduction of the 
 testTargets parameter and an extension of the existing swift test command 
 to support individual invocation of these targets.
 
 Detailed design
 
 The behavior of the new testDependencies parameter mirrors that of the 
 existing dependencies parameter with one important difference: fetched 
 dependencies are only built to support package-defined test targets as 
 part of an invocation of swift test.
 
 import PackageDescription
 
 let package = Package(
 name: "Foo",
 targets: [
 Target(name: "Foo")
 ],
 dependencies: [
 .Package(url: "https://github.com/org/ana.git;, versions: 
 Version(1,0,0)...Version(1,9,9)),
 ],
 testDependencies: [
 .Package(url: "https://github.com/org/anism.git;, versions: 
 Version(1,0,0)...Version(1,9,9)),
 ]
 )
>>> 
>>> I think this feature should be called local dependencies (or maybe dev 
>>> dependencies) because it can be used for tests as well as regular targets. 
>>> As an example say you have a networking library package and you want to 
>>> create an example CLI target which uses a JSON mapper package. You 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Ankit Agarwal via swift-evolution]

On Thu, Jan 26, 2017 at 1:55 PM, Robert Widmann via swift-evolution <
swift-evolution@swift.org> wrote:

> Three parts for three points:
>
> 1) Thanks!
>
> 2) I'm not sure generalizing this is particularly useful post-products.
> The larger point is not to have non-exported/local target dependencies but
> to make clear the divide between a package and its test suite in the
> manifest as well as on disk.  Local dependencies seem like a separate
> proposal entirely.
>

We do have a clear divide between normal modules and test modules.
I am not sure adding a special case test dependencies makes sense right
now. We might end up having conditional dependencies later w.r.t platform
and other parameters.

Daniel, do you want to pitch in?

3) Wow, I honestly should have noticed this: I took the time to peruse the
> source for the help text and still managed to miss it.  It has been struck
> from the gist of this proposal.
>
> ~Robert Widmann
>
> 2017/01/25 3:07、Ankit Aggarwal  のメッセージ:
>
>
> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> Hello Swift Community,
>
> Harlan Haskins and I have been working on libraries
>  to make interacting with LLVM and Clang’s
> APIs more elegant with native Swift interfaces.  While writing up the
> packages we realized the package manager wouldn’t allow us to specify
> testing targets and test-only dependencies.  To rectify that, I have
> attached a draft proposal for adding test-only targets and dependency
> fields to the Swift Package manager.  This proposal can also be read in
> gist form
> .
>
> Cheers,
>
> ~Robert Widmann
>
>
> Thanks for driving this! It is a very desirable feature which needs
> proposal work. Comments inline.
>
>
>
> Test-Only Package Dependencies and Targets
>
>- Proposal: SE- 
>- Authors: Harlan Haskins , Robert
>Widmann 
>- Review Manager: TBD
>- Status: Awaiting review
>
>
> 
> Introduction
>
> This proposal reinstates Swift package manager’s ability to fetch
> dependencies and build targets scoped exclusively to the testing module(s)
> of a given package.
>
> Swift-evolution thread: Discussion thread topic for that proposal
> 
>
> 
> Motivation
>
> Soon after SE-0019
> 
>  identified
> the need for richer test-only dependencies and targets, a decision was made
> to remove the package manager’s fledgling ability to treat certain
> dependencies as test-only. This has led to a myriad of
> clever-but-needlessly-complex workarounds ([1]
> 
> , [2] 
> , [3] )
> on the part of 3rd parties to recover the feature themselves. In addition,
> the Swift community has come up with a number of their own frameworks to
> augment functionality in XCTest but depending on these external testing
> frameworks is brittle and difficult to get right.
>
> Proposed
> solution
>
> We propose the re-introduction of the testDependencies parameter in
> Package Manifests to support external test-only dependencies. To support
> local test-only targets we also propose the introduction of the
> testTargets parameter and an extension of the existing swift test command
> to support individual invocation of these targets.
>
> Detailed
> design
>
> The behavior of the new testDependencies parameter mirrors that of the
> existing dependencies parameter with one important difference: fetched
> dependencies are only built to support package-defined test targets as part
> of an invocation of swift test.
>
> import PackageDescription
> let package = Package(
> name: "Foo",
> targets: [
> Target(name: "Foo")
> ],
> dependencies: [
> .Package(url: "https://github.com/org/ana.git;, versions: 
> Version(1,0,0)...Version(1,9,9)),
> ],
> testDependencies: [
> .Package(url: "https://github.com/org/anism.git;, versions: 
> Version(1,0,0)...Version(1,9,9)),
> ]
> )
>
>
> I think this feature should be called local dependencies (or maybe dev
> dependencies) because it can be used for tests as well as regular targets.
> As an example say you have a networking 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Robert Widmann via swift-evolution]

Three parts for three points:

1) Thanks!

2) I'm not sure generalizing this is particularly useful post-products.  The 
larger point is not to have non-exported/local target dependencies but to make 
clear the divide between a package and its test suite in the manifest as well 
as on disk.  Local dependencies seem like a separate proposal entirely.

3) Wow, I honestly should have noticed this: I took the time to peruse the 
source for the help text and still managed to miss it.  It has been struck from 
the gist of this proposal.

~Robert Widmann

2017/01/25 3:07、Ankit Aggarwal  のメッセージ:

> 
>> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution 
>>  wrote:
>> 
>> Hello Swift Community,
>> 
>> Harlan Haskins and I have been working on libraries to make interacting with 
>> LLVM and Clang’s APIs more elegant with native Swift interfaces.  While 
>> writing up the packages we realized the package manager wouldn’t allow us to 
>> specify testing targets and test-only dependencies.  To rectify that, I have 
>> attached a draft proposal for adding test-only targets and dependency fields 
>> to the Swift Package manager.  This proposal can also be read in gist form.
>> 
>> Cheers,
>> 
>> ~Robert Widmann
>> 
> 
> Thanks for driving this! It is a very desirable feature which needs proposal 
> work. Comments inline.
> 
> 
> 
>> Test-Only Package Dependencies and Targets
>> Proposal: SE-
>> Authors: Harlan Haskins, Robert Widmann
>> Review Manager: TBD
>> Status: Awaiting review
>> Introduction
>> 
>> This proposal reinstates Swift package manager’s ability to fetch 
>> dependencies and build targets scoped exclusively to the testing module(s) 
>> of a given package.
>> 
>> Swift-evolution thread: Discussion thread topic for that proposal
>> 
>> Motivation
>> 
>> Soon after SE-0019 identified the need for richer test-only dependencies and 
>> targets, a decision was made to remove the package manager’s fledgling 
>> ability to treat certain dependencies as test-only. This has led to a myriad 
>> of clever-but-needlessly-complex workarounds ([1], [2], [3]) on the part of 
>> 3rd parties to recover the feature themselves. In addition, the Swift 
>> community has come up with a number of their own frameworks to augment 
>> functionality in XCTest but depending on these external testing frameworks 
>> is brittle and difficult to get right.
>> 
>> Proposed solution
>> 
>> We propose the re-introduction of the testDependencies parameter in Package 
>> Manifests to support external test-only dependencies. To support local 
>> test-only targets we also propose the introduction of the testTargets 
>> parameter and an extension of the existing swift test command to support 
>> individual invocation of these targets.
>> 
>> Detailed design
>> 
>> The behavior of the new testDependencies parameter mirrors that of the 
>> existing dependencies parameter with one important difference: fetched 
>> dependencies are only built to support package-defined test targets as part 
>> of an invocation of swift test.
>> 
>> import PackageDescription
>> 
>> let package = Package(
>> name: "Foo",
>> targets: [
>> Target(name: "Foo")
>> ],
>> dependencies: [
>> .Package(url: "https://github.com/org/ana.git;, versions: 
>> Version(1,0,0)...Version(1,9,9)),
>> ],
>> testDependencies: [
>> .Package(url: "https://github.com/org/anism.git;, versions: 
>> Version(1,0,0)...Version(1,9,9)),
>> ]
>> )
> 
> I think this feature should be called local dependencies (or maybe dev 
> dependencies) because it can be used for tests as well as regular targets. 
> As an example say you have a networking library package and you want to 
> create an example CLI target which uses a JSON mapper package. You wouldn't 
> want to vend the CLI tool when you act as a dependency to other packages, or 
> include the JSON mapper in your dependencies. Test dependency doesn't sound 
> right in that context.
> 
> After the product proposal is implemented, you will be able to control what 
> you vend to your clients, and we thought about adding ability to define 
> dependencies in-line with target dependencies but left it out of the proposal 
> to keep it simpler. Maybe this proposal can add that instead of a separate 
> `testDependencies` property. Consider this manifest:
> 
> let package = Package(
> name: "FooNetworking",
> targets: [
> Target(
> name: "FooNetworking"),
> Target(
> name: "FooNetworkingExample",
> dependencies: [
> "FooNetworking", 
> .package(url: "/path/to/BarJSON.git", version: "1.0.0", 
> product: "BarJSON"), // Note: This doesn't actually exists right now.
> ]),
> Target(
> name: "FooNetworkingTests",
> dependencies: [
> "FooNetworking", 
> .package(url: 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Ankit Agarwal via swift-evolution]

We don't have support for inlining package dependencies in targets (see the
comment in above manifest example). I suggested that we turn this proposal
into adding that feature.

On Thu, 26 Jan 2017 at 1:04 PM, David Hart  wrote:

> I'm confused. I've followed this from afar as I don't have much SwiftPM
> experience, but if that is true, what's the point of the proposal?
>
>
>
> > On 26 Jan 2017, at 08:12, Ankit Agarwal via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> >
>
> > The test targets are *not* compiled when you run swift build. They're
> only complied on invocation of `swift test`. If a test target depends on a
> testing library (as in my example), they will *not* be built on invocation
> of `swift build`.
>
>
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[David Hart via swift-evolution]

I'm confused. I've followed this from afar as I don't have much SwiftPM 
experience, but if that is true, what's the point of the proposal?

> On 26 Jan 2017, at 08:12, Ankit Agarwal via swift-evolution 
>  wrote:
> 
> The test targets are *not* compiled when you run swift build. They're only 
> complied on invocation of `swift test`. If a test target depends on a testing 
> library (as in my example), they will *not* be built on invocation of `swift 
> build`.

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Ankit Agarwal via swift-evolution]

On Thu, Jan 26, 2017 at 6:37 AM, thislooksfun via swift-evolution <
swift-evolution@swift.org> wrote:

> I still much prefer 'testDependencies/Targets'. You seem to be confusing
> this proposal with the (already accepted) Package Manager Product
> Definitions
> 
>  proposal.
> This proposal is strictly for running under `swift test`. The issue
> currently is that running any swift command (specifically anything that
> ends up calling `swift build`), will attempt to build testing libraries /
> targets, which then fail because they rely on XCTest, which is only
> available for import when running `swift test`. The idea here is that the
> libraries / targets will *only* be compiled when running in a testing
> environment, without having to manually switch out Package.swift files.
>
>
The test targets are *not* compiled when you run swift build. They're only
complied on invocation of `swift test`. If a test target depends on a
testing library (as in my example), they will *not* be built on invocation
of `swift build`.

Also, you can import XCTest from your regular targets but that code will
only run from a test target. (see TestSupport module in SwiftPM).



> -thislooksfun (tlf)
>
> On Jan 25, 2017, at 2:07 AM, Ankit Aggarwal via swift-evolution <
> swift-evolution@swift.org> wrote:
>
>
> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> Hello Swift Community,
>
> Harlan Haskins and I have been working on libraries
>  to make interacting with LLVM and Clang’s
> APIs more elegant with native Swift interfaces.  While writing up the
> packages we realized the package manager wouldn’t allow us to specify
> testing targets and test-only dependencies.  To rectify that, I have
> attached a draft proposal for adding test-only targets and dependency
> fields to the Swift Package manager.  This proposal can also be read in
> gist form
> .
>
> Cheers,
>
> ~Robert Widmann
>
>
> Thanks for driving this! It is a very desirable feature which needs
> proposal work. Comments inline.
>
>
>
> Test-Only Package Dependencies and Targets
>
>- Proposal: SE- 
>- Authors: Harlan Haskins , Robert
>Widmann 
>- Review Manager: TBD
>- Status: Awaiting review
>
>
> 
> Introduction
>
> This proposal reinstates Swift package manager’s ability to fetch
> dependencies and build targets scoped exclusively to the testing module(s)
> of a given package.
>
> Swift-evolution thread: Discussion thread topic for that proposal
> 
>
> 
> Motivation
>
> Soon after SE-0019
> 
>  identified
> the need for richer test-only dependencies and targets, a decision was made
> to remove the package manager’s fledgling ability to treat certain
> dependencies as test-only. This has led to a myriad of
> clever-but-needlessly-complex workarounds ([1]
> 
> , [2] 
> , [3] )
> on the part of 3rd parties to recover the feature themselves. In addition,
> the Swift community has come up with a number of their own frameworks to
> augment functionality in XCTest but depending on these external testing
> frameworks is brittle and difficult to get right.
>
> Proposed
> solution
>
> We propose the re-introduction of the testDependencies parameter in
> Package Manifests to support external test-only dependencies. To support
> local test-only targets we also propose the introduction of the
> testTargets parameter and an extension of the existing swift test command
> to support individual invocation of these targets.
>
> Detailed
> design
>
> The behavior of the new testDependencies parameter mirrors that of the
> existing dependencies parameter with one important difference: fetched
> dependencies are only built to support package-defined test targets as part
> of an invocation of swift test.
>
> import PackageDescription
> let package = Package(
> name: "Foo",
> targets: [
> Target(name: "Foo")
> ],
> dependencies: [
> .Package(url: "https://github.com/org/ana.git;, versions: 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[thislooksfun via swift-evolution]

I still much prefer 'testDependencies/Targets'. You seem to be confusing this 
proposal with the (already accepted) Package Manager Product Definitions 

 proposal.
This proposal is strictly for running under `swift test`. The issue currently 
is that running any swift command (specifically anything that ends up calling 
`swift build`), will attempt to build testing libraries / targets, which then 
fail because they rely on XCTest, which is only available for import when 
running `swift test`. The idea here is that the libraries / targets will only 
be compiled when running in a testing environment, without having to manually 
switch out Package.swift files.

-thislooksfun (tlf)

> On Jan 25, 2017, at 2:07 AM, Ankit Aggarwal via swift-evolution 
>  wrote:
> 
>> 
>> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution 
>> > wrote:
>> 
>> Hello Swift Community,
>> 
>> Harlan Haskins and I have been working on libraries 
>>  to make interacting with LLVM and Clang’s 
>> APIs more elegant with native Swift interfaces.  While writing up the 
>> packages we realized the package manager wouldn’t allow us to specify 
>> testing targets and test-only dependencies.  To rectify that, I have 
>> attached a draft proposal for adding test-only targets and dependency fields 
>> to the Swift Package manager.  This proposal can also be read in gist form 
>> .
>> 
>> Cheers,
>> 
>> ~Robert Widmann
>> 
> 
> Thanks for driving this! It is a very desirable feature which needs proposal 
> work. Comments inline.
> 
> 
> 
>> Test-Only Package Dependencies and Targets
>> 
>> Proposal: SE- 
>> Authors: Harlan Haskins , Robert Widmann 
>> 
>> Review Manager: TBD
>> Status: Awaiting review
>>  
>> Introduction
>> 
>> This proposal reinstates Swift package manager’s ability to fetch 
>> dependencies and build targets scoped exclusively to the testing module(s) 
>> of a given package.
>> 
>> Swift-evolution thread: Discussion thread topic for that proposal 
>> 
>>  
>> Motivation
>> 
>> Soon after SE-0019 
>> 
>>  identified the need for richer test-only dependencies and targets, a 
>> decision was made to remove the package manager’s fledgling ability to treat 
>> certain dependencies as test-only. This has led to a myriad of 
>> clever-but-needlessly-complex workarounds ([1] 
>> ,
>>  [2] , 
>> [3] ) on the 
>> part of 3rd parties to recover the feature themselves. In addition, the 
>> Swift community has come up with a number of their own frameworks to augment 
>> functionality in XCTest but depending on these external testing frameworks 
>> is brittle and difficult to get right.
>> 
>>  
>> Proposed
>>  solution
>> 
>> We propose the re-introduction of the testDependencies parameter in Package 
>> Manifests to support external test-only dependencies. To support local 
>> test-only targets we also propose the introduction of the testTargets 
>> parameter and an extension of the existing swift test command to support 
>> individual invocation of these targets.
>> 
>>  
>> Detailed
>>  design
>> 
>> The behavior of the new testDependencies parameter mirrors that of the 
>> existing dependencies parameter with one important difference: fetched 
>> dependencies are only built to support package-defined test targets as part 
>> of an invocation of swift test.
>> 
>> import PackageDescription
>> 
>> let package = Package(
>> name: "Foo",
>> targets: [
>> Target(name: "Foo")
>> ],
>> dependencies: [
>> .Package(url: "https://github.com/org/ana.git 
>> ", versions: 
>> Version(1,0,0)...Version(1,9,9)),
>> ],
>> testDependencies: [
>> .Package(url: "https://github.com/org/anism.git 
>> ", versions: 
>> Version(1,0,0)...Version(1,9,9)),
>> ]
>> )
> 
> I think this feature should be called local dependencies (or maybe dev 
> 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Ankit Aggarwal via swift-evolution]

> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution 
>  wrote:
> 
> Hello Swift Community,
> 
> Harlan Haskins and I have been working on libraries 
>  to make interacting with LLVM and Clang’s 
> APIs more elegant with native Swift interfaces.  While writing up the 
> packages we realized the package manager wouldn’t allow us to specify testing 
> targets and test-only dependencies.  To rectify that, I have attached a draft 
> proposal for adding test-only targets and dependency fields to the Swift 
> Package manager.  This proposal can also be read in gist form 
> .
> 
> Cheers,
> 
> ~Robert Widmann
> 

Thanks for driving this! It is a very desirable feature which needs proposal 
work. Comments inline.



> Test-Only Package Dependencies and Targets
> 
> Proposal: SE- 
> Authors: Harlan Haskins , Robert Widmann 
> 
> Review Manager: TBD
> Status: Awaiting review
>  
> Introduction
> 
> This proposal reinstates Swift package manager’s ability to fetch 
> dependencies and build targets scoped exclusively to the testing module(s) of 
> a given package.
> 
> Swift-evolution thread: Discussion thread topic for that proposal 
> 
>  
> Motivation
> 
> Soon after SE-0019 
> 
>  identified the need for richer test-only dependencies and targets, a 
> decision was made to remove the package manager’s fledgling ability to treat 
> certain dependencies as test-only. This has led to a myriad of 
> clever-but-needlessly-complex workarounds ([1] 
> , 
> [2] , [3] 
> ) on the part 
> of 3rd parties to recover the feature themselves. In addition, the Swift 
> community has come up with a number of their own frameworks to augment 
> functionality in XCTest but depending on these external testing frameworks is 
> brittle and difficult to get right.
> 
>  
> Proposed
>  solution
> 
> We propose the re-introduction of the testDependencies parameter in Package 
> Manifests to support external test-only dependencies. To support local 
> test-only targets we also propose the introduction of the testTargets 
> parameter and an extension of the existing swift test command to support 
> individual invocation of these targets.
> 
>  
> Detailed
>  design
> 
> The behavior of the new testDependencies parameter mirrors that of the 
> existing dependencies parameter with one important difference: fetched 
> dependencies are only built to support package-defined test targets as part 
> of an invocation of swift test.
> 
> import PackageDescription
> 
> let package = Package(
> name: "Foo",
> targets: [
> Target(name: "Foo")
> ],
> dependencies: [
> .Package(url: "https://github.com/org/ana.git 
> ", versions: Version(1,0,0)...Version(1,9,9)),
> ],
> testDependencies: [
> .Package(url: "https://github.com/org/anism.git 
> ", versions: 
> Version(1,0,0)...Version(1,9,9)),
> ]
> )

I think this feature should be called local dependencies (or maybe dev 
dependencies) because it can be used for tests as well as regular targets. 
As an example say you have a networking library package and you want to create 
an example CLI target which uses a JSON mapper package. You wouldn't want to 
vend the CLI tool when you act as a dependency to other packages, or include 
the JSON mapper in your dependencies. Test dependency doesn't sound right in 
that context.

After the product proposal 

 is implemented, you will be able to control what you vend to your clients, and 
we thought about adding ability to define dependencies in-line with target 
dependencies but left it out of the proposal to keep it simpler. Maybe this 
proposal can add that instead of a separate `testDependencies` property. 
Consider this manifest:

let package = Package(
name: "FooNetworking",
targets: [
Target(
name: "FooNetworking"),
Target(
name: "FooNetworkingExample",

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Robert Widmann via swift-evolution]

Excellent.  I've added your name to the authors list on the gist.  That is the 
version I'll be submitting to evolution soon. 

~Robert Widmann

2017/01/25 0:58、thislooksfun  のメッセージ:

> No problem, it's been inactive for a couple weeks since I've been really 
> busy. And yeah, yours is more thorough. I didn't even consider adding a 
> `testTargets` section. That alone is worth merging.
> 
> -thislooksfun (tlf)
> 
>> On Jan 24, 2017, at 11:55 PM, Robert Widmann  
>> wrote:
>> 
>> Oh, I'm sorry.  I didn't know there was existing work in this space.  
>> Considering this proposal looks like it expands on your earlier work, would 
>> you like to sign on to this and merge the two efforts?
>> 
>> ~Robert Widmann
>> 
>> 2017/01/25 0:47、thislooksfun  のメッセージ:
>> 
>>> As the author of the Add support for test-only dependencies thread, and the 
>>> accompanying draft, this gets a big +1 from me.
>>> 
>>> -thislooksfun (tlf)
>>> 
 On Jan 24, 2017, at 4:32 PM, Robert Widmann via swift-evolution 
  wrote:
 
 Hello Swift Community,
 
 Harlan Haskins and I have been working on libraries to make interacting 
 with LLVM and Clang’s APIs more elegant with native Swift interfaces.  
 While writing up the packages we realized the package manager wouldn’t 
 allow us to specify testing targets and test-only dependencies.  To 
 rectify that, I have attached a draft proposal for adding test-only 
 targets and dependency fields to the Swift Package manager.  This proposal 
 can also be read in gist form.
 
 Cheers,
 
 ~Robert Widmann
 
 Test-Only Package Dependencies and Targets
 Proposal: SE-
 Authors: Harlan Haskins, Robert Widmann
 Review Manager: TBD
 Status: Awaiting review
 Introduction
 
 This proposal reinstates Swift package manager’s ability to fetch 
 dependencies and build targets scoped exclusively to the testing module(s) 
 of a given package.
 
 Swift-evolution thread: Discussion thread topic for that proposal
 
 Motivation
 
 Soon after SE-0019 identified the need for richer test-only dependencies 
 and targets, a decision was made to remove the package manager’s fledgling 
 ability to treat certain dependencies as test-only. This has led to a 
 myriad of clever-but-needlessly-complex workarounds ([1], [2], [3]) on the 
 part of 3rd parties to recover the feature themselves. In addition, the 
 Swift community has come up with a number of their own frameworks to 
 augment functionality in XCTest but depending on these external testing 
 frameworks is brittle and difficult to get right.
 
 Proposed solution
 
 We propose the re-introduction of the testDependencies parameter in 
 Package Manifests to support external test-only dependencies. To support 
 local test-only targets we also propose the introduction of the 
 testTargets parameter and an extension of the existing swift test command 
 to support individual invocation of these targets.
 
 Detailed design
 
 The behavior of the new testDependencies parameter mirrors that of the 
 existing dependencies parameter with one important difference: fetched 
 dependencies are only built to support package-defined test targets as 
 part of an invocation of swift test.
 
 import PackageDescription
 
 let package = Package(
 name: "Foo",
 targets: [
 Target(name: "Foo")
 ],
 dependencies: [
 .Package(url: "https://github.com/org/ana.git;, versions: 
 Version(1,0,0)...Version(1,9,9)),
 ],
 testDependencies: [
 .Package(url: "https://github.com/org/anism.git;, versions: 
 Version(1,0,0)...Version(1,9,9)),
 ]
 )
 Similarly, the behavior of the testTargets field mirrors that of the 
 existing targets field but defines a set of targets that are only built 
 during an invocation of swift test. Importantly, a target defined in 
 testTargets may reference a target defined in targets but not vice-versa. 
 Should that behavior be needed, the test target should be promoted to a 
 “full” target.
 
 import PackageDescription
 
 let package = Package(
 name: "SwiftPM",
 targets: [
 Target(
 name: "PackageDescription",
 dependencies: []),
 
 // MARK: Support libraries
 
 Target(
 /** Cross-platform access to bare `libc` functionality. */
 name: "libc",
 dependencies: []),
 Target(
 /** “Swifty” POSIX functions from libc */
 name: "POSIX",
 dependencies: ["libc"]),
 Target(
 /** Basic 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[thislooksfun via swift-evolution]

No problem, it's been inactive for a couple weeks since I've been really busy. 
And yeah, yours is more thorough. I didn't even consider adding a `testTargets` 
section. That alone is worth merging.

-thislooksfun (tlf)

> On Jan 24, 2017, at 11:55 PM, Robert Widmann  wrote:
> 
> Oh, I'm sorry.  I didn't know there was existing work in this space.  
> Considering this proposal looks like it expands on your earlier work, would 
> you like to sign on to this and merge the two efforts?
> 
> ~Robert Widmann
> 
> 2017/01/25 0:47、thislooksfun  > のメッセージ:
> 
>> As the author of the Add support for test-only dependencies 
>> 
>>  thread, and the accompanying draft 
>> ,
>>  this gets a big +1 from me.
>> 
>> -thislooksfun (tlf)
>> 
>>> On Jan 24, 2017, at 4:32 PM, Robert Widmann via swift-evolution 
>>> > wrote:
>>> 
>>> Hello Swift Community,
>>> 
>>> Harlan Haskins and I have been working on libraries 
>>>  to make interacting with LLVM and Clang’s 
>>> APIs more elegant with native Swift interfaces.  While writing up the 
>>> packages we realized the package manager wouldn’t allow us to specify 
>>> testing targets and test-only dependencies.  To rectify that, I have 
>>> attached a draft proposal for adding test-only targets and dependency 
>>> fields to the Swift Package manager.  This proposal can also be read in 
>>> gist form .
>>> 
>>> Cheers,
>>> 
>>> ~Robert Widmann
>>> 
>>> Test-Only Package Dependencies and Targets
>>> 
>>> Proposal: SE- 
>>> Authors: Harlan Haskins , Robert Widmann 
>>> 
>>> Review Manager: TBD
>>> Status: Awaiting review
>>>  
>>> Introduction
>>> 
>>> This proposal reinstates Swift package manager’s ability to fetch 
>>> dependencies and build targets scoped exclusively to the testing module(s) 
>>> of a given package.
>>> 
>>> Swift-evolution thread: Discussion thread topic for that proposal 
>>> 
>>>  
>>> Motivation
>>> 
>>> Soon after SE-0019 
>>> 
>>>  identified the need for richer test-only dependencies and targets, a 
>>> decision was made to remove the package manager’s fledgling ability to 
>>> treat certain dependencies as test-only. This has led to a myriad of 
>>> clever-but-needlessly-complex workarounds ([1] 
>>> ,
>>>  [2] , 
>>> [3] ) on 
>>> the part of 3rd parties to recover the feature themselves. In addition, the 
>>> Swift community has come up with a number of their own frameworks to 
>>> augment functionality in XCTest but depending on these external testing 
>>> frameworks is brittle and difficult to get right.
>>> 
>>>  
>>> Proposed
>>>  solution
>>> 
>>> We propose the re-introduction of the testDependencies parameter in Package 
>>> Manifests to support external test-only dependencies. To support local 
>>> test-only targets we also propose the introduction of the testTargets 
>>> parameter and an extension of the existing swift test command to support 
>>> individual invocation of these targets.
>>> 
>>>  
>>> Detailed
>>>  design
>>> 
>>> The behavior of the new testDependencies parameter mirrors that of the 
>>> existing dependencies parameter with one important difference: fetched 
>>> dependencies are only built to support package-defined test targets as part 
>>> of an invocation of swift test.
>>> 
>>> import PackageDescription
>>> 
>>> let package = Package(
>>> name: "Foo",
>>> targets: [
>>> Target(name: "Foo")
>>> ],
>>> dependencies: [
>>> .Package(url: "https://github.com/org/ana.git 
>>> ", versions: 
>>> Version(1,0,0)...Version(1,9,9)),
>>> ],
>>> testDependencies: [
>>> .Package(url: "https://github.com/org/anism.git 
>>> ", versions: 
>>> Version(1,0,0)...Version(1,9,9)),
>>> ]
>>> )
>>> Similarly, the behavior of the 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Russ Bishop via swift-evolution]

I never agreed with removing it so this gets a bit fat +1 from me.

Russ


> On Jan 24, 2017, at 2:32 PM, Robert Widmann via swift-evolution 
>  wrote:
> 
> Hello Swift Community,
> 
> Harlan Haskins and I have been working on libraries 
>  to make interacting with LLVM and Clang’s 
> APIs more elegant with native Swift interfaces.  While writing up the 
> packages we realized the package manager wouldn’t allow us to specify testing 
> targets and test-only dependencies.  To rectify that, I have attached a draft 
> proposal for adding test-only targets and dependency fields to the Swift 
> Package manager.  This proposal can also be read in gist form 
> .
> 
> Cheers,
> 
> ~Robert Widmann
> 
> Test-Only Package Dependencies and Targets
> 
> Proposal: SE- 
> Authors: Harlan Haskins , Robert Widmann 
> 
> Review Manager: TBD
> Status: Awaiting review
>  
> Introduction
> 
> This proposal reinstates Swift package manager’s ability to fetch 
> dependencies and build targets scoped exclusively to the testing module(s) of 
> a given package.
> 
> Swift-evolution thread: Discussion thread topic for that proposal 
> 
>  
> Motivation
> 
> Soon after SE-0019 
> 
>  identified the need for richer test-only dependencies and targets, a 
> decision was made to remove the package manager’s fledgling ability to treat 
> certain dependencies as test-only. This has led to a myriad of 
> clever-but-needlessly-complex workarounds ([1] 
> , 
> [2] , [3] 
> ) on the part 
> of 3rd parties to recover the feature themselves. In addition, the Swift 
> community has come up with a number of their own frameworks to augment 
> functionality in XCTest but depending on these external testing frameworks is 
> brittle and difficult to get right.
> 
>  
> Proposed
>  solution
> 
> We propose the re-introduction of the testDependencies parameter in Package 
> Manifests to support external test-only dependencies. To support local 
> test-only targets we also propose the introduction of the testTargets 
> parameter and an extension of the existing swift test command to support 
> individual invocation of these targets.
> 
>  
> Detailed
>  design
> 
> The behavior of the new testDependencies parameter mirrors that of the 
> existing dependencies parameter with one important difference: fetched 
> dependencies are only built to support package-defined test targets as part 
> of an invocation of swift test.
> 
> import PackageDescription
> 
> let package = Package(
> name: "Foo",
> targets: [
> Target(name: "Foo")
> ],
> dependencies: [
> .Package(url: "https://github.com/org/ana.git 
> ", versions: Version(1,0,0)...Version(1,9,9)),
> ],
> testDependencies: [
> .Package(url: "https://github.com/org/anism.git 
> ", versions: 
> Version(1,0,0)...Version(1,9,9)),
> ]
> )
> Similarly, the behavior of the testTargets field mirrors that of the existing 
> targets field but defines a set of targets that are only built during an 
> invocation of swift test. Importantly, a target defined in testTargets may 
> reference a target defined in targets but not vice-versa. Should that 
> behavior be needed, the test target should be promoted to a “full” target.
> 
> import PackageDescription
> 
> let package = Package(
> name: "SwiftPM",
> targets: [
> Target(
> name: "PackageDescription",
> dependencies: []),
> 
> // MARK: Support libraries
> 
> Target(
> /** Cross-platform access to bare `libc` functionality. */
> name: "libc",
> dependencies: []),
> Target(
> /** “Swifty” POSIX functions from libc */
> name: "POSIX",
> dependencies: ["libc"]),
> Target(
> /** Basic support library */
> name: "Basic",
> dependencies: ["libc", "POSIX"]),
> 
> /* Omitted for Brevity */
>  ],
>  testTargets: [
> Target(
> name: 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Robert Widmann via swift-evolution]

Oh, I'm sorry.  I didn't know there was existing work in this space.  
Considering this proposal looks like it expands on your earlier work, would you 
like to sign on to this and merge the two efforts?

~Robert Widmann

2017/01/25 0:47、thislooksfun  のメッセージ:

> As the author of the Add support for test-only dependencies thread, and the 
> accompanying draft, this gets a big +1 from me.
> 
> -thislooksfun (tlf)
> 
>> On Jan 24, 2017, at 4:32 PM, Robert Widmann via swift-evolution 
>>  wrote:
>> 
>> Hello Swift Community,
>> 
>> Harlan Haskins and I have been working on libraries to make interacting with 
>> LLVM and Clang’s APIs more elegant with native Swift interfaces.  While 
>> writing up the packages we realized the package manager wouldn’t allow us to 
>> specify testing targets and test-only dependencies.  To rectify that, I have 
>> attached a draft proposal for adding test-only targets and dependency fields 
>> to the Swift Package manager.  This proposal can also be read in gist form.
>> 
>> Cheers,
>> 
>> ~Robert Widmann
>> 
>> Test-Only Package Dependencies and Targets
>> Proposal: SE-
>> Authors: Harlan Haskins, Robert Widmann
>> Review Manager: TBD
>> Status: Awaiting review
>> Introduction
>> 
>> This proposal reinstates Swift package manager’s ability to fetch 
>> dependencies and build targets scoped exclusively to the testing module(s) 
>> of a given package.
>> 
>> Swift-evolution thread: Discussion thread topic for that proposal
>> 
>> Motivation
>> 
>> Soon after SE-0019 identified the need for richer test-only dependencies and 
>> targets, a decision was made to remove the package manager’s fledgling 
>> ability to treat certain dependencies as test-only. This has led to a myriad 
>> of clever-but-needlessly-complex workarounds ([1], [2], [3]) on the part of 
>> 3rd parties to recover the feature themselves. In addition, the Swift 
>> community has come up with a number of their own frameworks to augment 
>> functionality in XCTest but depending on these external testing frameworks 
>> is brittle and difficult to get right.
>> 
>> Proposed solution
>> 
>> We propose the re-introduction of the testDependencies parameter in Package 
>> Manifests to support external test-only dependencies. To support local 
>> test-only targets we also propose the introduction of the testTargets 
>> parameter and an extension of the existing swift test command to support 
>> individual invocation of these targets.
>> 
>> Detailed design
>> 
>> The behavior of the new testDependencies parameter mirrors that of the 
>> existing dependencies parameter with one important difference: fetched 
>> dependencies are only built to support package-defined test targets as part 
>> of an invocation of swift test.
>> 
>> import PackageDescription
>> 
>> let package = Package(
>> name: "Foo",
>> targets: [
>> Target(name: "Foo")
>> ],
>> dependencies: [
>> .Package(url: "https://github.com/org/ana.git;, versions: 
>> Version(1,0,0)...Version(1,9,9)),
>> ],
>> testDependencies: [
>> .Package(url: "https://github.com/org/anism.git;, versions: 
>> Version(1,0,0)...Version(1,9,9)),
>> ]
>> )
>> Similarly, the behavior of the testTargets field mirrors that of the 
>> existing targets field but defines a set of targets that are only built 
>> during an invocation of swift test. Importantly, a target defined in 
>> testTargets may reference a target defined in targets but not vice-versa. 
>> Should that behavior be needed, the test target should be promoted to a 
>> “full” target.
>> 
>> import PackageDescription
>> 
>> let package = Package(
>> name: "SwiftPM",
>> targets: [
>> Target(
>> name: "PackageDescription",
>> dependencies: []),
>> 
>> // MARK: Support libraries
>> 
>> Target(
>> /** Cross-platform access to bare `libc` functionality. */
>> name: "libc",
>> dependencies: []),
>> Target(
>> /** “Swifty” POSIX functions from libc */
>> name: "POSIX",
>> dependencies: ["libc"]),
>> Target(
>> /** Basic support library */
>> name: "Basic",
>> dependencies: ["libc", "POSIX"]),
>> 
>> /* Omitted for Brevity */
>>  ],
>>  testTargets: [
>> Target(
>> name: "BasicPerformanceTests",
>> dependencies: ["Basic"]),
>>/* Omitted for Brevity */
>>  ]
>>  )
>> Finally, with well-defined test targets in hand, we propose swift test be 
>> amended to support individual test execution.
>> 
>> We propose the following syntax to execute all tests of all known test 
>> targets.
>> 
>> $ swift test
>> To run a set of specific test cases, reference the module-defining test 
>> target and the specific name of a subclass of XCTestCase:
>> 
>> $ swift test TestModule.TestCase
>> To run an individual 

Re: [swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[thislooksfun via swift-evolution]

As the author of the Add support for test-only dependencies 

 thread, and the accompanying draft 
,
 this gets a big +1 from me.

-thislooksfun (tlf)

> On Jan 24, 2017, at 4:32 PM, Robert Widmann via swift-evolution 
>  wrote:
> 
> Hello Swift Community,
> 
> Harlan Haskins and I have been working on libraries 
>  to make interacting with LLVM and Clang’s 
> APIs more elegant with native Swift interfaces.  While writing up the 
> packages we realized the package manager wouldn’t allow us to specify testing 
> targets and test-only dependencies.  To rectify that, I have attached a draft 
> proposal for adding test-only targets and dependency fields to the Swift 
> Package manager.  This proposal can also be read in gist form 
> .
> 
> Cheers,
> 
> ~Robert Widmann
> 
> Test-Only Package Dependencies and Targets
> 
> Proposal: SE- 
> Authors: Harlan Haskins , Robert Widmann 
> 
> Review Manager: TBD
> Status: Awaiting review
>  
> Introduction
> 
> This proposal reinstates Swift package manager’s ability to fetch 
> dependencies and build targets scoped exclusively to the testing module(s) of 
> a given package.
> 
> Swift-evolution thread: Discussion thread topic for that proposal 
> 
>  
> Motivation
> 
> Soon after SE-0019 
> 
>  identified the need for richer test-only dependencies and targets, a 
> decision was made to remove the package manager’s fledgling ability to treat 
> certain dependencies as test-only. This has led to a myriad of 
> clever-but-needlessly-complex workarounds ([1] 
> , 
> [2] , [3] 
> ) on the part 
> of 3rd parties to recover the feature themselves. In addition, the Swift 
> community has come up with a number of their own frameworks to augment 
> functionality in XCTest but depending on these external testing frameworks is 
> brittle and difficult to get right.
> 
>  
> Proposed
>  solution
> 
> We propose the re-introduction of the testDependencies parameter in Package 
> Manifests to support external test-only dependencies. To support local 
> test-only targets we also propose the introduction of the testTargets 
> parameter and an extension of the existing swift test command to support 
> individual invocation of these targets.
> 
>  
> Detailed
>  design
> 
> The behavior of the new testDependencies parameter mirrors that of the 
> existing dependencies parameter with one important difference: fetched 
> dependencies are only built to support package-defined test targets as part 
> of an invocation of swift test.
> 
> import PackageDescription
> 
> let package = Package(
> name: "Foo",
> targets: [
> Target(name: "Foo")
> ],
> dependencies: [
> .Package(url: "https://github.com/org/ana.git 
> ", versions: Version(1,0,0)...Version(1,9,9)),
> ],
> testDependencies: [
> .Package(url: "https://github.com/org/anism.git 
> ", versions: 
> Version(1,0,0)...Version(1,9,9)),
> ]
> )
> Similarly, the behavior of the testTargets field mirrors that of the existing 
> targets field but defines a set of targets that are only built during an 
> invocation of swift test. Importantly, a target defined in testTargets may 
> reference a target defined in targets but not vice-versa. Should that 
> behavior be needed, the test target should be promoted to a “full” target.
> 
> import PackageDescription
> 
> let package = Package(
> name: "SwiftPM",
> targets: [
> Target(
> name: "PackageDescription",
> dependencies: []),
> 
> // MARK: Support libraries
> 
> Target(
> /** Cross-platform access to bare `libc` functionality. */
> name: "libc",
> dependencies: []),
> Target(
> /** “Swifty” POSIX functions from libc */
> name: "POSIX",
> dependencies: ["libc"]),
>  

[swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Robert Widmann via swift-evolution]

Hello Swift Community,

Harlan Haskins and I have been working on libraries 
 to make interacting with LLVM and Clang’s APIs 
more elegant with native Swift interfaces.  While writing up the packages we 
realized the package manager wouldn’t allow us to specify testing targets and 
test-only dependencies.  To rectify that, I have attached a draft proposal for 
adding test-only targets and dependency fields to the Swift Package manager.  
This proposal can also be read in gist form 
.

Cheers,

~Robert Widmann

Test-Only Package Dependencies and Targets

Proposal: SE- 
Authors: Harlan Haskins , Robert Widmann 

Review Manager: TBD
Status: Awaiting review
 
Introduction

This proposal reinstates Swift package manager’s ability to fetch dependencies 
and build targets scoped exclusively to the testing module(s) of a given 
package.

Swift-evolution thread: Discussion thread topic for that proposal 

 
Motivation

Soon after SE-0019 

 identified the need for richer test-only dependencies and targets, a decision 
was made to remove the package manager’s fledgling ability to treat certain 
dependencies as test-only. This has led to a myriad of 
clever-but-needlessly-complex workarounds ([1] 
, 
[2] , [3] 
) on the part 
of 3rd parties to recover the feature themselves. In addition, the Swift 
community has come up with a number of their own frameworks to augment 
functionality in XCTest but depending on these external testing frameworks is 
brittle and difficult to get right.

 
Proposed
 solution

We propose the re-introduction of the testDependencies parameter in Package 
Manifests to support external test-only dependencies. To support local 
test-only targets we also propose the introduction of the testTargets parameter 
and an extension of the existing swift test command to support individual 
invocation of these targets.

 
Detailed
 design

The behavior of the new testDependencies parameter mirrors that of the existing 
dependencies parameter with one important difference: fetched dependencies are 
only built to support package-defined test targets as part of an invocation of 
swift test.

import PackageDescription

let package = Package(
name: "Foo",
targets: [
Target(name: "Foo")
],
dependencies: [
.Package(url: "https://github.com/org/ana.git;, versions: 
Version(1,0,0)...Version(1,9,9)),
],
testDependencies: [
.Package(url: "https://github.com/org/anism.git;, versions: 
Version(1,0,0)...Version(1,9,9)),
]
)
Similarly, the behavior of the testTargets field mirrors that of the existing 
targets field but defines a set of targets that are only built during an 
invocation of swift test. Importantly, a target defined in testTargets may 
reference a target defined in targets but not vice-versa. Should that behavior 
be needed, the test target should be promoted to a “full” target.

import PackageDescription

let package = Package(
name: "SwiftPM",
targets: [
Target(
name: "PackageDescription",
dependencies: []),

// MARK: Support libraries

Target(
/** Cross-platform access to bare `libc` functionality. */
name: "libc",
dependencies: []),
Target(
/** “Swifty” POSIX functions from libc */
name: "POSIX",
dependencies: ["libc"]),
Target(
/** Basic support library */
name: "Basic",
dependencies: ["libc", "POSIX"]),

/* Omitted for Brevity */
 ],
 testTargets: [
Target(
name: "BasicPerformanceTests",
dependencies: ["Basic"]),
   /* Omitted for Brevity */
 ]
 )
Finally, with well-defined test targets in hand, we propose swift test be 
amended to support individual test execution.

We propose the following syntax to execute all tests of all known test targets.

$ swift test
To run a set of specific test cases, reference the module-defining test target 
and the specific name of a subclass of XCTestCase:

$ swift test TestModule.TestCase
To run