I have updated the proposal with everyone’s feedback:
SwiftPM System Module Search Paths
Proposal: SE-NNNN
<https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-swiftpm-system-module-search-paths.md>
Author: Max Howell <https://github.com/mxcl>
Status: Awaiting review
Review manager: Anders Bertelrud
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#introduction>Introduction
Swift is able to import C libraries in the same manner as Swift libraries.
For this to occur the library must be represented by a clang module-map file.
The current system for using these module-map files with SwiftPM works, but
with a number of caveats that must be addressed.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#motivation>Motivation
The current implementation of system module packages have a number of problems:
Install locations vary across platforms and modulemap files require absolute
paths
/usr/lib:/usr/local/lib is not always a sufficient -L search path
/usr/include:/usr/local/include is not always a sufficient -I C compiler search
path
Installing the system library is left up to the end-user to figure out
For example to import a module map representing the GTK library, the include
search path must be supplemented with -I/usr/include/gtk so that a number of
includes in the gtk.h header can be sourced for the complete modular definition
of GTK.
For example to import a module map representing the GTK library a user must
first have a copy of GTK and its headers installed. On Debian based systems the
install name for this system package is libgtk-3-0-dev which is not entirely
intuitive.
For example, Homebrew and MacPorts on OS X install to prefixes other than /usr.
.modulemap files must specify headers with absolute paths. The standard we
encourage with modulemaps is for the headers to be specified with an assumed
prefix of /usr, but you will not find eg. jpeglib.h at /usr/include/jpeglib.h
if it is installed with Homebrew or MacPorts.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#proposed-solution>Proposed
Solution
We propose that SwiftPM gains the ability to use the cross-platform pkg-config
tool so that it can query pkg-config for the missing path and flag arguments.
We propose that SwiftPM gains the ability to use the cross-platform pkg-config
tool to identify when the system package is not installed to a /usr and in such
a case preprocess the modulemap changing the prefix it uses.
We propose that Package.swift is supplemented with metadata that provides the
package-install-name for specific platforms.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#detailed-design>Detailed
Design
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#solving-pathflags-issues>Solving
Path/Flags Issues
Some of our problems can be solved by using the cross platform tool: pkg-config.
A C package can provide a pkg-config file (.pc) which describes:
Its install location
Supplementary C-flags that should be used when compiling against this library
Supplementary C-flags that should be used when linking against this library
If SwiftPM used the .pc file that comes with packages, this solves problems 1
through 3.
Of the tickets we currently have open describing issues using
Swift-system-module-packages, reading the .pc file would fix all of them.
It is a convention to name the .pc file after the library link-name, so we can
determine which .pc file to ask pkg-configfor by parsing the .modulemap file in
the Swift package. However sometimes this is not true, (eg. GTK-3 on Ubuntu),
so we will make it possible to specify the .pc file name in Package.swift.
pkg-config is not currently a dependency of the Swift toolchain, and thus to
avoid depending on it we will schedule work to interpret .pc files without
requiring pkg-config to be installed. The file format for .pc files is simple
and standard so despite reinventing the wheel, this is a low risk choice.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#providing-package-install-names>Providing
Package Install Names
Package.swift would be supplemented like so:
let package = Package(
name: "CFoo",
providers: .Brew(installName: "foo"),
.Apt(installName: "libfoo-dev"),
.PkgConfig("foo.pc"),
)
Thus, in the event of build failure for modules that depend on this package we
provide additional help to the user:
error: failed to build module `bar'
note: you may need to install `foo' using your system-packager:
apt-get install libfoo-dev
Since the syntax to provide this information uses an explicit enum we can add
code for each enum to detect which system packagers should be recommended. The
community will need to write the code for their own platforms. It also means
that if a specific packager requires additional parameters, they can be added
on a per enum basis.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#install-names-are-not-standard>Install-names
are not standard
apt is used across multiple distirbutions and the install-names for tools vary.
Even for the same distribution install-names may vary across releases (eg. from
Ubuntu 15.04 to Ubuntu 15.10) or even on ocassion at finer granularity.
We will not add explicit handling for this, but one can imagine the enums for
different system packagers could be supplemented in a backwards compatible way
to provide specific handling as real-world uses emerge, eg:
case Apt(installName: String)
// …could be adapted to:
struct Debian: Linux {}
struct Ubuntu: Debian {
enum Variant {
case Gubuntu
case Kubuntu(Version)
}
enum Version {
case v1510
case v1504
}
}
case Apt(installName: String, distribution: Linux? = nil)
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#impact-on-existing-code>Impact
on Existing Code
There will be no impact on existing code as this feature simply improves an
existing feature making new code possible.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#alternatives-considered>Alternatives
Considered
A clear alternative is allowing additional flags to be specified in a
system-module package’s Package.swift.
However since these paths and flags will vary by platform this would because a
large matrix that is quite a maintenance burden. Really this information is
recorded already, in the system package itself, and in fact almost all packages
nowadays provide it in a .pc pkg-config file.
Also we do not want to allow arbitrary flags to be specified in Package.swift,
this allows packages too much power to break a large dependency graph with bad
compiles. The only entity that understands the whole graph and can manage the
build without breakage is SwiftPM, and allowing packages themselves to add
arbitrary flags prevents SwiftPM from being able to understand and control the
build ensuring reliability and preventing “Dependency Hell”.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#unsolved-problems>Unsolved
Problems
Some (usually more legacy) C libraries do not provide .pc files instead they
may provide a tool named eg. foo-configthat can be queried for compile and link
flags. We do not yet support these tools, and would prefer to take a wait and
see approach to determine how important supporting them may be.
Some libraries on OS X do not come with .pc files. Again we'd like to see which
libraries are affected before potentially offering a solution here.
<https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#future-directions>Future
Directions
The build system could be made more reliable by having the specific packager
provide the information that this proposal garners from pkg-config. For
example, Homebrew installs everything into independent directories, using these
directories instead of more general POSIX search paths means there is no danger
of edge-case search path collisions and the wrong libraries being picked up.
If this was done pkg-config could become just one option for providing this
data, and be used only as a fallback.
We do not wish to provide a flag to automatically install dependencies via the
system packager. We feel this opens us up to security implications beyond the
scope of this tool.
Instead we can provide JSON output that can be parsed and executed by some
other tooling developed outside of Apple._______________________________________________
swift-evolution mailing list
[email protected]
https://lists.swift.org/mailman/listinfo/swift-evolution
