> On Nov 8, 2017, at 5:27 PM, Johannes Weiß via swift-dev <swift-dev@swift.org> > wrote: > > Hi Daniel, > >> On 2 Nov 2017, at 8:15 pm, Daniel Dunbar <daniel_dun...@apple.com> wrote: >> >> My personal preference is to: >> 1. Do nothing for now, but encourage publishing standardized protocols to >> solve this need. >> 2. Hope for a future with WMO+LTO magic which recovers the performance, for >> the case where the entire application ends up using one implementation. > > Hmm, but that'll only work if we get 'whole product optimisation', right?
yes. Even when we have cross-module optimizations (which would be comparable to thin-lto) we could not do that optimization. > If we still compile one module at the time I don't think the compiler will be > able to figure out that there's just one implementation of that protocol in > the whole program. exactly > In fact it can't as that module might be linked into different programs and > one of those programs might have a second implementation of that protocol. > This is extremely likely as the 'test program' might have a mock/fake or some > special implementation. Did I misunderstand you here? > > >> You can manage some of the “dependency injection” part by making the package >> which exports the common protocol also export a global variable for the >> concrete implementation in use (with setters). That could just be a >> “pattern” people follow. This wouldn’t be particularly pretty, but it would >> mean that intermediate packages could avoid declaring a concrete dependency >> on any one implementation, and leave it up to clients to pick. > > hmm, two questions: > - what would the type of that global variable be? All libraries in the > program will need to know and agree on that type > - sounds like ThreadSanitizer would trap here unless we synchronise it which > would make it slow again > > > -- Johannes > >> - Daniel >> >>> On Nov 2, 2017, at 5:57 PM, Johannes Weiß via swift-dev >>> <swift-dev@swift.org> wrote: >>> >>> Hi swift-dev, >>> >>> I talked to a few people about this problem and we agreed that it is a >>> problem and that it needs to be discussed. I didn't quite know where it >>> would fit best but let's go with swift-dev, please feel free to tell to >>> post it elsewhere if necessary. And apologies for the long mail, couldn't >>> come up with a sensible tl;dr... >>> >>> Let me briefly introduce the problem what for the lack of a better name I >>> call 'signature package' or 'Service Provider Interface' (SPI) as some >>> people from the Java community seem to be calling it >>> (https://en.wikipedia.org/wiki/Service_provider_interface). For the rest of >>> this email I'll use the term SPI. >>> >>> In a large ecosystem there is a few pieces that many libraries will depend >>> on and yet it seems pretty much impossible to standardise exactly one >>> implementation. Logging is a very good example as many people have >>> different ideas about how logging should and should not work. At the moment >>> I guess your best bet is to use your preferred logging API and hope that >>> all your other dependencies use the same one. If not you'll likely run into >>> annoying problems (different sub-systems logging to different places or >>> worse). >>> >>> Also, in a world where some dependencies might be closed source this is an >>> even bigger problem as clearly no open-source framework will depend on >>> something that's not open-source. >>> >>> >>> In Java the way seems to be to standardise on some logging interface (read >>> `protocol`) with different implementations. For logging that'd probably be >>> SLF4J [4]. In Swift: >>> >>> let logger: LoggerProtocol = MyFavouriteLoggingFramework(configuration) >>> >>> where `LoggerProtocol` comes from some SPI package and >>> `MyFavouriteLoggingFramework` is basically what the name says. And as a >>> general practise, everybody would only use `LoggerProtocol`. Then tomorrow >>> when I'll change my mind replacing `MyFavouriteLoggingFramework` by >>> `BetterFasterLoggingFramework` does the job. With 'dependency injection' >>> this 'logger' is handed through the whole program and there's a good chance >>> of it all working out. The benefits are that everybody just needs to agree >>> on a `protocol` instead of an implementation. 👍 >>> >>> In Swift the downside is that this means we're now getting a virtual >>> dispatch and the existential everywhere (which in Java will be optimised >>> away by the JIT). That might not be a huge problem but it might undermine >>> 'CrazyFastLoggingFramework's adoption as we always pay overhead. >>> >>> I don't think this problem can be elegantly solved today. What I could make >>> work today (and maybe we could add language/SwiftPM support to facilitate >>> it) is this (⚠️, it's ugly) >>> >>> - one SwiftPM package defines the SPI only, the only thing it exports is a >>> `public protocol` called say `_spi_Logger`, no implementation >>> - every implementation of that SPI defines a `public struct Logger: >>> _spi_Logger` (yes, they all share the _same_ name) >>> - every package that wants to log contains >>> >>> #if USE_FOO_LOGGER >>> import FooLogger >>> #elif USE_BAR_LOGGER >>> import BarLogger >>> #else >>> import BuzLogger >>> #endif >>> >>> where 'BuzLogger' is the preferred logging system of this package but if >>> either `USE_FOO_LOGGER` or `USE_BAR_LOGGER` was defined this package is >>> happy to use those as well. >>> - `Logger` is always used as the type, it might be provided by different >>> packages though >>> - in Package.swift of said package we'll need to define something like this: >>> >>> func loggingDependency() -> Package.Dependency { >>> #if USE_FOO_LOGGER >>> return .package(url: "github.com/...../foo.git", ...) >>> #elif USE_BAR_LOGGER >>> return ... >>> #else >>> return .package(url: "github.com/...../buz.git", ...) >>> #endif >>> } >>> >>> func loggingDependencyTarget() -> Target.Dependency { >>> #if USE_FOO_LOGGER >>> return "foo" >>> #elif USE_BAR_LOGGER >>> return "bar" >>> #else >>> return "buz" >>> #endif >>> } >>> - in the dependencies array of Package.swift we'll then use >>> `loggingDependency()` and in the target we use `loggingDependencyTarget()` >>> instead of the concrete one >>> >>> Yes, it's awful but even in a world with different opinions about the >>> implementation of a logger, we can make the program work. >>> In the happy case where application and all dependency agree that >>> 'AwesomeLogging' is the best framework we can just type `swift build` and >>> everything works. In the case where some dependencies think >>> 'AwesomeLogging' is the best but others prefer 'BestEverLogging' we can >>> force the whole application into one using `swift build -Xswiftc >>> -DUSE_AWESOME_LOGGING` or `swift build -Xswiftc -DUSE_BEST_EVER_LOGGING`. >>> >>> >>> Wrapping up, I can see a few different options: >>> >>> 1) do nothing and live with the situation (no Swift/SwiftPM changes >>> required) >>> 2) advertise something similar to what I propose above (no Swift/SwiftPM >>> changes required) >>> 3) do what Java does but optimise the existential away at compile time (if >>> the compiler can prove there's actually only one type that implements that >>> protocol) >>> 4) teach SwiftPM about those SPI packages and make everything work, maybe >>> by textually replacing the import statements in the source? >>> 5) do what Haskell did and retrofit a module system that can support this >>> 6) have 'special' `specialized protocol` for which a concrete >>> implementation needs to be selected by the primary source >>> 7) something I haven't thought of >>> >>> Btw, both Haskell (with the new 'backpack' [1, 2]) and ML have 'signatures' >>> to solve this problem. A signature is basically an SPI. For an example see >>> the backpack-str [3] module in Haskell which defines the signature >>> (str-sig) and a bunch of different implementations for that signature >>> (str-bytestring, str-string, str-foundation, str-text, ...). >>> >>> Let me know what you think! >>> >>> [1]: https://plv.mpi-sws.org/backpack/ >>> [2]: https://ghc.haskell.org/trac/ghc/wiki/Backpack >>> [3]: https://github.com/haskell-backpack/backpack-str >>> [4]: https://www.slf4j.org >>> >>> -- Johannes >>> PS: I attached a tar ball which contains the following 6 SwiftPM packages >>> that are created like I describe above: >>> >>> - app, the main application, prefers the 'foo' logging library >>> - somelibA, some library which logs and prefers the 'foo' logging library >>> - somelibB, some other library which prefers the 'bar' logging library >>> - foo, the 'foo' logging library >>> - bar, the 'bar' logging library >>> - spi, the logging SPI >>> >>> The dependency default graph looks like this: >>> +- somelibA ---+ foo >>> / / \ >>> app +--------------/ +-- spi >>> \ / >>> +- somelibB ---- bar >>> >>> that looks all good, except that 'foo' and 'bar' are two logging libraries >>> 🙈. In other words, we're in the unhappy case, therefore just typing `swift >>> build` gives this: >>> >>> --- SNIP --- >>> -1- johannes:~/devel/swift-spi-demo/app >>> $ swift build >>> Compile Swift Module 'app' (1 sources) >>> /Users/johannes/devel/swift-spi-demo/app/Sources/app/main.swift:14:23: >>> error: cannot convert value of type 'Logger' to expected argument type >>> 'Logger' >>> somelibB_func(logger: logger) >>> ^~~~~~ >>> error: terminated(1): >>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build-tool >>> -f /Users/johannes/devel/swift-spi-demo/app/.build/debug.yaml main >>> --- SNAP --- >>> >>> because there's two `Logger` types. But selecting `foo` gives (note that >>> all lines start with 'Foo:'): >>> >>> --- SNIP --- >>> $ swift build -Xswiftc -DUSE_FOO >>> Compile Swift Module 'spi' (1 sources) >>> Compile Swift Module 'foo' (1 sources) >>> Compile Swift Module 'bar' (1 sources) >>> Compile Swift Module 'somelibB' (1 sources) >>> Compile Swift Module 'somelibA' (1 sources) >>> Compile Swift Module 'app' (1 sources) >>> Linking ./.build/x86_64-apple-macosx10.10/debug/app >>> $ ./.build/x86_64-apple-macosx10.10/debug/app >>> Foo: info: hello from the app >>> Foo: info: hello from somelibA >>> Foo: info: hello from somelibB >>> Foo: info: hello from somelibA >>> Foo: info: hello from somelibB >>> --- SNAP --- >>> >>> and for 'bar' (note that all lines start with 'Bar:') >>> >>> --- SNIP --- >>> $ swift build -Xswiftc -DUSE_BAR >>> Compile Swift Module 'spi' (1 sources) >>> Compile Swift Module 'foo' (1 sources) >>> Compile Swift Module 'bar' (1 sources) >>> Compile Swift Module 'somelibA' (1 sources) >>> Compile Swift Module 'somelibB' (1 sources) >>> Compile Swift Module 'app' (1 sources) >>> Linking ./.build/x86_64-apple-macosx10.10/debug/app >>> $ ./.build/x86_64-apple-macosx10.10/debug/app >>> Bar: info: hello from the app >>> Bar: info: hello from somelibA >>> Bar: info: hello from somelibB >>> Bar: info: hello from somelibA >>> Bar: info: hello from somelibB >>> --- SNAP --- >>> >>> <swift-spi-demo.tar.gz> >>> >>> >>> _______________________________________________ >>> swift-dev mailing list >>> swift-dev@swift.org >>> https://lists.swift.org/mailman/listinfo/swift-dev >> > > _______________________________________________ > swift-dev mailing list > swift-dev@swift.org > https://lists.swift.org/mailman/listinfo/swift-dev _______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
