Hi Daniel, > On 2 Nov 2017, at 8:15 pm, Daniel Dunbar <daniel_dun...@apple.com> wrote: > > My personal preference is to: > 1. Do nothing for now, but encourage publishing standardized protocols to > solve this need. > 2. Hope for a future with WMO+LTO magic which recovers the performance, for > the case where the entire application ends up using one implementation.
Hmm, but that'll only work if we get 'whole product optimisation', right? If we still compile one module at the time I don't think the compiler will be able to figure out that there's just one implementation of that protocol in the whole program. In fact it can't as that module might be linked into different programs and one of those programs might have a second implementation of that protocol. This is extremely likely as the 'test program' might have a mock/fake or some special implementation. Did I misunderstand you here? > You can manage some of the “dependency injection” part by making the package > which exports the common protocol also export a global variable for the > concrete implementation in use (with setters). That could just be a “pattern” > people follow. This wouldn’t be particularly pretty, but it would mean that > intermediate packages could avoid declaring a concrete dependency on any one > implementation, and leave it up to clients to pick. hmm, two questions: - what would the type of that global variable be? All libraries in the program will need to know and agree on that type - sounds like ThreadSanitizer would trap here unless we synchronise it which would make it slow again -- Johannes > - Daniel > >> On Nov 2, 2017, at 5:57 PM, Johannes Weiß via swift-dev >> <swift-dev@swift.org> wrote: >> >> Hi swift-dev, >> >> I talked to a few people about this problem and we agreed that it is a >> problem and that it needs to be discussed. I didn't quite know where it >> would fit best but let's go with swift-dev, please feel free to tell to post >> it elsewhere if necessary. And apologies for the long mail, couldn't come up >> with a sensible tl;dr... >> >> Let me briefly introduce the problem what for the lack of a better name I >> call 'signature package' or 'Service Provider Interface' (SPI) as some >> people from the Java community seem to be calling it >> (https://en.wikipedia.org/wiki/Service_provider_interface). For the rest of >> this email I'll use the term SPI. >> >> In a large ecosystem there is a few pieces that many libraries will depend >> on and yet it seems pretty much impossible to standardise exactly one >> implementation. Logging is a very good example as many people have different >> ideas about how logging should and should not work. At the moment I guess >> your best bet is to use your preferred logging API and hope that all your >> other dependencies use the same one. If not you'll likely run into annoying >> problems (different sub-systems logging to different places or worse). >> >> Also, in a world where some dependencies might be closed source this is an >> even bigger problem as clearly no open-source framework will depend on >> something that's not open-source. >> >> >> In Java the way seems to be to standardise on some logging interface (read >> `protocol`) with different implementations. For logging that'd probably be >> SLF4J [4]. In Swift: >> >> let logger: LoggerProtocol = MyFavouriteLoggingFramework(configuration) >> >> where `LoggerProtocol` comes from some SPI package and >> `MyFavouriteLoggingFramework` is basically what the name says. And as a >> general practise, everybody would only use `LoggerProtocol`. Then tomorrow >> when I'll change my mind replacing `MyFavouriteLoggingFramework` by >> `BetterFasterLoggingFramework` does the job. With 'dependency injection' >> this 'logger' is handed through the whole program and there's a good chance >> of it all working out. The benefits are that everybody just needs to agree >> on a `protocol` instead of an implementation. 👍 >> >> In Swift the downside is that this means we're now getting a virtual >> dispatch and the existential everywhere (which in Java will be optimised >> away by the JIT). That might not be a huge problem but it might undermine >> 'CrazyFastLoggingFramework's adoption as we always pay overhead. >> >> I don't think this problem can be elegantly solved today. What I could make >> work today (and maybe we could add language/SwiftPM support to facilitate >> it) is this (⚠️, it's ugly) >> >> - one SwiftPM package defines the SPI only, the only thing it exports is a >> `public protocol` called say `_spi_Logger`, no implementation >> - every implementation of that SPI defines a `public struct Logger: >> _spi_Logger` (yes, they all share the _same_ name) >> - every package that wants to log contains >> >> #if USE_FOO_LOGGER >> import FooLogger >> #elif USE_BAR_LOGGER >> import BarLogger >> #else >> import BuzLogger >> #endif >> >> where 'BuzLogger' is the preferred logging system of this package but if >> either `USE_FOO_LOGGER` or `USE_BAR_LOGGER` was defined this package is >> happy to use those as well. >> - `Logger` is always used as the type, it might be provided by different >> packages though >> - in Package.swift of said package we'll need to define something like this: >> >> func loggingDependency() -> Package.Dependency { >> #if USE_FOO_LOGGER >> return .package(url: "github.com/...../foo.git", ...) >> #elif USE_BAR_LOGGER >> return ... >> #else >> return .package(url: "github.com/...../buz.git", ...) >> #endif >> } >> >> func loggingDependencyTarget() -> Target.Dependency { >> #if USE_FOO_LOGGER >> return "foo" >> #elif USE_BAR_LOGGER >> return "bar" >> #else >> return "buz" >> #endif >> } >> - in the dependencies array of Package.swift we'll then use >> `loggingDependency()` and in the target we use `loggingDependencyTarget()` >> instead of the concrete one >> >> Yes, it's awful but even in a world with different opinions about the >> implementation of a logger, we can make the program work. >> In the happy case where application and all dependency agree that >> 'AwesomeLogging' is the best framework we can just type `swift build` and >> everything works. In the case where some dependencies think 'AwesomeLogging' >> is the best but others prefer 'BestEverLogging' we can force the whole >> application into one using `swift build -Xswiftc -DUSE_AWESOME_LOGGING` or >> `swift build -Xswiftc -DUSE_BEST_EVER_LOGGING`. >> >> >> Wrapping up, I can see a few different options: >> >> 1) do nothing and live with the situation (no Swift/SwiftPM changes required) >> 2) advertise something similar to what I propose above (no Swift/SwiftPM >> changes required) >> 3) do what Java does but optimise the existential away at compile time (if >> the compiler can prove there's actually only one type that implements that >> protocol) >> 4) teach SwiftPM about those SPI packages and make everything work, maybe by >> textually replacing the import statements in the source? >> 5) do what Haskell did and retrofit a module system that can support this >> 6) have 'special' `specialized protocol` for which a concrete implementation >> needs to be selected by the primary source >> 7) something I haven't thought of >> >> Btw, both Haskell (with the new 'backpack' [1, 2]) and ML have 'signatures' >> to solve this problem. A signature is basically an SPI. For an example see >> the backpack-str [3] module in Haskell which defines the signature (str-sig) >> and a bunch of different implementations for that signature (str-bytestring, >> str-string, str-foundation, str-text, ...). >> >> Let me know what you think! >> >> [1]: https://plv.mpi-sws.org/backpack/ >> [2]: https://ghc.haskell.org/trac/ghc/wiki/Backpack >> [3]: https://github.com/haskell-backpack/backpack-str >> [4]: https://www.slf4j.org >> >> -- Johannes >> PS: I attached a tar ball which contains the following 6 SwiftPM packages >> that are created like I describe above: >> >> - app, the main application, prefers the 'foo' logging library >> - somelibA, some library which logs and prefers the 'foo' logging library >> - somelibB, some other library which prefers the 'bar' logging library >> - foo, the 'foo' logging library >> - bar, the 'bar' logging library >> - spi, the logging SPI >> >> The dependency default graph looks like this: >> +- somelibA ---+ foo >> / / \ >> app +--------------/ +-- spi >> \ / >> +- somelibB ---- bar >> >> that looks all good, except that 'foo' and 'bar' are two logging libraries >> 🙈. In other words, we're in the unhappy case, therefore just typing `swift >> build` gives this: >> >> --- SNIP --- >> -1- johannes:~/devel/swift-spi-demo/app >> $ swift build >> Compile Swift Module 'app' (1 sources) >> /Users/johannes/devel/swift-spi-demo/app/Sources/app/main.swift:14:23: >> error: cannot convert value of type 'Logger' to expected argument type >> 'Logger' >> somelibB_func(logger: logger) >> ^~~~~~ >> error: terminated(1): >> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build-tool >> -f /Users/johannes/devel/swift-spi-demo/app/.build/debug.yaml main >> --- SNAP --- >> >> because there's two `Logger` types. But selecting `foo` gives (note that all >> lines start with 'Foo:'): >> >> --- SNIP --- >> $ swift build -Xswiftc -DUSE_FOO >> Compile Swift Module 'spi' (1 sources) >> Compile Swift Module 'foo' (1 sources) >> Compile Swift Module 'bar' (1 sources) >> Compile Swift Module 'somelibB' (1 sources) >> Compile Swift Module 'somelibA' (1 sources) >> Compile Swift Module 'app' (1 sources) >> Linking ./.build/x86_64-apple-macosx10.10/debug/app >> $ ./.build/x86_64-apple-macosx10.10/debug/app >> Foo: info: hello from the app >> Foo: info: hello from somelibA >> Foo: info: hello from somelibB >> Foo: info: hello from somelibA >> Foo: info: hello from somelibB >> --- SNAP --- >> >> and for 'bar' (note that all lines start with 'Bar:') >> >> --- SNIP --- >> $ swift build -Xswiftc -DUSE_BAR >> Compile Swift Module 'spi' (1 sources) >> Compile Swift Module 'foo' (1 sources) >> Compile Swift Module 'bar' (1 sources) >> Compile Swift Module 'somelibA' (1 sources) >> Compile Swift Module 'somelibB' (1 sources) >> Compile Swift Module 'app' (1 sources) >> Linking ./.build/x86_64-apple-macosx10.10/debug/app >> $ ./.build/x86_64-apple-macosx10.10/debug/app >> Bar: info: hello from the app >> Bar: info: hello from somelibA >> Bar: info: hello from somelibB >> Bar: info: hello from somelibA >> Bar: info: hello from somelibB >> --- SNAP --- >> >> <swift-spi-demo.tar.gz> >> >> >> _______________________________________________ >> swift-dev mailing list >> swift-dev@swift.org >> https://lists.swift.org/mailman/listinfo/swift-dev > _______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
