Thanks Brent for the tips, I think those a well-documented starter guide. Two thing I couldn’t understand, and you may help.
I dug into the codebase as stated in the steps you mentioned, and tried to change one of the errors to see if it really change when I use Swift REPL, but nothing changed. Is it possible that I change something and directly affect the Swift compiler? (maybe I need to do a build first? Or maybe related to xcode-select?) Each type of diagnostics has 4 parts, ERROR(ID,Options,Text,Signature). Diagnostics use the first 3 parts and pass parentheses to the signature. What is meant by the signature and why it’s used? Thanks again for the help. > Message: 18 > Date: Thu, 9 Nov 2017 08:24:37 -0800 > From: Brent Royal-Gordon <br...@architechies.com > <mailto:br...@architechies.com>> > To: Mohammed Ennabah <ennaba...@gmail.com <mailto:ennaba...@gmail.com>> > Cc: swift-dev@swift.org <mailto:swift-dev@swift.org> > Subject: Re: [swift-dev] SR-3423 Starter questions > Message-ID: <1c627652-f5b7-42f3-8039-1ae7ba5f7...@architechies.com > <mailto:1c627652-f5b7-42f3-8039-1ae7ba5f7...@architechies.com>> > Content-Type: text/plain; charset=utf-8 > >> On Nov 7, 2017, at 4:18 AM, Mohammed Ennabah via swift-dev >> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote: >> >> Hi all, >> >> This is the first time I work on that deep of the compiler (in SR-3423 Enum >> with tuple raw value not allowed) and I’m not sure where to start and what >> do I need to do and what files this issue is related to. It would be great >> if anyone guided me. Thanks. > > This is *just* the very first step, but here's what I do when I want to debug > something that involves a compiler error message: > > 1. Write some code that triggers the bug and compile it. > > For instance, that bug includes a Stack Overflow link—copy that code > and paste it into the Swift REPL. > > 2. Pick some chunk of the error message that doesn't include any > identifiers and search the Swift source code for it. > > For instance, one of the errors is "raw value for enum case must be a > literal", so search for that. > > 3. Most of the hits will be tests, but one will be in a .def file; > that's where the error is defined. Open the file, go to that line, and get > the error identifier from it. > > In this case, include/swift/AST/DiagnosticsParse.swift has this line in > it: > > ERROR(nonliteral_enum_case_raw_value,PointsToFirstBadToken, > "raw value for enum case must be a literal", ()) > > The identifier I'm talking about is the first argument to the ERROR > macro—here, it's "nonliteral_enum_case_raw_value". > > (Before you move on to the next step, look around in this file a little > bit and try to get a sense for how error message strings are formatted. It > may help you with future error searches.) > > 4. Search the Swift source code for that error identifier. Each hit > (besides the one in the .def file) is one of the places in the compiler that > can generate that error message. Read the code around each of those hits > (including the name of the functions, types, files, and folders it's in) and > try to figure out if it handles the specific case you want to solve, and if > so, how you might be able to modify it to suit your needs. > > In this example, there is only one hit—in a .cpp file in the > parser—and, a couple lines above it, you can see a dyn_cast call that checks > if the expression is a literal. (`dyn_cast` is equivalent to an `as?` cast in > Swift—it returns `null` if the value doesn't belong to the indicated type.) > This is where the compiler is testing whether the raw value assigned to a > case is a literal. You'll need to modify the code here so that it also > accepts tuples of literals (and presumably, tuples of tuples of literals, and > so on). There will be more to do, but that's the first step. > > When you're reading code, don't be discouraged if you don't understand it > immediately. The Swift compiler is a large, complicated system, and there's > lots of complicated stuff going on. Be patient, don't be afraid to jump over > to another function to see what it does, and if you get stuck trying to > figure out one part, move on to something else. > > I'm still very much a beginner, but this has helped me get started on a few > different bugs. I hope it can help you too. > > -- > Brent Royal-Gordon > Architechies > > > > ------------------------------ > > Message: 19 > Date: Thu, 09 Nov 2017 08:48:33 -0800 > From: Daniel Dunbar <daniel_dun...@apple.com <mailto:daniel_dun...@apple.com>> > To: Johannes Weiss <johanneswe...@apple.com <mailto:johanneswe...@apple.com>> > Cc: swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> > Subject: Re: [swift-dev] Zero-cost 'Service Provider > Interface'/Signature Packages > Message-ID: <e2c40795-b1cd-43fe-856a-f4c34532f...@apple.com > <mailto:e2c40795-b1cd-43fe-856a-f4c34532f...@apple.com>> > Content-Type: text/plain; charset="utf-8" > > >> On Nov 8, 2017, at 5:27 PM, Johannes Weiß <johanneswe...@apple.com >> <mailto:johanneswe...@apple.com>> wrote: >> >> Hi Daniel, >> >>> On 2 Nov 2017, at 8:15 pm, Daniel Dunbar <daniel_dun...@apple.com >>> <mailto:daniel_dun...@apple.com>> wrote: >>> >>> My personal preference is to: >>> 1. Do nothing for now, but encourage publishing standardized protocols to >>> solve this need. >>> 2. Hope for a future with WMO+LTO magic which recovers the performance, for >>> the case where the entire application ends up using one implementation. >> >> Hmm, but that'll only work if we get 'whole product optimisation', right? If >> we still compile one module at the time I don't think the compiler will be >> able to figure out that there's just one implementation of that protocol in >> the whole program. In fact it can't as that module might be linked into >> different programs and one of those programs might have a second >> implementation of that protocol. This is extremely likely as the 'test >> program' might have a mock/fake or some special implementation. Did I >> misunderstand you here? > > That’s correct, that is what I meant by magic WMO+LTO future. > >>> You can manage some of the “dependency injection” part by making the >>> package which exports the common protocol also export a global variable for >>> the concrete implementation in use (with setters). That could just be a >>> “pattern” people follow. This wouldn’t be particularly pretty, but it would >>> mean that intermediate packages could avoid declaring a concrete dependency >>> on any one implementation, and leave it up to clients to pick. >> >> hmm, two questions: >> - what would the type of that global variable be? All libraries in the >> program will need to know and agree on that type > > It would be the type of the abstract protocol. > >> - sounds like ThreadSanitizer would trap here unless we synchronise it which >> would make it slow again > > Depends on how the protocol is phrased. The global instance could be a type > which is instantiated and then doesn’t require locking. > > An example of what I had in mind (Package.swifts left to the reader): > ``` > $ find . -name \*.swift -exec printf "**** %s ****\n" {} \; -exec cat {} \; > **** ./Client/Sources/Client/main.swift **** > import Log > import BadLogger > > Log.registerLogger(BadLogger.self) > let logger = Log.createLogger()! > logger.log("how quaint") > > > **** ./BadLogger/Sources/BadLogger/BadLogger.swift **** > import Log > > public struct BadLogger: Logger { > public init() {} > public func log(_ message: String) { > fatalError("logging considered harmful: \(message)") > } > } > > **** ./Log/Sources/Log/Log.swift **** > import Dispatch > > public protocol Logger { > // MARK: Logger API > > init() > func log(_ message: String) > } > > // MARK: Global Registration > > private let queue = DispatchQueue(label: "org.awesome.log") > private var theLoggerType: Logger.Type? = nil > > public func registerLogger(_ type: Logger.Type) { > queue.sync { > theLoggerType = type > } > } > > public func createLogger() -> Logger? { > return queue.sync{ theLoggerType }?.init() > } > ``` > > I’m not saying this is the best solution in the world, but it does work > currently without requiring new features. I agree there is a (small) > performance cost, but for most use cases I doubt that is the most important > consideration. > > - Daniel > >> >> >> -- Johannes >> >>> - Daniel >>> >>>> On Nov 2, 2017, at 5:57 PM, Johannes Weiß via swift-dev >>>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote: >>>> >>>> Hi swift-dev, >>>> >>>> I talked to a few people about this problem and we agreed that it is a >>>> problem and that it needs to be discussed. I didn't quite know where it >>>> would fit best but let's go with swift-dev, please feel free to tell to >>>> post it elsewhere if necessary. And apologies for the long mail, couldn't >>>> come up with a sensible tl;dr... >>>> >>>> Let me briefly introduce the problem what for the lack of a better name I >>>> call 'signature package' or 'Service Provider Interface' (SPI) as some >>>> people from the Java community seem to be calling it >>>> (https://en.wikipedia.org/wiki/Service_provider_interface >>>> <https://en.wikipedia.org/wiki/Service_provider_interface>). For the rest >>>> of this email I'll use the term SPI. >>>> >>>> In a large ecosystem there is a few pieces that many libraries will depend >>>> on and yet it seems pretty much impossible to standardise exactly one >>>> implementation. Logging is a very good example as many people have >>>> different ideas about how logging should and should not work. At the >>>> moment I guess your best bet is to use your preferred logging API and hope >>>> that all your other dependencies use the same one. If not you'll likely >>>> run into annoying problems (different sub-systems logging to different >>>> places or worse). >>>> >>>> Also, in a world where some dependencies might be closed source this is an >>>> even bigger problem as clearly no open-source framework will depend on >>>> something that's not open-source. >>>> >>>> >>>> In Java the way seems to be to standardise on some logging interface (read >>>> `protocol`) with different implementations. For logging that'd probably be >>>> SLF4J [4]. In Swift: >>>> >>>> let logger: LoggerProtocol = MyFavouriteLoggingFramework(configuration) >>>> >>>> where `LoggerProtocol` comes from some SPI package and >>>> `MyFavouriteLoggingFramework` is basically what the name says. And as a >>>> general practise, everybody would only use `LoggerProtocol`. Then tomorrow >>>> when I'll change my mind replacing `MyFavouriteLoggingFramework` by >>>> `BetterFasterLoggingFramework` does the job. With 'dependency injection' >>>> this 'logger' is handed through the whole program and there's a good >>>> chance of it all working out. The benefits are that everybody just needs >>>> to agree on a `protocol` instead of an implementation. 👍 >>>> >>>> In Swift the downside is that this means we're now getting a virtual >>>> dispatch and the existential everywhere (which in Java will be optimised >>>> away by the JIT). That might not be a huge problem but it might undermine >>>> 'CrazyFastLoggingFramework's adoption as we always pay overhead. >>>> >>>> I don't think this problem can be elegantly solved today. What I could >>>> make work today (and maybe we could add language/SwiftPM support to >>>> facilitate it) is this (⚠️, it's ugly) >>>> >>>> - one SwiftPM package defines the SPI only, the only thing it exports is a >>>> `public protocol` called say `_spi_Logger`, no implementation >>>> - every implementation of that SPI defines a `public struct Logger: >>>> _spi_Logger` (yes, they all share the _same_ name) >>>> - every package that wants to log contains >>>> >>>> #if USE_FOO_LOGGER >>>> import FooLogger >>>> #elif USE_BAR_LOGGER >>>> import BarLogger >>>> #else >>>> import BuzLogger >>>> #endif >>>> >>>> where 'BuzLogger' is the preferred logging system of this package but if >>>> either `USE_FOO_LOGGER` or `USE_BAR_LOGGER` was defined this package is >>>> happy to use those as well. >>>> - `Logger` is always used as the type, it might be provided by different >>>> packages though >>>> - in Package.swift of said package we'll need to define something like >>>> this: >>>> >>>> func loggingDependency() -> Package.Dependency { >>>> #if USE_FOO_LOGGER >>>> return .package(url: "github.com/...../foo.git >>>> <http://github.com/...../foo.git>", ...) >>>> #elif USE_BAR_LOGGER >>>> return ... >>>> #else >>>> return .package(url: "github.com/...../buz.git >>>> <http://github.com/...../buz.git>", ...) >>>> #endif >>>> } >>>> >>>> func loggingDependencyTarget() -> Target.Dependency { >>>> #if USE_FOO_LOGGER >>>> return "foo" >>>> #elif USE_BAR_LOGGER >>>> return "bar" >>>> #else >>>> return "buz" >>>> #endif >>>> } >>>> - in the dependencies array of Package.swift we'll then use >>>> `loggingDependency()` and in the target we use `loggingDependencyTarget()` >>>> instead of the concrete one >>>> >>>> Yes, it's awful but even in a world with different opinions about the >>>> implementation of a logger, we can make the program work. >>>> In the happy case where application and all dependency agree that >>>> 'AwesomeLogging' is the best framework we can just type `swift build` and >>>> everything works. In the case where some dependencies think >>>> 'AwesomeLogging' is the best but others prefer 'BestEverLogging' we can >>>> force the whole application into one using `swift build -Xswiftc >>>> -DUSE_AWESOME_LOGGING` or `swift build -Xswiftc -DUSE_BEST_EVER_LOGGING`. >>>> >>>> >>>> Wrapping up, I can see a few different options: >>>> >>>> 1) do nothing and live with the situation (no Swift/SwiftPM changes >>>> required) >>>> 2) advertise something similar to what I propose above (no Swift/SwiftPM >>>> changes required) >>>> 3) do what Java does but optimise the existential away at compile time (if >>>> the compiler can prove there's actually only one type that implements that >>>> protocol) >>>> 4) teach SwiftPM about those SPI packages and make everything work, maybe >>>> by textually replacing the import statements in the source? >>>> 5) do what Haskell did and retrofit a module system that can support this >>>> 6) have 'special' `specialized protocol` for which a concrete >>>> implementation needs to be selected by the primary source >>>> 7) something I haven't thought of >>>> >>>> Btw, both Haskell (with the new 'backpack' [1, 2]) and ML have >>>> 'signatures' to solve this problem. A signature is basically an SPI. For >>>> an example see the backpack-str [3] module in Haskell which defines the >>>> signature (str-sig) and a bunch of different implementations for that >>>> signature (str-bytestring, str-string, str-foundation, str-text, ...). >>>> >>>> Let me know what you think! >>>> >>>> [1]: https://plv.mpi-sws.org/backpack/ <https://plv.mpi-sws.org/backpack/> >>>> [2]: https://ghc.haskell.org/trac/ghc/wiki/Backpack >>>> <https://ghc.haskell.org/trac/ghc/wiki/Backpack> >>>> [3]: https://github.com/haskell-backpack/backpack-str >>>> <https://github.com/haskell-backpack/backpack-str> >>>> [4]: https://www.slf4j.org <https://www.slf4j.org/> >>>> >>>> -- Johannes >>>> PS: I attached a tar ball which contains the following 6 SwiftPM packages >>>> that are created like I describe above: >>>> >>>> - app, the main application, prefers the 'foo' logging library >>>> - somelibA, some library which logs and prefers the 'foo' logging library >>>> - somelibB, some other library which prefers the 'bar' logging library >>>> - foo, the 'foo' logging library >>>> - bar, the 'bar' logging library >>>> - spi, the logging SPI >>>> >>>> The dependency default graph looks like this: >>>> +- somelibA ---+ foo >>>> / / \ >>>> app +--------------/ +-- spi >>>> \ / >>>> +- somelibB ---- bar >>>> >>>> that looks all good, except that 'foo' and 'bar' are two logging libraries >>>> 🙈. In other words, we're in the unhappy case, therefore just typing `swift >>>> build` gives this: >>>> >>>> --- SNIP --- >>>> -1- johannes:~/devel/swift-spi-demo/app >>>> $ swift build >>>> Compile Swift Module 'app' (1 sources) >>>> /Users/johannes/devel/swift-spi-demo/app/Sources/app/main.swift:14:23: >>>> error: cannot convert value of type 'Logger' to expected argument type >>>> 'Logger' >>>> somelibB_func(logger: logger) >>>> ^~~~~~ >>>> error: terminated(1): >>>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build-tool >>>> -f /Users/johannes/devel/swift-spi-demo/app/.build/debug.yaml main >>>> --- SNAP --- >>>> >>>> because there's two `Logger` types. But selecting `foo` gives (note that >>>> all lines start with 'Foo:'): >>>> >>>> --- SNIP --- >>>> $ swift build -Xswiftc -DUSE_FOO >>>> Compile Swift Module 'spi' (1 sources) >>>> Compile Swift Module 'foo' (1 sources) >>>> Compile Swift Module 'bar' (1 sources) >>>> Compile Swift Module 'somelibB' (1 sources) >>>> Compile Swift Module 'somelibA' (1 sources) >>>> Compile Swift Module 'app' (1 sources) >>>> Linking ./.build/x86_64-apple-macosx10.10/debug/app >>>> $ ./.build/x86_64-apple-macosx10.10/debug/app >>>> Foo: info: hello from the app >>>> Foo: info: hello from somelibA >>>> Foo: info: hello from somelibB >>>> Foo: info: hello from somelibA >>>> Foo: info: hello from somelibB >>>> --- SNAP --- >>>> >>>> and for 'bar' (note that all lines start with 'Bar:') >>>> >>>> --- SNIP --- >>>> $ swift build -Xswiftc -DUSE_BAR >>>> Compile Swift Module 'spi' (1 sources) >>>> Compile Swift Module 'foo' (1 sources) >>>> Compile Swift Module 'bar' (1 sources) >>>> Compile Swift Module 'somelibA' (1 sources) >>>> Compile Swift Module 'somelibB' (1 sources) >>>> Compile Swift Module 'app' (1 sources) >>>> Linking ./.build/x86_64-apple-macosx10.10/debug/app >>>> $ ./.build/x86_64-apple-macosx10.10/debug/app >>>> Bar: info: hello from the app >>>> Bar: info: hello from somelibA >>>> Bar: info: hello from somelibB >>>> Bar: info: hello from somelibA >>>> Bar: info: hello from somelibB >>>> --- SNAP --- >>>> >>>> <swift-spi-demo.tar.gz> >>>> >>>> >>>> _______________________________________________ >>>> swift-dev mailing list >>>> swift-dev@swift.org <mailto:swift-dev@swift.org> >>>> https://lists.swift.org/mailman/listinfo/swift-dev >>> >> > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <https://lists.swift.org/pipermail/swift-dev/attachments/20171109/bfd886cf/attachment-0001.html > > <https://lists.swift.org/pipermail/swift-dev/attachments/20171109/bfd886cf/attachment-0001.html>> > > ------------------------------ > > Message: 20 > Date: Thu, 09 Nov 2017 09:12:27 -0800 > From: Breckin Loggins <brec...@apple.com <mailto:brec...@apple.com>> > To: Brent Royal-Gordon <br...@architechies.com > <mailto:br...@architechies.com>> > Cc: swift-dev@swift.org <mailto:swift-dev@swift.org>, Mohammed Ennabah > <ennaba...@gmail.com <mailto:ennaba...@gmail.com>> > Subject: Re: [swift-dev] SR-3423 Starter questions > Message-ID: <76a3c32f-9f9f-4db7-960f-de4cbc4e2...@apple.com > <mailto:76a3c32f-9f9f-4db7-960f-de4cbc4e2...@apple.com>> > Content-Type: text/plain; charset=utf-8 > > This would make a great contribution to the "how to contribute" > documentation; it's excellent advice. > >> On Nov 9, 2017, at 8:24 AM, Brent Royal-Gordon via swift-dev >> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote: >> >>> On Nov 7, 2017, at 4:18 AM, Mohammed Ennabah via swift-dev >>> <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote: >>> >>> Hi all, >>> >>> This is the first time I work on that deep of the compiler (in SR-3423 Enum >>> with tuple raw value not allowed) and I’m not sure where to start and what >>> do I need to do and what files this issue is related to. It would be great >>> if anyone guided me. Thanks. >> >> This is *just* the very first step, but here's what I do when I want to >> debug something that involves a compiler error message: >> >> 1. Write some code that triggers the bug and compile it. >> >> For instance, that bug includes a Stack Overflow link—copy that code >> and paste it into the Swift REPL. >> >> 2. Pick some chunk of the error message that doesn't include any >> identifiers and search the Swift source code for it. >> >> For instance, one of the errors is "raw value for enum case must be a >> literal", so search for that. >> >> 3. Most of the hits will be tests, but one will be in a .def file; >> that's where the error is defined. Open the file, go to that line, and get >> the error identifier from it. >> >> In this case, include/swift/AST/DiagnosticsParse.swift has this line in >> it: >> >> ERROR(nonliteral_enum_case_raw_value,PointsToFirstBadToken, >> "raw value for enum case must be a literal", ()) >> >> The identifier I'm talking about is the first argument to the ERROR >> macro—here, it's "nonliteral_enum_case_raw_value". >> >> (Before you move on to the next step, look around in this file a little >> bit and try to get a sense for how error message strings are formatted. It >> may help you with future error searches.) >> >> 4. Search the Swift source code for that error identifier. Each hit >> (besides the one in the .def file) is one of the places in the compiler that >> can generate that error message. Read the code around each of those hits >> (including the name of the functions, types, files, and folders it's in) and >> try to figure out if it handles the specific case you want to solve, and if >> so, how you might be able to modify it to suit your needs. >> >> In this example, there is only one hit—in a .cpp file in the >> parser—and, a couple lines above it, you can see a dyn_cast call that checks >> if the expression is a literal. (`dyn_cast` is equivalent to an `as?` cast >> in Swift—it returns `null` if the value doesn't belong to the indicated >> type.) This is where the compiler is testing whether the raw value assigned >> to a case is a literal. You'll need to modify the code here so that it also >> accepts tuples of literals (and presumably, tuples of tuples of literals, >> and so on). There will be more to do, but that's the first step. >> >> When you're reading code, don't be discouraged if you don't understand it >> immediately. The Swift compiler is a large, complicated system, and there's >> lots of complicated stuff going on. Be patient, don't be afraid to jump over >> to another function to see what it does, and if you get stuck trying to >> figure out one part, move on to something else. > > Another thing I've learned: don't be afraid to break it. On purpose. > > Think of the Swift repo on your machine as YOUR Swift compiler. You can make > it do anything you want! Wanna make Swift look like Pascal just to figure out > how the lexer and parser work? Have at it! Just because you never intend to > commit a change doesn't mean it isn't valuable. > > When I don't know where to start with something in a complicated codebase, I > always start by breaking something on purpose and putting a "DO NOT COMMIT" > comment or warning where I do that. Now that I know I won't inadvertently > check in a silly change and embarrass myself, I feel much better about > exploring the code with abandon and learning about how things work by > studying what happens when they don't... and that's where the learning > happens for me. > >> >> I'm still very much a beginner, but this has helped me get started on a few >> different bugs. I hope it can help you too. > > If it helps, I still do this on codebases I've been familiar with for years > when I'm changing something in an area I don't know fluently. :) > >> >> -- >> Brent Royal-Gordon >> Architechies >> >> _______________________________________________ >> swift-dev mailing list >> swift-dev@swift.org <mailto:swift-dev@swift.org> >> https://lists.swift.org/mailman/listinfo/swift-dev > > > > ------------------------------ > > _______________________________________________ > swift-dev mailing list > swift-dev@swift.org <mailto:swift-dev@swift.org> > https://lists.swift.org/mailman/listinfo/swift-dev > > > End of swift-dev Digest, Vol 24, Issue 10 > *****************************************
_______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
