Those two proposals are growing on me and they have the potential to make Swift even better at DSLs.
> On 15 Nov 2017, at 08:29, Chris Lattner via swift-evolution > <swift-evolution@swift.org> wrote: > > Hi All, > > As a peer to the DynamicCallable proposal > (https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d>), I’d like > to get your feedback on making member lookup dynamically extensible. My > primary motivation is to improve interoperability with dynamic languages like > Python, Perl, Ruby, Javascript, etc, but there are other use cases (e.g. when > working with untyped JSON). > > In addition to being a high impact on expressivity of Swift, I believe an > implementation can be done in a way with changes that are localized, and thus > not have significant impact on the maintainability of the compiler as a > whole. Once the pitch phase of this proposal helps refine the details, I’ll > be happy to prepare an implementation for consideration. > > In case it is useful, I’m working on cleaning up my current prototype Python > bindings. I’ll share them in the next day or two in case they are useful to > provide context. It is amazingly simple: less than 500 lines of Swift code > (plus some small additional C header glue to work around clang importer > limitations) enables impressive interoperability. The only problems are the > verbosity addressed by this proposal and the peer DynamicCallable proposal. > > > Here is the canonical proposal URL: > https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438 > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438> > > A snapshot of the proposal is included below in case it is useful. Thanks in > advance for help improving the proposal! > > -Chris > > > Introduce User-defined "Dynamic Member Lookup" Types > > Proposal: SE-NNNN > <https://gist.github.com/lattner/NNNN-DynamicMemberLookup.md> > Author: Chris Lattner <https://github.com/lattner> > Review Manager: TBD > Status: Awaiting implementation > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#introduction>Introduction > > This proposal introduces a new DynamicMemberLookupProtocol type to the > standard library. Types that conform to it provide "dot" syntax for arbitrary > names which are resolved at runtime. It is simple syntactic sugar which > allows the user to write: > > a = someValue.someMember > someValue.someMember = a > and have it be interpreted by the compiler as: > > a = someValue[dynamicMember: "someMember"] > someValue[dynamicMember: "someMember"] = a > Many other languages have analogous features (e.g. the composition of > Objective-C's explicit properties > <https://developer.apple.com/library/content/documentation/General/Conceptual/DevPedia-CocoaCore/DeclaredProperty.html> > and underlying messaging infrastructure > <https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtHowMessagingWorks.html>). > This sort of functionality is great for implementing dynamic language > interoperability, dynamic proxy APIs > <https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtForwarding.html>, > and other APIs (e.g. for JSON processing). > > Swift-evolution thread: Discussion thread topic for that proposal > <https://lists.swift.org/pipermail/swift-evolution/> > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#motivation-and-context>Motivation > and Context > > Swift is well known for being exceptional at interworking with existing C and > Objective-C APIs, but its support for calling APIs written in scripting > languages like Python, Perl, and Ruby is quite lacking. > > C and Objective-C are integrated into Swift by expending a heroic amount of > effort into integrating Clang ASTs, remapping existing APIs in an attempt to > feel "Swifty", and by providing a large number of attributes and > customization points for changing the behavior of this integration when > writing an Objective-C header. The end result of this massive investment of > effort is that Swift provides a better experience when programming against > these legacy APIs than Objective-C itself did. > > When considering the space of dynamic languages, three things are clear: 1) > there are several different languages of interest, and they each have > significant interest in different quarters: for example, Python is big in > data science and machine learning, Ruby is popular for building server side > apps, and even Perl is in still widely used. 2) These languages have decades > of library building behind them, sometimes with significant communities > <https://pandas.pydata.org/> and 3) there are one or two orders of magnitude > more users of these libraries than there are people currently using Swift. > > While it is theoretically possible to expend the same level of effort on each > of these languages and communities as has been spent on Objective-C, it is > quite clear that this would both ineffective as well as bad for Swift: It > would be ineffective, because the Swift community has not leverage over these > communities to force auditing and annotation of their APIs. It would be bad > for Swift because it would require a ton of language-specific support (and a > number of third-party dependencies) onto the compiler and runtime, each of > which makes the implementation significantly more complex, difficult to > reason about, difficult to maintain, and difficult to test the supported > permutations. In short, we'd end up with a mess. > > Fortunately for us, these scripting languages provide an extremely dynamic > programming model where almost everything is discovered at runtime, and many > of them are explicitly designed to be embedded into other languages and > applications. This aspect allows us to embed APIs from these languages > directly into Swift with no language support at all - without not the level > of effort, integration, and invasiveness that Objective-C has benefited from. > Instead of invasive importer work, we can write some language-specific Swift > APIs, and leave the interop details to that library. > > This offers a significant opportunity for us - the Swift community can > "embrace" these dynamic language APIs (making them directly available in > Swift) which reduces the pain of someone moving from one of those languages > into Swift. It is true that the APIs thus provided will not feel "Swifty", > but if that becomes a significant problem for any one API, then the community > behind it can evaluate the problem and come up with a solution (either a > Swift wrapper for the dynamic language, or a from-scratch Swift > reimplementation of the desired API). In any case, if/when we face this > challenge, it will be a good thing: we'll know that we've won a significant > new community of Swift developers. > > While it is possible today to import (nearly) arbitrary dynamic language APIs > into Swift today, the resultant API is unusable for two major reasons: member > lookup is too verbose to be acceptable, and calling behavior is similarly too > verbose to be acceptable. As such, we seek to provide two "syntactic sugar" > features that solve this problem. These sugars are specifically designed to > be dynamic language independent and, indeed, independent of dynamic languages > at all: we can imagine other usage for the same primitive capabilities. > > The two proposals in question are the introduction of the DynamicCallable > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> protocol > and a related DynamicMemberLookupProtocol proposal (this proposal). With > these two extensions, we think we can eliminate the need for invasive > importer magic by making interoperability with dynamic languages ergonomic > enough to be acceptable. > > For example, consider this Python code: > > class Dog: > def __init__(self, name): > self.name = name > self.tricks = [] # creates a new empty list for each dog > > def add_trick(self, trick): > self.tricks.append(trick) > we would like to be able to use this from Swift like this (the comments show > the corresponding syntax you would use in Python): > > // import DogModule > // import DogModule.Dog as Dog // an alternate > let Dog = Python.import(“DogModule.Dog") > > // dog = Dog("Brianna") > let dog = Dog("Brianna") > > // dog.add_trick("Roll over") > dog.add_trick("Roll over") > > // dog2 = Dog("Kaylee").add_trick("snore") > let dog2 = Dog("Kaylee").add_trick("snore") > Of course, this would also apply to standard Python APIs as well. Here is an > example working with the Python pickleAPI and the builtin Python function > open: > > // import pickle > let pickle = Python.import("pickle") > > // file = open(filename) > let file = Python.open(filename) > > // blob = file.read() > let blob = file.read() > > // result = pickle.loads(blob) > let result = pickle.loads(blob) > This can all be expressed today as library functionality written in Swift, > but without this proposal, the code required is unnecessarily verbose and > gross. Without it (but with the related DynamicCallable proposal > <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> the code > would have explicit member lookups all over the place: > > // import pickle > let pickle = Python.get(member: "import")("pickle") > > // file = open(filename) > let file = Python.get(member: "open")(filename) > > // blob = file.read() > let blob = file.get(member: "read")() > > // result = pickle.loads(blob) > let result = pickle.get(member: "loads")(blob) > > // dog2 = Dog("Kaylee").add_trick("snore") > let dog2 = Dog("Kaylee").get(member: "add_trick")("snore") > While this is a syntactic sugar proposal, we believe that this expands Swift > to be usable in important new domains. In addition to dynamic language > interoperability, this sort of functionality is useful for other APIs, e.g. > when working with dynamically typed unstructured data like JSON, which could > provide an API like jsonValue?.jsonField1?.jsonField2where each field is > dynamically looked up. > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#proposed-solution>Proposed > solution > > We propose introducing this protocol to the standard library: > > protocol DynamicMemberLookupProtocol { > associatedtype DynamicMemberLookupValue > > subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get set > } > } > It also extends the language such that member lookup syntax (x.y) - when it > otherwise fails (because there is no member y defined on the type of x) and > when applied to a value which conforms to DynamicMemberLookupProtocol- is > accepted and transformed into a call to the subscript in the protocol. This > ensures that no member lookup on such a type ever fails. > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#example-usage>Example > Usage > > While there are many potential uses of this sort of API (e.g. resolving JSON > members to named results, producing optional bindings) a motivating example > comes from a prototype Python interoperability layer. There are many ways to > implement this, and the details are not particularly important, but it is > perhaps useful to know that this is directly useful to address the motivation > section described above. Given a currency type of PyVal (and a conforming > implementation named PyRef), an implementation may look like: > > extension PyVal { > subscript(dynamicMember member: String) -> PyVal { > get { > let result = PyObject_GetAttrString(borrowedPyObject, member)! > return PyRef(owned: result) // PyObject_GetAttrString returns +1 > result. > } > set { > PyObject_SetAttrString(borrowedPyObject, member, > newValue.toPython().borrowedPyObject) > } > } > } > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#source-compatibility>Source > compatibility > > This is a strictly additive proposal with no source breaking changes. > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#effect-on-abi-stability>Effect > on ABI stability > > This is a strictly additive proposal with no ABI breaking changes. > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#effect-on-api-resilience>Effect > on API resilience > > This has no impact on API resilience which is not already captured by other > language features. > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#alternatives-considered>Alternatives > considered > > A few alternatives were considered: > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#add-ability-to-provide-read-only-members>Add > ability to provide read-only members > > The implementation above does not allow an implementation to statically > reject members that are read-only. If this was important to add, we could add > another protocol to model this, along the lines of: > > protocol DynamicMemberLookupGettableProtocol { > associatedtype DynamicMemberLookupValue > > // gettable only > subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get } > } > > protocol DynamicMemberLookupProtocol : DynamicMemberLookupGettableProtocol { > // gettable and settable. > subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get set > } > } > This would allow a type to implement one or the other based on their > capabilities. This proposal starts with a very simple design based on the > requirements of dynamic languages (which have no apparent immutability > model), but if there is demand for this (e.g. because we want input JSON > values to be gettable but not settalbe), the author is happy to switch to > this more general model. > > > <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#naming>Naming > > There is a lot of grounds to debate naming of the protocol and methods in > this type. Suggestions (along with rationale to support them) are more than > welcome. > > _______________________________________________ > swift-evolution mailing list > swift-evolution@swift.org > https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________ swift-evolution mailing list swift-evolution@swift.org https://lists.swift.org/mailman/listinfo/swift-evolution
