Re: [swift-evolution] DynamicMemberLookup proposal: status update

[Chris Lattner via swift-evolution]

Yes indeed!  He and I have been exchanging emails :-)

-Chris


> On Jan 5, 2018, at 5:20 AM, Ben Rimmington  wrote:
> 
> Have you seen John Holdsworth's experiments?
> 
> 
> -- Ben
> 
>> On 4 Jan 2018, at 20:52, Chris Lattner wrote:
>> 
>> Hi everyone,
>> 
>> With the holidays and many other things behind us, the core team had a 
>> chance to talk about python interop + the dynamic member lookup proposal 
>> recently.
>> 
>> Here’s where things stand: we specifically discussed whether a 
>> counter-proposal of using “automatically generated wrappers” or “foreign 
>> classes” to solve the problem would be better.  After discussion, the 
>> conclusion is no: the best approach appears to be 
>> DynamicMemberLookup/DynamicCallable or something similar in spirit to them.  
>> As such, I’ll be dusting off the proposal and we’ll eventually run it.
>> 
>> For transparency, I’m attaching the analysis below of what a wrapper 
>> facility could look like, and why it doesn’t work very well for Python 
>> interop.  I appologize in advance that this is sort of train-of-thought and 
>> not a well written doc. 
>> 
>> That said, it would be really great to get tighter integration between Swift 
>> and SwiftPM for other purposes!  I don’t have time to push this forward in 
>> the short term though, but if someone was interested in pushing it forward, 
>> many people would love to see it discussed seriously.
>> 
>> -Chris
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] DynamicMemberLookup proposal: status update

[Ben Rimmington via swift-evolution]

Have you seen John Holdsworth's experiments?


-- Ben

> On 4 Jan 2018, at 20:52, Chris Lattner wrote:
> 
> Hi everyone,
> 
> With the holidays and many other things behind us, the core team had a chance 
> to talk about python interop + the dynamic member lookup proposal recently.
> 
> Here’s where things stand: we specifically discussed whether a 
> counter-proposal of using “automatically generated wrappers” or “foreign 
> classes” to solve the problem would be better.  After discussion, the 
> conclusion is no: the best approach appears to be 
> DynamicMemberLookup/DynamicCallable or something similar in spirit to them.  
> As such, I’ll be dusting off the proposal and we’ll eventually run it.
> 
> For transparency, I’m attaching the analysis below of what a wrapper facility 
> could look like, and why it doesn’t work very well for Python interop.  I 
> appologize in advance that this is sort of train-of-thought and not a well 
> written doc. 
> 
> That said, it would be really great to get tighter integration between Swift 
> and SwiftPM for other purposes!  I don’t have time to push this forward in 
> the short term though, but if someone was interested in pushing it forward, 
> many people would love to see it discussed seriously.
> 
> -Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] DynamicMemberLookup proposal: status update

[Chris Lattner via swift-evolution]

On Jan 4, 2018, at 3:43 PM, Nevin Brackett-Rozinsky 
 wrote:
> 
> There’s a lot of information here and it’ll take some time to process it all. 
> My initial reaction is that a “strong type-alias” feature might help. If one 
> could write (strawman syntax):
> 
> strong typealias Dog = PyVal// A semantically independent new type
> 
> extension Dog {
> // Declarations here are only available on “Dog”, not on “PyVal”
> }
> 
> then most of the overload issues would evaporate.

Until and if there were an acceptable design for such a feature, it is 
impossible to say whether it would help.  I am personally skeptical that 
"strong type aliases" will ever make it into Swift.

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] DynamicMemberLookup proposal: status update

[Nevin Brackett-Rozinsky via swift-evolution]

There’s a lot of information here and it’ll take some time to process it
all. My initial reaction is that a “strong type-alias” feature might help.
If one could write (strawman syntax):

strong typealias Dog = PyVal// A semantically independent new type

extension Dog {
// Declarations here are only available on “Dog”, not on “PyVal”
}

then most of the overload issues would evaporate.

Nevin



On Thu, Jan 4, 2018 at 3:52 PM, Chris Lattner via swift-evolution <
swift-evolution@swift.org> wrote:

> Hi everyone,
>
> With the holidays and many other things behind us, the core team had a
> chance to talk about python interop + the dynamic member lookup proposal
> recently.
>
> Here’s where things stand: we specifically discussed whether a
> counter-proposal of using “automatically generated wrappers” or “foreign
> classes” to solve the problem would be better.  After discussion, the
> conclusion is no: the best approach appears to be 
> DynamicMemberLookup/DynamicCallable
> or something similar in spirit to them.  As such, I’ll be dusting off the
> proposal and we’ll eventually run it.
>
> For transparency, I’m attaching the analysis below of what a wrapper
> facility could look like, and why it doesn’t work very well for Python
> interop.  I appologize in advance that this is sort of train-of-thought and
> not a well written doc.
>
> That said, it would be really great to get tighter integration between
> Swift and SwiftPM for other purposes!  I don’t have time to push this
> forward in the short term though, but if someone was interested in pushing
> it forward, many people would love to see it discussed seriously.
>
> -Chris
>
>
> *A Swift automatic wrapper facility:*
>
> Requirements:
>  - We want the be able to run a user defined script to generate wrappers.
>  - This script can have arbitrary dependencies and should get updated when
> one of them change.
>  - These dependencies won’t be visible to the Xcode build system, so the
> compiler will have to manage them.
>  - In principle, one set of wrappers should be able to depend on another
> set, and wants “overlays”, so we need a pretty general model.
>
> I don’t think the clang modules based approach is a good way to go.
>
>
> *Proposed Approach: Tighter integration between SwiftPM and Swift*
>
> The model is that you should be able to say (strawman syntax):
>
>import Foo from http://github.com/whatever/mypackage
>import Bar from file:///some/path/on/my/machine
>
> and have the compiler ask SwiftPM to build and cache the specified module
> onto your local disk, then have the compiler load it like any other
> module.  This means that “generated wrappers” is now a SwiftPM/llbuild
> feature, and we can use the SwiftPM “language” to describe things like:
>
> 1. Information about what command line invocation is required to generate
> the wrappers.
> 2. Dependency information so that the compiler can regenerate the wrappers
> when they are out of date.
> 3. Platform abstraction tools since things are in different locations on
> linux vs mac, Python 2 vs Python 3 is also something that would have to be
> handled somehow.
> 4. The directory could contain manually written .swift code, serving the
> function similar to “overlays” to augment the automatic wrappers generated.
>
> We care about Playgrounds and the REPL, and they should be able to work
> with this model.
>
> I think that this would be a very nice and useful feature.
>
>
> *Using Wrappers to implement Python Interop:*
>
> While such a thing would be generally useful, it is important to explore
> how well this will work to solve the actual problem at hand, since this is
> being pitched as an alternative to DynamicMemberLookup.  Here is the
> example from the list:
>
> class BankAccount:
> def __init__(self, initial_balance: int = 0) -> None:
> self.balance = initial_balance
> def deposit(self, amount: int) -> None:
> self.balance += amount
> def withdraw(self, amount: int) -> None:
> self.balance -= amount
> def overdrawn(self) -> bool:
> return self.balance < 0
>
> my_account = BankAccount(15)
> my_account.withdraw(5)print(my_account.balance)
>
>
> The idea is to generate a wrapper like this (potentially including the
> type annotations as a refinement):
>
> typealias BankAccount = PyVal
> extension PyVal { // methods on BankAccount
>   init(initial_balance: PyVal) { … }
>   func deposit(amount: PyVal) -> PyVal { … }
>   func withdraw(amount: PyVal) -> PyVal { … }
>   func overdrawn() -> PyVal { … }
> }
>
> my_account = BankAccount(initial_balance: 15)
> my_account.withdraw(amount: 5)
> print(my_account.balance)
>
>
>
> It is worth pointing out that this approach is very analogous to the “type
> providers” feature that Joe pushed hard for months ago, it is just a
> different implementation approach.  The proposal specifically explains why
> this isn’t a great solution here:
> https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae54

[swift-evolution] DynamicMemberLookup proposal: status update

[Chris Lattner via swift-evolution]

Hi everyone,

With the holidays and many other things behind us, the core team had a chance 
to talk about python interop + the dynamic member lookup proposal recently.

Here’s where things stand: we specifically discussed whether a counter-proposal 
of using “automatically generated wrappers” or “foreign classes” to solve the 
problem would be better.  After discussion, the conclusion is no: the best 
approach appears to be DynamicMemberLookup/DynamicCallable or something similar 
in spirit to them.  As such, I’ll be dusting off the proposal and we’ll 
eventually run it.

For transparency, I’m attaching the analysis below of what a wrapper facility 
could look like, and why it doesn’t work very well for Python interop.  I 
appologize in advance that this is sort of train-of-thought and not a well 
written doc.  

That said, it would be really great to get tighter integration between Swift 
and SwiftPM for other purposes!  I don’t have time to push this forward in the 
short term though, but if someone was interested in pushing it forward, many 
people would love to see it discussed seriously.

-Chris


A Swift automatic wrapper facility:

Requirements:
 - We want the be able to run a user defined script to generate wrappers.
 - This script can have arbitrary dependencies and should get updated when one 
of them change.
 - These dependencies won’t be visible to the Xcode build system, so the 
compiler will have to manage them.
 - In principle, one set of wrappers should be able to depend on another set, 
and wants “overlays”, so we need a pretty general model.

I don’t think the clang modules based approach is a good way to go.


Proposed Approach: Tighter integration between SwiftPM and Swift

The model is that you should be able to say (strawman syntax):

   import Foo from http://github.com/whatever/mypackage 

   import Bar from file:///some/path/on/my/machine 


and have the compiler ask SwiftPM to build and cache the specified module onto 
your local disk, then have the compiler load it like any other module.  This 
means that “generated wrappers” is now a SwiftPM/llbuild feature, and we can 
use the SwiftPM “language” to describe things like:

1. Information about what command line invocation is required to generate the 
wrappers.
2. Dependency information so that the compiler can regenerate the wrappers when 
they are out of date.
3. Platform abstraction tools since things are in different locations on linux 
vs mac, Python 2 vs Python 3 is also something that would have to be handled 
somehow.
4. The directory could contain manually written .swift code, serving the 
function similar to “overlays” to augment the automatic wrappers generated.

We care about Playgrounds and the REPL, and they should be able to work with 
this model.

I think that this would be a very nice and useful feature.  


Using Wrappers to implement Python Interop:

While such a thing would be generally useful, it is important to explore how 
well this will work to solve the actual problem at hand, since this is being 
pitched as an alternative to DynamicMemberLookup.  Here is the example from the 
list:
> class BankAccount:
> def __init__(self, initial_balance: int = 0) -> None:
> self.balance = initial_balance
> def deposit(self, amount: int) -> None:
> self.balance += amount
> def withdraw(self, amount: int) -> None:
> self.balance -= amount
> def overdrawn(self) -> bool:
> return self.balance < 0
> 
> my_account = BankAccount(15)
> my_account.withdraw(5)
> print(my_account.balance)


The idea is to generate a wrapper like this (potentially including the type 
annotations as a refinement):

typealias BankAccount = PyVal
extension PyVal { // methods on BankAccount
  init(initial_balance: PyVal) { … }
  func deposit(amount: PyVal) -> PyVal { … }
  func withdraw(amount: PyVal) -> PyVal { … }
  func overdrawn() -> PyVal { … }
}

my_account = BankAccount(initial_balance: 15)
my_account.withdraw(amount: 5)
print(my_account.balance)


It is worth pointing out that this approach is very analogous to the “type 
providers” feature that Joe pushed hard for months ago, it is just a different 
implementation approach.  The proposal specifically explains why this isn’t a 
great solution here:
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#introduce-f-style-type-providers-into-swift
 


That said, while there are similarities, there are also differences with type 
providers.  Here are the problems that I foresee:


1) This design still requires DynamicMemberLookup

This is because Python doesn’t have property declarations for the wrapper 
generator to process.  The code above shows this on the last line: since there 
is no definition of the “balance" property, there will be no “balance member” 
declared in the PyVal extension.