On Dec 1, 2017, at 12:26 AM, Douglas Gregor <dgre...@apple.com> wrote:
>> On Nov 30, 2017, at 10:05 PM, Chris Lattner <clatt...@nondot.org 
>> <mailto:clatt...@nondot.org>> wrote:
>> Hi Doug,
>> 
>> Thank you for the detailed email.  I have been traveling today, so I haven’t 
>> had a chance to respond until now.  I haven’t read the down-thread emails, 
>> so I apologize if any of this was already discussed:
>> 
>>> I think better interoperability with Python (and other OO languages in 
>>> widespread use) is a good goal, and I agree that the implementation of the 
>>> feature described is straight-forward and not terribly invasive in the 
>>> compiler.
>> 
>> Fantastic, I’m really pleased to hear that!  I only care about solving the 
>> problem, so if we can find a good technical solution to the problems than 
>> I’ll be happy.

After thinking about your email a bit more, I think I might understand the 
disconnect we’re having.  I think we have different versions in mind of what 
“success” looks like:

I believe your view is that people start using Python APIs from their Swift 
code, but gradually add type annotations (either inline or in a sidecar 
database) to make those APIs progressively more Swifty.  New features will be 
added to Python (its type system, compilers, databases for common APIs, etc) to 
improve this interoperability, along the lines of what we’ve done for 
Objective-C over the years.  Fast forward several years, and large Python 
libraries would be nice to use from Swift - perhaps nicer than they are to use 
from Python itself.  This view aligns with what happened with Objective-C <-> 
Swift interoperability.

In contrast, I’m specifically interested in developers in certain large domains 
(e.g. data science and ML) which are “forced” to use Python because that is 
where all the libraries are.  I have spoken to many of these sorts of people, 
and a large number of them really *dislike* using Python for all the obvious 
reasons (including the tooling issues you point out).  My view of success is 
that we allow them to write all of *their code* in Swift, which will lead to a 
massive quality of life benefit for these frustrated people.  You’re right that 
they will still chaff when using imported Python APIs (which don’t feel 
“swifty” for LOTS of reasons - e.g. method naming and design patterns), but my 
view is that this will provide incentive for these people to provide a proper 
Swift implementation for these libraries over time.

In short, your end game is a pervasively intertwined Swift/Python world (like 
ObjC and Swift are).  My view is that Swift holds Python at arm's length, and 
wins over the hearts and minds of developers, leading to new Swift APIs 
designed for Swift.


I’m not sure if you agree with that portrayal of your position, and if not, I’m 
sorry and will try to understand another way.  However, if I’m close, then I 
have several concerns about that vision and don’t believe the end-game is 
achievable or better than what I’m proposing.  Consider:

- I don’t think there will be a lot of success getting people who *actually 
love* Python to use Swift, unless there is already an extrinsic reason for them 
to use it.
- Type hints are not widely used in Python, and there are believable reasons 
that they won’t ever be.  Because they are specifically poorly suited for 
libraries, their use seems to be in “user’s own code” - but my proposal solves 
this already!  See below for examples.
- Even if type hints were widely adopted, it would require massive extensions 
to them and to Python to make them be "good enough” to provide value for the 
things you’re envisioning.  Analogs to instancetype, objc generics, lots of the 
C macros, and many of the other things we’ve added to ObjC would have to be 
added.
- The Python community has no reason to do this work for the Swift community, 
or accept changes to Python that are required to make this actually great.
- Beyond the core type system, the Python and Objective-C languages work 
extremely differently in other ways (e.g. lack of umbrella headers making 
AnyObject-style dispatch questionable).
- Even with those annotations and all the work, the APIs we’d end up with are 
not going to be good Swift APIs.  I don’t think that “renamification” and "IUO 
audits" would ever actually happen in practice, for example.
- I believe the engineering effort required to implement this vision is so 
massive (including the changes to Swift, Python, and Python libraries) that it 
simply will never actually happen.


More concerning to me is that your design of using the existing AnyObject type 
presents a really concerning technical problem, scalability: It is not simply a 
Swift/ObjC/Python world, Javascript is also super important.  There are also a 
number of other less-widely used dynamic languages that are interesting.  Your 
design leads to them all being mashed together into a common runtime system.  I 
cannot imagine how this would end up being a good thing for system complexity, 
and would lead to each of them being jeopardized in different ways.  As I have 
mentioned before numerous times, it also opens the door for enormous Swift 
compiler complexity, as each of the object models need to be supported in the 
compiler (so you can subclass each languages’ types), and many other 
complexities.


If you are serious about improving the tooling situation for people using 
Python APIs in Swift, the most natural way to do so is to follow the approach 
that the MyPy community (they are the ones who have thought about this the 
most) is using: provide progressive typing extensions, use data flow analysis 
to propagate around types, and enhance the tooling to have support this.  I’m 
skeptical that this will ever be worthwhile, but a sketch could look like this:


Consider the first example from: http://mypy-lang.org/examples.html 
<http://mypy-lang.org/examples.html>

With my proposals you could write the dictionary part very similar to the 
Python part (other pieces of the example, e.g. list comprehensions are 
uninteresting to me):

let d: PyVal = {:}
…
d[word] = d.get(word, 0) + 1

We can teach the compiler simple data flow analysis, to know that ‘d’ is a 
Python dictionary.  We could then allow the user to go further, with some new 
syntax along the lines of:

let d: @type(Dict<String, Int>) PyVal = {:}        // Lots of other syntactic 
choices possible.
…
d[word] = d.get(word, 0) + 1



However, the really important thing to recognize about these examples it that 
the type annotations are really poor at representing “generic” code like common 
Python libraries.  They are designed for “user code” - like that on the mypy 
web page - which use concrete types.

As it turns out, my proposal provides a solution for this part of the problem, 
because we’re allowing user code to be written in Swift!  A Swift programmer 
working with Python APIs would actually write this as:


let d: Dictionary<String,Int> = {:}
…
d[word, default: 0] += 1


Similarly, there is no reason to do anything to make the second and third 
examples work nicely in Swift: you’d just define a Swift class, and a Swift 
function.

MyPy:

class BankAccount:
    def __init__(self, initial_balance: int = 0) -> None:
        self.balance = initial_balance
    def deposit(self, amount: int) -> None:
        self.balance += amount
    def withdraw(self, amount: int) -> None:
        self.balance -= amount
    def overdrawn(self) -> bool:
        return self.balance < 0

my_account = BankAccount(15)
my_account.withdraw(5)
print(my_account.balance)
Swift:

class BankAccount { 
   var balance : Int
   init(initialBalance: Int) { … }
  func deposit(….
}

Even the proposals that I’m making - simple and isolated though they are - are 
enough to provide a major quality of life improvement for people writing large 
amounts of code against Python APIs.  Beyond that, they are small extensions 
with low complexity, scale to supporting many different dynamic languages over 
time,  require a level of engineering effort that is plausible to be built, and 
do not require some sort of "executive buy in” from the Python community.

-Chris




_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Reply via email to