Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Nevin Brackett-Rozinsky via swift-evolution]

Without commenting on anything else, I have encountered one use-case where
it would be nice to be able to “call” an instance. And that is, while
modeling some mathematics, I made a DifferentiableFunction type which
stores a function and optionally a reference to another
DifferentiableFunction (its derivative).

I used the subscript notation for “calling” these DifferentiableFunctions,
because that’s what we have available, but it would certainly be nice to
use parentheses rather than square brackets.

Nevin
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 13, 2017, at 1:45 AM, Adrian Zubarev  
> wrote:
> 
> Hello Chris, I have some questions about this passage:
> 
> Before this proposal, the Swift language has two types that participate in 
> call syntax: functions and metatypes (for initialization). Neither of those 
> may conform to protocols at the moment, so this introduces no possible 
> ambiguity into the language.
> Can you shortly describe why it would be ambiguous if metatypes would conform 
> to protocols?
> 
It would only be ambiguous if a metatype conformed to the DynamicCallable 
protocol specifically.  This can’t happen today, and if it happened, we could 
easily define disambiguation rules.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

To close the loop on this pitch, I’ve gotten a lot of useful feedback from the 
discussion.  I’ll prepare a revised version and do pitch #2 when I have time, 
perhaps later this week.  Thanks!

-Chris


> On Nov 10, 2017, at 9:37 AM, Chris Lattner  wrote:
> 
> Hello all,
> 
> I have a couple of proposals cooking in a quest to make Swift interoperate 
> with dynamically typed languages like Python better.  Instead of baking in 
> hard coded support for one language or the other, I’m preferring to add a few 
> small but general purpose capabilities to Swift.  This is the first, which 
> allows a Swift type to become “callable”.
> 
> The proposal is here:
> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d 
> 
> 
> I’ve also attached a snapshot below, but it will drift out of date as the 
> proposal is refined.  Feedback and thoughts are appreciated, thanks!
> 
> -Chris
> 
> 
> 
> 
> 
> Introduce user-defined dynamically "callable" types
> 
> Proposal: SE- 
> Author: Chris Lattner 
> Review Manager: TBD
> Status: Awaiting implementation
>  
> Introduction
> 
> This proposal introduces a new DynamicCallable protocol to the standard 
> library. Types that conform to it are "callable" with the function call 
> syntax. It is simple syntactic sugar which allows the user to write:
> 
> a = someValue(keyword1: 42, "foo", keyword2: 19)
> and have it be interpreted by the compiler as:
> 
>   a = someValue.dynamicCall(arguments: [
> ("keyword1", 42), ("", "foo"), ("keyword2", 19)
>   ])
> Other languages have analogous features (e.g. Python "callables"), but the 
> primary motivation of this proposal is to allow elegant and natural 
> interoperation with dynamic languages in Swift.
> 
> Swift-evolution thread: Discussion thread topic for that proposal 
> 
>  
> Motivation
> 
> Swift is well known for being exceptional at interworking with existing C and 
> Objective-C APIs, but its support for calling APIs written in scripting 
> langauges like Python, Perl, and Ruby is quite lacking. These languages 
> provide an extremely dynamic programming model where almost everything is 
> discovered at runtime.
> 
> Through the introduction of this proposal, and the related 
> DynamicMemberLookupProtocol proposal, we seek to fix this problem. We believe 
> we can make many common APIs feel very natural to use directly from Swift 
> without all the complexity of implementing something like the Clang importer. 
> For example, consider this Python code:
> 
> class Dog:
> def __init__(self, name):
> self.name = name
> self.tricks = []# creates a new empty list for each dog
> 
> def add_trick(self, trick):
> self.tricks.append(trick)
> we would like to be able to use this from Swift like this (the comments show 
> the corresponding syntax you would use in Python):
> 
>   // import DogModule
>   // import DogModule.Dog as Dog// an alternate
>   let Dog = Python.import(“DogModule.Dog")
> 
>   // dog = Dog("Brianna")
>   let dog = Dog("Brianna")
> 
>   // dog.add_trick("Roll over")
>   dog.add_trick("Roll over")
> 
>   // dog2 = Dog("Kaylee").add_trick("snore")
>   let dog2 = Dog("Kaylee").add_trick("snore")
> Of course, this would also apply to standard Python APIs as well. Here is an 
> example working with the Python pickleAPI and the builtin Python function 
> open:
> 
>   // import pickle
>   let pickle = Python.import("pickle")
> 
>   // file = open(filename)
>   let file = Python.open(filename)
> 
>   // blob = file.read()
>   let blob = file.read()
> 
>   // result = pickle.loads(blob)
>   let result = pickle.loads(blob)
> This can all be expressed today as library functionality written in Swift, 
> but without this proposal, the code required is unnecessarily verbose and 
> gross. Without it (but with the related dynamic member lookup proposal) the 
> code would have a method name (like call) all over the code:
> 
>   // import pickle
>   let pickle = Python.import("pickle")  // normal method in Swift, no change.
> 
>   // file = open(filename)
>   let file = Python.open.call(filename)
> 
>   // blob = file.read()
>   let blob = file.read.call()
> 
>   // result = pickle.loads(blob)
>   let result = pickle.loads.call(blob)
> 
>   // dog2 = Dog("Kaylee").add_trick("snore")
>   let dog2 = Dog.call("Kaylee").add_trick.call("snore")
> While this is a syntactic sugar proposal, we believe that this expands Swift 
> to be usable in important new domains. This sort of capability is also highly 
> precedented in other languages, and is a generally useful language feature 
> that could be used 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Adrian Zubarev via swift-evolution]

Hello Chris, I have some questions about this passage:

Before this proposal, the Swift language has two types that participate in call 
syntax: functions and metatypes (for initialization). Neither of those may 
conform to protocols at the moment, so this introduces no possible ambiguity 
into the language.
Can you shortly describe why it would be ambiguous if metatypes would conform 
to protocols?

If metatypes were allowed to conform to protocols, how would this affect your 
proposal?

Last year I pitched a the idea to make metatypes conform to Hashable which 
seemed to be a welcome change.




Am 10. November 2017 um 18:37:26, Chris Lattner via swift-evolution 
(swift-evolution@swift.org) schrieb:

Hello all,

I have a couple of proposals cooking in a quest to make Swift interoperate with 
dynamically typed languages like Python better.  Instead of baking in hard 
coded support for one language or the other, I’m preferring to add a few small 
but general purpose capabilities to Swift.  This is the first, which allows a 
Swift type to become “callable”.

The proposal is here:
https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d

I’ve also attached a snapshot below, but it will drift out of date as the 
proposal is refined.  Feedback and thoughts are appreciated, thanks!

-Chris





Introduce user-defined dynamically "callable" types
Proposal: SE-
Author: Chris Lattner
Review Manager: TBD
Status: Awaiting implementation
Introduction

This proposal introduces a new DynamicCallable protocol to the standard 
library. Types that conform to it are "callable" with the function call syntax. 
It is simple syntactic sugar which allows the user to write:

a = someValue(keyword1: 42, "foo", keyword2: 19)
and have it be interpreted by the compiler as:

  a = someValue.dynamicCall(arguments: [
("keyword1", 42), ("", "foo"), ("keyword2", 19)
  ])
Other languages have analogous features (e.g. Python "callables"), but the 
primary motivation of this proposal is to allow elegant and natural 
interoperation with dynamic languages in Swift.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

Swift is well known for being exceptional at interworking with existing C and 
Objective-C APIs, but its support for calling APIs written in scripting 
langauges like Python, Perl, and Ruby is quite lacking. These languages provide 
an extremely dynamic programming model where almost everything is discovered at 
runtime.

Through the introduction of this proposal, and the related 
DynamicMemberLookupProtocol proposal, we seek to fix this problem. We believe 
we can make many common APIs feel very natural to use directly from Swift 
without all the complexity of implementing something like the Clang importer. 
For example, consider this Python code:

class Dog:
def __init__(self, name):
self.name = name
self.tricks = []# creates a new empty list for each dog
 
def add_trick(self, trick):
self.tricks.append(trick)
we would like to be able to use this from Swift like this (the comments show 
the corresponding syntax you would use in Python):

  // import DogModule
  // import DogModule.Dog as Dog// an alternate
  let Dog = Python.import(“DogModule.Dog")

  // dog = Dog("Brianna")
  let dog = Dog("Brianna")

  // dog.add_trick("Roll over")
  dog.add_trick("Roll over")

  // dog2 = Dog("Kaylee").add_trick("snore")
  let dog2 = Dog("Kaylee").add_trick("snore")
Of course, this would also apply to standard Python APIs as well. Here is an 
example working with the Python pickleAPI and the builtin Python function open:

  // import pickle
  let pickle = Python.import("pickle")

  // file = open(filename)
  let file = Python.open(filename)

  // blob = file.read()
  let blob = file.read()

  // result = pickle.loads(blob)
  let result = pickle.loads(blob)
This can all be expressed today as library functionality written in Swift, but 
without this proposal, the code required is unnecessarily verbose and gross. 
Without it (but with the related dynamic member lookup proposal) the code would 
have a method name (like call) all over the code:

  // import pickle
  let pickle = Python.import("pickle")  // normal method in Swift, no change.

  // file = open(filename)
  let file = Python.open.call(filename)

  // blob = file.read()
  let blob = file.read.call()

  // result = pickle.loads(blob)
  let result = pickle.loads.call(blob)

  // dog2 = Dog("Kaylee").add_trick("snore")
  let dog2 = Dog.call("Kaylee").add_trick.call("snore")
While this is a syntactic sugar proposal, we believe that this expands Swift to 
be usable in important new domains. This sort of capability is also highly 
precedented in other languages, and is a generally useful language feature that 
could be used for other purposes as well.

Proposed solution

We propose introducing this protocol to the standard library:

protocol DynamicCallable {
  associatedtype 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 12, 2017, at 3:33 AM, Tino Heth via swift-evolution 
>  wrote:
> 
> 
> 
>> Am 11.11.2017 um 19:03 schrieb Chris Lattner > >:
>> 
>>> Swift is quite flexible on what can act as a closure — would it be allowed 
>>> to use a dynamic callable in that context?
>>> I guess forwarding of
>>> 
>>> let closure: ([(String, Int)]) -> Int  = DynamicCallableType()
>>> 
>>> to the dynamicCall method of DynamicCallableType isn’t that hard, but 
>>> wouldn’t it be odd if the value of closure changes when you leave out its 
>>> type?
>> 
>> I’m not sure I understand what you’re getting at.  Can you show how this 
>> would work with the example in the motivation section?
> 
> I’m just asking if this should work ;-) — I didn’t see anything regarding it 
> in the proposal.
> But to take something from the document:
> 
>   // file = open(filename)
>   let file = Python.open.call(filename)
> let openFile: (String) -> FileHandle? = Python.open // is this be possible, 
> or an error?

This would be an error. It is perfectly fine to do something like this:

  func foo(a : PyRef) -> PyRef {
 let b = a.foo  // produces a PyRef
 let c = b(“string”)   // produces a PyRef
 return c
  }

The idea of this proposal is that in Python “everything is a python object”, 
but that some of the objects are callable.  This approach is modeling a python 
object with some type (PyRef in the example above).  It is totally possible to 
produce sugar on top of that, or to go farther.  For example, with no language 
extensions, it is possible to do:

let closure: (String) -> Int  = { arg in Int(pythonThing.call(args: 
arg))! }

Or with this proposal, you could do something like:

let closure: (String) -> Int  = { arg in Int(pythonThing(arg))! }

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[John Holdsworth via swift-evolution]

One last try at promoting code generators as an alternative means of importing….
It keeps the complexity out of the compiler and you are able to write the 
“importer” in
the language being imported allowing it to leverage it’s meta data 
introspection directly.

I’ve prepared a playground which shows how a generated class + limited 
supporting code
provides an effective way of bridging to Python. As the generated code is just 
Swift you
have the advantages of the Xcode editor such as code completion available.

https://github.com/johnno1962/SwiftPython/tree/master/SwiftPython.playground 

https://github.com/johnno1962/SwiftPython/blob/master/SwiftPython.playground/Sources/Complex.swift
 


The playground is standalone and the generated class is included as an 
auxiliary file.
The bridging code would only require updating when the surface of the api 
changes.
I’ve not written the actual generator script but it should be straightforward 
for Python.

John.

//: Playground - noun: a place where people can play

var str = "Hello, python integration"

// class Complex is implemented in python

let c = Complex(11.0, 12.0)
print(c.toString(extra: "123"))

c.r
c.add(c: Complex(1.0, 2.0))
c.r

print(c.toString(extra: [1,2,3]))
print(c.toString(extra: [1.0,2.0,3.0]))
print(c.toString(extra: ["a": 1.0, "b": 2.0, "c": [1,2,3]]))

c.toArray()
c.toDictionary()

(11.00 12.00 123)
(12.00 14.00 [1L, 2L, 3L])
(12.00 14.00 [1.0, 2.0, 3.0])
(12.00 14.00 {'a': 1.0, 'c': [1L, 2L, 3L], 'b': 2.0})

> On 12 Nov 2017, at 11:45, Brent Royal-Gordon via swift-evolution 
>  wrote:
> 
>> On Nov 11, 2017, at 7:02 AM, Joe Groff via swift-evolution 
>> > wrote:
>> 
>> Opening up the compiler architecture to make custom importers easier to 
>> write is a great solution to a ton of problems, including yours I think, 
>> without adding complexity to the core language. Experience in .NET land 
>> seems to show it's a great technique for integrating dynamic systems with 
>> static type systems, without poking unnecessary holes in the static 
>> language's type system
> 
> 
> I'm keenly interested in custom importers, if only because I would love to 
> treat storyboards and asset catalogs as "languages" and automatically 
> generate typesafe enums for their various stringly-typed identifiers. (Note 
> to anyone in the peanut gallery getting excited: I don't work at Apple and 
> can't make this a part of the toolchain; I just hope they'd do it if they 
> could.)
> 
> But at the same time, I have a hard time seeing that as a great solution for 
> dynamic languages. I've barely used Python, but I work in Ruby on Rails a 
> lot, and Rubyists rely pretty heavily on dynamic features. For instance, the 
> ActiveRecord ORM creates all its accessors on demand through Ruby's 
> `method_missing` feature. Static analysis would be useless for this, and I 
> can't imagine that profile-based analysis would be very reliable. Even the F# 
> Type Providers documentation warns that "You should avoid writing a type 
> provider where a schema isn't available."
> 
> A dynamic dispatch feature, on the other hand, could reliably reproduce 
> Ruby's behavior because every method call would simply be dispatched through 
> Ruby's method-calling machinery, just as a method call in Ruby would be. It's 
> a round hole for the round peg of a dynamic language's dispatch behavior to 
> fit into. Other languages, like Objective-C and C++, could use the square 
> hole of a custom importer.
> 
> Or maybe the right custom importer design could have it both ways, allowing 
> dynamic languages to accept and write a dispatch for unknown methods, while 
> permitting static languages to match against the more specific data they have 
> available. I don't know. All I'm saying is, static languages and dynamic 
> languages are different, and we need the features necessary to support both 
> without making either of them worse. It's quite possible that the best design 
> for static language bridges will look very, very different from the best 
> design for dynamic languages.
> 
> -- 
> Brent Royal-Gordon
> Architechies
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Brent Royal-Gordon via swift-evolution]

> On Nov 11, 2017, at 7:02 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> Opening up the compiler architecture to make custom importers easier to write 
> is a great solution to a ton of problems, including yours I think, without 
> adding complexity to the core language. Experience in .NET land seems to show 
> it's a great technique for integrating dynamic systems with static type 
> systems, without poking unnecessary holes in the static language's type system


I'm keenly interested in custom importers, if only because I would love to 
treat storyboards and asset catalogs as "languages" and automatically generate 
typesafe enums for their various stringly-typed identifiers. (Note to anyone in 
the peanut gallery getting excited: I don't work at Apple and can't make this a 
part of the toolchain; I just hope they'd do it if they could.)

But at the same time, I have a hard time seeing that as a great solution for 
dynamic languages. I've barely used Python, but I work in Ruby on Rails a lot, 
and Rubyists rely pretty heavily on dynamic features. For instance, the 
ActiveRecord ORM creates all its accessors on demand through Ruby's 
`method_missing` feature. Static analysis would be useless for this, and I 
can't imagine that profile-based analysis would be very reliable. Even the F# 
Type Providers documentation warns that "You should avoid writing a type 
provider where a schema isn't available."

A dynamic dispatch feature, on the other hand, could reliably reproduce Ruby's 
behavior because every method call would simply be dispatched through Ruby's 
method-calling machinery, just as a method call in Ruby would be. It's a round 
hole for the round peg of a dynamic language's dispatch behavior to fit into. 
Other languages, like Objective-C and C++, could use the square hole of a 
custom importer.

Or maybe the right custom importer design could have it both ways, allowing 
dynamic languages to accept and write a dispatch for unknown methods, while 
permitting static languages to match against the more specific data they have 
available. I don't know. All I'm saying is, static languages and dynamic 
languages are different, and we need the features necessary to support both 
without making either of them worse. It's quite possible that the best design 
for static language bridges will look very, very different from the best design 
for dynamic languages.

-- 
Brent Royal-Gordon
Architechies

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Tino Heth via swift-evolution]

> Am 11.11.2017 um 19:03 schrieb Chris Lattner :
> 
>> Swift is quite flexible on what can act as a closure — would it be allowed 
>> to use a dynamic callable in that context?
>> I guess forwarding of
>> 
>> let closure: ([(String, Int)]) -> Int  = DynamicCallableType()
>> 
>> to the dynamicCall method of DynamicCallableType isn’t that hard, but 
>> wouldn’t it be odd if the value of closure changes when you leave out its 
>> type?
> 
> I’m not sure I understand what you’re getting at.  Can you show how this 
> would work with the example in the motivation section?

I’m just asking if this should work ;-) — I didn’t see anything regarding it in 
the proposal.
But to take something from the document:

  // file = open(filename)
  let file = Python.open.call(filename)
let openFile: (String) -> FileHandle? = Python.open // is this be possible, or 
an error?

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 11, 2017, at 6:52 PM, Matthew Johnson  wrote:
 In point of fact, Swift already has the feature you are referring to.  It 
 just spells it with square brackets instead of parentheses.  A simple 
 change to the punctuation character has much less point than the proposal 
 that I’m pitching.
>>> 
>>> This is true if you squint, but I imagine a design for callable types would 
>>> include some 
>>>  differences other than just punctuation.
> 
> I think there was a pretty good discussion of this distinction in the  
> SE-0021 timeframe.  Specifically, subscripts model storage while callable 
> would model functions.  This means that subscripts can be used in a key path 
> while callable would not (at last for now).  

Sure, subscript getters are more general than calls in that respect, I agree.

> On the other hand, I would want to see callables implicitly convert to a 
> value of a compatible function type (possibly through a subtype relationship).

Subscripts are generalizations of the property model in Swift.  We’ve discussed 
being able to curry the getter of a property, so I don’t see why currying the 
getter of a subscript should work any different.  This would directly provide 
the functionality you’re referring to.

> Without this capability the syntactic illusion is only half complete.  The 
> callable feels like a function when used directly but not when it is assigned 
> to a function type or passed elsewhere - it must be explicitly be wrapped in 
> a closure and the arguments forwarded.  First-class callable types are a 
> syntactic sugar feature and as such if they exist they should be usable 
> syntactically in all of the same ways functions are.
> 
> Read-only subscripts are ok as a workaround in absence of first-class 
> callable types (although I think in many ways just using `call` is better).  
> Nevertheless, they are a workaround and can never be more than that because 
> of the fundamental difference in what they model.

Please don’t misunderstand me.  I’m not saying that:

let a = x[foo: 42, “bonk”, bar: 17]

is “better” than:

let a = x(foo: 42, “bonk”, bar: 17)

when x is a functor or something else function like.  Syntax matters: it is the 
principle way that language communicates to programmers, and sending an 
indexing connotation is very different than sending a call connotation.  My 
point is simply that we already have the ability to *express* this concept, so 
users have a way to solve a problem (even in a suboptimal way).  

In contrast, we do not have a way to express the dynamically callable form, 
which is the subject of my proposal.  The workaround is not a change of one 
punctuation character to another, it is a significantly heavier syntactic form 
like x.call(…) or perhaps Joe’s proposal of introducing weird punctuation.

In any case, as I have reiterated many times, I simply want to solve the Python 
import problem.  I am not wed to this approach.  If there is another approach 
that provides a equal (or better) result, then I’m super interested to hear 
about it.  The only alternative that I’m aware of is to burn support for Python 
into the Swift compiler, which I’m pretty opposed to doing: not only is it 
significantly more complexity than the proposals required to implement this in 
libraries, it also signs us up to do the same for Perl, Ruby, Javascript, …...

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 11, 2017, at 6:24 PM, Joe Groff  wrote:
>> In contrast, it does *not* seem like it would help with the problem of 
>> resolving members of “id” typed values in Objective-C, which is the crux of 
>> the dynamic language problem.  If my understanding is correct, type 
>> providers seem completely unhelpful for python interoperability, because you 
>> don’t know the types of python values: anything loaded from a Python 
>> property or returned by a Python function is the equivalent of “id”.
>> 
>> It’s entirely possible I’m missing something big.  If so, I’d appreciate a 
>> pointer to something to read that explains how this works.  Thanks!
> 
> A type provider should let you provide custom logic for something the 
> compiler sees as a module, which can answer name lookup queries by producing 
> decls based on externally-available information.

Right.

> While Python is dynamic, Python culture doesn't go as wild with dynamic 
> shenanigans as other languages do;

I’m not sure what you mean by this, but it is not my experience that Python 
programs are widely monomorphic.  In fact, several Python API design experts I 
respect have mentioned that a natural effect of dynamic languages is “interface 
creep” where APIs end up accepting broader and broader sets of stuff to make 
them “just work” in more cases.

Static languages solve this heads on by allowing API authors to express their 
requirements.  I prefer this approach of course, but we’re talking about a real 
existing world here, not my preference :-)

> there are established idioms for communicating some level of type information 
> in doc comments (or more recently with first class syntax in Python 3),

Yes, Python 3 does have optional type annotations.  I’m not sure how widely 
used they are though, and given that the Python 2 community is (far?) larger 
than Python 3, this isn’t something we can depend on IMO.

> and people like code completion to work with their Python text editors, so 
> there's incentive to use them. A type provider in Swift could address not 
> only the syntax gap but the tooling gap from directly writing Python, since 
> it could answer name lookup requests into an imported Python type by static 
> analysis, and it could bring that declaration into Swift with doc comments 
> and other metadata intact. This would let Python decls get wired up to 
> Swift's code completion, quick help, and other IDE tooling.

Absolutely, I can see how type providers would provide nice refinement on top 
of basic “anyobject” style dispatch.  We still need to solve the basic problem 
though before providing refinements.

I’m also extremely concerned about how type providers work in practice.  With 
the disclaimer that I barely understand what is going on, it seems like they 
cause an extreme amount of compile-time work.  Implementing them seems to 
require implementing an interpreter into the compiler.  This is extremely 
concerning to me: for the same amount of complexity, we could burn many 
languages worth of special cases into Swift.

Since you’re apparently familiar with F#, are you aware of any efforts to use 
F# type providers to interoperate with dynamic languages?  How well does that 
actually work in practice?

> Since statically-available type info still isn't going to be perfect in 
> Python, and we don't want to be boxed in to type-annotated APIs, a type 
> provider could be lax with its name lookup behavior and allow any known 
> Python method to be found on any Python type, much like AnyObject's behavior 
> for ObjC methods (and, like AnyObject, maybe tag them with IUO types so that 
> type safety aficionados can test for their presence, and prioritizing the 
> "good" type-matching methods in code completion results). 

Right, but there is a very simple and closed problem to solve here.  Type 
providers don’t solve it, and so now you’re basically saying that type 
providers could be extended to provide exactly the same functionality I’m 
describing, but in a far more opaque, difficult to use, and significantly more 
compile time intensive approach.

Type providers may be a cool thing in principle, but if you’re going to 
advocate for them, I think we should start with a discussion about whether they 
would ever be realistically implemented in Swift (ignoring schedule).  Further, 
whatever extension you’re imagining to solve the “anyobject dispatch” problem 
will require making Swift types “Callable" anyways (so the type provider can 
provide that type).  

At that point, I would ask why type providers are in the critical path.  It 
seems obvious to implement the underlying type that the providers would provide 
first.  It’s not like we said we should wait to introduce structs until type 
providers exist.

-Chris



___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Matthew Johnson via swift-evolution]

Sent from my iPad

> On Nov 11, 2017, at 5:27 PM, Chris Lattner  wrote:
> 
> Ok, which differences?
> 
> -Chris
> 
>> On Nov 11, 2017, at 2:19 PM, Matthew Johnson via swift-evolution 
>>  wrote:
>> 
>> 
>> 
>> Sent from my iPad
>> 
>>> On Nov 11, 2017, at 11:40 AM, Chris Lattner  wrote:
>>> 
>>> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>>>  wrote:
>> On Nov 10, 2017, at 11:25 AM, Matthew Johnson via swift-evolution 
>>  wrote:
>> 
 
 People have reasonably asked for the ability to make their own 
 function-like types in the past, such that "myvalue(...)" behaves like 
 sugar for "myvalue.call(...)" or something like that. In most cases, 
 they still want to have type system control over what arguments and 
 results their call operation produces. They don't really get that with 
 this proposal; they lose all control over the arity and argument 
 types. 
>>> 
>>> As I mentioned, this is directly addressed in the writeup. Here’s the 
>>> link:
>>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
>> 
>> That discusses why you didn’t include it in the present proposal but I 
>> think it’s reasonable to oppose adding a dynamic callable feature prior 
>> to a more Swifty static callable.
> 
> Why?  One does not preclude the other.
 
 For exactly the reason Joe articulates.  Some people will use what the 
 language offers to get the syntax they desire even if it sacrifices type 
 safety.  If we’re going to have first-class callable types in Swift (I 
 think it’s a great idea) type safety for native code should be prioritized 
 over syntactic convenience for dynamic language interop.  We can have 
 both, but the former should come first IMO.
>>> 
>>> Hi Matthew,
>>> 
>>> In point of fact, Swift already has the feature you are referring to.  It 
>>> just spells it with square brackets instead of parentheses.  A simple 
>>> change to the punctuation character has much less point than the proposal 
>>> that I’m pitching.
>> 
>> This is true if you squint, but I imagine a design for callable types would 
>> include some 
>>  differences other than just punctuation.

I think there was a pretty good discussion of this distinction in the  SE-0021 
timeframe.  Specifically, subscripts model storage while callable would model 
functions.  This means that subscripts can be used in a key path while callable 
would not (at last for now).  

On the other hand, I would want to see callables implicitly convert to a value 
of a compatible function type (possibly through a subtype relationship). 
Without this capability the syntactic illusion is only half complete.  The 
callable feels like a function when used directly but not when it is assigned 
to a function type or passed elsewhere - it must be explicitly be wrapped in a 
closure and the arguments forwarded.  First-class callable types are a 
syntactic sugar feature and as such if they exist they should be usable 
syntactically in all of the same ways functions are.

Read-only subscripts are ok as a workaround in absence of first-class callable 
types (although I think in many ways just using `call` is better).  
Nevertheless, they are a workaround and can never be more than that because of 
the fundamental difference in what they model.

Matthew

>> 
>>> 
>>> -Chris
>>> 
>>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Andrew Bennett via swift-evolution]

Thanks for taking the time to respond. I think my confusion started from a
misinterpretation that each method you're exposing from python would need a
new implementation of the protocol. I've worked out my mistake, and you
helped clarified with:

> The system I’ve sketched out does not incorporate Python declarations,
everything is dynamic

You've addressed the other things that didn't stem from that :)

Thanks,
Andrew

On Sun, Nov 12, 2017 at 12:27 PM, Chris Lattner  wrote:

> On Nov 11, 2017, at 4:15 PM, Andrew Bennett  wrote:
>
> HI, this proposal looks really interesting!
>
>
> Thanks!
>
> *Clarity on the proposal's intent*
> *Nice cheap bridges, or lowering barriers to bridging?*
>
>
> The later: I’m not overly concerned about constant time performance:
> dynamic languages are often slow (though exceptions exist, e.g.
> Javascript).  So if there is a beautiful approach that works great but is a
> bit slower, that is fine with me.
>
> My desire is to enable access to “any” API in Python directly from Swift
> with natural syntax.  This doesn’t mean it has to match Python of course:
> Python ranges and Swift ranges are syntactically very different, and
> Swift-in-Python should use Swift syntax.  That said, there are a LOT of
> expression level similarities between Swift and Python, so things that can
> be the same should be.
>
> This proposal isn’t the only way to achieve that, of course.  I’m open to
> other suggestions.
>
> I can see this providing a nice quick interface to Python from Swift, but
> I'm not sure if the exposed interface will be very Swifty (you probably
> have a much better idea of what is Swifty ;) than I do though). It seems
> you want it to be possible for everything to be dynamically exposed
>
>
> Right: Python APIs are not “Swifty”.  They do not use optionals or
> generics, use lowercase snake names, etc.  IMO, that’s perfectly fine.
> Unlike the Objective-C interop in Swift - which aims to make legacy ObjC
> APIs feel Swifty, it is perfectly acceptable (and far less impact on the
> compiler) to just import Python directly into Swift.  This is actually much
> better for the target audience anyway.
>
> Is it common for the the argument labels in other languages to be open
> ended, or are labels typically finite? If the answer is finite, why not use
> a Swift method as the wrapper?
>
>
> Python keyword arguments are open ended.  They get passed to the Python
> API as a dictionary.
>
> Do you want duck typing, and would it be better to expose this via a
> protocol?
>
>
> I’m not sure how that can work, can you elaborate?
>
> It seems like in almost every case you could do something like this:
>
> func myMethod(a: X?
> = nil, b: Y) {
> pythonBridge.call("myMethod", arguments: ["a": X, "b": Y])
> }
>
>
> The currency type is PyVal (name TBD of course), just like AnyObject for
> Objective-C. I’m not sure what the code above is getting at.
>
> It might be good to add some *use-cases* (a popular Python library
> perhaps) to the proposal where this type of bridge would be insufficient :).
>
>
> There are lots of examples.  In data science, NumPy and Pandas are two
> examples.  The Python community is 1 or 2 orders of magnitude larger than
> Swift’s community and there is 25 years of code out there to interop with.
> The point of this is to make it all accessible.
>
> It seems like this proposal pushes the responsibility of Swifty-ness and
> type-safety to the caller.
>
>
> Yes, welcome to dynamic languages :-)
>
> At some point you'll have to write a type-safe bridging layer, or write
> your entire program in non-Swifty code ("The most obvious way to write code
> should also behave in a safe manner"). Is the main goal to *lower the
> barrier to Python* and other dynamic languages? or is it to provide a
> cheap nice Swifty bridge? I have the above concerns about the latter.
>
>
> It’s the later.  Many Python APIs are already wrappers around C code, so
> if someone cares about investing a bunch of effort into making a great
> Swift API, it would generally make sense to wrap the C API in Swift, not
> wrap the Python API in Swift.
>
> *Alternative sugar*
>
> Ruby has Keyword Arguments for similar sugar:
>
> *def* foo(regular, hash={})
> *puts* "the hash: #{hash}"
>
>
> I'm sure you're aware of it, but I'll explain for completeness, any
> trailing argument labels are stored and passed as a hash:
>
> foo(regular, bar: "hello", bas: 123) *# outputs 'the hash: [**bar: "hello", 
> bas: 123]’*
>
> Python is similar, but allows them to be intermixed with positional
> arguments.
>
> Have you considered an alternative like this? For example:
>
> func myMethod(regular: Int, store: @argcapture [String: PythonConvertible])
> -> PythonConvertible
>
> I'm sure you have good reasons, it might make the implementation bleed out
> into other parts of the codebase. It would be good to include it in the
> proposal *alternatives* section though. At the moment most of the

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 1:57 PM, Chris Lattner  wrote:
> 
> 
> 
>>> On Nov 11, 2017, at 10:17 AM, Chris Lattner via swift-evolution 
>>>  wrote:
>>> 
>>> so I don't think there's a time constraint forcing us to consider the 
>>> "narrow" vs "huge" dimension. What's the best thing for the language and 
>>> tools in the long term? This is a feature that influences the semantics of 
>>> potentially any call site in all Swift code, which we'd have to live with 
>>> forever if we accepted it now. Opening up the compiler architecture to make 
>>> custom importers easier to write is a great solution to a ton of problems, 
>>> including yours I think, without adding complexity to the core language. 
>>> Experience in .NET land seems to show it's a great technique for 
>>> integrating dynamic systems with static type systems, without poking 
>>> unnecessary holes in the static language's type system
>> 
>> As I mentioned, I’m not familiar with F# type providers.  As I told you 
>> upthread, I will read about them and respond with an informed opinion when I 
>> have time.
> 
> I have done some quick reading about type providers, but I am certainly not 
> an expert.  I have a couple of questions that I’d appreciate your insight on:
> 
> Type providers seem like a (complicated!) compile-time form of meta 
> programming or compiler plugin.  How does this help interoperability with 
> dynamic languages, where the preponderance of values have types that are not 
> knowable at compile time?
> 
> To pick a specific example you are familiar with, this seems like it could 
> help in theory (though I suspect it would be be entirely impractical in 
> practice) to provide types for Objective-C methods that are coming from an 
> SDK where you have declarations.  
> 
> In contrast, it does *not* seem like it would help with the problem of 
> resolving members of “id” typed values in Objective-C, which is the crux of 
> the dynamic language problem.  If my understanding is correct, type providers 
> seem completely unhelpful for python interoperability, because you don’t know 
> the types of python values: anything loaded from a Python property or 
> returned by a Python function is the equivalent of “id”.
> 
> It’s entirely possible I’m missing something big.  If so, I’d appreciate a 
> pointer to something to read that explains how this works.  Thanks!

A type provider should let you provide custom logic for something the compiler 
sees as a module, which can answer name lookup queries by producing decls based 
on externally-available information. While Python is dynamic, Python culture 
doesn't go as wild with dynamic shenanigans as other languages do; there are 
established idioms for communicating some level of type information in doc 
comments (or more recently with first class syntax in Python 3), and people 
like code completion to work with their Python text editors, so there's 
incentive to use them. A type provider in Swift could address not only the 
syntax gap but the tooling gap from directly writing Python, since it could 
answer name lookup requests into an imported Python type by static analysis, 
and it could bring that declaration into Swift with doc comments and other 
metadata intact. This would let Python decls get wired up to Swift's code 
completion, quick help, and other IDE tooling.

Since statically-available type info still isn't going to be perfect in Python, 
and we don't want to be boxed in to type-annotated APIs, a type provider could 
be lax with its name lookup behavior and allow any known Python method to be 
found on any Python type, much like AnyObject's behavior for ObjC methods (and, 
like AnyObject, maybe tag them with IUO types so that type safety aficionados 
can test for their presence, and prioritizing the "good" type-matching methods 
in code completion results). I don't think that's a totally silly idea; much 
like AnyObject lookup is strictly an ObjC interop feature, I think there'd be 
benefit to keeping lax name lookup behavior a Python interop facilitator rather 
than a general language feature.

-Joe___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 11, 2017, at 4:15 PM, Andrew Bennett  wrote:
> HI, this proposal looks really interesting!

Thanks!

> Clarity on the proposal's intent
> Nice cheap bridges, or lowering barriers to bridging?

The later: I’m not overly concerned about constant time performance: dynamic 
languages are often slow (though exceptions exist, e.g. Javascript).  So if 
there is a beautiful approach that works great but is a bit slower, that is 
fine with me.

My desire is to enable access to “any” API in Python directly from Swift with 
natural syntax.  This doesn’t mean it has to match Python of course: Python 
ranges and Swift ranges are syntactically very different, and Swift-in-Python 
should use Swift syntax.  That said, there are a LOT of expression level 
similarities between Swift and Python, so things that can be the same should be.

This proposal isn’t the only way to achieve that, of course.  I’m open to other 
suggestions.

> I can see this providing a nice quick interface to Python from Swift, but I'm 
> not sure if the exposed interface will be very Swifty (you probably have a 
> much better idea of what is Swifty ;) than I do though). It seems you want it 
> to be possible for everything to be dynamically exposed


Right: Python APIs are not “Swifty”.  They do not use optionals or generics, 
use lowercase snake names, etc.  IMO, that’s perfectly fine.  Unlike the 
Objective-C interop in Swift - which aims to make legacy ObjC APIs feel Swifty, 
it is perfectly acceptable (and far less impact on the compiler) to just import 
Python directly into Swift.  This is actually much better for the target 
audience anyway.

> Is it common for the the argument labels in other languages to be open ended, 
> or are labels typically finite? If the answer is finite, why not use a Swift 
> method as the wrapper?

Python keyword arguments are open ended.  They get passed to the Python API as 
a dictionary.

> Do you want duck typing, and would it be better to expose this via a protocol?

I’m not sure how that can work, can you elaborate?

> It seems like in almost every case you could do something like this:
> 
> func myMethod(a: X? = 
> nil, b: Y) {
> pythonBridge.call("myMethod", arguments: ["a": X, "b": Y])
> }

The currency type is PyVal (name TBD of course), just like AnyObject for 
Objective-C. I’m not sure what the code above is getting at.

> It might be good to add some use-cases (a popular Python library perhaps) to 
> the proposal where this type of bridge would be insufficient :).

There are lots of examples.  In data science, NumPy and Pandas are two 
examples.  The Python community is 1 or 2 orders of magnitude larger than 
Swift’s community and there is 25 years of code out there to interop with.  The 
point of this is to make it all accessible.

> It seems like this proposal pushes the responsibility of Swifty-ness and 
> type-safety to the caller.

Yes, welcome to dynamic languages :-)

> At some point you'll have to write a type-safe bridging layer, or write your 
> entire program in non-Swifty code ("The most obvious way to write code should 
> also behave in a safe manner"). Is the main goal to lower the barrier to 
> Python and other dynamic languages? or is it to provide a cheap nice Swifty 
> bridge? I have the above concerns about the latter.

It’s the later.  Many Python APIs are already wrappers around C code, so if 
someone cares about investing a bunch of effort into making a great Swift API, 
it would generally make sense to wrap the C API in Swift, not wrap the Python 
API in Swift.

> Alternative sugar
> 
> Ruby has Keyword Arguments for similar sugar:
> 
> def foo(regular, hash={})
> puts "the hash: #{hash}"
> 
> 
> I'm sure you're aware of it, but I'll explain for completeness, any trailing 
> argument labels are stored and passed as a hash:
> 
> foo(regular, bar: "hello", bas: 123) # outputs 'the hash: [bar: "hello", bas: 
> 123]’
Python is similar, but allows them to be intermixed with positional arguments.

> Have you considered an alternative like this? For example:
> 
> func myMethod(regular: Int, store: @argcapture [String: PythonConvertible]) 
> -> PythonConvertible
> 
> I'm sure you have good reasons, it might make the implementation bleed out 
> into other parts of the codebase. It would be good to include it in the 
> proposal alternatives section though. At the moment most of the 
> "alternatives" in the proposal just seem to be extensions to the same 
> solution :)

I’m not sure what you’re getting at here.  The system I’ve sketched out does 
not incorporate Python declarations, everything is dynamic.  It is extremely 
simple.  This is a huge feature :-)  I think it is entirely non-scalable to do 
something like the ObjC importer for every language that Swift wants to interop 
with.

> Clarity
> Perhaps just that things are more clear to me now
> 
> If my extrapolation is correct a user will implement a single type that will 
> allow a subset of a good 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Howard Lovatt via swift-evolution]

In Java you can have annotation processors, user supplied compiler extensions, 
that are extensively used for making frameworks easier to use and for general 
interfacing to ‘foreign stuff’.

Would a limited form of user supplied compiler extension be an alternative, an 
import processor. Special import syntax, import beginning with ‘Foreign’, would 
be recognised by the compiler and a user supplied import processor would be 
called, in the case of the example for Python. The import processor would 
generate the glue code. Below is a basic example to give a feel for what I am 
proposing.

(Syntax, names, etc. in code below just the 1st that came into my head!)

  — Howard.

==

import Foreign.Python.Foo // Special import tag `Foreign` calls a user supplied 
compiler extension that parses the imported Python and generates something 
along these lines (its up to the importer the exact code generated!):

protocol _ForeignPythonCallable {
func call(method: Any, arguments: Any...) throws -> Any
}

extension _ForeignPythonCallable {
func call(method: Any, arguments: Any...) throws -> Any {
return "method: \(method), arguments: \(arguments)" // Method lookup 
and calling code for Python.
}
}

class Foo: _ForeignPythonCallable {
// Could override `call` if more efficient than complete general lookup was 
possible.
enum _ForeignPythonMethodNames { // This might be a C struct so that it 
matches perfectly what the python interpreter is expecting.
case bar
}
enum _ForeignPythonMethodBarArguments { // This might be a C struct so that 
it matches perfectly what the python interpreter is expecting.
case x(Any)
case noName1(Any)
case y(Any)
}
func bar(x: Any, _ noName1: Any, y: Any) -> Any {
do {
return try call(method: _ForeignPythonMethodNames.bar, arguments: 
_ForeignPythonMethodBarArguments.x(x), 
_ForeignPythonMethodBarArguments.noName1(noName1), 
_ForeignPythonMethodBarArguments.y(y))
} catch {
fatalError("Method `bar` does not throw, therefore Python importer 
bug.")
}
}
}

// Then the consumer of `Foo` uses as per normal Swift:

let foo = Foo()
foo.bar(x: 1, 23, y: 17)

class TypedFoo: Foo { // For many dynamic languages this wouldn't be possible 
to automatically generate via the import, therefore would have to be hand 
written.
func bar(x: Int, _ noName1: Int, y: Int) -> String {
return super.bar(x: x, noName1, y: y) as! String
}
}
let typedFoo = TypedFoo()
typedFoo.bar(x: 1, 23, y: 17)


> On 12 Nov 2017, at 11:15 am, Andrew Bennett via swift-evolution 
>  wrote:
> 
> HI, this proposal looks really interesting!
> 
> I have a few questions:
> 
> Clarity on the proposal's intent
> Nice cheap bridges, or lowering barriers to bridging?
> 
> I can see this providing a nice quick interface to Python from Swift, but I'm 
> not sure if the exposed interface will be very Swifty (you probably have a 
> much better idea of what is Swifty ;) than I do though). It seems you want it 
> to be possible for everything to be dynamically exposed, I've used similar 
> with Lua's meta methods, and I found it to be very powerful, you could 
> basically implement inheritance in the language, which wasn't necessarily a 
> good thing in retrospect.
> 
> Is it common for the the argument labels in other languages to be open ended, 
> or are labels typically finite? If the answer is finite, why not use a Swift 
> method as the wrapper?
> Do you want duck typing, and would it be better to expose this via a protocol?
> 
> It seems like in almost every case you could do something like this:
> 
> func myMethod(a: X? = 
> nil, b: Y) {
> pythonBridge.call("myMethod", arguments: ["a": X, "b": Y])
> }
> 
> It might be good to add some use-cases (a popular Python library perhaps) to 
> the proposal where this type of bridge would be insufficient :).
> 
> It seems like this proposal pushes the responsibility of Swifty-ness and 
> type-safety to the caller. At some point you'll have to write a type-safe 
> bridging layer, or write your entire program in non-Swifty code ("The most 
> obvious way to write code should also behave in a safe manner"). Is the main 
> goal to lower the barrier to Python and other dynamic languages? or is it to 
> provide a cheap nice Swifty bridge? I have the above concerns about the 
> latter.
> 
> Alternative sugar
> 
> Ruby has Keyword Arguments for similar sugar:
> 
> def foo(regular, hash={})
> puts "the hash: #{hash}"
> 
> 
> I'm sure you're aware of it, but I'll explain for completeness, any trailing 
> argument labels are stored and passed as a hash:
> 
> foo(regular, bar: "hello", bas: 123) # outputs 'the hash: [bar: "hello", bas: 
> 123]'
> Have you considered an alternative like this? For example:
> 
> func myMethod(regular: Int, store: @argcapture [String: PythonConvertible]) 
> -> 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Andrew Bennett via swift-evolution]

HI, this proposal looks really interesting!

I have a few questions:

*Clarity on the proposal's intent*
*Nice cheap bridges, or lowering barriers to bridging?*

I can see this providing a nice quick interface to Python from Swift, but
I'm not sure if the exposed interface will be very Swifty (you probably
have a much better idea of what is Swifty ;) than I do though). It seems
you want it to be possible for everything to be dynamically exposed, I've
used similar with Lua's meta methods, and I found it to be very powerful,
you could basically implement inheritance in the language, which wasn't
necessarily a good thing in retrospect.

Is it common for the the argument labels in other languages to be open
ended, or are labels typically finite? If the answer is finite, why not use
a Swift method as the wrapper?
Do you want duck typing, and would it be better to expose this via a
protocol?

It seems like in almost every case you could do something like this:

func myMethod(a: X?
= nil, b: Y) {

pythonBridge.call("myMethod", arguments: ["a": X, "b": Y])

}

It might be good to add some *use-cases* (a popular Python library perhaps)
to the proposal where this type of bridge would be insufficient :).

It seems like this proposal pushes the responsibility of Swifty-ness and
type-safety to the caller. At some point you'll have to write a type-safe
bridging layer, or write your entire program in non-Swifty code ("The most
obvious way to write code should also behave in a safe manner"). Is the
main goal to *lower the barrier to Python* and other dynamic languages? or
is it to provide a cheap nice Swifty bridge? I have the above concerns
about the latter.

*Alternative sugar*

Ruby has Keyword Arguments for similar sugar:

*def* foo(regular, hash={})
*puts* "the hash: #{hash}"


I'm sure you're aware of it, but I'll explain for completeness, any
trailing argument labels are stored and passed as a hash:

foo(regular, bar: "hello", bas: 123) *# outputs 'the hash: [**bar:
"hello", bas: 123]'*

Have you considered an alternative like this? For example:

func myMethod(regular: Int, store: @argcapture [String: PythonConvertible])
-> PythonConvertible

I'm sure you have good reasons, it might make the implementation bleed out
into other parts of the codebase. It would be good to include it in the
proposal *alternatives* section though. At the moment most of the
"alternatives" in the proposal just seem to be extensions to the same
solution :)

*Clarity*
*Perhaps just that things are more clear to me now*

If my extrapolation is correct a user will implement a single type that
will allow a subset of a good portion of another language to be exposed
(object method and property bridging). I'm guessing that the dynamic member
proposal you're planning will not work with methods, it will require a
property, I think this helps explain some of the motivations. It might be
nice to have a more complete example that includes dynamic members. I
didn't find it clear from the proposal that it would only be necessary to
implement this protocol once per language.

Thanks for considering my questions,
Andrew Bennett


On Sat, Nov 11, 2017 at 4:37 AM, Chris Lattner via swift-evolution <
swift-evolution@swift.org> wrote:

> Hello all,
>
> I have a couple of proposals cooking in a quest to make Swift interoperate
> with dynamically typed languages like Python better.  Instead of baking in
> hard coded support for one language or the other, I’m preferring to add a
> few small but general purpose capabilities to Swift.  This is the first,
> which allows a Swift type to become “callable”.
>
> The proposal is here:
> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
>
> I’ve also attached a snapshot below, but it will drift out of date as the
> proposal is refined.  Feedback and thoughts are appreciated, thanks!
>
> -Chris
>
>
>
>
>
> Introduce user-defined dynamically "callable" types
>
>- Proposal: SE-
>
>- Author: Chris Lattner 
>- Review Manager: TBD
>- Status: Awaiting implementation
>
>
> 
> Introduction
>
> This proposal introduces a new DynamicCallable protocol to the standard
> library. Types that conform to it are "callable" with the function call
> syntax. It is simple syntactic sugar which allows the user to write:
>
> a = someValue(keyword1: 42, "foo", keyword2: 19)
>
> and have it be interpreted by the compiler as:
>
>   a = someValue.dynamicCall(arguments: [
> ("keyword1", 42), ("", "foo"), ("keyword2", 19)
>   ])
>
> Other languages have analogous features (e.g. Python "callables"), but the
> primary motivation of this proposal is to allow elegant and natural
> interoperation with dynamic languages in Swift.
>
> Swift-evolution thread: Discussion thread topic for that proposal
> 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

That makes sense. I can see how type providers are very handy for that.  I’m 
just curious how they help with the goal that this proposal is intended to 
address: being able to use arbitrary Python APIs from Swift.

If the barrier to entry is the requirement for users to write type signatures 
for all APIs you want to use, then many will just immediately turn away at the 
gate.  It also doesn’t appear to help *actually* dynamically polymorphic 
values, which occur in Python just like the occur in ObjC.

Type providers seem like something that conceptually could be a useful 
refinement for my pitch: improving some APIs in some cases.  I don’t see how it 
is a replacement, which seems to be what Joe was saying.  

-Chris

> On Nov 11, 2017, at 2:17 PM, Alejandro Martinez  wrote:
> 
> If it helps, for what I understand (in Typescript with things like
> http://definitelytyped.org ) the idea with type providers is that
> someone write the type definitions for dynamic code to basically
> provide a more "typed" version of the API that can be consumed in a
> more static language.
> 
> On Sat, Nov 11, 2017 at 9:57 PM, Chris Lattner via swift-evolution
>  wrote:
>> 
>> 
>> On Nov 11, 2017, at 10:17 AM, Chris Lattner via swift-evolution
>>  wrote:
>> 
>> so I don't think there's a time constraint forcing us to consider the
>> "narrow" vs "huge" dimension. What's the best thing for the language and
>> tools in the long term? This is a feature that influences the semantics of
>> potentially any call site in all Swift code, which we'd have to live with
>> forever if we accepted it now. Opening up the compiler architecture to make
>> custom importers easier to write is a great solution to a ton of problems,
>> including yours I think, without adding complexity to the core language.
>> Experience in .NET land seems to show it's a great technique for integrating
>> dynamic systems with static type systems, without poking unnecessary holes
>> in the static language's type system
>> 
>> 
>> As I mentioned, I’m not familiar with F# type providers.  As I told you
>> upthread, I will read about them and respond with an informed opinion when I
>> have time.
>> 
>> 
>> I have done some quick reading about type providers, but I am certainly not
>> an expert.  I have a couple of questions that I’d appreciate your insight
>> on:
>> 
>> Type providers seem like a (complicated!) compile-time form of meta
>> programming or compiler plugin.  How does this help interoperability with
>> dynamic languages, where the preponderance of values have types that are not
>> knowable at compile time?
>> 
>> To pick a specific example you are familiar with, this seems like it could
>> help in theory (though I suspect it would be be entirely impractical in
>> practice) to provide types for Objective-C methods that are coming from an
>> SDK where you have declarations.
>> 
>> In contrast, it does *not* seem like it would help with the problem of
>> resolving members of “id” typed values in Objective-C, which is the crux of
>> the dynamic language problem.  If my understanding is correct, type
>> providers seem completely unhelpful for python interoperability, because you
>> don’t know the types of python values: anything loaded from a Python
>> property or returned by a Python function is the equivalent of “id”.
>> 
>> It’s entirely possible I’m missing something big.  If so, I’d appreciate a
>> pointer to something to read that explains how this works.  Thanks!
>> 
>> -Chris
>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
> 
> 
> 
> -- 
> Alejandro Martinez
> http://alejandromp.com
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

Ok, which differences?

-Chris

> On Nov 11, 2017, at 2:19 PM, Matthew Johnson via swift-evolution 
>  wrote:
> 
> 
> 
> Sent from my iPad
> 
>> On Nov 11, 2017, at 11:40 AM, Chris Lattner  wrote:
>> 
>> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>>  wrote:
> On Nov 10, 2017, at 11:25 AM, Matthew Johnson via swift-evolution 
>  wrote:
> 
>>> 
>>> People have reasonably asked for the ability to make their own 
>>> function-like types in the past, such that "myvalue(...)" behaves like 
>>> sugar for "myvalue.call(...)" or something like that. In most cases, 
>>> they still want to have type system control over what arguments and 
>>> results their call operation produces. They don't really get that with 
>>> this proposal; they lose all control over the arity and argument types. 
>> 
>> As I mentioned, this is directly addressed in the writeup. Here’s the 
>> link:
>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
> 
> That discusses why you didn’t include it in the present proposal but I 
> think it’s reasonable to oppose adding a dynamic callable feature prior 
> to a more Swifty static callable.
 
 Why?  One does not preclude the other.
>>> 
>>> For exactly the reason Joe articulates.  Some people will use what the 
>>> language offers to get the syntax they desire even if it sacrifices type 
>>> safety.  If we’re going to have first-class callable types in Swift (I 
>>> think it’s a great idea) type safety for native code should be prioritized 
>>> over syntactic convenience for dynamic language interop.  We can have both, 
>>> but the former should come first IMO.
>> 
>> Hi Matthew,
>> 
>> In point of fact, Swift already has the feature you are referring to.  It 
>> just spells it with square brackets instead of parentheses.  A simple change 
>> to the punctuation character has much less point than the proposal that I’m 
>> pitching.
> 
> This is true if you squint, but I imagine a design for callable types would 
> include some 
>  differences other than just punctuation.
> 
>> 
>> -Chris
>> 
>> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Matthew Johnson via swift-evolution]

Sent from my iPad

> On Nov 11, 2017, at 11:40 AM, Chris Lattner  wrote:
> 
> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>  wrote:
 On Nov 10, 2017, at 11:25 AM, Matthew Johnson via swift-evolution 
  wrote:
 
>> 
>> People have reasonably asked for the ability to make their own 
>> function-like types in the past, such that "myvalue(...)" behaves like 
>> sugar for "myvalue.call(...)" or something like that. In most cases, 
>> they still want to have type system control over what arguments and 
>> results their call operation produces. They don't really get that with 
>> this proposal; they lose all control over the arity and argument types. 
> 
> As I mentioned, this is directly addressed in the writeup. Here’s the 
> link:
> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
 
 That discusses why you didn’t include it in the present proposal but I 
 think it’s reasonable to oppose adding a dynamic callable feature prior to 
 a more Swifty static callable.
>>> 
>>> Why?  One does not preclude the other.
>> 
>> For exactly the reason Joe articulates.  Some people will use what the 
>> language offers to get the syntax they desire even if it sacrifices type 
>> safety.  If we’re going to have first-class callable types in Swift (I think 
>> it’s a great idea) type safety for native code should be prioritized over 
>> syntactic convenience for dynamic language interop.  We can have both, but 
>> the former should come first IMO.
> 
> Hi Matthew,
> 
> In point of fact, Swift already has the feature you are referring to.  It 
> just spells it with square brackets instead of parentheses.  A simple change 
> to the punctuation character has much less point than the proposal that I’m 
> pitching.

This is true if you squint, but I imagine a design for callable types would 
include some 
 differences other than just punctuation.

> 
> -Chris
> 
> 
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Alejandro Martinez via swift-evolution]

If it helps, for what I understand (in Typescript with things like
http://definitelytyped.org ) the idea with type providers is that
someone write the type definitions for dynamic code to basically
provide a more "typed" version of the API that can be consumed in a
more static language.

On Sat, Nov 11, 2017 at 9:57 PM, Chris Lattner via swift-evolution
 wrote:
>
>
> On Nov 11, 2017, at 10:17 AM, Chris Lattner via swift-evolution
>  wrote:
>
> so I don't think there's a time constraint forcing us to consider the
> "narrow" vs "huge" dimension. What's the best thing for the language and
> tools in the long term? This is a feature that influences the semantics of
> potentially any call site in all Swift code, which we'd have to live with
> forever if we accepted it now. Opening up the compiler architecture to make
> custom importers easier to write is a great solution to a ton of problems,
> including yours I think, without adding complexity to the core language.
> Experience in .NET land seems to show it's a great technique for integrating
> dynamic systems with static type systems, without poking unnecessary holes
> in the static language's type system
>
>
> As I mentioned, I’m not familiar with F# type providers.  As I told you
> upthread, I will read about them and respond with an informed opinion when I
> have time.
>
>
> I have done some quick reading about type providers, but I am certainly not
> an expert.  I have a couple of questions that I’d appreciate your insight
> on:
>
> Type providers seem like a (complicated!) compile-time form of meta
> programming or compiler plugin.  How does this help interoperability with
> dynamic languages, where the preponderance of values have types that are not
> knowable at compile time?
>
> To pick a specific example you are familiar with, this seems like it could
> help in theory (though I suspect it would be be entirely impractical in
> practice) to provide types for Objective-C methods that are coming from an
> SDK where you have declarations.
>
> In contrast, it does *not* seem like it would help with the problem of
> resolving members of “id” typed values in Objective-C, which is the crux of
> the dynamic language problem.  If my understanding is correct, type
> providers seem completely unhelpful for python interoperability, because you
> don’t know the types of python values: anything loaded from a Python
> property or returned by a Python function is the equivalent of “id”.
>
> It’s entirely possible I’m missing something big.  If so, I’d appreciate a
> pointer to something to read that explains how this works.  Thanks!
>
> -Chris
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>



-- 
Alejandro Martinez
http://alejandromp.com
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Eagle Offshore via swift-evolution]

They're called Blocks.  Objective C has them.  They're objects.  They're 
actually Objective C objects.

> On Nov 10, 2017, at 10:26 AM, Florent Vilmart via swift-evolution 
>  wrote:
> 
> Object that are functions too, Amazing! I wanted that in Javascript for a 
> while!
> 
> On Nov 10, 2017, 1:04 PM -0500, Joe Groff via swift-evolution 
> , wrote:
>> I don't like the idea of some calls having wildly different semantics from 
>> others; it's difficult enough to tell what exactly a call might be doing 
>> already. Since we also lack the more obvious static "Callable" protocol idea 
>> to give even well-typed call syntax to user-defined types, this also seems 
>> like it'd be easily abused for that purpose too.
>> 
>> I think a much better general solution to the problem of "make dynamic 
>> systems interact with type systems" is something like F#'s type providers 
>> which lets you write your own importers that look at dynamic information 
>> from a database, dynamic language VM, or some other system and generate type 
>> information usable by the compiler. Integration at the importer level could 
>> let you produce more well-typed Swift declarations by looking at the runtime 
>> information you get by importing a Python module.
>> 
>> -Joe
>> 
>>> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>>>  wrote:
>>> 
>>> Hello all,
>>> 
>>> I have a couple of proposals cooking in a quest to make Swift interoperate 
>>> with dynamically typed languages like Python better. Instead of baking in 
>>> hard coded support for one language or the other, I’m preferring to add a 
>>> few small but general purpose capabilities to Swift. This is the first, 
>>> which allows a Swift type to become “callable”.
>>> 
>>> The proposal is here:
>>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
>>> 
>>> I’ve also attached a snapshot below, but it will drift out of date as the 
>>> proposal is refined. Feedback and thoughts are appreciated, thanks!
>>> 
>>> -Chris
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Introduce user-defined dynamically "callable" types
>>> 
>>> • Proposal: SE-
>>> • Author: Chris Lattner
>>> • Review Manager: TBD
>>> • Status: Awaiting implementation
>>> Introduction
>>> 
>>> This proposal introduces a new DynamicCallable protocol to the standard 
>>> library. Types that conform to it are "callable" with the function call 
>>> syntax. It is simple syntactic sugar which allows the user to write:
>>> 
>>> a = someValue(keyword1: 42, "foo", keyword2: 19)
>>> and have it be interpreted by the compiler as:
>>> 
>>> a = someValue.dynamicCall(arguments
>>> : [
>>> (
>>> "keyword1", 42), ("", "foo"), ("keyword2", 19
>>> )
>>> ])
>>> 
>>> Other languages have analogous features (e.g. Python "callables"), but the 
>>> primary motivation of this proposal is to allow elegant and natural 
>>> interoperation with dynamic languages in Swift.
>>> 
>>> Swift-evolution thread: Discussion thread topic for that proposal
>>> 
>>> Motivation
>>> 
>>> Swift is well known for being exceptional at interworking with existing C 
>>> and Objective-C APIs, but its support for calling APIs written in scripting 
>>> langauges like Python, Perl, and Ruby is quite lacking. These languages 
>>> provide an extremely dynamic programming model where almost everything is 
>>> discovered at runtime.
>>> 
>>> Through the introduction of this proposal, and the related 
>>> DynamicMemberLookupProtocol proposal, we seek to fix this problem. We 
>>> believe we can make many common APIs feel very natural to use directly from 
>>> Swift without all the complexity of implementing something like the Clang 
>>> importer. For example, consider this Python code:
>>> 
>>> class Dog
>>> :
>>> 
>>> def __init__(self, name
>>> ):
>>> 
>>> self.name =
>>> name
>>> 
>>> self.tricks = [] # creates a new empty list for each dog
>>> 
>>> 
>>> 
>>> def add_trick(self, trick
>>> ):
>>> 
>>> self.tricks.append(trick)
>>> we would like to be able to use this from Swift like this (the comments 
>>> show the corresponding syntax you would use in Python):
>>> 
>>> // import DogModule
>>> // import DogModule.Dog as Dog // an alternate
>>> let Dog = Python.import(“DogModule.Dog")
>>> 
>>> // dog = Dog("Brianna")
>>> let dog = Dog("Brianna")
>>> 
>>> // dog.add_trick("Roll over")
>>> dog.add_trick("Roll over")
>>> 
>>> // dog2 = Dog("Kaylee").add_trick("snore")
>>> let dog2 = Dog("Kaylee").add_trick("snore")
>>> Of course, this would also apply to standard Python APIs as well. Here is 
>>> an example working with the Python pickleAPI and the builtin Python 
>>> function open:
>>> 
>>> // import pickle
>>> let pickle = Python.import("pickle"
>>> )
>>> 
>>> 
>>> // file = open(filename)
>>> let file = Python.open
>>> (filename)
>>> 
>>> 
>>> // blob = file.read()
>>> let blob = file.read
>>> ()
>>> 
>>> 
>>> // result = pickle.loads(blob)
>>> let result = pickle.loads(blob)

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 10:17 AM, Chris Lattner via swift-evolution 
>  wrote:
> 
>> so I don't think there's a time constraint forcing us to consider the 
>> "narrow" vs "huge" dimension. What's the best thing for the language and 
>> tools in the long term? This is a feature that influences the semantics of 
>> potentially any call site in all Swift code, which we'd have to live with 
>> forever if we accepted it now. Opening up the compiler architecture to make 
>> custom importers easier to write is a great solution to a ton of problems, 
>> including yours I think, without adding complexity to the core language. 
>> Experience in .NET land seems to show it's a great technique for integrating 
>> dynamic systems with static type systems, without poking unnecessary holes 
>> in the static language's type system
> 
> As I mentioned, I’m not familiar with F# type providers.  As I told you 
> upthread, I will read about them and respond with an informed opinion when I 
> have time.

I have done some quick reading about type providers, but I am certainly not an 
expert.  I have a couple of questions that I’d appreciate your insight on:

Type providers seem like a (complicated!) compile-time form of meta programming 
or compiler plugin.  How does this help interoperability with dynamic 
languages, where the preponderance of values have types that are not knowable 
at compile time?

To pick a specific example you are familiar with, this seems like it could help 
in theory (though I suspect it would be be entirely impractical in practice) to 
provide types for Objective-C methods that are coming from an SDK where you 
have declarations.  

In contrast, it does *not* seem like it would help with the problem of 
resolving members of “id” typed values in Objective-C, which is the crux of the 
dynamic language problem.  If my understanding is correct, type providers seem 
completely unhelpful for python interoperability, because you don’t know the 
types of python values: anything loaded from a Python property or returned by a 
Python function is the equivalent of “id”.

It’s entirely possible I’m missing something big.  If so, I’d appreciate a 
pointer to something to read that explains how this works.  Thanks!

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 10:52 AM, Xiaodi Wu  wrote:
> 
>> On Sat, Nov 11, 2017 at 12:39 PM, Joe Groff via swift-evolution 
>>  wrote:
>> 
>> 
>>> On Nov 11, 2017, at 10:12 AM, Chris Lattner  wrote:
>>> 
>>> On Nov 11, 2017, at 6:52 AM, Joe Groff via swift-evolution 
>>>  wrote:
>> I don't think it's that localized. It's going to make call resolution 
>> massively more complicated.  Through great pain and community anguish, 
>> we pushed ourselves to a model where argument labels are parts of the 
>> declaration name, not part of the call argument, and this would throw us 
>> straight back into the kinds of complications of dealing with overloaded 
>> name- and type-dependent behavior we're trying to get away from in the 
>> language.
> 
> I’m pretty sure it doesn’t, but good news: since Swift requires an 
> implementation before the proposal can be officially reviewed, you will 
> be able to look at the actual patch to make this decision.
 
 And the goal of the pitch phase is to hopefully save you the tears of 
 trying to implement something if it's a bad idea.
>>> 
>>> Appreciated.  Fortunately I enjoy coding so it’s not that big of a deal for 
>>> me.
>>> 
>> Swift is a dynamic language too, and tuples have enough metadata in them 
>> that you could get almost everything you want with an infix operator.
> 
> This is really gross.  The goal here is to make Python feel natural.  If 
> you oppose this proposal then we’ll have to find some other way to 
> provide an elegant experience:
>> x∫(0, "foo", bar: "bas”)
> 
> This is not going to be acceptable to users.
 
 Is it? My choice of operator character is silly, but it's the same amount 
 of syntactic overhead as an optional chain/force call. It's a fair sight 
 better than x.call(args: ..., keywordArgs: ...), I hope you'd agree, and 
 is something that works without any language change.
>>> 
>>> Yes, I’m happy to explore that, and I agree this approach doesn’t require 
>>> language changes.  That said, there are three things you should understand:
>>> 
>>> 1) Details matter.  While you claim that there is some other operator 
>>> character that could be awesome, I’m not seeing it.  Can you try harder to 
>>> come up with something that you think would be acceptable?
>>> 
>>> 2) The point of the entire proposal is that a certain audience will be 
>>> using the feature enabled by this (python interop) *a lot*, so ergonomics 
>>> matter.
>>> 
>>> 3) This feature (python interop) is aimed to get Python programmers to 
>>> convert to Swift, not simply to ease friction for existing Swift 
>>> programmers (most of whom are iOS developers, to whom this feature is 
>>> completely irrelevant).  Getting people to move from Python to Swift is a 
>>> challenge, and syntax does actually matter. I hope we can agree that 
>>> expanding the community of Swift developers is a laudable goal.  
>> 
>> Sure. I suggest it not as a final endpoint but as incremental progress you 
>> can make with the language today. If you can get some early adopters 
>> interested in moving from Python to Swift with some small warts now and the 
>> promise of refined syntax in the future, that could let you make progress 
>> while we work out the right final model for dynamic language interop.
> 
> As a community member, I have to say that this part of the core team response 
> to the pitch has been the most dismaying part. It is quite evident to all, 
> I'm sure, that the ergonomics and approachability of Python is one of the 
> major draws for its audience. If Python interop in Swift is to gain traction 
> with that audience, it is pretty clear that anything short of `foo.bar(42)` 
> is not going to cut it. Overall, the response to a proposed design to achieve 
> that goal seems to be focused not on the design but on that goal itself, and 
> the gist of the response I'm sensing is "not now, if ever," or "current Swift 
> users first, new Swift users later." If a final model for dynamic language 
> interop is what's necessary before any headway is made in this space, then it 
> would seem to me that it is tantamount to shutting down the conversation here.

I'm sorry for discouraging discussion. I shouldn't have implied that this was a 
desirable endpoint, I was trying to figure out how far you can get in the 
language today, and highlight the availability of runtime metadata that could 
make other approaches more viable than the proposal suggested, and I should've 
connected those dots more clearly. To be clear, I think this is an interesting 
problem to solve, and I don't want to shut down conversation. Chris evaluated 
several approaches and picked one, and it's not the one I would pick. 
Ultimately, I want to do what's right for the language. I am not "the Core 
Team" in whole, I'm a 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Xiaodi Wu via swift-evolution]

On Sat, Nov 11, 2017 at 12:39 PM, Joe Groff via swift-evolution <
swift-evolution@swift.org> wrote:

>
>
> On Nov 11, 2017, at 10:12 AM, Chris Lattner  wrote:
>
> On Nov 11, 2017, at 6:52 AM, Joe Groff via swift-evolution <
> swift-evolution@swift.org> wrote:
>
> I don't think it's that localized. It's going to make call resolution
> massively more complicated.  Through great pain and community anguish, we
> pushed ourselves to a model where argument labels are parts of the
> declaration name, not part of the call argument, and this would throw us
> straight back into the kinds of complications of dealing with overloaded
> name- and type-dependent behavior we're trying to get away from in the
> language.
>
>
> I’m pretty sure it doesn’t, but good news: since Swift requires an
> implementation before the proposal can be officially reviewed, you will be
> able to look at the actual patch to make this decision.
>
>
> And the goal of the pitch phase is to hopefully save you the tears of
> trying to implement something if it's a bad idea.
>
>
> Appreciated.  Fortunately I enjoy coding so it’s not that big of a deal
> for me.
>
> Swift is a dynamic language too, and tuples have enough metadata in them
> that you could get almost everything you want with an infix operator.
>
>
> This is really gross.  The goal here is to make Python feel natural.  If
> you oppose this proposal then we’ll have to find some other way to provide
> an elegant experience:
>
> x∫(0, "foo", bar: "bas”)
>
>
> This is not going to be acceptable to users.
>
>
> Is it? My choice of operator character is silly, but it's the same amount
> of syntactic overhead as an optional chain/force call. It's a fair sight
> better than x.call(args: ..., keywordArgs: ...), I hope you'd agree, and is
> something that works without any language change.
>
>
> Yes, I’m happy to explore that, and I agree this approach doesn’t require
> language changes.  That said, there are three things you should understand:
>
> 1) Details matter.  While you claim that there is some other operator
> character that could be awesome, I’m not seeing it.  Can you try harder to
> come up with something that you think would be acceptable?
>
> 2) The point of the entire proposal is that a certain audience will be
> using the feature enabled by this (python interop) *a lot*, so ergonomics
> matter.
>
> 3) This feature (python interop) is aimed to get Python programmers to
> convert to Swift, not simply to ease friction for existing Swift
> programmers (most of whom are iOS developers, to whom this feature is
> completely irrelevant).  Getting people to move from Python to Swift is a
> challenge, and syntax does actually matter. I hope we can agree that
> expanding the community of Swift developers is a laudable goal.
>
>
> Sure. I suggest it not as a final endpoint but as incremental progress you
> can make with the language today. If you can get some early adopters
> interested in moving from Python to Swift with some small warts now and the
> promise of refined syntax in the future, that could let you make progress
> while we work out the right final model for dynamic language interop.
>

As a community member, I have to say that this part of the core team
response to the pitch has been the most dismaying part. It is quite evident
to all, I'm sure, that the ergonomics and approachability of Python is one
of the major draws for its audience. If Python interop in Swift is to gain
traction with that audience, it is pretty clear that anything short of
`foo.bar(42)` is not going to cut it. Overall, the response to a proposed
design to achieve that goal seems to be focused not on the design but on
that goal itself, and the gist of the response I'm sensing is "not now, if
ever," or "current Swift users first, new Swift users later." If a final
model for dynamic language interop is what's necessary before any headway
is made in this space, then it would seem to me that it is tantamount to
shutting down the conversation here.

Since you have long term ambitions in this space, I think it'd pay off to
> consider more infrastructurally-heavy features with bigger long-term
> payoffs.
>
>
> You mentioned variadics as being inadequate for your purposes as a
> considered alternative, but a variadics model that wasn't based on type
> erasure to arrays and kept runtime type information about the argument
> tuple that was passed in, along with better reflection APIs for accessing
> that type info, might lead to something you can use by incrementally
> improving the expressivity of existing language features instead of adding
> new ones.
>
>
> I’m not sure what you mean by this: variadics are fine. The hard part is
> getting the keyword arguments for a call dynamically.  Can you point to
> what you’re referring to in the proposal?
>
>
> I was referring to your comment in this section:
>
> Statically checking for exact signatures
>
> This proposal does not allow a type to specify an 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 10:33 AM, Chris Lattner  wrote:
> 
> 
> 
>> On Nov 11, 2017, at 10:15 AM, Joe Groff via swift-evolution 
>>  wrote:
>> 
>> 
>> 
>>> On Nov 11, 2017, at 10:04 AM, Chris Lattner  wrote:
>>> 
>>> 
>>> 
> On Nov 11, 2017, at 6:29 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> below the fold as far as the rest of the language is concerned. You could 
> just as well written what the importer synths up in Swift directly:
> 
> func foo(bar: String) {
> unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
> NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
> }
> 
> and the rest of the language would be none the wiser.
 
 Though, since you bring up objc_msgSend, the way it works in ObjC might be 
 a better fit for Swift's name lookup model, since their keyword argument 
 models are similar. If Swift had a 'method of last resort' like ObjC's
>>> 
>>> In fact, this is the subject of the next proposal that I will write up when 
>>> I have time.
>> 
>> It seems to me like a dynamic method-of-last-resort feature would subsume 
>> this proposal. Out of curiosity, why would we need both?
> 
> It seems that you missed the "[Discussion] Swift for Data Science / ML / Big 
> Data analytics” thread, in which a lot of context was discussed, as well as 
> multiple design approaches.  

Mea culpa, though data science is not my forte so I wouldn't have noised up a 
thread with that topic without being asked to.

> In any case, I will summarize here:
> 
> There are two fundamental operations that need to be supported to make a big 
> impact on dynamic language interop: member lookup and calls.
> 
> Many languages treat these as separable concepts.  In Python for example, 
> these are equivalent:
> 
>   return foo.bar(x: 1, 23, y: 17)
> and:
>   a = foo.bar
>   return a(x: 1, 23, y: 17)
> 
> Swift and the other Smalltalk inspired languages are the odd ones out who tie 
> method arguments into the base name of a function.  With no language changes, 
> it is possible to implement a pretty reasonable Python interop layer with the 
> call and member operations explicit, it looks like this:
> 
>   func example(foo : PyVal) -> PyVal {
> return  foo.get(member: “bar”).call(args: (“x”, 1), (“”, 23), (“y”, 
> 17))
>   }
> 
> and while it is totally possible to sugar common cases, e.g.:
> 
>   func example(foo : PyVal) -> PyVal {
> return  foo.call(member: “bar”, args: (“x”, 1), (“”, 23), (“y”, 17))
>   }
> 
> you still need to support the basic model because Python supports currying 
> and has global functions with keyword arguments.  FWIW, Perl follows a 
> similar model, but doesn’t have keyword arguments.

Sure, the language models are different, but they're also different languages 
with different idioms, and bridging one to the other IMO doesn't mean it's a 
good idea to port over all the idioms literally. Allowing:

> a = foo.bar
> return a(x: 1, 23, y: 17)

may provide some initial familiarity to someone fresh from Python, but it will 
delay their ability to transition to Swift's way of doing things and confuse 
them when their own Swift code they write doesn't work the Python way. We 
should be able to make the more Swift-idiomatic `a = foo.bar(x:y:); a(1, 23, 
17)` form work.
-Joe

> I don’t really care much about the implementation approach (if this proposal 
> isn’t the right one, fine!) but it is important to get this down to:
> 
>   func example(foo : PyVal) -> PyVal {
> return  foo.bar(x: 1, 23, y: 17)
>   }
> 
> which I hope you’ll agree is a *HUGE* readability and writability benefit, so 
> much so that users would reject the long hand notion.
> 
> 
> The discussion phase in the previous thread considered several approaches, 
> including burning python specific support in, implementing a python importer, 
> and implementing a single “dynamic thing” protocol.  These are all bad ideas 
> for reasons described in the thread.  If you’re curious, I’m happy to 
> elaborate in alternatives considered.
> 
> As such, discussion converged on pitching two proposals, which are generally 
> useful beyond this niche:
> 
> 1. The dynamically callable one already written.
> 2. A dynamically member lookupable proposal to sugar things like 
> foo.get(member: “bar”) - and the corresponding setter.
> 
> The former one is presumably useful for things outside dynamic languages, 
> e.g. implementing dynamic “proxy” type interfaces.
> The second one is useful for far more than just dynamic languages, e.g. 
> sugaring json keypath traversals.
> 
> 
> -Chris
> 
> 
> 
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 10:12 AM, Chris Lattner  wrote:
> 
> On Nov 11, 2017, at 6:52 AM, Joe Groff via swift-evolution 
>  wrote:
 I don't think it's that localized. It's going to make call resolution 
 massively more complicated.  Through great pain and community anguish, we 
 pushed ourselves to a model where argument labels are parts of the 
 declaration name, not part of the call argument, and this would throw us 
 straight back into the kinds of complications of dealing with overloaded 
 name- and type-dependent behavior we're trying to get away from in the 
 language.
>>> 
>>> I’m pretty sure it doesn’t, but good news: since Swift requires an 
>>> implementation before the proposal can be officially reviewed, you will be 
>>> able to look at the actual patch to make this decision.
>> 
>> And the goal of the pitch phase is to hopefully save you the tears of trying 
>> to implement something if it's a bad idea.
> 
> Appreciated.  Fortunately I enjoy coding so it’s not that big of a deal for 
> me.
> 
 Swift is a dynamic language too, and tuples have enough metadata in them 
 that you could get almost everything you want with an infix operator.
>>> 
>>> This is really gross.  The goal here is to make Python feel natural.  If 
>>> you oppose this proposal then we’ll have to find some other way to provide 
>>> an elegant experience:
 x∫(0, "foo", bar: "bas”)
>>> 
>>> This is not going to be acceptable to users.
>> 
>> Is it? My choice of operator character is silly, but it's the same amount of 
>> syntactic overhead as an optional chain/force call. It's a fair sight better 
>> than x.call(args: ..., keywordArgs: ...), I hope you'd agree, and is 
>> something that works without any language change.
> 
> Yes, I’m happy to explore that, and I agree this approach doesn’t require 
> language changes.  That said, there are three things you should understand:
> 
> 1) Details matter.  While you claim that there is some other operator 
> character that could be awesome, I’m not seeing it.  Can you try harder to 
> come up with something that you think would be acceptable?
> 
> 2) The point of the entire proposal is that a certain audience will be using 
> the feature enabled by this (python interop) *a lot*, so ergonomics matter.
> 
> 3) This feature (python interop) is aimed to get Python programmers to 
> convert to Swift, not simply to ease friction for existing Swift programmers 
> (most of whom are iOS developers, to whom this feature is completely 
> irrelevant).  Getting people to move from Python to Swift is a challenge, and 
> syntax does actually matter. I hope we can agree that expanding the community 
> of Swift developers is a laudable goal.  

Sure. I suggest it not as a final endpoint but as incremental progress you can 
make with the language today. If you can get some early adopters interested in 
moving from Python to Swift with some small warts now and the promise of 
refined syntax in the future, that could let you make progress while we work 
out the right final model for dynamic language interop. Since you have long 
term ambitions in this space, I think it'd pay off to consider more 
infrastructurally-heavy features with bigger long-term payoffs.
> 
>> You mentioned variadics as being inadequate for your purposes as a 
>> considered alternative, but a variadics model that wasn't based on type 
>> erasure to arrays and kept runtime type information about the argument tuple 
>> that was passed in, along with better reflection APIs for accessing that 
>> type info, might lead to something you can use by incrementally improving 
>> the expressivity of existing language features instead of adding new ones.
> 
> I’m not sure what you mean by this: variadics are fine. The hard part is 
> getting the keyword arguments for a call dynamically.  Can you point to what 
> you’re referring to in the proposal?

I was referring to your comment in this section:

> Statically checking for exact signatures
> 
> This proposal does not allow a type to specify an exact signature for the 
> callable - a specific number of parameters with specific types. If we went 
> down that route, the best approach would be to introduce a new declaration 
> kind (which would end up being very similar to get-only subscripts) since, in 
> general, a type could want multiple concrete callable signatures, and those 
> signatures should participate in overload resolution.
> 
> While such a feature could be interesting for some use cases, it is almost 
> entirely orthogonal from this proposal: it addresses different use cases and 
> does not solve the needs of this proposal. It does not address our needs 
> because even a variadic callable declaration would not provide access to the 
> keyword argument labels we need.
> 
We know our variadics model is inadequate in a lot of ways. I was suggesting 
there are ways of extending it that could allow the 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 10:33 AM, Chris Lattner via swift-evolution 
>  wrote:
> 
> 
> The discussion phase in the previous thread considered several approaches, 
> including burning python specific support in, implementing a python importer, 
> and implementing a single “dynamic thing” protocol.  These are all bad ideas 
> for reasons described in the thread.  If you’re curious, I’m happy to 
> elaborate in alternatives considered.
> 
> As such, discussion converged on pitching two proposals, which are generally 
> useful beyond this niche:
> 
> 1. The dynamically callable one already written.
> 2. A dynamically member lookupable proposal to sugar things like 
> foo.get(member: “bar”) - and the corresponding setter.
> 

One other pertinent thing from the discussion thread: it seriously isn’t lost 
on me how the Swift type checker works.  One of the the alternate designs that 
I intend to explore is to add a second method to the protocol, to encapsulate 
the Swift member dispatch idiom:

  func dynamicCallMember(member: String, arguments: [(String, 
DynamicCallableArgument)]) throws ->
 DynamicCallableResult


This puts more effort onto the implementer of the protocol, but I think that is 
a totally acceptable tradeoff to save compiler complexity.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 11, 2017, at 10:35 AM, John Holdsworth  wrote:
> Integration with Xcode would be straightforward but it’s been Android Studio
> I’ve most experience with. For that it was just a case of adapting a gradle
> plugin to add a rebuild phase. If you can’t automate it running a generator
> when you change the surface of your api is not a big overhead.
> 
> Supporting playgrounds would involve being able to import a user module
> to publish the api and link with python. Is that possible?

Playgrounds do not have a build system if used outside the context of a larger 
app project.

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[John Holdsworth via swift-evolution]

Integration with Xcode would be straightforward but it’s been Android Studio
I’ve most experience with. For that it was just a case of adapting a gradle
plugin to add a rebuild phase. If you can’t automate it running a generator
when you change the surface of your api is not a big overhead.

Supporting playgrounds would involve being able to import a user module
to publish the api and link with python. Is that possible?

John

> On 11 Nov 2017, at 18:20, Chris Lattner  wrote:
> 
> 
> 
>> On Nov 11, 2017, at 9:53 AM, Joe Groff via swift-evolution 
>> > wrote:
>> 
>> 
>> 
>> On Nov 11, 2017, at 9:44 AM, John Holdsworth > > wrote:
>> 
>>> Isn’t there a third way? I wrote a bridge between Swift and Java without 
>>> having to
>>> change the compiler at all by using a code generator to generate bridging 
>>> Swift
>>> rather than having a Java importer.
>> 
>> Good point. For unidirectional importing without deep support for overriding 
>> or runtime integration like what we do with ObjC, which AIUI is the extent 
>> of Chris's ambition with Python interfacing here, a code generator can get 
>> the job done without requiring deep compiler integration, though at the 
>> expense of needing an added tool in your build process. Have you looked at 
>> all into making SwiftJava integrate with the package manager or with Xcode 
>> yet? How's that experience?
> 
> Also, what is the playground experience like?
> 
> -Chris
> 
> 

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 10:15 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
> 
> On Nov 11, 2017, at 10:04 AM, Chris Lattner  > wrote:
> 
>> 
>> 
>>> On Nov 11, 2017, at 6:29 AM, Joe Groff via swift-evolution 
>>> > wrote:
>>> 
 below the fold as far as the rest of the language is concerned. You could 
 just as well written what the importer synths up in Swift directly:
 
 func foo(bar: String) {
 unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
 NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
 }
 
 and the rest of the language would be none the wiser.
>>> 
>>> Though, since you bring up objc_msgSend, the way it works in ObjC might be 
>>> a better fit for Swift's name lookup model, since their keyword argument 
>>> models are similar. If Swift had a 'method of last resort' like ObjC's
>> 
>> In fact, this is the subject of the next proposal that I will write up when 
>> I have time.
> 
> It seems to me like a dynamic method-of-last-resort feature would subsume 
> this proposal. Out of curiosity, why would we need both?

It seems that you missed the "[Discussion] Swift for Data Science / ML / Big 
Data analytics” thread, in which a lot of context was discussed, as well as 
multiple design approaches.  In any case, I will summarize here:

There are two fundamental operations that need to be supported to make a big 
impact on dynamic language interop: member lookup and calls.

Many languages treat these as separable concepts.  In Python for example, these 
are equivalent:

return foo.bar(x: 1, 23, y: 17)
and:
a = foo.bar
return a(x: 1, 23, y: 17)

Swift and the other Smalltalk inspired languages are the odd ones out who tie 
method arguments into the base name of a function.  With no language changes, 
it is possible to implement a pretty reasonable Python interop layer with the 
call and member operations explicit, it looks like this:

func example(foo : PyVal) -> PyVal {
  return  foo.get(member: “bar”).call(args: (“x”, 1), (“”, 23), (“y”, 
17))
}

and while it is totally possible to sugar common cases, e.g.:

func example(foo : PyVal) -> PyVal {
  return  foo.call(member: “bar”, args: (“x”, 1), (“”, 23), (“y”, 17))
}

you still need to support the basic model because Python supports currying and 
has global functions with keyword arguments.  FWIW, Perl follows a similar 
model, but doesn’t have keyword arguments.

I don’t really care much about the implementation approach (if this proposal 
isn’t the right one, fine!) but it is important to get this down to:

func example(foo : PyVal) -> PyVal {
  return  foo.bar(x: 1, 23, y: 17)
}

which I hope you’ll agree is a *HUGE* readability and writability benefit, so 
much so that users would reject the long hand notion.


The discussion phase in the previous thread considered several approaches, 
including burning python specific support in, implementing a python importer, 
and implementing a single “dynamic thing” protocol.  These are all bad ideas 
for reasons described in the thread.  If you’re curious, I’m happy to elaborate 
in alternatives considered.

As such, discussion converged on pitching two proposals, which are generally 
useful beyond this niche:

1. The dynamically callable one already written.
2. A dynamically member lookupable proposal to sugar things like 
foo.get(member: “bar”) - and the corresponding setter.

The former one is presumably useful for things outside dynamic languages, e.g. 
implementing dynamic “proxy” type interfaces.
The second one is useful for far more than just dynamic languages, e.g. 
sugaring json keypath traversals.


-Chris



___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 9:53 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
> 
> On Nov 11, 2017, at 9:44 AM, John Holdsworth  > wrote:
> 
>> Isn’t there a third way? I wrote a bridge between Swift and Java without 
>> having to
>> change the compiler at all by using a code generator to generate bridging 
>> Swift
>> rather than having a Java importer.
> 
> Good point. For unidirectional importing without deep support for overriding 
> or runtime integration like what we do with ObjC, which AIUI is the extent of 
> Chris's ambition with Python interfacing here, a code generator can get the 
> job done without requiring deep compiler integration, though at the expense 
> of needing an added tool in your build process. Have you looked at all into 
> making SwiftJava integrate with the package manager or with Xcode yet? How's 
> that experience?

Also, what is the playground experience like?

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 7:02 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
> 
> On Nov 10, 2017, at 8:35 PM, Chris Lattner via swift-evolution 
> > wrote:
> 
>> 
>>> On Nov 10, 2017, at 6:12 PM, Slava Pestov via swift-evolution 
>>> > wrote:
>>> 
>>> 
>>> 
 On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
 > wrote:
 
 Setting this aside, I’m very curious to hear whether type providers 
 influence your thinking after you’ve had a chance to look into them.  I 
 have always thought they were very cool.
>>> 
>>> I’m in favor of solving this problem with something like type providers 
>>> also. The required compiler changes would be significant but would also 
>>> clean up the interface between the ClangImporter, Sema and Serialization. 
>>> If done right it would be a net gain that would benefit all users, instead 
>>> of just adding YetAnotherCornerCase™ that makes implementation maintainers 
>>> curse and scream.
>> 
>> I find it ironic that you’re talking pejoratively about a feature that has 
>> very narrow impact, complaining about how much of an impact on the compiler 
>> it would have, and then pine for a hugely invasive feature - one that would 
>> cause a ton of code churn, and probably wouldn’t actually be enough to 
>> eliminate the special cases in place because of ObjC interop.
> 
> You underestimate the impact this would have on function call type checking, 
> but since this is an additive, non-ABI-stability feature, I have trouble 
> considering it a candidate for Swift 5,

The guidelines for Swift 5 are publicly documented here:
https://github.com/apple/swift-evolution/blob/master/README.md#evolution-process-for-swift-5
 


This is relevant and seems reasonable to me:
Syntactic additions. Syntactic changes do not increase the expressive power of 
the language but do increase its complexity. Consequently, such changes must be 
extremely well-motivated and will be subject to additional scrutiny. We will 
expect proposals to include concrete data about how wide spread the positive 
impact will be.


I explicitly point out that this is a sugar change.

> so I don't think there's a time constraint forcing us to consider the 
> "narrow" vs "huge" dimension. What's the best thing for the language and 
> tools in the long term? This is a feature that influences the semantics of 
> potentially any call site in all Swift code, which we'd have to live with 
> forever if we accepted it now. Opening up the compiler architecture to make 
> custom importers easier to write is a great solution to a ton of problems, 
> including yours I think, without adding complexity to the core language. 
> Experience in .NET land seems to show it's a great technique for integrating 
> dynamic systems with static type systems, without poking unnecessary holes in 
> the static language's type system

As I mentioned, I’m not familiar with F# type providers.  As I told you 
upthread, I will read about them and respond with an informed opinion when I 
have time.

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 10:04 AM, Chris Lattner  wrote:
> 
> 
> 
>>> On Nov 11, 2017, at 6:29 AM, Joe Groff via swift-evolution 
>>>  wrote:
>>> 
>>> below the fold as far as the rest of the language is concerned. You could 
>>> just as well written what the importer synths up in Swift directly:
>>> 
>>> func foo(bar: String) {
>>> unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
>>> NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
>>> }
>>> 
>>> and the rest of the language would be none the wiser.
>> 
>> Though, since you bring up objc_msgSend, the way it works in ObjC might be a 
>> better fit for Swift's name lookup model, since their keyword argument 
>> models are similar. If Swift had a 'method of last resort' like ObjC's
> 
> In fact, this is the subject of the next proposal that I will write up when I 
> have time.

It seems to me like a dynamic method-of-last-resort feature would subsume this 
proposal. Out of curiosity, why would we need both?

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 11, 2017, at 6:52 AM, Joe Groff via swift-evolution 
 wrote:
>>> I don't think it's that localized. It's going to make call resolution 
>>> massively more complicated.  Through great pain and community anguish, we 
>>> pushed ourselves to a model where argument labels are parts of the 
>>> declaration name, not part of the call argument, and this would throw us 
>>> straight back into the kinds of complications of dealing with overloaded 
>>> name- and type-dependent behavior we're trying to get away from in the 
>>> language.
>> 
>> I’m pretty sure it doesn’t, but good news: since Swift requires an 
>> implementation before the proposal can be officially reviewed, you will be 
>> able to look at the actual patch to make this decision.
> 
> And the goal of the pitch phase is to hopefully save you the tears of trying 
> to implement something if it's a bad idea.

Appreciated.  Fortunately I enjoy coding so it’s not that big of a deal for me.

>>> Swift is a dynamic language too, and tuples have enough metadata in them 
>>> that you could get almost everything you want with an infix operator.
>> 
>> This is really gross.  The goal here is to make Python feel natural.  If you 
>> oppose this proposal then we’ll have to find some other way to provide an 
>> elegant experience:
>>> x∫(0, "foo", bar: "bas”)
>> 
>> This is not going to be acceptable to users.
> 
> Is it? My choice of operator character is silly, but it's the same amount of 
> syntactic overhead as an optional chain/force call. It's a fair sight better 
> than x.call(args: ..., keywordArgs: ...), I hope you'd agree, and is 
> something that works without any language change.

Yes, I’m happy to explore that, and I agree this approach doesn’t require 
language changes.  That said, there are three things you should understand:

1) Details matter.  While you claim that there is some other operator character 
that could be awesome, I’m not seeing it.  Can you try harder to come up with 
something that you think would be acceptable?

2) The point of the entire proposal is that a certain audience will be using 
the feature enabled by this (python interop) *a lot*, so ergonomics matter.

3) This feature (python interop) is aimed to get Python programmers to convert 
to Swift, not simply to ease friction for existing Swift programmers (most of 
whom are iOS developers, to whom this feature is completely irrelevant).  
Getting people to move from Python to Swift is a challenge, and syntax does 
actually matter. I hope we can agree that expanding the community of Swift 
developers is a laudable goal.  

> You mentioned variadics as being inadequate for your purposes as a considered 
> alternative, but a variadics model that wasn't based on type erasure to 
> arrays and kept runtime type information about the argument tuple that was 
> passed in, along with better reflection APIs for accessing that type info, 
> might lead to something you can use by incrementally improving the 
> expressivity of existing language features instead of adding new ones.

I’m not sure what you mean by this: variadics are fine. The hard part is 
getting the keyword arguments for a call dynamically.  Can you point to what 
you’re referring to in the proposal?

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 6:29 AM, Joe Groff via swift-evolution 
>  wrote:
> 
>> below the fold as far as the rest of the language is concerned. You could 
>> just as well written what the importer synths up in Swift directly:
>> 
>> func foo(bar: String) {
>> unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
>> NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
>> }
>> 
>> and the rest of the language would be none the wiser.
> 
> Though, since you bring up objc_msgSend, the way it works in ObjC might be a 
> better fit for Swift's name lookup model, since their keyword argument models 
> are similar. If Swift had a 'method of last resort' like ObjC's

In fact, this is the subject of the next proposal that I will write up when I 
have time.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 11, 2017, at 2:19 AM, Tino Heth via swift-evolution 
>  wrote:
> 
>> Example Usage
>> 
> 
> Swift is quite flexible on what can act as a closure — would it be allowed to 
> use a dynamic callable in that context?
> I guess forwarding of
> 
> let closure: ([(String, Int)]) -> Int  = DynamicCallableType()
> 
> to the dynamicCall method of DynamicCallableType isn’t that hard, but 
> wouldn’t it be odd if the value of closure changes when you leave out its 
> type?

I’m not sure I understand what you’re getting at.  Can you show how this would 
work with the example in the motivation section?

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 11, 2017, at 2:04 AM, Tino Heth via swift-evolution 
 wrote:
> I generally dislike „protocol magic“, but I think in this case, there is also 
> a precedence for an (imho) better way:
> There’s no „Subscriptable“ protocol, and I would prefer callable types to be 
> handled in the same way (afaics, subscripts are already used as „callable 
> with different braces“ today in some cases).
> 
> You mention subscripts in „Alternatives considered“, but without an 
> explanation why you don’t want a new declaration kind for dynamicCall — and I 
> think it could co-exist with the orthogonal use case that you mention.

Sure, this is a great question.  We could definitely introduce a new 
declaration kind.  The tradeoff I see is that that is far more language 
invasive, and that none of our existing decls work this way: packaging up 
syntax (keyword labels) and handing it off.

One theoretical way to address this is to introduce a macro system, but that 
would be far more invasive and isn’t going to happen now (and perhaps never?).

Despite the claims, the impact of this proposal on the compiler is extremely 
narrow.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 9:44 AM, John Holdsworth  wrote:
> 
> Isn’t there a third way? I wrote a bridge between Swift and Java without 
> having to
> change the compiler at all by using a code generator to generate bridging 
> Swift
> rather than having a Java importer.

Good point. For unidirectional importing without deep support for overriding or 
runtime integration like what we do with ObjC, which AIUI is the extent of 
Chris's ambition with Python interfacing here, a code generator can get the job 
done without requiring deep compiler integration, though at the expense of 
needing an added tool in your build process. Have you looked at all into making 
SwiftJava integrate with the package manager or with Xcode yet? How's that 
experience?

-Joe

> The script introspects the classes/packages
> you’re interested in using a script and generates code such as the following:
> 
> /// public java.lang.String java.lang.Object.toString()
> 
> private static var toString_MethodID_9: jmethodID?
> 
> open func toString() -> String! {
> var __locals = [jobject]()
> var __args = [jvalue]( repeating: jvalue(), count: 1 )
> let __return = JNIMethod.CallObjectMethod( object: javaObject, 
> methodName: "toString", methodSig: "()Ljava/lang/String;", methodCache: 
> _MethodID_9, args: &__args, locals: &__locals )
> defer { JNI.DeleteLocalRef( __return ) }
> return __return != nil ? String( javaObject: __return ) : nil
> }
> 
> This bridging code, along with some supporting classes is compiled 
> conventionally
> along with your app. See:
> 
> https://github.com/SwiftJava/SwiftJava
> https://github.com/SwiftJava/java_swift/blob/master/Sources/JavaObject.swift
> https://github.com/SwiftJava/SwiftJava/blob/master/src/genswift.java
> 
> Python’s introspection seems to be reasonably capable and could certainly 
> support
> this approach. You’d need some sort of omni-type enum for the arguments and 
> return.
> 
> https://stackoverflow.com/questions/196960/can-you-list-the-keyword-arguments-a-python-function-receives
> https://docs.python.org/2/c-api/intro.html
> 
> John
> 
>> On 11 Nov 2017, at 16:13, David Hart via swift-evolution 
>>  wrote:
>> 
>> 
>> On 11 Nov 2017, at 16:02, Joe Groff via swift-evolution 
>>  wrote:
>> 
>>> 
>>> 
 On Nov 10, 2017, at 8:35 PM, Chris Lattner via swift-evolution 
  wrote:
 
 
>> On Nov 10, 2017, at 6:12 PM, Slava Pestov via swift-evolution 
>>  wrote:
>> 
>> 
>> 
>> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>>  wrote:
>> 
>> Setting this aside, I’m very curious to hear whether type providers 
>> influence your thinking after you’ve had a chance to look into them.  I 
>> have always thought they were very cool.
> 
> I’m in favor of solving this problem with something like type providers 
> also. The required compiler changes would be significant but would also 
> clean up the interface between the ClangImporter, Sema and Serialization. 
> If done right it would be a net gain that would benefit all users, 
> instead of just adding YetAnotherCornerCase™ that makes implementation 
> maintainers curse and scream.
 
 I find it ironic that you’re talking pejoratively about a feature that has 
 very narrow impact, complaining about how much of an impact on the 
 compiler it would have, and then pine for a hugely invasive feature - one 
 that would cause a ton of code churn, and probably wouldn’t actually be 
 enough to eliminate the special cases in place because of ObjC interop.
>>> 
>>> You underestimate the impact this would have on function call type 
>>> checking, but since this is an additive, non-ABI-stability feature, I have 
>>> trouble considering it a candidate for Swift 5, so I don't think there's a 
>>> time constraint forcing us to consider the "narrow" vs "huge" dimension. 
>>> What's the best thing for the language and tools in the long term? This is 
>>> a feature that influences the semantics of potentially any call site in all 
>>> Swift code, which we'd have to live with forever if we accepted it now. 
>>> Opening up the compiler architecture to make custom importers easier to 
>>> write is a great solution to a ton of problems, including yours I think, 
>>> without adding complexity to the core language. Experience in .NET land 
>>> seems to show it's a great technique for integrating dynamic systems with 
>>> static type systems, without poking unnecessary holes in the static 
>>> language's type system
>> 
>> I agree with Joe. I also think that it would be very important to compare 
>> both approaches in detail before choosing one, because the last thing we 
>> want is to end up with both in the language. And if this proposal is 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[John Holdsworth via swift-evolution]

Isn’t there a third way? I wrote a bridge between Swift and Java without having 
to
change the compiler at all by using a code generator to generate bridging Swift
rather than having a Java importer. The script introspects the classes/packages
you’re interested in using a script and generates code such as the following:

/// public java.lang.String java.lang.Object.toString()

private static var toString_MethodID_9: jmethodID?

open func toString() -> String! {
var __locals = [jobject]()
var __args = [jvalue]( repeating: jvalue(), count: 1 )
let __return = JNIMethod.CallObjectMethod( object: javaObject, 
methodName: "toString", methodSig: "()Ljava/lang/String;", methodCache: 
_MethodID_9, args: &__args, locals: &__locals )
defer { JNI.DeleteLocalRef( __return ) }
return __return != nil ? String( javaObject: __return ) : nil
}

This bridging code, along with some supporting classes is compiled 
conventionally
along with your app. See:

https://github.com/SwiftJava/SwiftJava 
https://github.com/SwiftJava/java_swift/blob/master/Sources/JavaObject.swift 

https://github.com/SwiftJava/SwiftJava/blob/master/src/genswift.java 


Python’s introspection seems to be reasonably capable and could certainly 
support
this approach. You’d need some sort of omni-type enum for the arguments and 
return.

https://stackoverflow.com/questions/196960/can-you-list-the-keyword-arguments-a-python-function-receives
 

https://docs.python.org/2/c-api/intro.html 


John

> On 11 Nov 2017, at 16:13, David Hart via swift-evolution 
>  wrote:
> 
> 
> On 11 Nov 2017, at 16:02, Joe Groff via swift-evolution 
> > wrote:
> 
>> 
>> 
>> On Nov 10, 2017, at 8:35 PM, Chris Lattner via swift-evolution 
>> > wrote:
>> 
>>> 
 On Nov 10, 2017, at 6:12 PM, Slava Pestov via swift-evolution 
 > wrote:
 
 
 
> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
> > wrote:
> 
> Setting this aside, I’m very curious to hear whether type providers 
> influence your thinking after you’ve had a chance to look into them.  I 
> have always thought they were very cool.
 
 I’m in favor of solving this problem with something like type providers 
 also. The required compiler changes would be significant but would also 
 clean up the interface between the ClangImporter, Sema and Serialization. 
 If done right it would be a net gain that would benefit all users, instead 
 of just adding YetAnotherCornerCase™ that makes implementation maintainers 
 curse and scream.
>>> 
>>> I find it ironic that you’re talking pejoratively about a feature that has 
>>> very narrow impact, complaining about how much of an impact on the compiler 
>>> it would have, and then pine for a hugely invasive feature - one that would 
>>> cause a ton of code churn, and probably wouldn’t actually be enough to 
>>> eliminate the special cases in place because of ObjC interop.
>> 
>> You underestimate the impact this would have on function call type checking, 
>> but since this is an additive, non-ABI-stability feature, I have trouble 
>> considering it a candidate for Swift 5, so I don't think there's a time 
>> constraint forcing us to consider the "narrow" vs "huge" dimension. What's 
>> the best thing for the language and tools in the long term? This is a 
>> feature that influences the semantics of potentially any call site in all 
>> Swift code, which we'd have to live with forever if we accepted it now. 
>> Opening up the compiler architecture to make custom importers easier to 
>> write is a great solution to a ton of problems, including yours I think, 
>> without adding complexity to the core language. Experience in .NET land 
>> seems to show it's a great technique for integrating dynamic systems with 
>> static type systems, without poking unnecessary holes in the static 
>> language's type system
> 
> I agree with Joe. I also think that it would be very important to compare 
> both approaches in detail before choosing one, because the last thing we want 
> is to end up with both in the language. And if this proposal is accepted, we 
> might refrain from introducing custom importers later, even if they are the 
> better long term solution.
> 
> I want Swift to continue to shine for a very long time because I enjoy 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Goffredo Marocchi via swift-evolution]

Very interesting indeed...

class VehicleUtilities {

int numberOfAxles(Vehicle v) { return 2;} // a plausible default

int numberOfAxles (Truck t){ return 3;}

}




Vehicle v = new Truck();

VehicleUtilities u = new VehicleUtilities();

u.numberOfAxles(v); // returns 2


--> this is a nice concise example of even simple code can suffer from a
nasty code issue such as this...

On Sat, Nov 11, 2017 at 4:32 PM, Gwendal Roué via swift-evolution <
swift-evolution@swift.org> wrote:

>
> Le 11 nov. 2017 à 16:48, Joe Groff via swift-evolution <
> swift-evolution@swift.org> a écrit :
>
> That'd be great, but Swift violates Gilad Bracha's golden rule by having
> static overloading, and throws bidirectional type inference on top, so our
> static name resolution can't be treated as a specialization of a dynamic
> name lookup mechanism in all cases.
>
>
> I didn't know of Gilad Bracha, so you made me curious.
>
> I guess that the "golden rule" you refer to is here, for anyone curious:
> https://gbracha.blogspot.fr/2009/09/systemic-overload.html
>
> Gwendal Roué
>
>
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
 wrote:
>>> On Nov 10, 2017, at 11:25 AM, Matthew Johnson via swift-evolution 
>>> > wrote:
>>> 
> 
> People have reasonably asked for the ability to make their own 
> function-like types in the past, such that "myvalue(...)" behaves like 
> sugar for "myvalue.call(...)" or something like that. In most cases, they 
> still want to have type system control over what arguments and results 
> their call operation produces. They don't really get that with this 
> proposal; they lose all control over the arity and argument types. 
 
 As I mentioned, this is directly addressed in the writeup. Here’s the link:
 https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
  
 
>>> That discusses why you didn’t include it in the present proposal but I 
>>> think it’s reasonable to oppose adding a dynamic callable feature prior to 
>>> a more Swifty static callable.
>> 
>> Why?  One does not preclude the other.
> 
> For exactly the reason Joe articulates.  Some people will use what the 
> language offers to get the syntax they desire even if it sacrifices type 
> safety.  If we’re going to have first-class callable types in Swift (I think 
> it’s a great idea) type safety for native code should be prioritized over 
> syntactic convenience for dynamic language interop.  We can have both, but 
> the former should come first IMO.

Hi Matthew,

In point of fact, Swift already has the feature you are referring to.  It just 
spells it with square brackets instead of parentheses.  A simple change to the 
punctuation character has much less point than the proposal that I’m pitching.

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Gwendal Roué via swift-evolution]

> Le 11 nov. 2017 à 16:48, Joe Groff via swift-evolution 
>  a écrit :
> 
> That'd be great, but Swift violates Gilad Bracha's golden rule by having 
> static overloading, and throws bidirectional type inference on top, so our 
> static name resolution can't be treated as a specialization of a dynamic name 
> lookup mechanism in all cases.

I didn't know of Gilad Bracha, so you made me curious. 

I guess that the "golden rule" you refer to is here, for anyone curious: 
https://gbracha.blogspot.fr/2009/09/systemic-overload.html 


Gwendal Roué

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[David Hart via swift-evolution]

> On 11 Nov 2017, at 16:02, Joe Groff via swift-evolution 
>  wrote:
> 
> 
> 
>> On Nov 10, 2017, at 8:35 PM, Chris Lattner via swift-evolution 
>>  wrote:
>> 
>> 
>>> On Nov 10, 2017, at 6:12 PM, Slava Pestov via swift-evolution 
>>>  wrote:
>>> 
>>> 
>>> 
 On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
  wrote:
 
 Setting this aside, I’m very curious to hear whether type providers 
 influence your thinking after you’ve had a chance to look into them.  I 
 have always thought they were very cool.
>>> 
>>> I’m in favor of solving this problem with something like type providers 
>>> also. The required compiler changes would be significant but would also 
>>> clean up the interface between the ClangImporter, Sema and Serialization. 
>>> If done right it would be a net gain that would benefit all users, instead 
>>> of just adding YetAnotherCornerCase™ that makes implementation maintainers 
>>> curse and scream.
>> 
>> I find it ironic that you’re talking pejoratively about a feature that has 
>> very narrow impact, complaining about how much of an impact on the compiler 
>> it would have, and then pine for a hugely invasive feature - one that would 
>> cause a ton of code churn, and probably wouldn’t actually be enough to 
>> eliminate the special cases in place because of ObjC interop.
> 
> You underestimate the impact this would have on function call type checking, 
> but since this is an additive, non-ABI-stability feature, I have trouble 
> considering it a candidate for Swift 5, so I don't think there's a time 
> constraint forcing us to consider the "narrow" vs "huge" dimension. What's 
> the best thing for the language and tools in the long term? This is a feature 
> that influences the semantics of potentially any call site in all Swift code, 
> which we'd have to live with forever if we accepted it now. Opening up the 
> compiler architecture to make custom importers easier to write is a great 
> solution to a ton of problems, including yours I think, without adding 
> complexity to the core language. Experience in .NET land seems to show it's a 
> great technique for integrating dynamic systems with static type systems, 
> without poking unnecessary holes in the static language's type system

I agree with Joe. I also think that it would be very important to compare both 
approaches in detail before choosing one, because the last thing we want is to 
end up with both in the language. And if this proposal is accepted, we might 
refrain from introducing custom importers later, even if they are the better 
long term solution.

I want Swift to continue to shine for a very long time because I enjoy this 
language and I just want to make sure we don’t jeopardize that by choosing a 
quick solution without taking in consideration other solutions.

> -Joe
> 
>> -Chris
>> 
>> 
>> ___
>> swift-evolution mailing list
>> swift-evolution@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 11, 2017, at 4:21 AM, Marcel Weiher  wrote:
> 
> 
> 
>> On Nov 10, 2017, at 19:04 , Joe Groff via swift-evolution 
>>  wrote:
>> 
>> I don't like the idea of some calls having wildly different semantics from 
>> others;
> 
> The obvious solution then is to make this proposal the general case and 
> current calls a specialization of that.

That'd be great, but Swift violates Gilad Bracha's golden rule by having static 
overloading, and throws bidirectional type inference on top, so our static name 
resolution can't be treated as a specialization of a dynamic name lookup 
mechanism in all cases.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[C. Keith Ray via swift-evolution]

how does c code (and Python code, etc, other than ObjC) call Swift functions, 
methods, etc? 

C. Keith Ray
https://leanpub.com/wepntk <- buy my book?
http://agilesolutionspace.blogspot.com/
twitter: @ckeithray
http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf

> On Nov 11, 2017, at 6:29 AM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
> 
>> On Nov 10, 2017, at 4:20 PM, Joe Groff  wrote:
>> 
>> 
>> 
 On Nov 10, 2017, at 4:12 PM, Charles Srstka  
 wrote:
 
 On Nov 10, 2017, at 5:51 PM, Joe Groff  wrote:
 
 
 
>> On Nov 10, 2017, at 3:45 PM, Charles Srstka  
>> wrote:
>> 
>> On Nov 10, 2017, at 5:36 PM, Joe Groff  wrote:
>> 
>> How `MyObject.foo(_:bar:)` gets implemented is its own business, as far 
>> as the compiler is concerned. The compile-time name resolution for the 
>> method isn't impacted.
>> 
>> -Joe
> 
> The compile-time name resolution for the method doesn’t happen *at all.*
 
 You declared the method in your @interface, and the compiler saw that and 
 brought it in as what Swift considers to be a regular method, and your 
 call on the Swift side was resolved to it by Swift's usual lookup rules. 
 To do what Chris is suggesting requires changing the way calls get 
 resolved in the compiler before the call is even formed.
>>> 
>>> The only thing that makes this the “usual lookup rules” is that the 
>>> Objective-C bridge has already been implemented.
>> 
>> As I mentioned in my original reply, I personally think the "importer" 
>> approach would be superior, and that in a perfect world we'd have type 
>> providers to make writing something like the ObjC importer but for a 
>> different language or other dynamic data source something that doesn't 
>> require invasive compiler hackery. The importer puts all of this:
>> 
>>> - It’s changing the compile-time name resolution! The Swift name is 
>>> foo(bar:), but it’s changing that to fooWithBar:!
>>> 
>>> - It’s changing the signature! The argument took a String, but now it’s 
>>> passing an NSString!
>>> 
>>> - It’s not resolving the method at compile-time! It’s passing the modified 
>>> method name and the arg list to some objc_msgSend() function, which 
>>> resolves it dynamically in a way that user code can intercept and interpret 
>>> at runtime!
>>> 
>>> I’m just not seeing the conceptual difference here.
>> 
>> below the fold as far as the rest of the language is concerned. You could 
>> just as well written what the importer synths up in Swift directly:
>> 
>> func foo(bar: String) {
>> unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
>> NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
>> }
>> 
>> and the rest of the language would be none the wiser.
> 
> Though, since you bring up objc_msgSend, the way it works in ObjC might be a 
> better fit for Swift's name lookup model, since their keyword argument models 
> are similar. If Swift had a 'method of last resort' like ObjC's, say as a 
> strawman you could overload the '.' operator, then you could use it to 
> provide an implementation for a method given a compound name in a similar 
> way. So if you had:
> 
> struct Dynamic { func .(methodName: String) -> (Any...) -> Int }
> 
> let x = Dynamic()
> x.foo(x: 0, y: 1)
> 
> Then, when we do name lookup into x for foo(x:y:) and that fails, we'd fall 
> back to turning this into x.`func .`("foo(x:y:)")(0, 1). It would take a bit 
> more work to turn this into something like a Python call, but would fit 
> Swift's language model better.
> 
> -Joe
> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 8:35 PM, Chris Lattner via swift-evolution 
>  wrote:
> 
> 
>> On Nov 10, 2017, at 6:12 PM, Slava Pestov via swift-evolution 
>>  wrote:
>> 
>> 
>> 
>>> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>>>  wrote:
>>> 
>>> Setting this aside, I’m very curious to hear whether type providers 
>>> influence your thinking after you’ve had a chance to look into them.  I 
>>> have always thought they were very cool.
>> 
>> I’m in favor of solving this problem with something like type providers 
>> also. The required compiler changes would be significant but would also 
>> clean up the interface between the ClangImporter, Sema and Serialization. If 
>> done right it would be a net gain that would benefit all users, instead of 
>> just adding YetAnotherCornerCase™ that makes implementation maintainers 
>> curse and scream.
> 
> I find it ironic that you’re talking pejoratively about a feature that has 
> very narrow impact, complaining about how much of an impact on the compiler 
> it would have, and then pine for a hugely invasive feature - one that would 
> cause a ton of code churn, and probably wouldn’t actually be enough to 
> eliminate the special cases in place because of ObjC interop.

You underestimate the impact this would have on function call type checking, 
but since this is an additive, non-ABI-stability feature, I have trouble 
considering it a candidate for Swift 5, so I don't think there's a time 
constraint forcing us to consider the "narrow" vs "huge" dimension. What's the 
best thing for the language and tools in the long term? This is a feature that 
influences the semantics of potentially any call site in all Swift code, which 
we'd have to live with forever if we accepted it now. Opening up the compiler 
architecture to make custom importers easier to write is a great solution to a 
ton of problems, including yours I think, without adding complexity to the core 
language. Experience in .NET land seems to show it's a great technique for 
integrating dynamic systems with static type systems, without poking 
unnecessary holes in the static language's type system

-Joe

> -Chris
> 
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 5:44 PM, Chris Lattner  wrote:
> 
>> On Nov 10, 2017, at 11:42 AM, Joe Groff via swift-evolution 
>>  wrote:
 
 On Nov 10, 2017, at 10:57 AM, Alejandro Martinez via swift-evolution 
  wrote:
 
 This seems a really interesting solution Chris.
 Similar to what Joe mentions I think this would also be appreciated by
 the community to make even nicer DSLs in Swift, which may or may not
 be a good side effect of the proposal.
 Also, I'm just wondering, how much complication adds this to the
 compiler itself that would have to be maintained in the future?
>>> 
>>> This is a very localized and simple change to the compiler.  Assuming the 
>>> pitch process goes well, I will provide an implementation.
>> 
>> I don't think it's that localized. It's going to make call resolution 
>> massively more complicated.  Through great pain and community anguish, we 
>> pushed ourselves to a model where argument labels are parts of the 
>> declaration name, not part of the call argument, and this would throw us 
>> straight back into the kinds of complications of dealing with overloaded 
>> name- and type-dependent behavior we're trying to get away from in the 
>> language.
> 
> I’m pretty sure it doesn’t, but good news: since Swift requires an 
> implementation before the proposal can be officially reviewed, you will be 
> able to look at the actual patch to make this decision.

And the goal of the pitch phase is to hopefully save you the tears of trying to 
implement something if it's a bad idea.

>> Swift is a dynamic language too, and tuples have enough metadata in them 
>> that you could get almost everything you want with an infix operator.
> 
> This is really gross.  The goal here is to make Python feel natural.  If you 
> oppose this proposal then we’ll have to find some other way to provide an 
> elegant experience:
>> x∫(0, "foo", bar: "bas”)
> 
> This is not going to be acceptable to users.

Is it? My choice of operator character is silly, but it's the same amount of 
syntactic overhead as an optional chain/force call. It's a fair sight better 
than x.call(args: ..., keywordArgs: ...), I hope you'd agree, and is something 
that works without any language change. You mentioned variadics as being 
inadequate for your purposes as a considered alternative, but a variadics model 
that wasn't based on type erasure to arrays and kept runtime type information 
about the argument tuple that was passed in, along with better reflection APIs 
for accessing that type info, might lead to something you can use by 
incrementally improving the expressivity of existing language features instead 
of adding new ones.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 4:20 PM, Joe Groff  wrote:
> 
> 
> 
>>> On Nov 10, 2017, at 4:12 PM, Charles Srstka  
>>> wrote:
>>> 
>>> On Nov 10, 2017, at 5:51 PM, Joe Groff  wrote:
>>> 
>>> 
>>> 
> On Nov 10, 2017, at 3:45 PM, Charles Srstka  
> wrote:
> 
> On Nov 10, 2017, at 5:36 PM, Joe Groff  wrote:
> 
> How `MyObject.foo(_:bar:)` gets implemented is its own business, as far 
> as the compiler is concerned. The compile-time name resolution for the 
> method isn't impacted.
> 
> -Joe
 
 The compile-time name resolution for the method doesn’t happen *at all.*
>>> 
>>> You declared the method in your @interface, and the compiler saw that and 
>>> brought it in as what Swift considers to be a regular method, and your call 
>>> on the Swift side was resolved to it by Swift's usual lookup rules. To do 
>>> what Chris is suggesting requires changing the way calls get resolved in 
>>> the compiler before the call is even formed.
>> 
>> The only thing that makes this the “usual lookup rules” is that the 
>> Objective-C bridge has already been implemented.
> 
> As I mentioned in my original reply, I personally think the "importer" 
> approach would be superior, and that in a perfect world we'd have type 
> providers to make writing something like the ObjC importer but for a 
> different language or other dynamic data source something that doesn't 
> require invasive compiler hackery. The importer puts all of this:
> 
>> - It’s changing the compile-time name resolution! The Swift name is 
>> foo(bar:), but it’s changing that to fooWithBar:!
>> 
>> - It’s changing the signature! The argument took a String, but now it’s 
>> passing an NSString!
>> 
>> - It’s not resolving the method at compile-time! It’s passing the modified 
>> method name and the arg list to some objc_msgSend() function, which resolves 
>> it dynamically in a way that user code can intercept and interpret at 
>> runtime!
>> 
>> I’m just not seeing the conceptual difference here.
> 
> below the fold as far as the rest of the language is concerned. You could 
> just as well written what the importer synths up in Swift directly:
> 
> func foo(bar: String) {
>  unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
> NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
> }
> 
> and the rest of the language would be none the wiser.

Though, since you bring up objc_msgSend, the way it works in ObjC might be a 
better fit for Swift's name lookup model, since their keyword argument models 
are similar. If Swift had a 'method of last resort' like ObjC's, say as a 
strawman you could overload the '.' operator, then you could use it to provide 
an implementation for a method given a compound name in a similar way. So if 
you had:

struct Dynamic { func .(methodName: String) -> (Any...) -> Int }

let x = Dynamic()
x.foo(x: 0, y: 1)

Then, when we do name lookup into x for foo(x:y:) and that fails, we'd fall 
back to turning this into x.`func .`("foo(x:y:)")(0, 1). It would take a bit 
more work to turn this into something like a Python call, but would fit Swift's 
language model better.

-Joe


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Marcel Weiher via swift-evolution]

> On Nov 10, 2017, at 19:04 , Joe Groff via swift-evolution 
>  wrote:
> 
> I don't like the idea of some calls having wildly different semantics from 
> others;

The obvious solution then is to make this proposal the general case and current 
calls a specialization of that.

Marcel

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Tino Heth via swift-evolution]

> Example Usage
> 

Swift is quite flexible on what can act as a closure — would it be allowed to 
use a dynamic callable in that context?
I guess forwarding of

let closure: ([(String, Int)]) -> Int  = DynamicCallableType()

to the dynamicCall method of DynamicCallableType isn’t that hard, but wouldn’t 
it be odd if the value of closure changes when you leave out its type?___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Tino Heth via swift-evolution]

I generally dislike „protocol magic“, but I think in this case, there is also a 
precedence for an (imho) better way:
There’s no „Subscriptable“ protocol, and I would prefer callable types to be 
handled in the same way (afaics, subscripts are already used as „callable with 
different braces“ today in some cases).

You mention subscripts in „Alternatives considered“, but without an explanation 
why you don’t want a new declaration kind for dynamicCall — and I think it 
could co-exist with the orthogonal use case that you mention.

- Tino

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 10, 2017, at 9:10 PM, Brent Royal-Gordon  
> wrote:
> Leaving the "should we do it at all" question aside…

Thanks, most of the comments in this thread haven’t been constructive at all, 
I’m glad to get one that is :-)

>> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>>  wrote:
>> 
>> We propose introducing this protocol to the standard library:
>> 
>> protocol DynamicCallable {
>>  associatedtype DynamicCallableArgument
>>  associatedtype DynamicCallableResult
>> 
>>  func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> 
>> DynamicCallableResult
>> }
>> 
>> It also extends the language such that function call syntax - when applied 
>> to a value of DynamicCallable type - is accepted and transformed into a call 
>> to the dynamicCall member. The dynamicCall method takes a list of tuples: 
>> the first element is the keyword label (or an empty string if absent) and 
>> the second value is the formal parameter specified at the call site.
> 
> I don't really like passing the argument names to the parameter list. 
> Instead, I would prefer to see them handled in the member-resolution method. 
> You don't give a design for DynamicMemberLookupProtocol here, but assuming 
> that you're imagining that a call like this:
> 
>   foreignObject.someValue(keyword1: 42, "foo", keyword2: 19)
> 
> Would translate to something like this:
> 
>   foreignObject.dynamicMethodLookup(name: 
> "someValue").dynamicCall(arguments: [("keyword1", 42), ("", "foo"), 
> ("keyword2", 19)])

Right, that was the idea.

> I would instead like it to look like this:
> 
>   foreignObject.dynamicMethodLookup(name: "someValue", arguments: 
> ["keyword1", "", "keyword2"]).dynamicCall(arguments: [42, "foo", 19])
> 
> This would better suit languages like Squeak (or Swift!) where you cannot 
> fully look up a method without its keywords.

Good point.

> (Languages like Python would simply store the keywords inside the object 
> returned by `dynamicMethodLookup`.)

While I don’t care greatly about making Python efficient, doing this storage 
would be grossly inefficient and doesn’t have an obvious implementation.  
Python also supports currying, and global functions with keyword arguments, 
which means that:

fn(keyword1: 42, "foo", keyword2: 19)

… needs to work, even if x didn’t come from Swift.

I guess we can solve both use cases by passing the keyword in both cases for a 
Swift style call.  Your example would look like this:

foreignObject.dynamicMethodLookup(name: "someValue", arguments: ["keyword1", 
"", "keyword2"]).dynamicCall(arguments: [(“keyword1”, 42), (“”, “foo"), 
(“keyword2”: 19)])

The fn example above would look like this:

fn.dynamicCall(arguments: [(“keyword1”, 42), (“”, “foo"), (“keyword2”: 
19)]

And a member lookup with no call would simply pass no arguments.

> It would also allow you to use our "get-a-closure" syntax:
> 
>   let uncalledMethod = foreignObject.someValue(keyword1:_:keyword2:)

Yep, that would pass the keywords to member lookup.  They’d be ignored by 
Python but presumably used for Squeak.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Brent Royal-Gordon via swift-evolution]

Leaving the "should we do it at all" question aside…

> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>  wrote:
> 
> We propose introducing this protocol to the standard library:
> 
> protocol DynamicCallable {
>   associatedtype DynamicCallableArgument
>   associatedtype DynamicCallableResult
> 
>   func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> 
> DynamicCallableResult
> }
> 
> It also extends the language such that function call syntax - when applied to 
> a value of DynamicCallable type - is accepted and transformed into a call to 
> the dynamicCall member. The dynamicCall method takes a list of tuples: the 
> first element is the keyword label (or an empty string if absent) and the 
> second value is the formal parameter specified at the call site.

I don't really like passing the argument names to the parameter list. Instead, 
I would prefer to see them handled in the member-resolution method. You don't 
give a design for DynamicMemberLookupProtocol here, but assuming that you're 
imagining that a call like this:

foreignObject.someValue(keyword1: 42, "foo", keyword2: 19)

Would translate to something like this:

foreignObject.dynamicMethodLookup(name: 
"someValue").dynamicCall(arguments: [("keyword1", 42), ("", "foo"), 
("keyword2", 19)])

I would instead like it to look like this:

foreignObject.dynamicMethodLookup(name: "someValue", arguments: 
["keyword1", "", "keyword2"]).dynamicCall(arguments: [42, "foo", 19])

This would better suit languages like Squeak (or Swift!) where you cannot fully 
look up a method without its keywords. (Languages like Python would simply 
store the keywords inside the object returned by `dynamicMethodLookup`.) It 
would also allow you to use our "get-a-closure" syntax:

let uncalledMethod = foreignObject.someValue(keyword1:_:keyword2:)

-- 
Brent Royal-Gordon
Architechies

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 10, 2017, at 6:12 PM, Slava Pestov via swift-evolution 
>  wrote:
> 
> 
> 
>> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>> > wrote:
>> 
>> Setting this aside, I’m very curious to hear whether type providers 
>> influence your thinking after you’ve had a chance to look into them.  I have 
>> always thought they were very cool.
> 
> I’m in favor of solving this problem with something like type providers also. 
> The required compiler changes would be significant but would also clean up 
> the interface between the ClangImporter, Sema and Serialization. If done 
> right it would be a net gain that would benefit all users, instead of just 
> adding YetAnotherCornerCase™ that makes implementation maintainers curse and 
> scream.

I find it ironic that you’re talking pejoratively about a feature that has very 
narrow impact, complaining about how much of an impact on the compiler it would 
have, and then pine for a hugely invasive feature - one that would cause a ton 
of code churn, and probably wouldn’t actually be enough to eliminate the 
special cases in place because of ObjC interop.

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 10, 2017, at 6:05 PM, Slava Pestov via swift-evolution 
 wrote:
>> On Nov 10, 2017, at 6:01 PM, Chris Lattner  wrote:
>> 
>> It is.  It is strictly sugar, as mentioned in the proposal.
> 
> All new code has a cost.

Indeed, I’m quite aware of that!

>> I do not expect any SIL or IRGen changes associated with this proposal, just 
>> type checker changes.  The type checker changes should be straight-forward, 
>> but you can evaluate that when there is a patch.
> 
> Like I said, the type checker’s code for solving member constraints and 
> applying the solution is already a rats-nest of special cases to deal with 
> the rich semantics of method calls in Swift. We’ve been trying to simplify it 
> over time and fix bugs, and adding new special cases is a step in the wrong 
> direction.

A strict interpretation of your paragraph above says that no new thing should 
ever be added to the language, which I am sure is not what  you mean.

That said, I understand what you’re concerned with.  However, your approach 
with this pitch has been completely unconstructive and unhelpful.  There is a 
time and place where evaluation of the impact can take place: that’s when the 
patch is available.  You are correct that all extensions come at a cost, which 
is why the cost and the *benefit* have to be weighed.

>> Our goal is for Swift to be awesome, not comparable to “other statically 
>> typed languages”.
> 
> Ok, but being awesome is not the same as adding every possible feature to the 
> language.

I’m scratching my head and wondering if you seriously believe that you’re 
informing something that isn’t obvious.

>> swift-evolution isn’t about priorities.  I’m not asking someone else to 
>> implement this, and you don’t tell me how I spend my engineering time :-) :-)
> 
> Again, my point here is that implementing a new feature is just the first 
> step. New code also has to be maintained going forward.

Again, I’m pretty aware of how long term software projects works.

>> I’m open to requiring these to be the same type if there is a benefit to 
>> doing so. What benefit do you see?
> 
> Can you think of a case where they have to be different?

Yes, it is easy to imagine cases where the parameters are all that matters.  In 
that case the result would be Void.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Slava Pestov via swift-evolution]

> On Nov 10, 2017, at 6:10 PM, Matthew Johnson via swift-evolution 
>  wrote:
> 
> Setting this aside, I’m very curious to hear whether type providers influence 
> your thinking after you’ve had a chance to look into them.  I have always 
> thought they were very cool.

I’m in favor of solving this problem with something like type providers also. 
The required compiler changes would be significant but would also clean up the 
interface between the ClangImporter, Sema and Serialization. If done right it 
would be a net gain that would benefit all users, instead of just adding 
YetAnotherCornerCase™ that makes implementation maintainers curse and scream.

Slava___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Matthew Johnson via swift-evolution]

Sent from my iPad

> On Nov 10, 2017, at 7:41 PM, Chris Lattner  wrote:
> 
> 
>> On Nov 10, 2017, at 11:25 AM, Matthew Johnson via swift-evolution 
>>  wrote:
>> 
 
 People have reasonably asked for the ability to make their own 
 function-like types in the past, such that "myvalue(...)" behaves like 
 sugar for "myvalue.call(...)" or something like that. In most cases, they 
 still want to have type system control over what arguments and results 
 their call operation produces. They don't really get that with this 
 proposal; they lose all control over the arity and argument types. 
>>> 
>>> As I mentioned, this is directly addressed in the writeup. Here’s the link:
>>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
>> 
>> That discusses why you didn’t include it in the present proposal but I think 
>> it’s reasonable to oppose adding a dynamic callable feature prior to a more 
>> Swifty static callable.
> 
> Why?  One does not preclude the other.

For exactly the reason Joe articulates.  Some people will use what the language 
offers to get the syntax they desire even if it sacrifices type safety.  If 
we’re going to have first-class callable types in Swift (I think it’s a great 
idea) type safety for native code should be prioritized over syntactic 
convenience for dynamic language interop.  We can have both, but the former 
should come first IMO.

Setting this aside, I’m very curious to hear whether type providers influence 
your thinking after you’ve had a chance to look into them.  I have always 
thought they were very cool.

> 
> -Chris
> 
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Slava Pestov via swift-evolution]

> On Nov 10, 2017, at 6:01 PM, Chris Lattner  wrote:
> 
> It is.  It is strictly sugar, as mentioned in the proposal.

All new code has a cost.

> I do not expect any SIL or IRGen changes associated with this proposal, just 
> type checker changes.  The type checker changes should be straight-forward, 
> but you can evaluate that when there is a patch.

Like I said, the type checker’s code for solving member constraints and 
applying the solution is already a rats-nest of special cases to deal with the 
rich semantics of method calls in Swift. We’ve been trying to simplify it over 
time and fix bugs, and adding new special cases is a step in the wrong 
direction.

> Our goal is for Swift to be awesome, not comparable to “other statically 
> typed languages”.

Ok, but being awesome is not the same as adding every possible feature to the 
language.

> swift-evolution isn’t about priorities.  I’m not asking someone else to 
> implement this, and you don’t tell me how I spend my engineering time :-) :-)

Again, my point here is that implementing a new feature is just the first step. 
New code also has to be maintained going forward.

> I’m open to requiring these to be the same type if there is a benefit to 
> doing so. What benefit do you see?

Can you think of a case where they have to be different?

Slava
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 10, 2017, at 5:51 PM, Slava Pestov via swift-evolution 
 wrote:
> Hi Chris,
> 
>> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>> > wrote:
>> 
>> Hello all,
>> 
>> I have a couple of proposals cooking in a quest to make Swift interoperate 
>> with dynamically typed languages like Python better.  Instead of baking in 
>> hard coded support for one language or the other, I’m preferring to add a 
>> few small but general purpose capabilities to Swift.  This is the first, 
>> which allows a Swift type to become “callable”.
> 
> I’m generally in favor of adding new features if they simplify the language 
> model or subsume existing special cases, either moving them into the standard 
> library or simplifying their implementation.

Great!

> However this proposal looks like a strictly additive feature,

It is.  It is strictly sugar, as mentioned in the proposal.

> which introduces yet another kind of callable thing. We already have:
> 
> - Statically dispatched functions
> - VTable-dispatched class methods
> - Witness table dispatched protocol methods
> - ObjC methods
> - Dynamic method dispatch on AnyObject
> - Enum case constructors
> - Curried functions and various thunks, etc
> 
> I don’t see the new dynamic callable you are proposing replacing or 
> generalizing any of the above, it will simply be a whole new code path.

These things are completely different in character.  They are not sugar: they 
are invasive changes that spread through the rest of the compiler.  They are 
also (generally) not part of the user exposed semantic model for swift, they 
are implementation mechanics.  This proposal if far less invasive, and also 
quite different than these.

> This all comes at a great cost. If you look at the implementation of calls in 
> lib/SILGen/SILGenApply.cpp you will see there is a great deal of complexity 
> there to deal with all the different special cases. The type checker also has 
> a lot of complexity related to method calls and member accesses.

I do not expect any SIL or IRGen changes associated with this proposal, just 
type checker changes.  The type checker changes should be straight-forward, but 
you can evaluate that when there is a patch.

> 
>> Swift is well known for being exceptional at interworking with existing C 
>> and Objective-C APIs, but its support for calling APIs written in scripting 
>> langauges like Python, Perl, and Ruby is quite lacking. These languages 
>> provide an extremely dynamic programming model where almost everything is 
>> discovered at runtime.
> 
> Most other statically compiled languages don’t attempt to solve this problem 
> of interoperating with Python and Ruby either.

Our goal is for Swift to be awesome, not comparable to “other statically typed 
languages”.

> I’m not sure this is a feature users expect or one that should be 
> prioritized, given all the other work that remains in the implementation that 
> will actually improve the day to day experience of developers.

swift-evolution isn’t about priorities.  I’m not asking someone else to 
implement this, and you don’t tell me how I spend my engineering time :-) :-)

> 
>> We propose introducing this protocol to the standard library:
>> 
>> protocol DynamicCallable {
>>   associatedtype DynamicCallableArgument
>>   associatedtype DynamicCallableResult
>> 
>>   func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> 
>> DynamicCallableResult
>> }
> This is not really very general at all, because it assumes all arguments have 
> the same type, along with all results.

Correct.  This is important and common for interoperating with dynamic 
languages.

> Why would arguments and results have different types if they’re type erased 
> anyway?

I’m open to requiring these to be the same type if there is a benefit to doing 
so. What benefit do you see?

> And why are string keyword names privileged in any way? What about varargs, 
> inout parameters and other Swift-specific modifiers on calls?

Because dynamic languages support keywords, and this is all about reflecting 
APIs written in those languages into Swift.  This API works fine for varargs.  
Dynamic languages do not support Swift specific keywords and (generally) do not 
support inout.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 10, 2017, at 5:37 PM, Susan Cheng via swift-evolution 
>  wrote:
> 
> Hi all,
> 
> I have few problems with this proposal.
> 
> How we guarantee the arguments type are what we need?
> If we passing the type to the dynamicCall that are not acceptable, what will 
> happened?

Dynamic languages do not provide static guarantees: they check these things at 
runtime and generally abort/throw on a mismatch.

> Should we always write the dynamicCall as a throwing function to check the 
> precondition?

Sure, that’s totally supported by the proposal.

> Instead of introducing dynamic calls, I would like introducing mirror of 
> instance methods.

I don’t know what that means, but moving forward with this proposal does not 
preclude solving other people’s problems.  You are welcome to write a proposal 
for something if you are interested in it.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Slava Pestov via swift-evolution]

Hi Chris,

> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>  wrote:
> 
> Hello all,
> 
> I have a couple of proposals cooking in a quest to make Swift interoperate 
> with dynamically typed languages like Python better.  Instead of baking in 
> hard coded support for one language or the other, I’m preferring to add a few 
> small but general purpose capabilities to Swift.  This is the first, which 
> allows a Swift type to become “callable”.

I’m generally in favor of adding new features if they simplify the language 
model or subsume existing special cases, either moving them into the standard 
library or simplifying their implementation. However this proposal looks like a 
strictly additive feature, which introduces yet another kind of callable thing. 
We already have:

- Statically dispatched functions
- VTable-dispatched class methods
- Witness table dispatched protocol methods
- ObjC methods
- Dynamic method dispatch on AnyObject
- Enum case constructors
- Curried functions and various thunks, etc

I don’t see the new dynamic callable you are proposing replacing or 
generalizing any of the above, it will simply be a whole new code path.

This all comes at a great cost. If you look at the implementation of calls in 
lib/SILGen/SILGenApply.cpp you will see there is a great deal of complexity 
there to deal with all the different special cases. The type checker also has a 
lot of complexity related to method calls and member accesses.

I would be against adding yet another new type of call to the language at this 
point. Even if the implementation is relatively small it is yet another special 
case we have to maintain forever.

> Swift is well known for being exceptional at interworking with existing C and 
> Objective-C APIs, but its support for calling APIs written in scripting 
> langauges like Python, Perl, and Ruby is quite lacking. These languages 
> provide an extremely dynamic programming model where almost everything is 
> discovered at runtime.

Most other statically compiled languages don’t attempt to solve this problem of 
interoperating with Python and Ruby either. I’m not sure this is a feature 
users expect or one that should be prioritized, given all the other work that 
remains in the implementation that will actually improve the day to day 
experience of developers.

> We propose introducing this protocol to the standard library:
> 
> protocol DynamicCallable {
>   associatedtype DynamicCallableArgument
>   associatedtype DynamicCallableResult
> 
>   func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> 
> DynamicCallableResult
> }
This is not really very general at all, because it assumes all arguments have 
the same type, along with all results. Why would arguments and results have 
different types if they’re type erased anyway? And why are string keyword names 
privileged in any way? What about varargs, inout parameters and other 
Swift-specific modifiers on calls?


> Before this proposal, the Swift language has two types that participate in 
> call syntax: functions and metatypes (for initialization). Neither of those 
> may conform to protocols at the moment, so this introduces no possible 
> ambiguity into the language.
> 
> 

However, it rules out a possible type checker optimization — currently we can 
assume that if a type variable is the subject of both a conformance constraint 
and an apply constraint, the constraint system is invalid. With your change we 
could not make this assumption anymore.

> It is worth noting that this does not introduce the ability to provide 
> dynamicly callable static/class members. We don't believe that this is 
> important given the goal of supporting dynamic languages like Python, but if 
> there is a usecase discovered in the future, it could be explored as future 
> work. Such future work should keep in mind that call syntax on metatypes is 
> already meaningful, and that ambiguity would have to be resolved somehow.
> 
> 

This is the problem with this proposal — it solves a very narrow use case, but 
it introduces a piece of ABI that we have to maintain forever. When future 
generalizations are discovered, we will have to add yet another mechanism, 
complicating the language model further.
>  
> Effect on ABI stability
> 
> This is a strictly additive proposal with no ABI breaking changes.
> 
> 

However, it adds *new* ABI which we will have to maintain forever.

A strong -1 from me here.

Slava

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 10, 2017, at 11:42 AM, Joe Groff via swift-evolution 
 wrote:
>> 
>>> On Nov 10, 2017, at 10:57 AM, Alejandro Martinez via swift-evolution 
>>>  wrote:
>>> 
>>> This seems a really interesting solution Chris.
>>> Similar to what Joe mentions I think this would also be appreciated by
>>> the community to make even nicer DSLs in Swift, which may or may not
>>> be a good side effect of the proposal.
>>> Also, I'm just wondering, how much complication adds this to the
>>> compiler itself that would have to be maintained in the future?
>> 
>> This is a very localized and simple change to the compiler.  Assuming the 
>> pitch process goes well, I will provide an implementation.
> 
> I don't think it's that localized. It's going to make call resolution 
> massively more complicated.  Through great pain and community anguish, we 
> pushed ourselves to a model where argument labels are parts of the 
> declaration name, not part of the call argument, and this would throw us 
> straight back into the kinds of complications of dealing with overloaded 
> name- and type-dependent behavior we're trying to get away from in the 
> language.

I’m pretty sure it doesn’t, but good news: since Swift requires an 
implementation before the proposal can be officially reviewed, you will be able 
to look at the actual patch to make this decision.

> Swift is a dynamic language too, and tuples have enough metadata in them that 
> you could get almost everything you want with an infix operator.

This is really gross.  The goal here is to make Python feel natural.  If you 
oppose this proposal then we’ll have to find some other way to provide an 
elegant experience:

> x∫(0, "foo", bar: "bas”)

This is not going to be acceptable to users.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 10, 2017, at 11:25 AM, Matthew Johnson via swift-evolution 
>  wrote:
> 
>>> 
>>> People have reasonably asked for the ability to make their own 
>>> function-like types in the past, such that "myvalue(...)" behaves like 
>>> sugar for "myvalue.call(...)" or something like that. In most cases, they 
>>> still want to have type system control over what arguments and results 
>>> their call operation produces. They don't really get that with this 
>>> proposal; they lose all control over the arity and argument types. 
>> 
>> As I mentioned, this is directly addressed in the writeup. Here’s the link:
>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
>>  
>> 
> That discusses why you didn’t include it in the present proposal but I think 
> it’s reasonable to oppose adding a dynamic callable feature prior to a more 
> Swifty static callable.

Why?  One does not preclude the other.

-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Susan Cheng via swift-evolution]

Hi all,

I have few problems with this proposal.

How we guarantee the arguments type are what we need?

If we passing the type to the dynamicCall that are not acceptable, what will 
happened?

Should we always write the dynamicCall as a throwing function to check the 
precondition?

Instead of introducing dynamic calls, I would like introducing mirror of 
instance methods.

> Joe Groff via swift-evolution  於 2017年11月11日 
> 上午8:20 寫道：
> 
> 
> 
>>> On Nov 10, 2017, at 4:12 PM, Charles Srstka  
>>> wrote:
>>> 
>>> On Nov 10, 2017, at 5:51 PM, Joe Groff  wrote:
>>> 
>>> 
>>> 
> On Nov 10, 2017, at 3:45 PM, Charles Srstka  
> wrote:
> 
> On Nov 10, 2017, at 5:36 PM, Joe Groff  wrote:
> 
> How `MyObject.foo(_:bar:)` gets implemented is its own business, as far 
> as the compiler is concerned. The compile-time name resolution for the 
> method isn't impacted.
> 
> -Joe
 
 The compile-time name resolution for the method doesn’t happen *at all.*
>>> 
>>> You declared the method in your @interface, and the compiler saw that and 
>>> brought it in as what Swift considers to be a regular method, and your call 
>>> on the Swift side was resolved to it by Swift's usual lookup rules. To do 
>>> what Chris is suggesting requires changing the way calls get resolved in 
>>> the compiler before the call is even formed.
>> 
>> The only thing that makes this the “usual lookup rules” is that the 
>> Objective-C bridge has already been implemented.
> 
> As I mentioned in my original reply, I personally think the "importer" 
> approach would be superior, and that in a perfect world we'd have type 
> providers to make writing something like the ObjC importer but for a 
> different language or other dynamic data source something that doesn't 
> require invasive compiler hackery. The importer puts all of this:
> 
>> - It’s changing the compile-time name resolution! The Swift name is 
>> foo(bar:), but it’s changing that to fooWithBar:!
>> 
>> - It’s changing the signature! The argument took a String, but now it’s 
>> passing an NSString!
>> 
>> - It’s not resolving the method at compile-time! It’s passing the modified 
>> method name and the arg list to some objc_msgSend() function, which resolves 
>> it dynamically in a way that user code can intercept and interpret at 
>> runtime!
>> 
>> I’m just not seeing the conceptual difference here.
> 
> below the fold as far as the rest of the language is concerned. You could 
> just as well written what the importer synths up in Swift directly:
> 
> func foo(bar: String) {
>  unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
> NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
> }
> 
> and the rest of the language would be none the wiser.
> 
> -Joe
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 4:12 PM, Charles Srstka  wrote:
> 
>> On Nov 10, 2017, at 5:51 PM, Joe Groff  wrote:
>> 
>> 
>> 
>>> On Nov 10, 2017, at 3:45 PM, Charles Srstka  
>>> wrote:
>>> 
 On Nov 10, 2017, at 5:36 PM, Joe Groff  wrote:
 
 How `MyObject.foo(_:bar:)` gets implemented is its own business, as far as 
 the compiler is concerned. The compile-time name resolution for the method 
 isn't impacted.
 
 -Joe
>>> 
>>> The compile-time name resolution for the method doesn’t happen *at all.*
>> 
>> You declared the method in your @interface, and the compiler saw that and 
>> brought it in as what Swift considers to be a regular method, and your call 
>> on the Swift side was resolved to it by Swift's usual lookup rules. To do 
>> what Chris is suggesting requires changing the way calls get resolved in the 
>> compiler before the call is even formed.
> 
> The only thing that makes this the “usual lookup rules” is that the 
> Objective-C bridge has already been implemented.

As I mentioned in my original reply, I personally think the "importer" approach 
would be superior, and that in a perfect world we'd have type providers to make 
writing something like the ObjC importer but for a different language or other 
dynamic data source something that doesn't require invasive compiler hackery. 
The importer puts all of this:

> - It’s changing the compile-time name resolution! The Swift name is 
> foo(bar:), but it’s changing that to fooWithBar:!
> 
> - It’s changing the signature! The argument took a String, but now it’s 
> passing an NSString!
> 
> - It’s not resolving the method at compile-time! It’s passing the modified 
> method name and the arg list to some objc_msgSend() function, which resolves 
> it dynamically in a way that user code can intercept and interpret at runtime!
> 
> I’m just not seeing the conceptual difference here.

below the fold as far as the rest of the language is concerned. You could just 
as well written what the importer synths up in Swift directly:

func foo(bar: String) {
  unsafeBitCast(objc_msgSend, to: @convention(c) (AnyObject, Selector, 
NSString) -> ().self)(self, "fooWithBar:", NSString(bar))
}

and the rest of the language would be none the wiser.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Charles Srstka via swift-evolution]

> On Nov 10, 2017, at 5:51 PM, Joe Groff  wrote:
> 
> 
> 
>> On Nov 10, 2017, at 3:45 PM, Charles Srstka > > wrote:
>> 
>>> On Nov 10, 2017, at 5:36 PM, Joe Groff >> > wrote:
>>> 
>>> How `MyObject.foo(_:bar:)` gets implemented is its own business, as far as 
>>> the compiler is concerned. The compile-time name resolution for the method 
>>> isn't impacted.
>>> 
>>> -Joe
>> 
>> The compile-time name resolution for the method doesn’t happen *at all.*
> 
> You declared the method in your @interface, and the compiler saw that and 
> brought it in as what Swift considers to be a regular method, and your call 
> on the Swift side was resolved to it by Swift's usual lookup rules. To do 
> what Chris is suggesting requires changing the way calls get resolved in the 
> compiler before the call is even formed.

The only thing that makes this the “usual lookup rules” is that the Objective-C 
bridge has already been implemented. Consider scenario in which the Objective-C 
bridge does not exist, and someone is proposing it. As I see it, all these 
objections will apply to it just the same.

For the following method:

@objc dynamic func foo(bar: String)

- It’s changing the compile-time name resolution! The Swift name is foo(bar:), 
but it’s changing that to fooWithBar:!

- It’s changing the signature! The argument took a String, but now it’s passing 
an NSString!

- It’s not resolving the method at compile-time! It’s passing the modified 
method name and the arg list to some objc_msgSend() function, which resolves it 
dynamically in a way that user code can intercept and interpret at runtime!

I’m just not seeing the conceptual difference here.

Charles

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 3:45 PM, Charles Srstka  wrote:
> 
>> On Nov 10, 2017, at 5:36 PM, Joe Groff  wrote:
>> 
>> How `MyObject.foo(_:bar:)` gets implemented is its own business, as far as 
>> the compiler is concerned. The compile-time name resolution for the method 
>> isn't impacted.
>> 
>> -Joe
> 
> The compile-time name resolution for the method doesn’t happen *at all.*

You declared the method in your @interface, and the compiler saw that and 
brought it in as what Swift considers to be a regular method, and your call on 
the Swift side was resolved to it by Swift's usual lookup rules. To do what 
Chris is suggesting requires changing the way calls get resolved in the 
compiler before the call is even formed.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Charles Srstka via swift-evolution]

> On Nov 10, 2017, at 5:36 PM, Joe Groff  wrote:
> 
> How `MyObject.foo(_:bar:)` gets implemented is its own business, as far as 
> the compiler is concerned. The compile-time name resolution for the method 
> isn't impacted.
> 
> -Joe

The compile-time name resolution for the method doesn’t happen *at all.* If the 
method is dynamic, it’s resolved at runtime. The method may or may not even 
exist; all you know is that the object address, the method name, and a list of 
the arguments will be passed to a function called objc_msgSend(), which 
conceptually is remarkably similar to Chris’s dynamicCall().

Charles

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 3:27 PM, Charles Srstka  wrote:
> 
>> On Nov 10, 2017, at 2:57 PM, Joe Groff  wrote:
>> 
>>> On Nov 10, 2017, at 12:37 PM, Charles Srstka  
>>> wrote:
>>> 
 On Nov 10, 2017, at 12:04 PM, Joe Groff via swift-evolution 
  wrote:
 
 I don't like the idea of some calls having wildly different semantics from 
 others; it's difficult enough to tell what exactly a call might be doing 
 already. Since we also lack the more obvious static "Callable" protocol 
 idea to give even well-typed call syntax to user-defined types, this also 
 seems like it'd be easily abused for that purpose too.
>>> 
>>> We already have that though, with the Objective-C bridge. How is the 
>>> proposed behavior here more wildly different than the semantics of 
>>> non-@objc, @objc, and @objc dynamic calls?
>> 
>> The language semantics aren't any different for non-@objc or @objc calls. 
>> The dispatch mechanism is an implementation detail. `dynamic` admits the 
>> possibility of late binding to change the method implementation dynamically, 
>> but doesn't change the type system behavior of the method, or radically 
>> change the implementation mechanism; it's still ultimately an indirect call, 
>> it doesn't turn your argument list into a dictionary that can be arbitrarily 
>> interpreted by user code.
>> 
>> -Joe
> 
> You sure about that? ;-)
> 
> MyObject.h:
> 
> #import 
> 
> @interface MyObject : NSObject
> 
> @property (nonatomic, copy) void (^callback)(NSDictionary *);
> 
> @end
> 
> @interface MyObject (MyCategory)
> 
> - (void)foo:(NSString *)foo bar:(NSString *)bar;
> 
> @end
> 
> MyObject.m:
> 
> #import "MyObject.h"
> 
> @implementation MyObject
> 
> - (void)baz:(NSString *)baz qux:(NSString *)qux {}
> 
> - (NSMethodSignature *)methodSignatureForSelector:(SEL)selector {
> if (selector == @selector(foo:bar:)) {
> return [super methodSignatureForSelector:@selector(baz:qux:)];
> } else {
> return [super methodSignatureForSelector:selector];
> }
> }
> 
> - (void)forwardInvocation:(NSInvocation *)invocation {
> NSString *name = NSStringFromSelector(invocation.selector);
> NSMutableArray *args = [NSMutableArray new];
> 
> for (NSUInteger i = 2; i < invocation.methodSignature.numberOfArguments; 
> i++) {
> __unsafe_unretained id obj = nil;
> 
> [invocation getArgument: atIndex:i];
> 
> [args addObject:obj];
> }
> 
> self.callback(@{ @"name" : name, @"arguments" : args });
> }
> 
> @end
> 
> main.swift:
> 
> import Foundation
> 
> let obj = MyObject()
> 
> obj.callback = { dict in
> print("got this dictionary: \(dict as? [String : Any] ?? [:])")
> }
> 
> obj.foo("Foo", bar: "Baz”)

How `MyObject.foo(_:bar:)` gets implemented is its own business, as far as the 
compiler is concerned. The compile-time name resolution for the method isn't 
impacted.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Charles Srstka via swift-evolution]

> On Nov 10, 2017, at 2:57 PM, Joe Groff  wrote:
> 
>> On Nov 10, 2017, at 12:37 PM, Charles Srstka > > wrote:
>> 
>>> On Nov 10, 2017, at 12:04 PM, Joe Groff via swift-evolution 
>>> > wrote:
>>> 
>>> I don't like the idea of some calls having wildly different semantics from 
>>> others; it's difficult enough to tell what exactly a call might be doing 
>>> already. Since we also lack the more obvious static "Callable" protocol 
>>> idea to give even well-typed call syntax to user-defined types, this also 
>>> seems like it'd be easily abused for that purpose too.
>> 
>> We already have that though, with the Objective-C bridge. How is the 
>> proposed behavior here more wildly different than the semantics of 
>> non-@objc, @objc, and @objc dynamic calls?
> 
> The language semantics aren't any different for non-@objc or @objc calls. The 
> dispatch mechanism is an implementation detail. `dynamic` admits the 
> possibility of late binding to change the method implementation dynamically, 
> but doesn't change the type system behavior of the method, or radically 
> change the implementation mechanism; it's still ultimately an indirect call, 
> it doesn't turn your argument list into a dictionary that can be arbitrarily 
> interpreted by user code.
> 
> -Joe

You sure about that? ;-)

MyObject.h:

#import 

@interface MyObject : NSObject

@property (nonatomic, copy) void (^callback)(NSDictionary *);

@end

@interface MyObject (MyCategory)

- (void)foo:(NSString *)foo bar:(NSString *)bar;

@end

MyObject.m:

#import "MyObject.h"

@implementation MyObject

- (void)baz:(NSString *)baz qux:(NSString *)qux {}

- (NSMethodSignature *)methodSignatureForSelector:(SEL)selector {
if (selector == @selector(foo:bar:)) {
return [super methodSignatureForSelector:@selector(baz:qux:)];
} else {
return [super methodSignatureForSelector:selector];
}
}

- (void)forwardInvocation:(NSInvocation *)invocation {
NSString *name = NSStringFromSelector(invocation.selector);
NSMutableArray *args = [NSMutableArray new];

for (NSUInteger i = 2; i < invocation.methodSignature.numberOfArguments; 
i++) {
__unsafe_unretained id obj = nil;

[invocation getArgument: atIndex:i];

[args addObject:obj];
}

self.callback(@{ @"name" : name, @"arguments" : args });
}

@end

main.swift:

import Foundation

let obj = MyObject()

obj.callback = { dict in
print("got this dictionary: \(dict as? [String : Any] ?? [:])")
}

obj.foo("Foo", bar: "Baz”)

Charles

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 12:37 PM, Charles Srstka  wrote:
> 
>> On Nov 10, 2017, at 12:04 PM, Joe Groff via swift-evolution 
>>  wrote:
>> 
>> I don't like the idea of some calls having wildly different semantics from 
>> others; it's difficult enough to tell what exactly a call might be doing 
>> already. Since we also lack the more obvious static "Callable" protocol idea 
>> to give even well-typed call syntax to user-defined types, this also seems 
>> like it'd be easily abused for that purpose too.
> 
> We already have that though, with the Objective-C bridge. How is the proposed 
> behavior here more wildly different than the semantics of non-@objc, @objc, 
> and @objc dynamic calls?

The language semantics aren't any different for non-@objc or @objc calls. The 
dispatch mechanism is an implementation detail. `dynamic` admits the 
possibility of late binding to change the method implementation dynamically, 
but doesn't change the type system behavior of the method, or radically change 
the implementation mechanism; it's still ultimately an indirect call, it 
doesn't turn your argument list into a dictionary that can be arbitrarily 
interpreted by user code.

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 12:16 PM, Goffredo Marocchi  wrote:
> 
> 
> Sent from my iPhone
> 
>> On 10 Nov 2017, at 19:42, Joe Groff via swift-evolution 
>>  wrote:
>> 
>> Through great pain and community anguish, we pushed ourselves to a model 
>> where argument labels are parts of the declaration name, not part of the 
>> call argument
> 
> Hey Joe, does this mean it is likely there is little to no chance to have 
> argument labels for closures and stored functions back? Sorry to bring noise 
> in here, but your words put a grim outlook on something the core team said it 
> would be revisited last year in the next Swift iteration.

Not at all. The idea for closure labels is that the labels would still be part 
of the name, but that you could give compound names to closure variables in 
addition to named functions. That would let you write something like:

let foo(bar:) : (Int) -> String = { "\($0)" }
foo(bar: 0)

Or, with a bit more sugar:

let foo : (bar: Int) -> String = { "\($0)" } // Notationally, we write 
the labels with the type, but they become part of the 'let' name
foo(bar: 0)

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Charles Srstka via swift-evolution]

> On Nov 10, 2017, at 12:04 PM, Joe Groff via swift-evolution 
>  wrote:
> 
> I don't like the idea of some calls having wildly different semantics from 
> others; it's difficult enough to tell what exactly a call might be doing 
> already. Since we also lack the more obvious static "Callable" protocol idea 
> to give even well-typed call syntax to user-defined types, this also seems 
> like it'd be easily abused for that purpose too.

We already have that though, with the Objective-C bridge. How is the proposed 
behavior here more wildly different than the semantics of non-@objc, @objc, and 
@objc dynamic calls?

Charles

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Goffredo Marocchi via swift-evolution]

Sent from my iPhone

> On 10 Nov 2017, at 19:42, Joe Groff via swift-evolution 
>  wrote:
> 
> Through great pain and community anguish, we pushed ourselves to a model 
> where argument labels are parts of the declaration name, not part of the call 
> argument

Hey Joe, does this mean it is likely there is little to no chance to have 
argument labels for closures and stored functions back? Sorry to bring noise in 
here, but your words put a grim outlook on something the core team said it 
would be revisited last year in the next Swift iteration.
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 11:20 AM, Chris Lattner via swift-evolution 
>  wrote:
> 
> 
>> On Nov 10, 2017, at 10:57 AM, Alejandro Martinez via swift-evolution 
>>  wrote:
>> 
>> This seems a really interesting solution Chris.
>> Similar to what Joe mentions I think this would also be appreciated by
>> the community to make even nicer DSLs in Swift, which may or may not
>> be a good side effect of the proposal.
>> Also, I'm just wondering, how much complication adds this to the
>> compiler itself that would have to be maintained in the future?
> 
> This is a very localized and simple change to the compiler.  Assuming the 
> pitch process goes well, I will provide an implementation.

I don't think it's that localized. It's going to make call resolution massively 
more complicated. Through great pain and community anguish, we pushed ourselves 
to a model where argument labels are parts of the declaration name, not part of 
the call argument, and this would throw us straight back into the kinds of 
complications of dealing with overloaded name- and type-dependent behavior 
we're trying to get away from in the language.

Swift is a dynamic language too, and tuples have enough metadata in them that 
you could get almost everything you want with an infix operator. Mirror doesn't 
expose the reflection information particularly elegantly, but you can do 
something like this to decompose a Swift tuple into positional and keyword 
arguments to fit the Python model:

infix operator ∫ // a "snake". get it

protocol Pythonic {
  func call(positionalArgs: [Pythonic], kwArgs: [(String, Pythonic)]) -> 
Pythonic
}

func ∫(function: Pythonic, args: Any) -> Pythonic {
  var positionalArgs: [Pythonic] = []
  var kwArgs: [(String, Pythonic)] = []
  
  let argMirror = Mirror(reflecting: args)
  
  if argMirror.displayStyle == .tuple {
// Tuple argument
for child in argMirror.children {
  if let label = child.label, let first = label.first, first != "." {
kwArgs.append((label, child.value as! Pythonic))
  } else {
positionalArgs.append(child.value as! Pythonic)
  }
}
  } else {
// One argument
positionalArgs = [args as! Pythonic]
  }
  
  return function.call(positionalArgs: positionalArgs, kwArgs: kwArgs)
}

x∫(0, "foo", bar: "bas")

-Joe
___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Matthew Johnson via swift-evolution]

> On Nov 10, 2017, at 1:19 PM, Chris Lattner via swift-evolution 
>  wrote:
> 
>> On Nov 10, 2017, at 10:51 AM, Joe Groff via swift-evolution 
>> > wrote:
>>> 
 Since we also lack the more obvious static "Callable" protocol idea to 
 give even well-typed call syntax to user-defined types, this also seems 
 like it'd be easily abused for that purpose too.
>>> 
>>> Similarly, I’d love for you to elaborate on how the potential for abuse of 
>>> this feature is different than anything else in Swift (e.g. operator 
>>> overloading).  Potential for abuse hasn’t been a particularly guiding force 
>>> in the design on Swift, and for good reasons.
>>> 
>>> I also don’t understand what you mean by a static Callable protocol.  I 
>>> specifically address what I think you might mean in the “alternatives” part 
>>> of the proposal, did you read that?
>> 
>> People have reasonably asked for the ability to make their own function-like 
>> types in the past, such that "myvalue(...)" behaves like sugar for 
>> "myvalue.call(...)" or something like that. In most cases, they still want 
>> to have type system control over what arguments and results their call 
>> operation produces. They don't really get that with this proposal; they lose 
>> all control over the arity and argument types.
> 
> As I mentioned, this is directly addressed in the writeup. Here’s the link:
> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
>  
> 
That discusses why you didn’t include it in the present proposal but I think 
it’s reasonable to oppose adding a dynamic callable feature prior to a more 
Swifty static callable.

> 
> -Chris
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 10, 2017, at 10:57 AM, Alejandro Martinez via swift-evolution 
>  wrote:
> 
> This seems a really interesting solution Chris.
> Similar to what Joe mentions I think this would also be appreciated by
> the community to make even nicer DSLs in Swift, which may or may not
> be a good side effect of the proposal.
> Also, I'm just wondering, how much complication adds this to the
> compiler itself that would have to be maintained in the future?

This is a very localized and simple change to the compiler.  Assuming the pitch 
process goes well, I will provide an implementation.

-Chris


___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

> On Nov 10, 2017, at 10:51 AM, Joe Groff via swift-evolution 
>  wrote:
>> 
>>> Since we also lack the more obvious static "Callable" protocol idea to give 
>>> even well-typed call syntax to user-defined types, this also seems like 
>>> it'd be easily abused for that purpose too.
>> 
>> Similarly, I’d love for you to elaborate on how the potential for abuse of 
>> this feature is different than anything else in Swift (e.g. operator 
>> overloading).  Potential for abuse hasn’t been a particularly guiding force 
>> in the design on Swift, and for good reasons.
>> 
>> I also don’t understand what you mean by a static Callable protocol.  I 
>> specifically address what I think you might mean in the “alternatives” part 
>> of the proposal, did you read that?
> 
> People have reasonably asked for the ability to make their own function-like 
> types in the past, such that "myvalue(...)" behaves like sugar for 
> "myvalue.call(...)" or something like that. In most cases, they still want to 
> have type system control over what arguments and results their call operation 
> produces. They don't really get that with this proposal; they lose all 
> control over the arity and argument types.

As I mentioned, this is directly addressed in the writeup. Here’s the link:
https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d#staticly-checking-for-exact-signatures
 


-Chris

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Matthew Johnson via swift-evolution]

Sent from my iPad

> On Nov 10, 2017, at 12:51 PM, Joe Groff via swift-evolution 
>  wrote:
> 
> 
> 
>> On Nov 10, 2017, at 10:41 AM, Chris Lattner  wrote:
>> 
>> On Nov 10, 2017, at 10:03 AM, Joe Groff  wrote:
>>> 
>>> I don't like the idea of some calls having wildly different semantics from 
>>> others; it's difficult enough to tell what exactly a call might be doing 
>>> already.
>> 
>> This isn’t particularly useful feedback.  Can you elaborate more on what 
>> your concern is, and how calls are unlike anything else in Swift that would 
>> have this potential problem?
>> 
>>> Since we also lack the more obvious static "Callable" protocol idea to give 
>>> even well-typed call syntax to user-defined types, this also seems like 
>>> it'd be easily abused for that purpose too.
>> 
>> Similarly, I’d love for you to elaborate on how the potential for abuse of 
>> this feature is different than anything else in Swift (e.g. operator 
>> overloading).  Potential for abuse hasn’t been a particularly guiding force 
>> in the design on Swift, and for good reasons.
>> 
>> I also don’t understand what you mean by a static Callable protocol.  I 
>> specifically address what I think you might mean in the “alternatives” part 
>> of the proposal, did you read that?
> 
> People have reasonably asked for the ability to make their own function-like 
> types in the past, such that "myvalue(...)" behaves like sugar for 
> "myvalue.call(...)" or something like that. In most cases, they still want to 
> have type system control over what arguments and results their call operation 
> produces. They don't really get that with this proposal; they lose all 
> control over the arity and argument types. That may be what you want for 
> calling into a dynamically-typed black hole, but it's not what you want in 
> general. I fear that people would be willing to compromise their designs in 
> order to get the sugar they want by contorting to fit this design.

+1.  This exact concern is the first thing that came to mind while reading this 
proposal.  

The general problem of interop with dynamic languages is an important one but a 
solution shouldn’t come at the cost of introducing syntactic sugar that is 
sub-optimal for native Swift code.

> 
>> 
>>> I think a much better general solution to the problem of "make dynamic 
>>> systems interact with type systems" is something like F#'s type providers 
>>> which lets you write your own importers that look at dynamic information 
>>> from a database, dynamic language VM, or some other system and generate 
>>> type information usable by the compiler.
>> 
>> Thanks! I wasn’t aware of type providers, I’ll investigate them.
>> 
>>> Integration at the importer level could let you produce more well-typed 
>>> Swift declarations by looking at the runtime information you get by 
>>> importing a Python module.
>> 
>> This is also addressed directly in the proposal.  I don’t want to add Python 
>> specific support to Swift.  The motivations are explained in the proposal.
> 
> I'm not suggesting a Python-specific feature, I'm suggesting that type 
> providers could be a general framework for the kinds of problems you 
> encounter trying to make dynamic systems work with Swift, including Python.
> 
> -Joe
> 
> ___
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Alejandro Martinez via swift-evolution]

This seems a really interesting solution Chris.
Similar to what Joe mentions I think this would also be appreciated by
the community to make even nicer DSLs in Swift, which may or may not
be a good side effect of the proposal.
Also, I'm just wondering, how much complication adds this to the
compiler itself that would have to be maintained in the future? I've
seen some threads lately about removing some "debt" with the
Objective-C interoperability, so I guess it would be nice to analyse
that also (not a compiler engineer here but I guess is a thing that
affects everyone so I'm just raising it :P)
And following up on Joe idea of type providers is that is something
that the web community has been using to interop JS with more typed
compile-to-JS langs, it also helps them give autocomplition on IDEs so
it may be worth investigating that.
Sorry for not having more direct feedback on the proposal. For what
can understand it looks reasonable and a good solution for the issue
in hand.
Cheers.

On Fri, Nov 10, 2017 at 6:41 PM, Chris Lattner via swift-evolution
 wrote:
> On Nov 10, 2017, at 10:03 AM, Joe Groff  wrote:
>>
>> I don't like the idea of some calls having wildly different semantics from 
>> others; it's difficult enough to tell what exactly a call might be doing 
>> already.
>
> This isn’t particularly useful feedback.  Can you elaborate more on what your 
> concern is, and how calls are unlike anything else in Swift that would have 
> this potential problem?
>
>> Since we also lack the more obvious static "Callable" protocol idea to give 
>> even well-typed call syntax to user-defined types, this also seems like it'd 
>> be easily abused for that purpose too.
>
> Similarly, I’d love for you to elaborate on how the potential for abuse of 
> this feature is different than anything else in Swift (e.g. operator 
> overloading).  Potential for abuse hasn’t been a particularly guiding force 
> in the design on Swift, and for good reasons.
>
> I also don’t understand what you mean by a static Callable protocol.  I 
> specifically address what I think you might mean in the “alternatives” part 
> of the proposal, did you read that?
>
>
>> I think a much better general solution to the problem of "make dynamic 
>> systems interact with type systems" is something like F#'s type providers 
>> which lets you write your own importers that look at dynamic information 
>> from a database, dynamic language VM, or some other system and generate type 
>> information usable by the compiler.
>
> Thanks! I wasn’t aware of type providers, I’ll investigate them.
>
>> Integration at the importer level could let you produce more well-typed 
>> Swift declarations by looking at the runtime information you get by 
>> importing a Python module.
>
> This is also addressed directly in the proposal.  I don’t want to add Python 
> specific support to Swift.  The motivations are explained in the proposal.
>
>
> Do you have any actual feedback on the proposal?
>
> -Chris
>
>
>
>
>>
>> -Joe
>>
>>> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>>>  wrote:
>>>
>>> Hello all,
>>>
>>> I have a couple of proposals cooking in a quest to make Swift interoperate 
>>> with dynamically typed languages like Python better.  Instead of baking in 
>>> hard coded support for one language or the other, I’m preferring to add a 
>>> few small but general purpose capabilities to Swift.  This is the first, 
>>> which allows a Swift type to become “callable”.
>>>
>>> The proposal is here:
>>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
>>>
>>> I’ve also attached a snapshot below, but it will drift out of date as the 
>>> proposal is refined.  Feedback and thoughts are appreciated, thanks!
>>>
>>> -Chris
>>>
>>>
>>>
>>>
>>>
>>> Introduce user-defined dynamically "callable" types
>>>
>>>  • Proposal: SE-
>>>  • Author: Chris Lattner
>>>  • Review Manager: TBD
>>>  • Status: Awaiting implementation
>>> Introduction
>>>
>>> This proposal introduces a new DynamicCallable protocol to the standard 
>>> library. Types that conform to it are "callable" with the function call 
>>> syntax. It is simple syntactic sugar which allows the user to write:
>>>
>>>a = someValue(keyword1: 42, "foo", keyword2: 19)
>>> and have it be interpreted by the compiler as:
>>>
>>>  a = someValue.dynamicCall(arguments
>>> : [
>>>(
>>> "keyword1", 42), ("", "foo"), ("keyword2", 19
>>> )
>>>  ])
>>>
>>> Other languages have analogous features (e.g. Python "callables"), but the 
>>> primary motivation of this proposal is to allow elegant and natural 
>>> interoperation with dynamic languages in Swift.
>>>
>>> Swift-evolution thread: Discussion thread topic for that proposal
>>>
>>> Motivation
>>>
>>> Swift is well known for being exceptional at interworking with existing C 
>>> and Objective-C APIs, but its support for calling APIs written in scripting 
>>> 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

> On Nov 10, 2017, at 10:41 AM, Chris Lattner  wrote:
> 
> On Nov 10, 2017, at 10:03 AM, Joe Groff  wrote:
>> 
>> I don't like the idea of some calls having wildly different semantics from 
>> others; it's difficult enough to tell what exactly a call might be doing 
>> already.
> 
> This isn’t particularly useful feedback.  Can you elaborate more on what your 
> concern is, and how calls are unlike anything else in Swift that would have 
> this potential problem?
> 
>> Since we also lack the more obvious static "Callable" protocol idea to give 
>> even well-typed call syntax to user-defined types, this also seems like it'd 
>> be easily abused for that purpose too.
> 
> Similarly, I’d love for you to elaborate on how the potential for abuse of 
> this feature is different than anything else in Swift (e.g. operator 
> overloading).  Potential for abuse hasn’t been a particularly guiding force 
> in the design on Swift, and for good reasons.
> 
> I also don’t understand what you mean by a static Callable protocol.  I 
> specifically address what I think you might mean in the “alternatives” part 
> of the proposal, did you read that?

People have reasonably asked for the ability to make their own function-like 
types in the past, such that "myvalue(...)" behaves like sugar for 
"myvalue.call(...)" or something like that. In most cases, they still want to 
have type system control over what arguments and results their call operation 
produces. They don't really get that with this proposal; they lose all control 
over the arity and argument types. That may be what you want for calling into a 
dynamically-typed black hole, but it's not what you want in general. I fear 
that people would be willing to compromise their designs in order to get the 
sugar they want by contorting to fit this design.

> 
>> I think a much better general solution to the problem of "make dynamic 
>> systems interact with type systems" is something like F#'s type providers 
>> which lets you write your own importers that look at dynamic information 
>> from a database, dynamic language VM, or some other system and generate type 
>> information usable by the compiler.
> 
> Thanks! I wasn’t aware of type providers, I’ll investigate them.
> 
>> Integration at the importer level could let you produce more well-typed 
>> Swift declarations by looking at the runtime information you get by 
>> importing a Python module.
> 
> This is also addressed directly in the proposal.  I don’t want to add Python 
> specific support to Swift.  The motivations are explained in the proposal.

I'm not suggesting a Python-specific feature, I'm suggesting that type 
providers could be a general framework for the kinds of problems you encounter 
trying to make dynamic systems work with Swift, including Python.

-Joe

___
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

On Nov 10, 2017, at 10:03 AM, Joe Groff  wrote:
> 
> I don't like the idea of some calls having wildly different semantics from 
> others; it's difficult enough to tell what exactly a call might be doing 
> already.

This isn’t particularly useful feedback.  Can you elaborate more on what your 
concern is, and how calls are unlike anything else in Swift that would have 
this potential problem?

> Since we also lack the more obvious static "Callable" protocol idea to give 
> even well-typed call syntax to user-defined types, this also seems like it'd 
> be easily abused for that purpose too.

Similarly, I’d love for you to elaborate on how the potential for abuse of this 
feature is different than anything else in Swift (e.g. operator overloading).  
Potential for abuse hasn’t been a particularly guiding force in the design on 
Swift, and for good reasons.

I also don’t understand what you mean by a static Callable protocol.  I 
specifically address what I think you might mean in the “alternatives” part of 
the proposal, did you read that?


> I think a much better general solution to the problem of "make dynamic 
> systems interact with type systems" is something like F#'s type providers 
> which lets you write your own importers that look at dynamic information from 
> a database, dynamic language VM, or some other system and generate type 
> information usable by the compiler.

Thanks! I wasn’t aware of type providers, I’ll investigate them.

> Integration at the importer level could let you produce more well-typed Swift 
> declarations by looking at the runtime information you get by importing a 
> Python module.

This is also addressed directly in the proposal.  I don’t want to add Python 
specific support to Swift.  The motivations are explained in the proposal.


Do you have any actual feedback on the proposal?

-Chris




> 
> -Joe
> 
>> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>>  wrote:
>> 
>> Hello all,
>> 
>> I have a couple of proposals cooking in a quest to make Swift interoperate 
>> with dynamically typed languages like Python better.  Instead of baking in 
>> hard coded support for one language or the other, I’m preferring to add a 
>> few small but general purpose capabilities to Swift.  This is the first, 
>> which allows a Swift type to become “callable”.
>> 
>> The proposal is here:
>> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
>> 
>> I’ve also attached a snapshot below, but it will drift out of date as the 
>> proposal is refined.  Feedback and thoughts are appreciated, thanks!
>> 
>> -Chris
>> 
>> 
>> 
>> 
>> 
>> Introduce user-defined dynamically "callable" types
>> 
>>  • Proposal: SE-
>>  • Author: Chris Lattner
>>  • Review Manager: TBD
>>  • Status: Awaiting implementation
>> Introduction
>> 
>> This proposal introduces a new DynamicCallable protocol to the standard 
>> library. Types that conform to it are "callable" with the function call 
>> syntax. It is simple syntactic sugar which allows the user to write:
>> 
>>a = someValue(keyword1: 42, "foo", keyword2: 19)
>> and have it be interpreted by the compiler as:
>> 
>>  a = someValue.dynamicCall(arguments
>> : [
>>(
>> "keyword1", 42), ("", "foo"), ("keyword2", 19
>> )
>>  ])
>> 
>> Other languages have analogous features (e.g. Python "callables"), but the 
>> primary motivation of this proposal is to allow elegant and natural 
>> interoperation with dynamic languages in Swift.
>> 
>> Swift-evolution thread: Discussion thread topic for that proposal
>> 
>> Motivation
>> 
>> Swift is well known for being exceptional at interworking with existing C 
>> and Objective-C APIs, but its support for calling APIs written in scripting 
>> langauges like Python, Perl, and Ruby is quite lacking. These languages 
>> provide an extremely dynamic programming model where almost everything is 
>> discovered at runtime.
>> 
>> Through the introduction of this proposal, and the related 
>> DynamicMemberLookupProtocol proposal, we seek to fix this problem. We 
>> believe we can make many common APIs feel very natural to use directly from 
>> Swift without all the complexity of implementing something like the Clang 
>> importer. For example, consider this Python code:
>> 
>> class Dog
>> :
>> 
>> def __init__(self, name
>> ):
>> 
>> self.name =
>> name
>> 
>> self.tricks = []# creates a new empty list for each dog
>> 
>> 
>> 
>> def add_trick(self, trick
>> ):
>> 
>> self.tricks.append(trick)
>> we would like to be able to use this from Swift like this (the comments show 
>> the corresponding syntax you would use in Python):
>> 
>>  // import DogModule
>>  // import DogModule.Dog as Dog// an alternate
>>  let Dog = Python.import(“DogModule.Dog")
>> 
>>  // dog = Dog("Brianna")
>>  let dog = Dog("Brianna")
>> 
>>  // dog.add_trick("Roll over")
>>  dog.add_trick("Roll over")
>> 
>>  // dog2 = Dog("Kaylee").add_trick("snore")

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Florent Vilmart via swift-evolution]

Object that are functions too, Amazing! I wanted that in Javascript for a while!

On Nov 10, 2017, 1:04 PM -0500, Joe Groff via swift-evolution 
, wrote:
> I don't like the idea of some calls having wildly different semantics from 
> others; it's difficult enough to tell what exactly a call might be doing 
> already. Since we also lack the more obvious static "Callable" protocol idea 
> to give even well-typed call syntax to user-defined types, this also seems 
> like it'd be easily abused for that purpose too.
>
> I think a much better general solution to the problem of "make dynamic 
> systems interact with type systems" is something like F#'s type providers 
> which lets you write your own importers that look at dynamic information from 
> a database, dynamic language VM, or some other system and generate type 
> information usable by the compiler. Integration at the importer level could 
> let you produce more well-typed Swift declarations by looking at the runtime 
> information you get by importing a Python module.
>
> -Joe
>
> > On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
> >  wrote:
> >
> > Hello all,
> >
> > I have a couple of proposals cooking in a quest to make Swift interoperate 
> > with dynamically typed languages like Python better. Instead of baking in 
> > hard coded support for one language or the other, I’m preferring to add a 
> > few small but general purpose capabilities to Swift. This is the first, 
> > which allows a Swift type to become “callable”.
> >
> > The proposal is here:
> > https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
> >
> > I’ve also attached a snapshot below, but it will drift out of date as the 
> > proposal is refined. Feedback and thoughts are appreciated, thanks!
> >
> > -Chris
> >
> >
> >
> >
> >
> > Introduce user-defined dynamically "callable" types
> >
> > • Proposal: SE-
> > • Author: Chris Lattner
> > • Review Manager: TBD
> > • Status: Awaiting implementation
> > Introduction
> >
> > This proposal introduces a new DynamicCallable protocol to the standard 
> > library. Types that conform to it are "callable" with the function call 
> > syntax. It is simple syntactic sugar which allows the user to write:
> >
> > a = someValue(keyword1: 42, "foo", keyword2: 19)
> > and have it be interpreted by the compiler as:
> >
> > a = someValue.dynamicCall(arguments
> > : [
> > (
> > "keyword1", 42), ("", "foo"), ("keyword2", 19
> > )
> > ])
> >
> > Other languages have analogous features (e.g. Python "callables"), but the 
> > primary motivation of this proposal is to allow elegant and natural 
> > interoperation with dynamic languages in Swift.
> >
> > Swift-evolution thread: Discussion thread topic for that proposal
> >
> > Motivation
> >
> > Swift is well known for being exceptional at interworking with existing C 
> > and Objective-C APIs, but its support for calling APIs written in scripting 
> > langauges like Python, Perl, and Ruby is quite lacking. These languages 
> > provide an extremely dynamic programming model where almost everything is 
> > discovered at runtime.
> >
> > Through the introduction of this proposal, and the related 
> > DynamicMemberLookupProtocol proposal, we seek to fix this problem. We 
> > believe we can make many common APIs feel very natural to use directly from 
> > Swift without all the complexity of implementing something like the Clang 
> > importer. For example, consider this Python code:
> >
> > class Dog
> > :
> >
> > def __init__(self, name
> > ):
> >
> > self.name =
> > name
> >
> > self.tricks = [] # creates a new empty list for each dog
> >
> >
> >
> > def add_trick(self, trick
> > ):
> >
> > self.tricks.append(trick)
> > we would like to be able to use this from Swift like this (the comments 
> > show the corresponding syntax you would use in Python):
> >
> > // import DogModule
> > // import DogModule.Dog as Dog // an alternate
> > let Dog = Python.import(“DogModule.Dog")
> >
> > // dog = Dog("Brianna")
> > let dog = Dog("Brianna")
> >
> > // dog.add_trick("Roll over")
> > dog.add_trick("Roll over")
> >
> > // dog2 = Dog("Kaylee").add_trick("snore")
> > let dog2 = Dog("Kaylee").add_trick("snore")
> > Of course, this would also apply to standard Python APIs as well. Here is 
> > an example working with the Python pickleAPI and the builtin Python 
> > function open:
> >
> > // import pickle
> > let pickle = Python.import("pickle"
> > )
> >
> >
> > // file = open(filename)
> > let file = Python.open
> > (filename)
> >
> >
> > // blob = file.read()
> > let blob = file.read
> > ()
> >
> >
> > // result = pickle.loads(blob)
> > let result = pickle.loads(blob)
> > This can all be expressed today as library functionality written in Swift, 
> > but without this proposal, the code required is unnecessarily verbose and 
> > gross. Without it (but with the related dynamic member lookup proposal) the 
> > code would have a method name (like 

Re: [swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Joe Groff via swift-evolution]

I don't like the idea of some calls having wildly different semantics from 
others; it's difficult enough to tell what exactly a call might be doing 
already. Since we also lack the more obvious static "Callable" protocol idea to 
give even well-typed call syntax to user-defined types, this also seems like 
it'd be easily abused for that purpose too.

I think a much better general solution to the problem of "make dynamic systems 
interact with type systems" is something like F#'s type providers which lets 
you write your own importers that look at dynamic information from a database, 
dynamic language VM, or some other system and generate type information usable 
by the compiler. Integration at the importer level could let you produce more 
well-typed Swift declarations by looking at the runtime information you get by 
importing a Python module.

-Joe

> On Nov 10, 2017, at 9:37 AM, Chris Lattner via swift-evolution 
>  wrote:
> 
> Hello all,
> 
> I have a couple of proposals cooking in a quest to make Swift interoperate 
> with dynamically typed languages like Python better.  Instead of baking in 
> hard coded support for one language or the other, I’m preferring to add a few 
> small but general purpose capabilities to Swift.  This is the first, which 
> allows a Swift type to become “callable”.
> 
> The proposal is here:
> https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d
> 
> I’ve also attached a snapshot below, but it will drift out of date as the 
> proposal is refined.  Feedback and thoughts are appreciated, thanks!
> 
> -Chris
> 
> 
> 
> 
> 
> Introduce user-defined dynamically "callable" types
> 
>   • Proposal: SE-
>   • Author: Chris Lattner
>   • Review Manager: TBD
>   • Status: Awaiting implementation
> Introduction
> 
> This proposal introduces a new DynamicCallable protocol to the standard 
> library. Types that conform to it are "callable" with the function call 
> syntax. It is simple syntactic sugar which allows the user to write:
> 
> a = someValue(keyword1: 42, "foo", keyword2: 19)
> and have it be interpreted by the compiler as:
> 
>   a = someValue.dynamicCall(arguments
> : [
> (
> "keyword1", 42), ("", "foo"), ("keyword2", 19
> )
>   ])
> 
> Other languages have analogous features (e.g. Python "callables"), but the 
> primary motivation of this proposal is to allow elegant and natural 
> interoperation with dynamic languages in Swift.
> 
> Swift-evolution thread: Discussion thread topic for that proposal
> 
> Motivation
> 
> Swift is well known for being exceptional at interworking with existing C and 
> Objective-C APIs, but its support for calling APIs written in scripting 
> langauges like Python, Perl, and Ruby is quite lacking. These languages 
> provide an extremely dynamic programming model where almost everything is 
> discovered at runtime.
> 
> Through the introduction of this proposal, and the related 
> DynamicMemberLookupProtocol proposal, we seek to fix this problem. We believe 
> we can make many common APIs feel very natural to use directly from Swift 
> without all the complexity of implementing something like the Clang importer. 
> For example, consider this Python code:
> 
> class Dog
> :
> 
> def __init__(self, name
> ):
> 
> self.name =
>  name
> 
> self.tricks = []# creates a new empty list for each dog
> 
> 
> 
> def add_trick(self, trick
> ):
> 
> self.tricks.append(trick)
> we would like to be able to use this from Swift like this (the comments show 
> the corresponding syntax you would use in Python):
> 
>   // import DogModule
>   // import DogModule.Dog as Dog// an alternate
>   let Dog = Python.import(“DogModule.Dog")
> 
>   // dog = Dog("Brianna")
>   let dog = Dog("Brianna")
> 
>   // dog.add_trick("Roll over")
>   dog.add_trick("Roll over")
> 
>   // dog2 = Dog("Kaylee").add_trick("snore")
>   let dog2 = Dog("Kaylee").add_trick("snore")
> Of course, this would also apply to standard Python APIs as well. Here is an 
> example working with the Python pickleAPI and the builtin Python function 
> open:
> 
>   // import pickle
>   let pickle = Python.import("pickle"
> )
> 
>   
> // file = open(filename)
>   let file = Python.open
> (filename)
> 
>   
> // blob = file.read()
>   let blob = file.read
> ()
> 
>   
> // result = pickle.loads(blob)
>   let result = pickle.loads(blob)
> This can all be expressed today as library functionality written in Swift, 
> but without this proposal, the code required is unnecessarily verbose and 
> gross. Without it (but with the related dynamic member lookup proposal) the 
> code would have a method name (like call) all over the code:
> 
>   // import pickle
>   let pickle = Python.import("pickle")  // normal method in Swift, no change.
> 
>   
> // file = open(filename)
>   let file = Python.open.call
> (filename)
> 
>   
> // blob = file.read()
>   let blob = file.read.call
> ()
> 
>   
> // result = 

[swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner via swift-evolution]

Hello all,

I have a couple of proposals cooking in a quest to make Swift interoperate with 
dynamically typed languages like Python better.  Instead of baking in hard 
coded support for one language or the other, I’m preferring to add a few small 
but general purpose capabilities to Swift.  This is the first, which allows a 
Swift type to become “callable”.

The proposal is here:
https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d 


I’ve also attached a snapshot below, but it will drift out of date as the 
proposal is refined.  Feedback and thoughts are appreciated, thanks!

-Chris





Introduce user-defined dynamically "callable" types

Proposal: SE- 
Author: Chris Lattner 
Review Manager: TBD
Status: Awaiting implementation
 
Introduction

This proposal introduces a new DynamicCallable protocol to the standard 
library. Types that conform to it are "callable" with the function call syntax. 
It is simple syntactic sugar which allows the user to write:

a = someValue(keyword1: 42, "foo", keyword2: 19)
and have it be interpreted by the compiler as:

  a = someValue.dynamicCall(arguments: [
("keyword1", 42), ("", "foo"), ("keyword2", 19)
  ])
Other languages have analogous features (e.g. Python "callables"), but the 
primary motivation of this proposal is to allow elegant and natural 
interoperation with dynamic languages in Swift.

Swift-evolution thread: Discussion thread topic for that proposal 

 
Motivation

Swift is well known for being exceptional at interworking with existing C and 
Objective-C APIs, but its support for calling APIs written in scripting 
langauges like Python, Perl, and Ruby is quite lacking. These languages provide 
an extremely dynamic programming model where almost everything is discovered at 
runtime.

Through the introduction of this proposal, and the related 
DynamicMemberLookupProtocol proposal, we seek to fix this problem. We believe 
we can make many common APIs feel very natural to use directly from Swift 
without all the complexity of implementing something like the Clang importer. 
For example, consider this Python code:

class Dog:
def __init__(self, name):
self.name = name
self.tricks = []# creates a new empty list for each dog

def add_trick(self, trick):
self.tricks.append(trick)
we would like to be able to use this from Swift like this (the comments show 
the corresponding syntax you would use in Python):

  // import DogModule
  // import DogModule.Dog as Dog// an alternate
  let Dog = Python.import(“DogModule.Dog")

  // dog = Dog("Brianna")
  let dog = Dog("Brianna")

  // dog.add_trick("Roll over")
  dog.add_trick("Roll over")

  // dog2 = Dog("Kaylee").add_trick("snore")
  let dog2 = Dog("Kaylee").add_trick("snore")
Of course, this would also apply to standard Python APIs as well. Here is an 
example working with the Python pickleAPI and the builtin Python function open:

  // import pickle
  let pickle = Python.import("pickle")

  // file = open(filename)
  let file = Python.open(filename)

  // blob = file.read()
  let blob = file.read()

  // result = pickle.loads(blob)
  let result = pickle.loads(blob)
This can all be expressed today as library functionality written in Swift, but 
without this proposal, the code required is unnecessarily verbose and gross. 
Without it (but with the related dynamic member lookup proposal) the code would 
have a method name (like call) all over the code:

  // import pickle
  let pickle = Python.import("pickle")  // normal method in Swift, no change.

  // file = open(filename)
  let file = Python.open.call(filename)

  // blob = file.read()
  let blob = file.read.call()

  // result = pickle.loads(blob)
  let result = pickle.loads.call(blob)

  // dog2 = Dog("Kaylee").add_trick("snore")
  let dog2 = Dog.call("Kaylee").add_trick.call("snore")
While this is a syntactic sugar proposal, we believe that this expands Swift to 
be usable in important new domains. This sort of capability is also highly 
precedented in other languages, and is a generally useful language feature that 
could be used for other purposes as well.

 
Proposed
 solution

We propose introducing this protocol to the standard library:

protocol DynamicCallable {
  associatedtype DynamicCallableArgument
  associatedtype DynamicCallableResult

  func dynamicCall(arguments: [(String, DynamicCallableArgument)]) throws -> 
DynamicCallableResult
}
It also extends the language such that function call syntax - when applied to a 
value of DynamicCallable type