Re: [julia-users] Re: Julia 0.5 Highlights

[Stefan Karpinski]

Yes, that's essentially it – except that since we haven't converged on a
particular design, it's hard to say exactly what interfaces are at this
point. But yes, it's something that provides a first class representation
of some protocol/interface.

On Mon, Oct 17, 2016 at 11:33 AM, Brian Rogoff  wrote:

> On Thursday, October 13, 2016 at 1:00:21 PM UTC-7, Stefan Karpinski wrote:
>>
>> No, Function doesn't have signatures, arity or return type as part of its
>> type. The signature of a function is the union of its method signatures,
>> which is potentially very complicated. Type parameters are not
>> contravariant, so they can't be described without massively complicated
>> Julia's (already complicated) type system. Worse still, adding any form of
>> contravariance would almost certainly make important predicates like
>> subtype and type intersection undecidable. There are still things that
>> could be done to get some of the features that you probably want from
>> function types, but dispatching on the return type is unlikely to ever be
>> allowed. Two things that may happen are:
>>
>
> Got it, thanks! I want method signatures for documentation, debugging, and
> as constraints or hints for the compiler, which are exactly what your two
> things provide.
>
> Are these are what you call 'interfaces' in your JuliaCon 2016 keynote,
> discussed here https://github.com/JuliaLang/julia/issues/6975?
>
>
>> 1. Constraining the type signature of a generic function, raising an
>> error if any method returns something that doesn't match:
>>
>> convert{T} :: (T, Any)-->T
>>
>>
>> or whatever syntax makes sense. This would implicitly mean that any call
>> to convert(T,x) would be translated to convert(T,x)::T so that we know
>> convert always returns the type one would expect for it. This is what I was
>> alluding to above.
>>
>> 2. Intersecting a function signature on an argument with a generic
>> function to extract a "sub-function" that will either behave the way we
>> expect it to or raise an error:
>>
>> function mysort!{T}(lt::(T,T)-->Bool, Vector{T})
>> ...
>>
>> end
>>
>>
>> This would mean that any use like lt(a, b) in the function body would
>> implicitly be wrapped as lt(a::T, b::T)::Bool or something like that. This
>> extra type information could potentially allow the compiler to reason
>> better about the function's behavior even in cases where it otherwise can't
>> figure out that much. Of course, in the case that's already fast, we don't
>> need that information since the type of function calls can already be
>> completely inferred.
>>
>> Note that neither of these allow you to dispatch on the type of lt.
>>
>

Re: [julia-users] Re: Julia 0.5 Highlights

[Brian Rogoff]

On Thursday, October 13, 2016 at 1:00:21 PM UTC-7, Stefan Karpinski wrote:
>
> No, Function doesn't have signatures, arity or return type as part of its 
> type. The signature of a function is the union of its method signatures, 
> which is potentially very complicated. Type parameters are not 
> contravariant, so they can't be described without massively complicated 
> Julia's (already complicated) type system. Worse still, adding any form of 
> contravariance would almost certainly make important predicates like 
> subtype and type intersection undecidable. There are still things that 
> could be done to get some of the features that you probably want from 
> function types, but dispatching on the return type is unlikely to ever be 
> allowed. Two things that may happen are:
>

Got it, thanks! I want method signatures for documentation, debugging, and 
as constraints or hints for the compiler, which are exactly what your two 
things provide.

Are these are what you call 'interfaces' in your JuliaCon 2016 keynote, 
discussed here https://github.com/JuliaLang/julia/issues/6975? 


> 1. Constraining the type signature of a generic function, raising an error 
> if any method returns something that doesn't match:
>
> convert{T} :: (T, Any)-->T
>
>
> or whatever syntax makes sense. This would implicitly mean that any call 
> to convert(T,x) would be translated to convert(T,x)::T so that we know 
> convert always returns the type one would expect for it. This is what I was 
> alluding to above.
>
> 2. Intersecting a function signature on an argument with a generic 
> function to extract a "sub-function" that will either behave the way we 
> expect it to or raise an error:
>
> function mysort!{T}(lt::(T,T)-->Bool, Vector{T})
> ...
>
> end
>
>
> This would mean that any use like lt(a, b) in the function body would 
> implicitly be wrapped as lt(a::T, b::T)::Bool or something like that. This 
> extra type information could potentially allow the compiler to reason 
> better about the function's behavior even in cases where it otherwise can't 
> figure out that much. Of course, in the case that's already fast, we don't 
> need that information since the type of function calls can already be 
> completely inferred.
>
> Note that neither of these allow you to dispatch on the type of lt.
>

Re: [julia-users] Re: Julia 0.5 Highlights

[Stefan Karpinski]

On Thu, Oct 13, 2016 at 12:26 PM, Brian Rogoff  wrote:

> Great summary, thanks so much!
>
> Being a fan of typeful functional programming, I really like the return
> type annotations and FP performance improvements. Is there a way to
> describe a precise return type for a higher order function? The examples of
> Function I've seen have neither the arguments type/arity or return types.
>

No, Function doesn't have signatures, arity or return type as part of its
type. The signature of a function is the union of its method signatures,
which is potentially very complicated. Type parameters are not
contravariant, so they can't be described without massively complicated
Julia's (already complicated) type system. Worse still, adding any form of
contravariance would almost certainly make important predicates like
subtype and type intersection undecidable. There are still things that
could be done to get some of the features that you probably want from
function types, but dispatching on the return type is unlikely to ever be
allowed. Two things that may happen are:

1. Constraining the type signature of a generic function, raising an error
if any method returns something that doesn't match:

convert{T} :: (T, Any)-->T


or whatever syntax makes sense. This would implicitly mean that any call to
convert(T,x) would be translated to convert(T,x)::T so that we know convert
always returns the type one would expect for it. This is what I was
alluding to above.

2. Intersecting a function signature on an argument with a generic function
to extract a "sub-function" that will either behave the way we expect it to
or raise an error:

function mysort!{T}(lt::(T,T)-->Bool, Vector{T})
...

end


This would mean that any use like lt(a, b) in the function body would
implicitly be wrapped as lt(a::T, b::T)::Bool or something like that. This
extra type information could potentially allow the compiler to reason
better about the function's behavior even in cases where it otherwise can't
figure out that much. Of course, in the case that's already fast, we don't
need that information since the type of function calls can already be
completely inferred.

Note that neither of these allow you to dispatch on the type of lt.

Re: [julia-users] Re: Julia 0.5 Highlights

[Steven G. Johnson]

On Wednesday, October 12, 2016 at 9:40:27 PM UTC-4, Steven G. Johnson wrote:
>
>
>
> On Wednesday, October 12, 2016 at 9:26:54 PM UTC-4, Stefan Karpinski wrote:
>>
>> That's a fair point. It seems like it could/should be handled by the same 
>> (not-yet-implemented) mechanism that ensures that `convert(T,x)::T` is 
>> true. Of course, we could choose to enforce this fact via lowering in this 
>> case, independent of enforcing it for convert.
>>
>
Update: this was a bug that occurred for small, inlined functions.   Now 
fixed, and will be fixed in the next 0.5.x release: 
https://github.com/JuliaLang/julia/pull/18899 

[julia-users] Re: Julia 0.5 Highlights

[Brian Rogoff]

Great summary, thanks so much!

Being a fan of typeful functional programming, I really like the return 
type annotations and FP performance improvements. Is there a way to 
describe a precise return type for a higher order function? The examples of 
Function I've seen have neither the arguments type/arity or return types.

Re: [julia-users] Re: Julia 0.5 Highlights

[Steven G. Johnson]

On Wednesday, October 12, 2016 at 9:26:54 PM UTC-4, Stefan Karpinski wrote:
>
> That's a fair point. It seems like it could/should be handled by the same 
> (not-yet-implemented) mechanism that ensures that `convert(T,x)::T` is 
> true. Of course, we could choose to enforce this fact via lowering in this 
> case, independent of enforcing it for convert.
>

I think we should add a typeassert in the lowering for this syntax.   I'm 
confused because Jeff's PR actually claimed it was using convert(T, val)::T --- 
see https://github.com/JuliaLang/julia/pull/16432

Re: [julia-users] Re: Julia 0.5 Highlights

[Stefan Karpinski]

That's a fair point. It seems like it could/should be handled by the same
(not-yet-implemented) mechanism that ensures that `convert(T,x)::T` is
true. Of course, we could choose to enforce this fact via lowering in this
case, independent of enforcing it for convert.

On Wed, Oct 12, 2016 at 7:40 PM, Cedric St-Jean 
wrote:

> Very nice summary!
>
> I assume that there's a mile-long issue discussing this somewhere, but why
> doesn't the return type also assert that convert returns a value of the
> correct type?
>
> type A end
> Base.convert(::Type{Int}, ::A) = "hey"
> foo()::Int = A()
> foo()  # returns "hey"
>
>
> On Wednesday, October 12, 2016 at 4:29:09 PM UTC-4, Jared Crean wrote:
>>
>>   Perfect, thanks.
>>
>>   Jared Crean
>>
>> On Wednesday, October 12, 2016 at 2:40:03 PM UTC-4, harven wrote:
>>>
>>>
>>>
>>> Le mercredi 12 octobre 2016 01:45:25 UTC+2, Jared Crean a écrit :

 Very nice summary, thanks for posting.  One question I had was what
 should the signature of a function be to receive a generator?  For example,
 if the only method of extrema is extrema(A::AbstractArray), is that too
 restrictive?

   Jared Crean


>>> Any functions working with iterables will work with generators.
>>>
>>> julia> methods(extrema)
>>> # 4 methods for generic function "extrema":
>>> extrema(r::Range) at reduce.jl:345
>>> extrema(x::Real) at reduce.jl:346
>>> extrema(A::AbstractArray, dims) at reduce.jl:388
>>> extrema(itr) at reduce.jl:362
>>>
>>>
>>> The last line tells you that extrema will work. An object is iterable if
>>> it implements the methods start, next and done. There are in fact a few
>>> other objects that also work on generators.
>>>
>>>  julia> methodswith(Base.Generator)
>>> 8-element Array{Method,1}:
>>>  collect(itr::Base.Generator) at array.jl:298
>>>  done(g::Base.Generator, s) at generator.jl:22
>>>  indices(g::Base.Generator) at generator.jl:91
>>>  length(g::Base.Generator) at generator.jl:89
>>>  ndims(g::Base.Generator) at generator.jl:92
>>>  next(g::Base.Generator, s) at generator.jl:24
>>>  size(g::Base.Generator) at generator.jl:90
>>>  start(g::Base.Generator) at generator.jl:21
>>>
>>> There are a few functions that work on arrays but not on iterables. You
>>> should not expect these to work on generators.
>>>
>>> julia> show(reverse([1:10;]))
>>> [10,9,8,7,6,5,4,3,2,1]
>>> julia> show(reverse(i for i = 1:10))
>>> ERROR: MethodError: no method matching reverse(::Base.Generator{UnitR
>>> ange{Int64},##9#10})
>>> Closest candidates are:
>>>   reverse(!Matched::String) at strings/string.jl:209
>>>   reverse(!Matched::BitArray{1}) at bitarray.jl:1416
>>>   reverse(!Matched::Tuple) at tuple.jl:199
>>>   ...
>>>
>>

[julia-users] Re: Julia 0.5 Highlights

[Cedric St-Jean]

Very nice summary!

I assume that there's a mile-long issue discussing this somewhere, but why 
doesn't the return type also assert that convert returns a value of the 
correct type?

type A end
Base.convert(::Type{Int}, ::A) = "hey"
foo()::Int = A()
foo()  # returns "hey"


On Wednesday, October 12, 2016 at 4:29:09 PM UTC-4, Jared Crean wrote:
>
>   Perfect, thanks.
>
>   Jared Crean
>
> On Wednesday, October 12, 2016 at 2:40:03 PM UTC-4, harven wrote:
>>
>>
>>
>> Le mercredi 12 octobre 2016 01:45:25 UTC+2, Jared Crean a écrit :
>>>
>>> Very nice summary, thanks for posting.  One question I had was what 
>>> should the signature of a function be to receive a generator?  For example, 
>>> if the only method of extrema is extrema(A::AbstractArray), is that too 
>>> restrictive?
>>>
>>>   Jared Crean
>>>
>>>
>> Any functions working with iterables will work with generators. 
>>
>> julia> methods(extrema)
>> # 4 methods for generic function "extrema":
>> extrema(r::Range) at reduce.jl:345
>> extrema(x::Real) at reduce.jl:346
>> extrema(A::AbstractArray, dims) at reduce.jl:388
>> extrema(itr) at reduce.jl:362
>>
>>
>> The last line tells you that extrema will work. An object is iterable if 
>> it implements the methods start, next and done. There are in fact a few 
>> other objects that also work on generators.
>>
>>  julia> methodswith(Base.Generator)
>> 8-element Array{Method,1}:
>>  collect(itr::Base.Generator) at array.jl:298 
>>  done(g::Base.Generator, s) at generator.jl:22
>>  indices(g::Base.Generator) at generator.jl:91
>>  length(g::Base.Generator) at generator.jl:89 
>>  ndims(g::Base.Generator) at generator.jl:92  
>>  next(g::Base.Generator, s) at generator.jl:24
>>  size(g::Base.Generator) at generator.jl:90   
>>  start(g::Base.Generator) at generator.jl:21  
>>
>> There are a few functions that work on arrays but not on iterables. You 
>> should not expect these to work on generators.
>>
>> julia> show(reverse([1:10;]))
>> [10,9,8,7,6,5,4,3,2,1]
>> julia> show(reverse(i for i = 1:10))
>> ERROR: MethodError: no method matching 
>> reverse(::Base.Generator{UnitRange{Int64},##9#10})
>> Closest candidates are:
>>   reverse(!Matched::String) at strings/string.jl:209
>>   reverse(!Matched::BitArray{1}) at bitarray.jl:1416
>>   reverse(!Matched::Tuple) at tuple.jl:199
>>   ...
>>
>

[julia-users] Re: Julia 0.5 Highlights

[Jared Crean]

  Perfect, thanks.

  Jared Crean

On Wednesday, October 12, 2016 at 2:40:03 PM UTC-4, harven wrote:
>
>
>
> Le mercredi 12 octobre 2016 01:45:25 UTC+2, Jared Crean a écrit :
>>
>> Very nice summary, thanks for posting.  One question I had was what 
>> should the signature of a function be to receive a generator?  For example, 
>> if the only method of extrema is extrema(A::AbstractArray), is that too 
>> restrictive?
>>
>>   Jared Crean
>>
>>
> Any functions working with iterables will work with generators. 
>
> julia> methods(extrema)
> # 4 methods for generic function "extrema":
> extrema(r::Range) at reduce.jl:345
> extrema(x::Real) at reduce.jl:346
> extrema(A::AbstractArray, dims) at reduce.jl:388
> extrema(itr) at reduce.jl:362
>
>
> The last line tells you that extrema will work. An object is iterable if 
> it implements the methods start, next and done. There are in fact a few 
> other objects that also work on generators.
>
>  julia> methodswith(Base.Generator)
> 8-element Array{Method,1}:
>  collect(itr::Base.Generator) at array.jl:298 
>  done(g::Base.Generator, s) at generator.jl:22
>  indices(g::Base.Generator) at generator.jl:91
>  length(g::Base.Generator) at generator.jl:89 
>  ndims(g::Base.Generator) at generator.jl:92  
>  next(g::Base.Generator, s) at generator.jl:24
>  size(g::Base.Generator) at generator.jl:90   
>  start(g::Base.Generator) at generator.jl:21  
>
> There are a few functions that work on arrays but not on iterables. You 
> should not expect these to work on generators.
>
> julia> show(reverse([1:10;]))
> [10,9,8,7,6,5,4,3,2,1]
> julia> show(reverse(i for i = 1:10))
> ERROR: MethodError: no method matching 
> reverse(::Base.Generator{UnitRange{Int64},##9#10})
> Closest candidates are:
>   reverse(!Matched::String) at strings/string.jl:209
>   reverse(!Matched::BitArray{1}) at bitarray.jl:1416
>   reverse(!Matched::Tuple) at tuple.jl:199
>   ...
>

[julia-users] Re: Julia 0.5 Highlights

[harven]

Le mercredi 12 octobre 2016 01:45:25 UTC+2, Jared Crean a écrit :
>
> Very nice summary, thanks for posting.  One question I had was what should 
> the signature of a function be to receive a generator?  For example, if the 
> only method of extrema is extrema(A::AbstractArray), is that too 
> restrictive?
>
>   Jared Crean
>
>
Any functions working with iterables will work with generators. 

julia> methods(extrema)
# 4 methods for generic function "extrema":
extrema(r::Range) at reduce.jl:345
extrema(x::Real) at reduce.jl:346
extrema(A::AbstractArray, dims) at reduce.jl:388
extrema(itr) at reduce.jl:362


The last line tells you that extrema will work. An object is iterable if it 
implements the methods start, next and done. There are in fact a few other 
objects that also work on generators.

 julia> methodswith(Base.Generator)
8-element Array{Method,1}:
 collect(itr::Base.Generator) at array.jl:298 
 done(g::Base.Generator, s) at generator.jl:22
 indices(g::Base.Generator) at generator.jl:91
 length(g::Base.Generator) at generator.jl:89 
 ndims(g::Base.Generator) at generator.jl:92  
 next(g::Base.Generator, s) at generator.jl:24
 size(g::Base.Generator) at generator.jl:90   
 start(g::Base.Generator) at generator.jl:21  

There are a few functions that work on arrays but not on iterables. You 
should not expect these to work on generators.

julia> show(reverse([1:10;]))
[10,9,8,7,6,5,4,3,2,1]
julia> show(reverse(i for i = 1:10))
ERROR: MethodError: no method matching 
reverse(::Base.Generator{UnitRange{Int64},##9#10})
Closest candidates are:
  reverse(!Matched::String) at strings/string.jl:209
  reverse(!Matched::BitArray{1}) at bitarray.jl:1416
  reverse(!Matched::Tuple) at tuple.jl:199
  ...

[julia-users] Re: Julia 0.5 Highlights

[Evan Fields]

Thanks for writing this up; it's helpful to see certain things highlighted 
and explained in more detail than news.md gives!

[julia-users] Re: Julia 0.5 Highlights

[Christoph Ortner]

However, 
g(n) = sum( i^2 for i = 1:n )
julia> g(0)
ERROR: MethodError: no method matching zero(::Type{Any})
Closest candidates are:
  zero(::Type{Base.LibGit2.Oid}) at libgit2/oid.jl:88
  zero(::Type{Base.Pkg.Resolve.VersionWeights.VWPreBuildItem}) at 
pkg/resolve/versionweight.jl:80
  zero(::Type{Base.Pkg.Resolve.VersionWeights.VWPreBuild}) at 
pkg/resolve/versionweight.jl:120
  ...
 in mr_empty(::Base.#identity, ::Base.#+, ::Type{T}) at ./reduce.jl:130
 in mr_empty(::Base.#identity, ::Base.#+, ::Type{T}) at 
/Users/ortner/gits/julia/usr/lib/julia/sys.dylib:?
 in mapfoldl(::Base.#identity, ::Function, 
::Base.Generator{UnitRange{Int64},##3#4}) at ./reduce.jl:60
 in g(::Int64) at ./REPL[17]:1

though this seems to have been fixed with JuliaLang/julia#18873    (I 
haven't tested it yet)

[julia-users] Re: Julia 0.5 Highlights

[Christoph Ortner]

f(n) = [ i^2 for i = 1:n ]
julia> f(0)
0-element Array{Int64,1}


On Wednesday, 12 October 2016 07:10:37 UTC+1, Jussi Piitulainen wrote:
>
> Does that mean that an empty array comprehension is always Array{Any}?
>
> that array comprehensions are now type-inference-independent. That means 
>> that the type of the resulting array only depends on the actual types of 
>> values produced, not what the compiler can prove about the expression in 
>> advance.
>>
>  
>

[julia-users] Re: Julia 0.5 Highlights

[Jussi Piitulainen]

Does that mean that an empty array comprehension is always Array{Any}?

that array comprehensions are now type-inference-independent. That means 
> that the type of the resulting array only depends on the actual types of 
> values produced, not what the compiler can prove about the expression in 
> advance.
>
 

[julia-users] Re: Julia 0.5 Highlights

[Jared Crean]

Very nice summary, thanks for posting.  One question I had was what should 
the signature of a function be to receive a generator?  For example, if the 
only method of extrema is extrema(A::AbstractArray), is that too 
restrictive?

  Jared Crean

On Tuesday, October 11, 2016 at 1:05:03 PM UTC-4, Stefan Karpinski wrote:
>
> Since the 0.5 release affects everyone here, I wrote a longish blog post 
> about what the major changes are: 
> http://julialang.org/blog/2016/10/julia-0.5-highlights.
>
> One other change that I left out of the post because it was getting pretty 
> long and it seems a bit esoteric is that array comprehensions are now 
> type-inference-independent. That means that the type of the resulting array 
> only depends on the actual types of values produced, not what the compiler 
> can prove about the expression in advance. In particular, this means that 
> comprehensions behave the same way in global scope as in local scope now, 
> which is a fairly major relief to anyone who's struggled with that.
>