[julia-users] Re: Warning since 0.5

[Ralph Smith]

This is indeed fairly confusing, partly because the terminological 
conventions of functional programming are somewhat arbitrary. Some of my 
discussion upthread
uses these conventions, but some (e.g. my use of "attach" and 
"associate/dissociate") may not be conventional.

In Julia, and as Steven J & I used it above, "name" is a field of a 
function object. Thus
f(x)=x^2
sets the name and other fields all at once, so it's never anonymous. In 
particular, it attaches a method for argument type "Any" (since none was 
specified).

Methods are effectively rules for applying the function to tuples of 
argument types.
Methods in turn have "specializations" which seem to be (essentially) the 
compiled code for the specific leaf types of arguments, so
a = f(2.0)
gets Julia to compile a specialization of the (f,Any) method for 
(f,Float64), and save it for later use by stashing it in a subfield of the 
function object.

f(x::Int)=x^2+1
would add another method to the function named "f", just to be used with 
integer arguments.  But
f(x)=x^3
replaces the first method (because it has the same nominal argument type, 
i.e. Any) That is, it replaces it within the object we set up with the 
original definition.
Operations like map, broadcast, and closure construction use and store 
references to parts of the function object (in particular, methods), and 
associate the name "f"
with those references. In Julia v0.5, these references are not updated 
correctly on redefinition. That's what I meant when I said the 
disassociation is incomplete.


On Monday, October 10, 2016 at 3:09:24 PM UTC-4, digxx wrote:
>
> The function object then points into
>> method tables. You can't assign the name "f" to a different function 
>> object, just attach different methods to it.  The troubles seem to arise 
>> from cached references to
>> orphaned method table entries, which are not completely dissociated from 
>> the object named "f". 
>> f(x)=x^2
>>
>
>  so this definition associates the anonymous function with a name (and 
> then it is not anoymous anymore?)
> However how do these method tables look like? In particular the name f is 
> then associated to these methods and for some reason one does not want to 
> be able to change f to refer to different methods?
> How would different mathod attaching look like?
> when I write
>
> f(x)=x^2
> and then
> f(x)=x^3
>
> the old method (is the method here the operation x^2 or x^3?) is still 
> there and x^3 is just attached to it?!?
> is it possible to completely dissociate the old method (in this case as I 
> understand it correctly it is x^2?) with the name f?
>

[julia-users] Re: Warning since 0.5

[digxx]

>
> The function object then points into
> method tables. You can't assign the name "f" to a different function 
> object, just attach different methods to it.  The troubles seem to arise 
> from cached references to
> orphaned method table entries, which are not completely dissociated from 
> the object named "f". 
> f(x)=x^2
>

 so this definition associates the anonymous function with a name (and then 
it is not anoymous anymore?)
However how do these method tables look like? In particular the name f is 
then associated to these methods and for some reason one does not want to 
be able to change f to refer to different methods?
How would different mathod attaching look like?
when I write

f(x)=x^2
and then
f(x)=x^3

the old method (is the method here the operation x^2 or x^3?) is still 
there and x^3 is just attached to it?!?
is it possible to completely dissociate the old method (in this case as I 
understand it correctly it is x^2?) with the name f?

[julia-users] Re: Warning since 0.5

[digxx]

>
> No, it has a name "f".  An anonymous function is an expression like "x -> 
> x^2" that creates a function object without binding it to a constant name. 
>

Ok, but from my feeling also

f=x->x^2
binds this function object  to the name f since thats how it is called? 
Right?

[julia-users] Re: Warning since 0.5

[Steven G. Johnson]

On Saturday, October 8, 2016 at 5:09:19 PM UTC-4, digxx wrote:
>
> is f(x)=x^2 not an anonymous function?!?!
>

No, it has a name "f".  An anonymous function is an expression like "x -> 
x^2" that creates a function object without binding it to a constant name. 

[julia-users] Re: Warning since 0.5

[Ralph Smith]

By "module file" I just meant a source code file where all definitions are 
enclosed in modules, so if you "include" it, it replaces the whole module. 
Thus

module M
function f(x)
   x^2
end
end


then references to M.f are more likely to be consistent than a bare 
function f defined at the top level (the Main module).

As you surmised, 

g = x -> x^2


binds the mutable variable with symbol :g  to an anonymous function (i.e. 
one with a hidden name and type).  If you bind :g to a different function, 
references to :g should
always use the new association.  Defining a regular function (like M.f 
above) embeds "f" as a name in the function object itself. The function 
object then points into
method tables. You can't assign the name "f" to a different function 
object, just attach different methods to it.  The troubles seem to arise 
from cached references to
orphaned method table entries, which are not completely dissociated from 
the object named "f". 

On Saturday, October 8, 2016 at 5:09:19 PM UTC-4, digxx wrote:
 | what do u mean by "module files"?

 | So what is the difference between
 | f(x)=x^2
 | and 
 | f=x->x^2

is f(x)=x^2 not an anonymous function?!?!
>

[julia-users] Re: Warning since 0.5

[digxx]

is f(x)=x^2 not an anonymous function?!?!

[julia-users] Re: Warning since 0.5

[digxx]

Maybe one sidenote:
So what is the difference between
f(x)=x^2
and 
f=x->x^2
since the last one does not return that warning?

[julia-users] Re: Warning since 0.5

[digxx]

Thanks for ur example
I'm not that familiar with it but what do u mean by "module files"?
Do you have an example and why are they less dangerous?

[julia-users] Re: Warning since 0.5

[Ralph Smith]

The problem is that some Julia processing stores references to definitions 
in hidden locations which are not updated consistently, so you get 
inconsistency like this:

julia> f(x)=x^2
f (generic function with 1 method)


julia> map(f,[1,2,3])
3-element Array{Int64,1}:
 1
 4
 9


julia> f(x)=x^2+1
WARNING: Method definition f(Any) in module Main at REPL[7]:1 overwritten 
at REPL[9]:1.
f (generic function with 1 method)


julia> [f(x) for x in [1,2,3]]
3-element Array{Int64,1}:
 1
 4
 9


julia> for x in [1,2,3]; println(f(x)); end
2
5
10


julia> map(f,[1,2,3])
3-element Array{Int64,1}:
 1
 4
 9


(Thanks to fcard for pointing this out.  See this issue 
 for more discussion.) 
 Fixing this and related problems is hard, but there is hope for v0.6.

Anyway, my point was that "function files" are more dangerous than "module 
files", and that's why they generate more warnings.

In the REPL one can also drop *all* old definitions & bindings by using 
workspace(), but I find that more painful than just using modules and 
restarting Julia when
necessary.

If you're convinced that all this doesn't apply to your case, you can 
suppress messages: 
https://github.com/cstjean/ClobberingReload.jl#silencing-warnings 

On Saturday, October 8, 2016 at 9:35:10 AM UTC-4, digxx wrote:
>
> Hey,
> Thx for ur answer. So The first time I call my program which includes a 
> file with function definitions there is no problem. I do this because with 
> 0.4 parallel loops didnt work with functions which are defined in the same 
> file even though an @everywhere is prefixed.
> I still dont understand why this should happen at all. Isnt it totally 
> natural that a function file where things like
> g(x)=x.^2
> are defined and not much more complex it might happen that I want to 
> change it to 
> g(x)=x.^2+1
> without restarting Julia in order to not get the error.
> This also happens in the REPL when I overwrite a simple function. So it is 
> probably not correlated to the @everywhere.
> How can I avoid the warning or is it possible to "free" all allocated 
> function definitions before I redefine them?
>

[julia-users] Re: Warning since 0.5

[digxx]

Hey,
Thx for ur answer. So The first time I call my program which includes a 
file with function definitions there is no problem. I do this because with 
0.4 parallel loops didnt work with functions which are defined in the same 
file even though an @everywhere is prefixed.
I still dont understand why this should happen at all. Isnt it totally 
natural that a function file where things like
g(x)=x.^2
are defined and not much more complex it might happen that I want to change 
it to 
g(x)=x.^2+1
without restarting Julia in order to not get the error.
This also happens in the REPL when I overwrite a simple function. So it is 
probably not correlated to the @everywhere.
How can I avoid the warning or is it possible to "free" all allocated 
function definitions before I redefine them?

[julia-users] Re: Warning since 0.5

[Ralph Smith]

TL;DR: put everything in modules.

"WARNING: Method definition ... overwritten..."

This warning was extended to more cases in v0.5.  It has caused some 
confusion, and some available explanations are themselves confusing.  Here 
is another try.

Overwriting a method (or type) has always been a formally undefined action 
- that is, there is no guarantee of consistency when you refer to the 
method after
overwriting it (  issue 265 
).  In v0.5 there were 
changes to the implementation of functions which make it more likely that 
this will cause conflicts and problems, 
so it is good that the warning is more prevalent.

If you put your method definitions in modules, and reload the module files, 
you just get warnings for the module redefinitions. You would then also 
have to reload
any modules which use the redefined ones, and so on. It is possible to 
cause conflicts with consistently redefined modules, but it takes some 
effort - unless you
are foolish enough to type
using MyVolatileModule
in interactive sessions; that is asking for trouble.  If even your global 
variables are in modules, consistent reloading will redefine them in terms 
of
updated used modules. For production work (i.e. if you care about valid 
results) you should not overwrite definitions.  If you use packages which 
themselves have
overwrites, please file an issue.

In your case, if the `@everywhere` clause includes definitions which were 
really not changed (not just the file text, but also any global context it 
refers to), you might not
encounter conflicts. It is nevertheless a deprecated usage.  It would seem 
to be either pointless or dangerous.

Caveat: I am not a Julia developer, just a user with a tendency to stumble 
into edge cases.  I happily defer to corrections from the experts, but 
regret that they didn't
explain this issue in the regular documentation.

On Thursday, October 6, 2016 at 7:42:36 PM UTC-4, digxx wrote:
>
> Something has changed since the code which ran with 0.4 does return a 
> warning now:
>
> include("C:\\Users\\Diger\\Documents\\Julia\\Diffusion\\bessel0_expansion_nlsolve.jl")
> WARNING: Method definition WARNING: Method definition WARNING: Method 
> definition WARNING: Method definition WARNING: Method definition g_r(Any) 
> in module Main at 
> C:\Users\Diger\Documents\Julia\Rohrstroemung_function.jl:11 overwritten at 
> C:\Users\Diger\Documents\Julia\Rohrstroemung_function.jl:11.
> WARNING: Method definition g_i(Any, Any) in module Main at 
> C:\Users\Diger\Documents\Julia\Rohrstroemung_function.jl:36 overwritten at 
> C:\Users\Diger\Documents\Julia\Rohrstroemung_function.jl:36.
>
>
> I had a function file which i included with @everywhere
> Now when I read it after the first time it seems to be that he doesnt like 
> to "overwrite" it again?
> Why?
>