Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Cool, workspace() seems very helpful. Yeah, I don't really understand 
exactly what that code would look like either, because I'm not the most 
familiar with time-series statistics. My point was that to work easily with 
flexible models that mix environmental variables with dataframe variables, 
it is convenient (though again not necessary) to have environment access.

On Saturday, July 25, 2015 at 7:34:44 AM UTC-4, Milan Bouchet-Valat wrote:
>
> Le samedi 25 juillet 2015 à 01:58 -0700, Brandon Taylor a écrit : 
> > Well a somewhat unique feature or R is automatic argument recycling, 
> > so that when doing operations on dataframes, it's a lot easier to 
> > incorporate data from outside of the dataframe into evaluation. Let's 
> > say you are doing a stepwise logisitic growth model, like dP/dt = 
> > r*P*(1 - P/K). Let's say that r a known single value in the global 
> > environment, K is going be predicted from some environmental 
> > variables within your dataframe, and P is going to be given as an 
> > initial condition and its evolution modeled. The R way of looking at 
> > this would be to set up an environment where the dataframe 
> > environment inherits from the global environment, and to evaluate 
> > within this environment regardless of calling environment. That means 
> > that within a sub sub function, you can use r to represent, maybe, a 
> > row number, and K, to represent, maybe, a column number, and that you 
> > can still evaluate an expression like r*P*(1 - P/K) with no problem. 
> > This is most useful when writing your own functions. For example, you 
> > could build some sort of time-series predictive package that could 
> > handle not only logisitic models, but other models, like the Lotka 
> > -Volterra models. Now, I'm not saying this is impossible to do in 
> > Julia: in fact, I'm quite sure it is. It's just less convenient. 
> Thanks for the example. I don't think I would use an environment for 
> that. Why not have the function take a data frame with the variables, a 
> nd constants as an argument? As I understand it, the only interest of 
> environments is that you don't have to make a difference between 
> variables in the data frame and variables in the global scope. But I 
> also find it confusing that you don't know where the data comes from, 
> and I've seen students make mistakes because of that. 
>
> That said, I don't understand exactly what the code you describe would 
> look like, so I might be missing something. If you can provide a few 
> lines to illustrate a toy example, it could help. 
>
> > The other nice things about environments are more trivial. For 
> > example, you might be doing a homework assignment with several 
> > problems. At the start of each problem, you could define a new 
> > environment. At the end of the problem, you could display the entire 
> > content of the problem's environment as a way to track down errors. 
> > When looking at an answer sheet, you could compare intermediate 
> > results and see where you went wrong. Again, not impossible to do 
> > with Julia; you would probably want to set up a dict for each 
> > problem, and do all operations within that dict. Which, I guess, is 
> > basically what is happening under the hood in a language with 
> > environments. 
> You should be able to do this by working inside modules in Julia. In 
> particular, workspace() looks exactly like what you're describing: once 
> you're done with one problem, call it to get a clean environment, 
> storing globals in the LastMain module. 
>
>
> Regards 
>
> > I'll just nod in agreement with the fexpr stuff and pretend I 
> > understand. 
> > 
> > On Friday, July 24, 2015 at 4:11:37 PM UTC-4, Scott Jones wrote: 
> > > I should give you a hard time about arguing from authority ;-), but 
> > > I was there at the same time and knew KMP, so I wouldn't want to 
> > > argue too much with him either, and I am in agreement with you 
> > > about fexprs vs. macros, and the equivalent issues in Julia. 
> > > 
> > > On Friday, July 24, 2015 at 2:31:54 PM UTC-4, Stefan Karpinski 
> > > wrote: 
> > > > Lisp has already been down this path – fexprs were special forms 
> > > > with behavior similar to R's functions, and they were deprecated 
> > > > in the 1980s in favor of macros, https://en.wikipedia.org/wiki/Fe 
> > > > xpr#Mainstream_use_and_deprecation: 
> > > > 
> > > > > His central objection was that, in a Lisp dialect that allows 
> > > > > fexprs, static analysis cannot determine generally whether an 
> > > > > operator represents an ordinary function or a fexpr — 
> > > > > therefore, static analysis cannot determine whether or not the 
> > > > > operands will be evaluated. In particular, the compiler cannot 
> > > > > tell whether a subexpression can be safely optimized, since the 
> > > > > subexpression might be treated as unevaluated data at run-time. 
> > > > If Lispers think a feature is too dynamic and hard to understand, 
> > > > that's a danger sign. In

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le samedi 25 juillet 2015 à 01:58 -0700, Brandon Taylor a écrit :
> Well a somewhat unique feature or R is automatic argument recycling, 
> so that when doing operations on dataframes, it's a lot easier to 
> incorporate data from outside of the dataframe into evaluation. Let's 
> say you are doing a stepwise logisitic growth model, like dP/dt = 
> r*P*(1 - P/K). Let's say that r a known single value in the global 
> environment, K is going be predicted from some environmental 
> variables within your dataframe, and P is going to be given as an 
> initial condition and its evolution modeled. The R way of looking at 
> this would be to set up an environment where the dataframe 
> environment inherits from the global environment, and to evaluate 
> within this environment regardless of calling environment. That means 
> that within a sub sub function, you can use r to represent, maybe, a 
> row number, and K, to represent, maybe, a column number, and that you 
> can still evaluate an expression like r*P*(1 - P/K) with no problem. 
> This is most useful when writing your own functions. For example, you 
> could build some sort of time-series predictive package that could 
> handle not only logisitic models, but other models, like the Lotka
> -Volterra models. Now, I'm not saying this is impossible to do in 
> Julia: in fact, I'm quite sure it is. It's just less convenient.
Thanks for the example. I don't think I would use an environment for
that. Why not have the function take a data frame with the variables, a
nd constants as an argument? As I understand it, the only interest of
environments is that you don't have to make a difference between
variables in the data frame and variables in the global scope. But I
also find it confusing that you don't know where the data comes from,
and I've seen students make mistakes because of that.

That said, I don't understand exactly what the code you describe would
look like, so I might be missing something. If you can provide a few
lines to illustrate a toy example, it could help.

> The other nice things about environments are more trivial. For 
> example, you might be doing a homework assignment with several 
> problems. At the start of each problem, you could define a new 
> environment. At the end of the problem, you could display the entire 
> content of the problem's environment as a way to track down errors. 
> When looking at an answer sheet, you could compare intermediate 
> results and see where you went wrong. Again, not impossible to do 
> with Julia; you would probably want to set up a dict for each 
> problem, and do all operations within that dict. Which, I guess, is 
> basically what is happening under the hood in a language with 
> environments.
You should be able to do this by working inside modules in Julia. In
particular, workspace() looks exactly like what you're describing: once
you're done with one problem, call it to get a clean environment,
storing globals in the LastMain module.


Regards

> I'll just nod in agreement with the fexpr stuff and pretend I 
> understand.
> 
> On Friday, July 24, 2015 at 4:11:37 PM UTC-4, Scott Jones wrote:
> > I should give you a hard time about arguing from authority ;-), but 
> > I was there at the same time and knew KMP, so I wouldn't want to 
> > argue too much with him either, and I am in agreement with you 
> > about fexprs vs. macros, and the equivalent issues in Julia.
> > 
> > On Friday, July 24, 2015 at 2:31:54 PM UTC-4, Stefan Karpinski 
> > wrote:
> > > Lisp has already been down this path – fexprs were special forms 
> > > with behavior similar to R's functions, and they were deprecated 
> > > in the 1980s in favor of macros, https://en.wikipedia.org/wiki/Fe
> > > xpr#Mainstream_use_and_deprecation:
> > > 
> > > > His central objection was that, in a Lisp dialect that allows 
> > > > fexprs, static analysis cannot determine generally whether an 
> > > > operator represents an ordinary function or a fexpr — 
> > > > therefore, static analysis cannot determine whether or not the 
> > > > operands will be evaluated. In particular, the compiler cannot 
> > > > tell whether a subexpression can be safely optimized, since the 
> > > > subexpression might be treated as unevaluated data at run-time.
> > > If Lispers think a feature is too dynamic and hard to understand, 
> > > that's a danger sign. In general, I wouldn't want to argue too 
> > > much with Kent Pitman :-)
> > > 
> > > On the subject of Kent Pitman, here's a random bit of Lisp wisdom 
> > > from this interview with him:
> > > 
> > > > I like Lisp's willingness to represent itself. People often 
> > > > explain this as its ability to represent itself, but I think 
> > > > that's wrong. Most languages are capable of representing 
> > > > themselves, but they simply don't have the will to. Lisp 
> > > > programs are represented by lists and programmers are aware of 
> > > > that. It wouldn't matter if it had been arrays. It does matter 
> > > > that

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Well a somewhat unique feature or R is automatic argument recycling, so 
that when doing operations on dataframes, it's a lot easier to incorporate 
data from outside of the dataframe into evaluation. Let's say you are doing 
a stepwise logisitic growth model, like dP/dt = r*P*(1 - P/K). Let's say 
that r a known single value in the global environment, K is going be 
predicted from some environmental variables within your dataframe, and P is 
going to be given as an initial condition and its evolution modeled. The R 
way of looking at this would be to set up an environment where the 
dataframe environment inherits from the global environment, and to evaluate 
within this environment regardless of calling environment. That means that 
within a sub sub function, you can use r to represent, maybe, a row number, 
and K, to represent, maybe, a column number, and that you can still 
evaluate an expression like r*P*(1 - P/K) with no problem. This is most 
useful when writing your own functions. For example, you could build some 
sort of time-series predictive package that could handle not only logisitic 
models, but other models, like the Lotka-Volterra models. Now, I'm not 
saying this is impossible to do in Julia: in fact, I'm quite sure it is. 
It's just less convenient.

The other nice things about environments are more trivial. For example, you 
might be doing a homework assignment with several problems. At the start of 
each problem, you could define a new environment. At the end of the 
problem, you could display the entire content of the problem's environment 
as a way to track down errors. When looking at an answer sheet, you could 
compare intermediate results and see where you went wrong. Again, not 
impossible to do with Julia; you would probably want to set up a dict for 
each problem, and do all operations within that dict. Which, I guess, is 
basically what is happening under the hood in a language with environments.

I'll just nod in agreement with the fexpr stuff and pretend I understand.

On Friday, July 24, 2015 at 4:11:37 PM UTC-4, Scott Jones wrote:
>
> I should give you a hard time about arguing from authority ;-), but I was 
> there at the same time and knew KMP, so I wouldn't want to argue too much 
> with him either, and I am in agreement with you about fexprs vs. macros, 
> and the equivalent issues in Julia.
>
> On Friday, July 24, 2015 at 2:31:54 PM UTC-4, Stefan Karpinski wrote:
>>
>> Lisp has already been down this path – fexprs 
>>  were special forms with behavior 
>> similar to R's functions, and they were deprecated in the 1980s in favor of 
>> macros, 
>> https://en.wikipedia.org/wiki/Fexpr#Mainstream_use_and_deprecation:
>>
>> His central objection was that, in a Lisp dialect that allows fexprs, 
>>> static analysis cannot determine generally whether an operator represents 
>>> an ordinary function or a fexpr — therefore, static analysis cannot 
>>> determine whether or not the operands will be evaluated. In particular, the 
>>> compiler cannot tell whether a subexpression can be safely optimized, since 
>>> the subexpression might be treated as unevaluated data at run-time.
>>
>>
>> If Lispers think a feature is too dynamic and hard to understand, that's 
>> a danger sign. In general, I wouldn't want to argue too much with Kent 
>> Pitman :-)
>>
>> On the subject of Kent Pitman, here's a random bit of Lisp wisdom from this 
>> interview 
>> 
>>  
>> with him:
>>
>> I like Lisp's willingness to represent itself. People often explain this 
>>> as its ability to represent itself, but I think that's wrong. Most 
>>> languages are capable of representing themselves, but they simply don't 
>>> have the will to. Lisp programs are represented by lists and programmers 
>>> are aware of that. It wouldn't matter if it had been arrays. It does matter 
>>> that it's program structure that is represented, and not character syntax, 
>>> but beyond that the choice is pretty arbitrary. It's not important that the 
>>> representation be the Right® choice. It's just important that it be a 
>>> common, agreed-upon choice so that there can be a rich community of 
>>> program-manipulating programs that "do trade" in this common representation.
>>
>>
>> This is probably the best quote about homoiconicity I've read anywhere 
>> and explains why the term is tricky to pin down.
>>
>> On Fri, Jul 24, 2015 at 2:21 PM, Brandon Taylor  
>> wrote:
>>
>>> Macros seem to have a bunch of limitations. They can't be overloaded to 
>>> take advantage of Julia's typing system. Their functionality can somewhat 
>>> mimic evaluation within a function in its direct parent environment, but 
>>> what if you want to write functions within your macro, or macros inside 
>>> your macro, etc? Probably still possible to do many things, but it would 
>>> invovle passing expressions back in fo

Re: [julia-users] Re: Environment reification and lazy evaluation

[Scott Jones]

I should give you a hard time about arguing from authority ;-), but I was 
there at the same time and knew KMP, so I wouldn't want to argue too much 
with him either, and I am in agreement with you about fexprs vs. macros, 
and the equivalent issues in Julia.

On Friday, July 24, 2015 at 2:31:54 PM UTC-4, Stefan Karpinski wrote:
>
> Lisp has already been down this path – fexprs 
>  were special forms with behavior 
> similar to R's functions, and they were deprecated in the 1980s in favor of 
> macros, https://en.wikipedia.org/wiki/Fexpr#Mainstream_use_and_deprecation
> :
>
> His central objection was that, in a Lisp dialect that allows fexprs, 
>> static analysis cannot determine generally whether an operator represents 
>> an ordinary function or a fexpr — therefore, static analysis cannot 
>> determine whether or not the operands will be evaluated. In particular, the 
>> compiler cannot tell whether a subexpression can be safely optimized, since 
>> the subexpression might be treated as unevaluated data at run-time.
>
>
> If Lispers think a feature is too dynamic and hard to understand, that's a 
> danger sign. In general, I wouldn't want to argue too much with Kent Pitman 
> :-)
>
> On the subject of Kent Pitman, here's a random bit of Lisp wisdom from this 
> interview 
> 
>  
> with him:
>
> I like Lisp's willingness to represent itself. People often explain this 
>> as its ability to represent itself, but I think that's wrong. Most 
>> languages are capable of representing themselves, but they simply don't 
>> have the will to. Lisp programs are represented by lists and programmers 
>> are aware of that. It wouldn't matter if it had been arrays. It does matter 
>> that it's program structure that is represented, and not character syntax, 
>> but beyond that the choice is pretty arbitrary. It's not important that the 
>> representation be the Right® choice. It's just important that it be a 
>> common, agreed-upon choice so that there can be a rich community of 
>> program-manipulating programs that "do trade" in this common representation.
>
>
> This is probably the best quote about homoiconicity I've read anywhere and 
> explains why the term is tricky to pin down.
>
> On Fri, Jul 24, 2015 at 2:21 PM, Brandon Taylor  > wrote:
>
>> Macros seem to have a bunch of limitations. They can't be overloaded to 
>> take advantage of Julia's typing system. Their functionality can somewhat 
>> mimic evaluation within a function in its direct parent environment, but 
>> what if you want to write functions within your macro, or macros inside 
>> your macro, etc? Probably still possible to do many things, but it would 
>> invovle passing expressions back in forth in confusing ways, where as with 
>> environments, as long as an environment is kept attached to every 
>> expression, they can be immediately evaluated in any sub or subsub etc. 
>> environment.
>>
>
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Stefan Karpinski]

I agree, it would be very helpful to the discussion to have some examples
of the problems Brandon is looking to solve.

On Fri, Jul 24, 2015 at 3:09 PM, Milan Bouchet-Valat 
wrote:

> Le vendredi 24 juillet 2015 à 11:21 -0700, Brandon Taylor a écrit :
> > Macros seem to have a bunch of limitations. They can't be overloaded
> > to take advantage of Julia's typing system. Their functionality can
> > somewhat mimic evaluation within a function in its direct parent
> > environment, but what if you want to write functions within your
> > macro, or macros inside your macro, etc? Probably still possible to
> > do many things, but it would invovle passing expressions back in
> > forth in confusing ways, where as with environments, as long as an
> > environment is kept attached to every expression, they can be
> > immediately evaluated in any sub or subsub etc. environment.
> I'm still looking for examples of situations where these limitations
> are indeed a problem... Discussing APIs in full generality, without
> concrete applications, is very hard.
>
>
> Regards
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le vendredi 24 juillet 2015 à 11:21 -0700, Brandon Taylor a écrit :
> Macros seem to have a bunch of limitations. They can't be overloaded 
> to take advantage of Julia's typing system. Their functionality can 
> somewhat mimic evaluation within a function in its direct parent 
> environment, but what if you want to write functions within your 
> macro, or macros inside your macro, etc? Probably still possible to 
> do many things, but it would invovle passing expressions back in 
> forth in confusing ways, where as with environments, as long as an 
> environment is kept attached to every expression, they can be 
> immediately evaluated in any sub or subsub etc. environment.
I'm still looking for examples of situations where these limitations
are indeed a problem... Discussing APIs in full generality, without
concrete applications, is very hard.


Regards

Re: [julia-users] Re: Environment reification and lazy evaluation

[Stefan Karpinski]

Lisp has already been down this path – fexprs
 were special forms with behavior
similar to R's functions, and they were deprecated in the 1980s in favor of
macros, https://en.wikipedia.org/wiki/Fexpr#Mainstream_use_and_deprecation:

His central objection was that, in a Lisp dialect that allows fexprs,
> static analysis cannot determine generally whether an operator represents
> an ordinary function or a fexpr — therefore, static analysis cannot
> determine whether or not the operands will be evaluated. In particular, the
> compiler cannot tell whether a subexpression can be safely optimized, since
> the subexpression might be treated as unevaluated data at run-time.


If Lispers think a feature is too dynamic and hard to understand, that's a
danger sign. In general, I wouldn't want to argue too much with Kent Pitman
:-)

On the subject of Kent Pitman, here's a random bit of Lisp wisdom from this
interview

with him:

I like Lisp's willingness to represent itself. People often explain this as
> its ability to represent itself, but I think that's wrong. Most languages
> are capable of representing themselves, but they simply don't have the will
> to. Lisp programs are represented by lists and programmers are aware of
> that. It wouldn't matter if it had been arrays. It does matter that it's
> program structure that is represented, and not character syntax, but beyond
> that the choice is pretty arbitrary. It's not important that the
> representation be the Right® choice. It's just important that it be a
> common, agreed-upon choice so that there can be a rich community of
> program-manipulating programs that "do trade" in this common representation.


This is probably the best quote about homoiconicity I've read anywhere and
explains why the term is tricky to pin down.

On Fri, Jul 24, 2015 at 2:21 PM, Brandon Taylor  wrote:

> Macros seem to have a bunch of limitations. They can't be overloaded to
> take advantage of Julia's typing system. Their functionality can somewhat
> mimic evaluation within a function in its direct parent environment, but
> what if you want to write functions within your macro, or macros inside
> your macro, etc? Probably still possible to do many things, but it would
> invovle passing expressions back in forth in confusing ways, where as with
> environments, as long as an environment is kept attached to every
> expression, they can be immediately evaluated in any sub or subsub etc.
> environment.
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Macros seem to have a bunch of limitations. They can't be overloaded to take 
advantage of Julia's typing system. Their functionality can somewhat mimic 
evaluation within a function in its direct parent environment, but what if you 
want to write functions within your macro, or macros inside your macro, etc? 
Probably still possible to do many things, but it would invovle passing 
expressions back in forth in confusing ways, where as with environments, as 
long as an environment is kept attached to every expression, they can be 
immediately evaluated in any sub or subsub etc. environment.

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Macros seem to have a bunch of limitations. They can't be overloaded to take 
advantage of Julia's typing system. Their functionality can somewhat mimic 
evaluation within a function in its direct parent environment, but what if you 
want to write functions within your macro, or macros inside your macro, etc? 
Probably still possible to do many things, but it would invovle passing 
expressions back in forth in confusing ways, where as with environments, as 
long as an environment is kept attached to every expression, they can be 
immediately evaluated in any sub or subsub etc. environment.

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le vendredi 24 juillet 2015 à 08:48 -0700, Brandon Taylor a écrit :
> Ok, maybe I was being harsh when I used the word mis-marketing. The 
> post here: http://julialang.org/blog/2012/02/why-we-created-julia/ sp
> ecifically mentions that one of Julia's goals is to be as easy for 
> statistics as R. Building a domain specific language is more 
> difficult (but clearly not impossible) without environment access. 
> Again, the examples I'm most familiar with are in the Hadleyverse 
> (see previous posts in this thread for links to vignettes). Also, 
> this is probably the fifth time that me being stubborn seems to have 
> caused tensions on threads, and I'm sorry for that. Learning a bit 
> about C compilers convinced me that I was completely wrong about 
> environments needing to be implemented for scoping to work.
No worries, but that still doesn't give me a very concrete example of
what you're trying to do. :-)

I'm asking because if you only want to do things like dplyr offers,
you'll likely work only on data frames or similar structures. Then I
don't think you really need reified environments. DataFramesMeta.jl
already shows that macros can be quite powerful to reproduce dplyr-like
features the Julian way. If you show us a use case where this strategy
wouldn't work, maybe we can make more helpful comments.


Regards

> On Friday, July 24, 2015 at 3:46:28 AM UTC-4, Milan Bouchet-Valat 
> wrote:
> > Le jeudi 23 juillet 2015 à 15:31 -0700, Brandon Taylor a écrit : 
> > > I'm not saying inherent slowness will be particularly useful. I'm 
> > > saying environment access will be particularly useful. The point 
> > of 
> > > Julia is no compromise between speed and code-ability. If it is 
> > not 
> > > possible to integrate environment access and speed, then I think 
> > that 
> > > Julia is mis-marketing itself, at least as a viable R 
> > alternative. 
> > Could you give us a concrete example of what you would like to do 
> > with 
> > environments? I haven't encountered cases where I would personally 
> > miss 
> > this R feature in Julia, so I don't understand what you're trying 
> > to 
> > achieve. 
> > 
> > Also, note that when we say that a solution will be slow in Julia, 
> > it 
> > doesn't mean that it would be slower than in R. It's just that to 
> > write 
> > code approximately equivalent to C, you cannot use the same 
> > solutions 
> > as R. 
> > 
> > 
> > Regards 
> > 
> > 
> > > On Friday, July 24, 2015 at 12:35:13 AM UTC+4, Yichao Yu wrote: 
> > > > On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor 
> > > >  wrote: 
> > > > > Will it cause a slowdown because there is a better way to do 
> > it 
> > > > or will it 
> > > > > cause a slowdown because something inherent about environment 
> > > > access? 
> > > > > Because if the second is the case, then it would be worth it, 
> > at 
> > > > least to 
> > > > > me. 
> > > > 
> > > > AFAICT, environment access is basically global variable and it 
> > will 
> > > > trash any optimization julia can do so in some sense the slow 
> > down 
> > > > is 
> > > > inherent about environment access. 
> > > > 
> > > > I don't see why the inherent slowness would be particularly 
> > useful 
> > > > though. 
> > > > 
> > > > > 
> > > > > On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu 
> > wrote: 
> > > > >> 
> > > > >> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor 
> > > > >>  wrote: 
> > > > >> > Ok, made some progress. Still having trouble with a _ENV_ 
> > not 
> > > > defined 
> > > > >> > error: 
> > > > >> > how could it not be defined if it's a global variable??? 
> > > > Figuring out 
> > > > >> > argument passing is going to be tricky. Is this kind of 
> > system 
> > > > feasible 
> > > > >> > or 
> > > > >> > is it going to cause huge slowdowns? 
> > > > >> 
> > > > >> It'll cause a huge slowdown. 
> > > > >> 
> > > > >> > 
> > > > >> > using DataFrames 
> > > > >> > 
> > > > >> > import Base.convert 
> > > > >> > 
> > > > >> > _TYPES_ = Symbol[] 
> > > > >> > 
> > > > >> > type _LAZY_ 
> > > > >> >   _E_::Expr 
> > > > >> >   _ENV_::Symbol 
> > > > >> > end 
> > > > >> > 
> > > > >> > # modify dicts such that if a key is not found, search the 
> > > > parent 
> > > > >> > function Base.getindex{K,V}(h::Dict{K,V}, key) 
> > > > >> >   index = Base.ht_keyindex(h, key) 
> > > > >> >   if index < 0 
> > > > >> > if :_parent in keys(h) 
> > > > >> >   Base.getindex(eval(h[:_parent]), key) 
> > > > >> > else 
> > > > >> >   throw(KeyError(key)) 
> > > > >> > end 
> > > > >> >   else 
> > > > >> > h.vals[index]::V 
> > > > >> >   end 
> > > > >> > end 
> > > > >> > 
> > > > >> > # allow inheritance from modules 
> > > > >> > function convert(::Type{Dict}, m::Module) 
> > > > >> >   dict = Dict() 
> > > > >> >   for name in names(m) 
> > > > >> > dict[name] = eval( :(Base.$name) ) 
> > > > >> > # add types to type list 
> > > > >> > if typeof(dict[name]) <: DataType 
> > > > >> >   push!(_TYPES_, name

Re: [julia-users] Re: Environment reification and lazy evaluation

[Stefan Karpinski]

I can understand that if you're used to building interfaces that use R's
non-standard evaluation and reified environments heavily, it can be
frustrating to not have those features. But I don't think that means that
you cannot build convenient and safe interfaces for data analysis.

On Fri, Jul 24, 2015 at 11:48 AM, Brandon Taylor <
brandon.taylor...@gmail.com> wrote:

> Ok, maybe I was being harsh when I used the word mis-marketing. The post
> here: http://julialang.org/blog/2012/02/why-we-created-julia/
> specifically mentions that one of Julia's goals is to be as easy for
> statistics as R. Building a domain specific language is more difficult (but
> clearly not impossible) without environment access. Again, the examples I'm
> most familiar with are in the Hadleyverse (see previous posts in this
> thread for links to vignettes). Also, this is probably the fifth time that
> me being stubborn seems to have caused tensions on threads, and I'm sorry
> for that. Learning a bit about C compilers convinced me that I was
> completely wrong about environments needing to be implemented for scoping
> to work.
>
>
> On Friday, July 24, 2015 at 3:46:28 AM UTC-4, Milan Bouchet-Valat wrote:
>
>> Le jeudi 23 juillet 2015 à 15:31 -0700, Brandon Taylor a écrit :
>> > I'm not saying inherent slowness will be particularly useful. I'm
>> > saying environment access will be particularly useful. The point of
>> > Julia is no compromise between speed and code-ability. If it is not
>> > possible to integrate environment access and speed, then I think that
>> > Julia is mis-marketing itself, at least as a viable R alternative.
>> Could you give us a concrete example of what you would like to do with
>> environments? I haven't encountered cases where I would personally miss
>> this R feature in Julia, so I don't understand what you're trying to
>> achieve.
>>
>> Also, note that when we say that a solution will be slow in Julia, it
>> doesn't mean that it would be slower than in R. It's just that to write
>> code approximately equivalent to C, you cannot use the same solutions
>> as R.
>>
>>
>> Regards
>>
>>
>> > On Friday, July 24, 2015 at 12:35:13 AM UTC+4, Yichao Yu wrote:
>> > > On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor
>> > >  wrote:
>> > > > Will it cause a slowdown because there is a better way to do it
>> > > or will it
>> > > > cause a slowdown because something inherent about environment
>> > > access?
>> > > > Because if the second is the case, then it would be worth it, at
>> > > least to
>> > > > me.
>> > >
>> > > AFAICT, environment access is basically global variable and it will
>> > > trash any optimization julia can do so in some sense the slow down
>> > > is
>> > > inherent about environment access.
>> > >
>> > > I don't see why the inherent slowness would be particularly useful
>> > > though.
>> > >
>> > > >
>> > > > On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote:
>> > > >>
>> > > >> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor
>> > > >>  wrote:
>> > > >> > Ok, made some progress. Still having trouble with a _ENV_ not
>> > > defined
>> > > >> > error:
>> > > >> > how could it not be defined if it's a global variable???
>> > > Figuring out
>> > > >> > argument passing is going to be tricky. Is this kind of system
>> > > feasible
>> > > >> > or
>> > > >> > is it going to cause huge slowdowns?
>> > > >>
>> > > >> It'll cause a huge slowdown.
>> > > >>
>> > > >> >
>> > > >> > using DataFrames
>> > > >> >
>> > > >> > import Base.convert
>> > > >> >
>> > > >> > _TYPES_ = Symbol[]
>> > > >> >
>> > > >> > type _LAZY_
>> > > >> >   _E_::Expr
>> > > >> >   _ENV_::Symbol
>> > > >> > end
>> > > >> >
>> > > >> > # modify dicts such that if a key is not found, search the
>> > > parent
>> > > >> > function Base.getindex{K,V}(h::Dict{K,V}, key)
>> > > >> >   index = Base.ht_keyindex(h, key)
>> > > >> >   if index < 0
>> > > >> > if :_parent in keys(h)
>> > > >> >   Base.getindex(eval(h[:_parent]), key)
>> > > >> > else
>> > > >> >   throw(KeyError(key))
>> > > >> > end
>> > > >> >   else
>> > > >> > h.vals[index]::V
>> > > >> >   end
>> > > >> > end
>> > > >> >
>> > > >> > # allow inheritance from modules
>> > > >> > function convert(::Type{Dict}, m::Module)
>> > > >> >   dict = Dict()
>> > > >> >   for name in names(m)
>> > > >> > dict[name] = eval( :(Base.$name) )
>> > > >> > # add types to type list
>> > > >> > if typeof(dict[name]) <: DataType
>> > > >> >   push!(_TYPES_, name)
>> > > >> > end
>> > > >> >   end
>> > > >> >   dict
>> > > >> > end
>> > > >> >
>> > > >> > # allow inheritance from DataFrames
>> > > >> > function convert(::Type{Dict}, d::DataFrame)
>> > > >> > dict = Dict()
>> > > >> >   for name in names(d)
>> > > >> > dict[name] = d[name]
>> > > >> > # add types to type list
>> > > >> > if typeof(dict[name]) <: DataType
>> > > >> >   push!(_TYPES_, name)
>> > > >> > end
>> > > >> >   end
>> > > >> >   dict
>> > > >> > en

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Ok, maybe I was being harsh when I used the word mis-marketing. The post 
here: http://julialang.org/blog/2012/02/why-we-created-julia/ specifically 
mentions that one of Julia's goals is to be as easy for statistics as 
R. Building a domain specific language is more difficult (but clearly not 
impossible) without environment access. Again, the examples I'm most 
familiar with are in the Hadleyverse (see previous posts in this thread for 
links to vignettes). Also, this is probably the fifth time that me being 
stubborn seems to have caused tensions on threads, and I'm sorry for that. 
Learning a bit about C compilers convinced me that I was completely wrong 
about environments needing to be implemented for scoping to work.

On Friday, July 24, 2015 at 3:46:28 AM UTC-4, Milan Bouchet-Valat wrote:
>
> Le jeudi 23 juillet 2015 à 15:31 -0700, Brandon Taylor a écrit : 
> > I'm not saying inherent slowness will be particularly useful. I'm 
> > saying environment access will be particularly useful. The point of 
> > Julia is no compromise between speed and code-ability. If it is not 
> > possible to integrate environment access and speed, then I think that 
> > Julia is mis-marketing itself, at least as a viable R alternative. 
> Could you give us a concrete example of what you would like to do with 
> environments? I haven't encountered cases where I would personally miss 
> this R feature in Julia, so I don't understand what you're trying to 
> achieve. 
>
> Also, note that when we say that a solution will be slow in Julia, it 
> doesn't mean that it would be slower than in R. It's just that to write 
> code approximately equivalent to C, you cannot use the same solutions 
> as R. 
>
>
> Regards 
>
>
> > On Friday, July 24, 2015 at 12:35:13 AM UTC+4, Yichao Yu wrote: 
> > > On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor 
> > >  wrote: 
> > > > Will it cause a slowdown because there is a better way to do it 
> > > or will it 
> > > > cause a slowdown because something inherent about environment 
> > > access? 
> > > > Because if the second is the case, then it would be worth it, at 
> > > least to 
> > > > me. 
> > > 
> > > AFAICT, environment access is basically global variable and it will 
> > > trash any optimization julia can do so in some sense the slow down 
> > > is 
> > > inherent about environment access. 
> > > 
> > > I don't see why the inherent slowness would be particularly useful 
> > > though. 
> > > 
> > > > 
> > > > On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote: 
> > > >> 
> > > >> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor 
> > > >>  wrote: 
> > > >> > Ok, made some progress. Still having trouble with a _ENV_ not 
> > > defined 
> > > >> > error: 
> > > >> > how could it not be defined if it's a global variable??? 
> > > Figuring out 
> > > >> > argument passing is going to be tricky. Is this kind of system 
> > > feasible 
> > > >> > or 
> > > >> > is it going to cause huge slowdowns? 
> > > >> 
> > > >> It'll cause a huge slowdown. 
> > > >> 
> > > >> > 
> > > >> > using DataFrames 
> > > >> > 
> > > >> > import Base.convert 
> > > >> > 
> > > >> > _TYPES_ = Symbol[] 
> > > >> > 
> > > >> > type _LAZY_ 
> > > >> >   _E_::Expr 
> > > >> >   _ENV_::Symbol 
> > > >> > end 
> > > >> > 
> > > >> > # modify dicts such that if a key is not found, search the 
> > > parent 
> > > >> > function Base.getindex{K,V}(h::Dict{K,V}, key) 
> > > >> >   index = Base.ht_keyindex(h, key) 
> > > >> >   if index < 0 
> > > >> > if :_parent in keys(h) 
> > > >> >   Base.getindex(eval(h[:_parent]), key) 
> > > >> > else 
> > > >> >   throw(KeyError(key)) 
> > > >> > end 
> > > >> >   else 
> > > >> > h.vals[index]::V 
> > > >> >   end 
> > > >> > end 
> > > >> > 
> > > >> > # allow inheritance from modules 
> > > >> > function convert(::Type{Dict}, m::Module) 
> > > >> >   dict = Dict() 
> > > >> >   for name in names(m) 
> > > >> > dict[name] = eval( :(Base.$name) ) 
> > > >> > # add types to type list 
> > > >> > if typeof(dict[name]) <: DataType 
> > > >> >   push!(_TYPES_, name) 
> > > >> > end 
> > > >> >   end 
> > > >> >   dict 
> > > >> > end 
> > > >> > 
> > > >> > # allow inheritance from DataFrames 
> > > >> > function convert(::Type{Dict}, d::DataFrame) 
> > > >> > dict = Dict() 
> > > >> >   for name in names(d) 
> > > >> > dict[name] = d[name] 
> > > >> > # add types to type list 
> > > >> > if typeof(dict[name]) <: DataType 
> > > >> >   push!(_TYPES_, name) 
> > > >> > end 
> > > >> >   end 
> > > >> >   dict 
> > > >> > end 
> > > >> > 
> > > >> > # establish the base environment, save it as global, and point 
> > > it to an 
> > > >> > empty dictionary 
> > > >> > macro _ENV_MACRO_() 
> > > >> >   esc(quote 
> > > >> > _ENV_ = gensym() 
> > > >> > _GLOBAL_ = _ENV_ 
> > > >> > eval(quote 
> > > >> >$_ENV_ = Dict() 
> > > >> >  end) 
> > > >> >   end) 
> > > >> > end 
> > > >> > 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le jeudi 23 juillet 2015 à 15:31 -0700, Brandon Taylor a écrit :
> I'm not saying inherent slowness will be particularly useful. I'm 
> saying environment access will be particularly useful. The point of 
> Julia is no compromise between speed and code-ability. If it is not 
> possible to integrate environment access and speed, then I think that 
> Julia is mis-marketing itself, at least as a viable R alternative.
Could you give us a concrete example of what you would like to do with
environments? I haven't encountered cases where I would personally miss
this R feature in Julia, so I don't understand what you're trying to
achieve.

Also, note that when we say that a solution will be slow in Julia, it
doesn't mean that it would be slower than in R. It's just that to write
code approximately equivalent to C, you cannot use the same solutions
as R.


Regards


> On Friday, July 24, 2015 at 12:35:13 AM UTC+4, Yichao Yu wrote:
> > On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor 
> >  wrote: 
> > > Will it cause a slowdown because there is a better way to do it 
> > or will it 
> > > cause a slowdown because something inherent about environment 
> > access? 
> > > Because if the second is the case, then it would be worth it, at 
> > least to 
> > > me. 
> > 
> > AFAICT, environment access is basically global variable and it will 
> > trash any optimization julia can do so in some sense the slow down 
> > is 
> > inherent about environment access. 
> > 
> > I don't see why the inherent slowness would be particularly useful 
> > though. 
> > 
> > > 
> > > On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote: 
> > >> 
> > >> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor 
> > >>  wrote: 
> > >> > Ok, made some progress. Still having trouble with a _ENV_ not 
> > defined 
> > >> > error: 
> > >> > how could it not be defined if it's a global variable??? 
> > Figuring out 
> > >> > argument passing is going to be tricky. Is this kind of system 
> > feasible 
> > >> > or 
> > >> > is it going to cause huge slowdowns? 
> > >> 
> > >> It'll cause a huge slowdown. 
> > >> 
> > >> > 
> > >> > using DataFrames 
> > >> > 
> > >> > import Base.convert 
> > >> > 
> > >> > _TYPES_ = Symbol[] 
> > >> > 
> > >> > type _LAZY_ 
> > >> >   _E_::Expr 
> > >> >   _ENV_::Symbol 
> > >> > end 
> > >> > 
> > >> > # modify dicts such that if a key is not found, search the 
> > parent 
> > >> > function Base.getindex{K,V}(h::Dict{K,V}, key) 
> > >> >   index = Base.ht_keyindex(h, key) 
> > >> >   if index < 0 
> > >> > if :_parent in keys(h) 
> > >> >   Base.getindex(eval(h[:_parent]), key) 
> > >> > else 
> > >> >   throw(KeyError(key)) 
> > >> > end 
> > >> >   else 
> > >> > h.vals[index]::V 
> > >> >   end 
> > >> > end 
> > >> > 
> > >> > # allow inheritance from modules 
> > >> > function convert(::Type{Dict}, m::Module) 
> > >> >   dict = Dict() 
> > >> >   for name in names(m) 
> > >> > dict[name] = eval( :(Base.$name) ) 
> > >> > # add types to type list 
> > >> > if typeof(dict[name]) <: DataType 
> > >> >   push!(_TYPES_, name) 
> > >> > end 
> > >> >   end 
> > >> >   dict 
> > >> > end 
> > >> > 
> > >> > # allow inheritance from DataFrames 
> > >> > function convert(::Type{Dict}, d::DataFrame) 
> > >> > dict = Dict() 
> > >> >   for name in names(d) 
> > >> > dict[name] = d[name] 
> > >> > # add types to type list 
> > >> > if typeof(dict[name]) <: DataType 
> > >> >   push!(_TYPES_, name) 
> > >> > end 
> > >> >   end 
> > >> >   dict 
> > >> > end 
> > >> > 
> > >> > # establish the base environment, save it as global, and point 
> > it to an 
> > >> > empty dictionary 
> > >> > macro _ENV_MACRO_() 
> > >> >   esc(quote 
> > >> > _ENV_ = gensym() 
> > >> > _GLOBAL_ = _ENV_ 
> > >> > eval(quote 
> > >> >$_ENV_ = Dict() 
> > >> >  end) 
> > >> >   end) 
> > >> > end 
> > >> > 
> > >> > # establish a new environment and point it towards a dict 
> > daughter of 
> > >> > the 
> > >> > old environment 
> > >> > macro _NEW_ENV_MACRO_() 
> > >> >   esc(quote 
> > >> > _NEW_ENV_ = gensym() 
> > >> > eval(quote 
> > >> >$_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) } 
> > >> >  end) 
> > >> > _ENV_ = _NEW_ENV_ 
> > >> >   end) 
> > >> > end 
> > >> > 
> > >> > # establish a new environment and point it towards a dict 
> > daughter of 
> > >> > the 
> > >> > old environment with dict contents 
> > >> > macro _ADD_ENV_MACRO_(dict) 
> > >> >   esc(quote 
> > >> > _DICT_ = $dict 
> > >> > _ADD_ENV_ = gensym() 
> > >> > eval(quote 
> > >> >$_ADD_ENV_ = convert(Dict, $_DICT_) 
> > >> >$_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_)) 
> > >> >  end) 
> > >> > _ENV_ = _ADD_ENV_ 
> > >> >   end) 
> > >> > end 
> > >> > 
> > >> > # jump back in time to the previous generation 
> > >> > macro _REMOVE_ENV_MACRO_() 
> > >> >   esc(quote 
> > >> > _EN

Re: [julia-users] Re: Environment reification and lazy evaluation

[Stefan Karpinski]

What kind of mismarketing do you think is happening? I've been pretty clear on 
my position (which seems to be representative of others here): environment 
reification is a bad idea. It's basically just a tempting hack based on the 
fact that slow language implementations often use dictionaries to implement 
environments – if the environment is just a dictionary, why not give people 
access to that dictionary? But that's not how environments are represented in 
fast language implementations. Once you've taken the bait, though, you're stuck 
with the slow implementation, or trying to emulate it somehow. Rather than 
willfully putting ourselves in the slow lane, we're opting not to take the 
performance-poisoning bait in the first place.

> On Jul 23, 2015, at 6:31 PM, Brandon Taylor  
> wrote:
> 
> I'm not saying inherent slowness will be particularly useful. I'm saying 
> environment access will be particularly useful. The point of Julia is no 
> compromise between speed and code-ability. If it is not possible to integrate 
> environment access and speed, then I think that Julia is mis-marketing 
> itself, at least as a viable R alternative.
> 
>> On Friday, July 24, 2015 at 12:35:13 AM UTC+4, Yichao Yu wrote:
>> On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor 
>>  wrote: 
>> > Will it cause a slowdown because there is a better way to do it or will it 
>> > cause a slowdown because something inherent about environment access? 
>> > Because if the second is the case, then it would be worth it, at least to 
>> > me. 
>> 
>> AFAICT, environment access is basically global variable and it will 
>> trash any optimization julia can do so in some sense the slow down is 
>> inherent about environment access. 
>> 
>> I don't see why the inherent slowness would be particularly useful though. 
>> 
>> > 
>> > On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote: 
>> >> 
>> >> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor 
>> >>  wrote: 
>> >> > Ok, made some progress. Still having trouble with a _ENV_ not defined 
>> >> > error: 
>> >> > how could it not be defined if it's a global variable??? Figuring out 
>> >> > argument passing is going to be tricky. Is this kind of system feasible 
>> >> > or 
>> >> > is it going to cause huge slowdowns? 
>> >> 
>> >> It'll cause a huge slowdown. 
>> >> 
>> >> > 
>> >> > using DataFrames 
>> >> > 
>> >> > import Base.convert 
>> >> > 
>> >> > _TYPES_ = Symbol[] 
>> >> > 
>> >> > type _LAZY_ 
>> >> >   _E_::Expr 
>> >> >   _ENV_::Symbol 
>> >> > end 
>> >> > 
>> >> > # modify dicts such that if a key is not found, search the parent 
>> >> > function Base.getindex{K,V}(h::Dict{K,V}, key) 
>> >> >   index = Base.ht_keyindex(h, key) 
>> >> >   if index < 0 
>> >> > if :_parent in keys(h) 
>> >> >   Base.getindex(eval(h[:_parent]), key) 
>> >> > else 
>> >> >   throw(KeyError(key)) 
>> >> > end 
>> >> >   else 
>> >> > h.vals[index]::V 
>> >> >   end 
>> >> > end 
>> >> > 
>> >> > # allow inheritance from modules 
>> >> > function convert(::Type{Dict}, m::Module) 
>> >> >   dict = Dict() 
>> >> >   for name in names(m) 
>> >> > dict[name] = eval( :(Base.$name) ) 
>> >> > # add types to type list 
>> >> > if typeof(dict[name]) <: DataType 
>> >> >   push!(_TYPES_, name) 
>> >> > end 
>> >> >   end 
>> >> >   dict 
>> >> > end 
>> >> > 
>> >> > # allow inheritance from DataFrames 
>> >> > function convert(::Type{Dict}, d::DataFrame) 
>> >> > dict = Dict() 
>> >> >   for name in names(d) 
>> >> > dict[name] = d[name] 
>> >> > # add types to type list 
>> >> > if typeof(dict[name]) <: DataType 
>> >> >   push!(_TYPES_, name) 
>> >> > end 
>> >> >   end 
>> >> >   dict 
>> >> > end 
>> >> > 
>> >> > # establish the base environment, save it as global, and point it to an 
>> >> > empty dictionary 
>> >> > macro _ENV_MACRO_() 
>> >> >   esc(quote 
>> >> > _ENV_ = gensym() 
>> >> > _GLOBAL_ = _ENV_ 
>> >> > eval(quote 
>> >> >$_ENV_ = Dict() 
>> >> >  end) 
>> >> >   end) 
>> >> > end 
>> >> > 
>> >> > # establish a new environment and point it towards a dict daughter of 
>> >> > the 
>> >> > old environment 
>> >> > macro _NEW_ENV_MACRO_() 
>> >> >   esc(quote 
>> >> > _NEW_ENV_ = gensym() 
>> >> > eval(quote 
>> >> >$_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) } 
>> >> >  end) 
>> >> > _ENV_ = _NEW_ENV_ 
>> >> >   end) 
>> >> > end 
>> >> > 
>> >> > # establish a new environment and point it towards a dict daughter of 
>> >> > the 
>> >> > old environment with dict contents 
>> >> > macro _ADD_ENV_MACRO_(dict) 
>> >> >   esc(quote 
>> >> > _DICT_ = $dict 
>> >> > _ADD_ENV_ = gensym() 
>> >> > eval(quote 
>> >> >$_ADD_ENV_ = convert(Dict, $_DICT_) 
>> >> >$_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_)) 
>> >> >  end) 
>> >> > _ENV_ = _ADD_ENV_ 
>> >> >   end) 
>> >> > end 
>> >> > 
>> >> > # jump ba

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

I'm not saying inherent slowness will be particularly useful. I'm saying 
environment access will be particularly useful. The point of Julia is no 
compromise between speed and code-ability. If it is not possible to 
integrate environment access and speed, then I think that Julia is 
mis-marketing itself, at least as a viable R alternative.

On Friday, July 24, 2015 at 12:35:13 AM UTC+4, Yichao Yu wrote:
>
> On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor 
> > wrote: 
> > Will it cause a slowdown because there is a better way to do it or will 
> it 
> > cause a slowdown because something inherent about environment access? 
> > Because if the second is the case, then it would be worth it, at least 
> to 
> > me. 
>
> AFAICT, environment access is basically global variable and it will 
> trash any optimization julia can do so in some sense the slow down is 
> inherent about environment access. 
>
> I don't see why the inherent slowness would be particularly useful though. 
>
> > 
> > On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote: 
> >> 
> >> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor 
> >>  wrote: 
> >> > Ok, made some progress. Still having trouble with a _ENV_ not defined 
> >> > error: 
> >> > how could it not be defined if it's a global variable??? Figuring out 
> >> > argument passing is going to be tricky. Is this kind of system 
> feasible 
> >> > or 
> >> > is it going to cause huge slowdowns? 
> >> 
> >> It'll cause a huge slowdown. 
> >> 
> >> > 
> >> > using DataFrames 
> >> > 
> >> > import Base.convert 
> >> > 
> >> > _TYPES_ = Symbol[] 
> >> > 
> >> > type _LAZY_ 
> >> >   _E_::Expr 
> >> >   _ENV_::Symbol 
> >> > end 
> >> > 
> >> > # modify dicts such that if a key is not found, search the parent 
> >> > function Base.getindex{K,V}(h::Dict{K,V}, key) 
> >> >   index = Base.ht_keyindex(h, key) 
> >> >   if index < 0 
> >> > if :_parent in keys(h) 
> >> >   Base.getindex(eval(h[:_parent]), key) 
> >> > else 
> >> >   throw(KeyError(key)) 
> >> > end 
> >> >   else 
> >> > h.vals[index]::V 
> >> >   end 
> >> > end 
> >> > 
> >> > # allow inheritance from modules 
> >> > function convert(::Type{Dict}, m::Module) 
> >> >   dict = Dict() 
> >> >   for name in names(m) 
> >> > dict[name] = eval( :(Base.$name) ) 
> >> > # add types to type list 
> >> > if typeof(dict[name]) <: DataType 
> >> >   push!(_TYPES_, name) 
> >> > end 
> >> >   end 
> >> >   dict 
> >> > end 
> >> > 
> >> > # allow inheritance from DataFrames 
> >> > function convert(::Type{Dict}, d::DataFrame) 
> >> > dict = Dict() 
> >> >   for name in names(d) 
> >> > dict[name] = d[name] 
> >> > # add types to type list 
> >> > if typeof(dict[name]) <: DataType 
> >> >   push!(_TYPES_, name) 
> >> > end 
> >> >   end 
> >> >   dict 
> >> > end 
> >> > 
> >> > # establish the base environment, save it as global, and point it to 
> an 
> >> > empty dictionary 
> >> > macro _ENV_MACRO_() 
> >> >   esc(quote 
> >> > _ENV_ = gensym() 
> >> > _GLOBAL_ = _ENV_ 
> >> > eval(quote 
> >> >$_ENV_ = Dict() 
> >> >  end) 
> >> >   end) 
> >> > end 
> >> > 
> >> > # establish a new environment and point it towards a dict daughter of 
> >> > the 
> >> > old environment 
> >> > macro _NEW_ENV_MACRO_() 
> >> >   esc(quote 
> >> > _NEW_ENV_ = gensym() 
> >> > eval(quote 
> >> >$_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) } 
> >> >  end) 
> >> > _ENV_ = _NEW_ENV_ 
> >> >   end) 
> >> > end 
> >> > 
> >> > # establish a new environment and point it towards a dict daughter of 
> >> > the 
> >> > old environment with dict contents 
> >> > macro _ADD_ENV_MACRO_(dict) 
> >> >   esc(quote 
> >> > _DICT_ = $dict 
> >> > _ADD_ENV_ = gensym() 
> >> > eval(quote 
> >> >$_ADD_ENV_ = convert(Dict, $_DICT_) 
> >> >$_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_)) 
> >> >  end) 
> >> > _ENV_ = _ADD_ENV_ 
> >> >   end) 
> >> > end 
> >> > 
> >> > # jump back in time to the previous generation 
> >> > macro _REMOVE_ENV_MACRO_() 
> >> >   esc(quote 
> >> > _ENV_ = eval(_ENV_)[:_parent] 
> >> >   end) 
> >> > end 
> >> > 
> >> > 
> >> > # new types will have to be included in a module at the beginning of 
> >> > code 
> >> > # that module will need to be converted to a Dict along with base 
> >> > # a namespace will need to be created such that module dicts inherit 
> >> > from 
> >> > each other, with base at the top 
> >> > 
> >> > @_ENV_MACRO_() 
> >> > @_ADD_ENV_MACRO_(Base) 
> >> > @_NEW_ENV_MACRO_ 
> >> > 
> >> > # test expression 
> >> > e = 
> >> >   quote 
> >> > a = 1 
> >> > b = 2 
> >> > test = function() 
> >> >   b = a 
> >> > end 
> >> >   end 
> >> > 
> >> > 
> >> > # reformat code to use dict scoping 
> >> > function _ENV_REPLACE_(_Lazy_::_LAZY_) 
> >> > 
> >> >   e = copy(_Lazy_._E_) 
> >> >   _ENV_ = _Lazy_._ENV_ 
> >> > 
> 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Thu, Jul 23, 2015 at 4:23 PM, Brandon Taylor
 wrote:
> Will it cause a slowdown because there is a better way to do it or will it
> cause a slowdown because something inherent about environment access?
> Because if the second is the case, then it would be worth it, at least to
> me.

AFAICT, environment access is basically global variable and it will
trash any optimization julia can do so in some sense the slow down is
inherent about environment access.

I don't see why the inherent slowness would be particularly useful though.

>
> On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote:
>>
>> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor
>>  wrote:
>> > Ok, made some progress. Still having trouble with a _ENV_ not defined
>> > error:
>> > how could it not be defined if it's a global variable??? Figuring out
>> > argument passing is going to be tricky. Is this kind of system feasible
>> > or
>> > is it going to cause huge slowdowns?
>>
>> It'll cause a huge slowdown.
>>
>> >
>> > using DataFrames
>> >
>> > import Base.convert
>> >
>> > _TYPES_ = Symbol[]
>> >
>> > type _LAZY_
>> >   _E_::Expr
>> >   _ENV_::Symbol
>> > end
>> >
>> > # modify dicts such that if a key is not found, search the parent
>> > function Base.getindex{K,V}(h::Dict{K,V}, key)
>> >   index = Base.ht_keyindex(h, key)
>> >   if index < 0
>> > if :_parent in keys(h)
>> >   Base.getindex(eval(h[:_parent]), key)
>> > else
>> >   throw(KeyError(key))
>> > end
>> >   else
>> > h.vals[index]::V
>> >   end
>> > end
>> >
>> > # allow inheritance from modules
>> > function convert(::Type{Dict}, m::Module)
>> >   dict = Dict()
>> >   for name in names(m)
>> > dict[name] = eval( :(Base.$name) )
>> > # add types to type list
>> > if typeof(dict[name]) <: DataType
>> >   push!(_TYPES_, name)
>> > end
>> >   end
>> >   dict
>> > end
>> >
>> > # allow inheritance from DataFrames
>> > function convert(::Type{Dict}, d::DataFrame)
>> > dict = Dict()
>> >   for name in names(d)
>> > dict[name] = d[name]
>> > # add types to type list
>> > if typeof(dict[name]) <: DataType
>> >   push!(_TYPES_, name)
>> > end
>> >   end
>> >   dict
>> > end
>> >
>> > # establish the base environment, save it as global, and point it to an
>> > empty dictionary
>> > macro _ENV_MACRO_()
>> >   esc(quote
>> > _ENV_ = gensym()
>> > _GLOBAL_ = _ENV_
>> > eval(quote
>> >$_ENV_ = Dict()
>> >  end)
>> >   end)
>> > end
>> >
>> > # establish a new environment and point it towards a dict daughter of
>> > the
>> > old environment
>> > macro _NEW_ENV_MACRO_()
>> >   esc(quote
>> > _NEW_ENV_ = gensym()
>> > eval(quote
>> >$_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) }
>> >  end)
>> > _ENV_ = _NEW_ENV_
>> >   end)
>> > end
>> >
>> > # establish a new environment and point it towards a dict daughter of
>> > the
>> > old environment with dict contents
>> > macro _ADD_ENV_MACRO_(dict)
>> >   esc(quote
>> > _DICT_ = $dict
>> > _ADD_ENV_ = gensym()
>> > eval(quote
>> >$_ADD_ENV_ = convert(Dict, $_DICT_)
>> >$_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_))
>> >  end)
>> > _ENV_ = _ADD_ENV_
>> >   end)
>> > end
>> >
>> > # jump back in time to the previous generation
>> > macro _REMOVE_ENV_MACRO_()
>> >   esc(quote
>> > _ENV_ = eval(_ENV_)[:_parent]
>> >   end)
>> > end
>> >
>> >
>> > # new types will have to be included in a module at the beginning of
>> > code
>> > # that module will need to be converted to a Dict along with base
>> > # a namespace will need to be created such that module dicts inherit
>> > from
>> > each other, with base at the top
>> >
>> > @_ENV_MACRO_()
>> > @_ADD_ENV_MACRO_(Base)
>> > @_NEW_ENV_MACRO_
>> >
>> > # test expression
>> > e =
>> >   quote
>> > a = 1
>> > b = 2
>> > test = function()
>> >   b = a
>> > end
>> >   end
>> >
>> >
>> > # reformat code to use dict scoping
>> > function _ENV_REPLACE_(_Lazy_::_LAZY_)
>> >
>> >   e = copy(_Lazy_._E_)
>> >   _ENV_ = _Lazy_._ENV_
>> >
>> >   # expressions wrapped in _esc will be left alone
>> >   if length(e.args) > 0
>> > if (e.head == :call) & (e.args[1] == :_esc)
>> >   return e.args[2]
>> > end
>> >   end
>> >
>> >   # set a new scope for a new function. This will also have to be done
>> > with
>> > for loops, modules, etc.
>> >   if (e.head == :function)
>> > # insert a new scope definition into the function definition
>> > e.args[2].args = [
>> >   e.args[2].args[1],
>> >   :(@_NEW_ENV_MACRO_),
>> >   :(_ENV_REPLACE!_(
>> >   $(Expr(:block,
>> >  e.args[2].args[2:end]...,
>> >   :(@_REMOVE_ENV_MACRO_)]
>> >
>> >   # ignore line numbers
>> > elseif e.head != :line
>> > # for each sentence
>> > for i in 1:length(e.args)
>> >   # replace symbols with their dict scoped version
>> >   if typeof(e.args[i]) == Symbol
>> 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Will it cause a slowdown because there is a better way to do it or will it 
cause a slowdown because something inherent about environment access? 
Because if the second is the case, then it would be worth it, at least to 
me.

On Thursday, July 23, 2015 at 11:13:14 AM UTC+8, Yichao Yu wrote:
>
> On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor 
> > wrote: 
> > Ok, made some progress. Still having trouble with a _ENV_ not defined 
> error: 
> > how could it not be defined if it's a global variable??? Figuring out 
> > argument passing is going to be tricky. Is this kind of system feasible 
> or 
> > is it going to cause huge slowdowns? 
>
> It'll cause a huge slowdown. 
>
> > 
> > using DataFrames 
> > 
> > import Base.convert 
> > 
> > _TYPES_ = Symbol[] 
> > 
> > type _LAZY_ 
> >   _E_::Expr 
> >   _ENV_::Symbol 
> > end 
> > 
> > # modify dicts such that if a key is not found, search the parent 
> > function Base.getindex{K,V}(h::Dict{K,V}, key) 
> >   index = Base.ht_keyindex(h, key) 
> >   if index < 0 
> > if :_parent in keys(h) 
> >   Base.getindex(eval(h[:_parent]), key) 
> > else 
> >   throw(KeyError(key)) 
> > end 
> >   else 
> > h.vals[index]::V 
> >   end 
> > end 
> > 
> > # allow inheritance from modules 
> > function convert(::Type{Dict}, m::Module) 
> >   dict = Dict() 
> >   for name in names(m) 
> > dict[name] = eval( :(Base.$name) ) 
> > # add types to type list 
> > if typeof(dict[name]) <: DataType 
> >   push!(_TYPES_, name) 
> > end 
> >   end 
> >   dict 
> > end 
> > 
> > # allow inheritance from DataFrames 
> > function convert(::Type{Dict}, d::DataFrame) 
> > dict = Dict() 
> >   for name in names(d) 
> > dict[name] = d[name] 
> > # add types to type list 
> > if typeof(dict[name]) <: DataType 
> >   push!(_TYPES_, name) 
> > end 
> >   end 
> >   dict 
> > end 
> > 
> > # establish the base environment, save it as global, and point it to an 
> > empty dictionary 
> > macro _ENV_MACRO_() 
> >   esc(quote 
> > _ENV_ = gensym() 
> > _GLOBAL_ = _ENV_ 
> > eval(quote 
> >$_ENV_ = Dict() 
> >  end) 
> >   end) 
> > end 
> > 
> > # establish a new environment and point it towards a dict daughter of 
> the 
> > old environment 
> > macro _NEW_ENV_MACRO_() 
> >   esc(quote 
> > _NEW_ENV_ = gensym() 
> > eval(quote 
> >$_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) } 
> >  end) 
> > _ENV_ = _NEW_ENV_ 
> >   end) 
> > end 
> > 
> > # establish a new environment and point it towards a dict daughter of 
> the 
> > old environment with dict contents 
> > macro _ADD_ENV_MACRO_(dict) 
> >   esc(quote 
> > _DICT_ = $dict 
> > _ADD_ENV_ = gensym() 
> > eval(quote 
> >$_ADD_ENV_ = convert(Dict, $_DICT_) 
> >$_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_)) 
> >  end) 
> > _ENV_ = _ADD_ENV_ 
> >   end) 
> > end 
> > 
> > # jump back in time to the previous generation 
> > macro _REMOVE_ENV_MACRO_() 
> >   esc(quote 
> > _ENV_ = eval(_ENV_)[:_parent] 
> >   end) 
> > end 
> > 
> > 
> > # new types will have to be included in a module at the beginning of 
> code 
> > # that module will need to be converted to a Dict along with base 
> > # a namespace will need to be created such that module dicts inherit 
> from 
> > each other, with base at the top 
> > 
> > @_ENV_MACRO_() 
> > @_ADD_ENV_MACRO_(Base) 
> > @_NEW_ENV_MACRO_ 
> > 
> > # test expression 
> > e = 
> >   quote 
> > a = 1 
> > b = 2 
> > test = function() 
> >   b = a 
> > end 
> >   end 
> > 
> > 
> > # reformat code to use dict scoping 
> > function _ENV_REPLACE_(_Lazy_::_LAZY_) 
> > 
> >   e = copy(_Lazy_._E_) 
> >   _ENV_ = _Lazy_._ENV_ 
> > 
> >   # expressions wrapped in _esc will be left alone 
> >   if length(e.args) > 0 
> > if (e.head == :call) & (e.args[1] == :_esc) 
> >   return e.args[2] 
> > end 
> >   end 
> > 
> >   # set a new scope for a new function. This will also have to be done 
> with 
> > for loops, modules, etc. 
> >   if (e.head == :function) 
> > # insert a new scope definition into the function definition 
> > e.args[2].args = [ 
> >   e.args[2].args[1], 
> >   :(@_NEW_ENV_MACRO_), 
> >   :(_ENV_REPLACE!_( 
> >   $(Expr(:block, 
> >  e.args[2].args[2:end]..., 
> >   :(@_REMOVE_ENV_MACRO_)] 
> > 
> >   # ignore line numbers 
> > elseif e.head != :line 
> > # for each sentence 
> > for i in 1:length(e.args) 
> >   # replace symbols with their dict scoped version 
> >   if typeof(e.args[i]) == Symbol 
> > 
> > #avoid types 
> > if !(e.args[i] in _TYPES_) 
> >   e.args[i] = :($_ENV_[$(string(e.args[i]))]) 
> > end 
> > 
> >   # recur into new expressions 
> >   elseif typeof(e.args[i]) == Expr 
> > e.args[i] = _ENV_REPLACE!_(e.args[i], _ENV_) 
> >   end 
> > end 
> >   end 
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Wed, Jul 22, 2015 at 10:51 PM, Brandon Taylor
 wrote:
> Ok, made some progress. Still having trouble with a _ENV_ not defined error:
> how could it not be defined if it's a global variable??? Figuring out
> argument passing is going to be tricky. Is this kind of system feasible or
> is it going to cause huge slowdowns?

It'll cause a huge slowdown.

>
> using DataFrames
>
> import Base.convert
>
> _TYPES_ = Symbol[]
>
> type _LAZY_
>   _E_::Expr
>   _ENV_::Symbol
> end
>
> # modify dicts such that if a key is not found, search the parent
> function Base.getindex{K,V}(h::Dict{K,V}, key)
>   index = Base.ht_keyindex(h, key)
>   if index < 0
> if :_parent in keys(h)
>   Base.getindex(eval(h[:_parent]), key)
> else
>   throw(KeyError(key))
> end
>   else
> h.vals[index]::V
>   end
> end
>
> # allow inheritance from modules
> function convert(::Type{Dict}, m::Module)
>   dict = Dict()
>   for name in names(m)
> dict[name] = eval( :(Base.$name) )
> # add types to type list
> if typeof(dict[name]) <: DataType
>   push!(_TYPES_, name)
> end
>   end
>   dict
> end
>
> # allow inheritance from DataFrames
> function convert(::Type{Dict}, d::DataFrame)
> dict = Dict()
>   for name in names(d)
> dict[name] = d[name]
> # add types to type list
> if typeof(dict[name]) <: DataType
>   push!(_TYPES_, name)
> end
>   end
>   dict
> end
>
> # establish the base environment, save it as global, and point it to an
> empty dictionary
> macro _ENV_MACRO_()
>   esc(quote
> _ENV_ = gensym()
> _GLOBAL_ = _ENV_
> eval(quote
>$_ENV_ = Dict()
>  end)
>   end)
> end
>
> # establish a new environment and point it towards a dict daughter of the
> old environment
> macro _NEW_ENV_MACRO_()
>   esc(quote
> _NEW_ENV_ = gensym()
> eval(quote
>$_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) }
>  end)
> _ENV_ = _NEW_ENV_
>   end)
> end
>
> # establish a new environment and point it towards a dict daughter of the
> old environment with dict contents
> macro _ADD_ENV_MACRO_(dict)
>   esc(quote
> _DICT_ = $dict
> _ADD_ENV_ = gensym()
> eval(quote
>$_ADD_ENV_ = convert(Dict, $_DICT_)
>$_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_))
>  end)
> _ENV_ = _ADD_ENV_
>   end)
> end
>
> # jump back in time to the previous generation
> macro _REMOVE_ENV_MACRO_()
>   esc(quote
> _ENV_ = eval(_ENV_)[:_parent]
>   end)
> end
>
>
> # new types will have to be included in a module at the beginning of code
> # that module will need to be converted to a Dict along with base
> # a namespace will need to be created such that module dicts inherit from
> each other, with base at the top
>
> @_ENV_MACRO_()
> @_ADD_ENV_MACRO_(Base)
> @_NEW_ENV_MACRO_
>
> # test expression
> e =
>   quote
> a = 1
> b = 2
> test = function()
>   b = a
> end
>   end
>
>
> # reformat code to use dict scoping
> function _ENV_REPLACE_(_Lazy_::_LAZY_)
>
>   e = copy(_Lazy_._E_)
>   _ENV_ = _Lazy_._ENV_
>
>   # expressions wrapped in _esc will be left alone
>   if length(e.args) > 0
> if (e.head == :call) & (e.args[1] == :_esc)
>   return e.args[2]
> end
>   end
>
>   # set a new scope for a new function. This will also have to be done with
> for loops, modules, etc.
>   if (e.head == :function)
> # insert a new scope definition into the function definition
> e.args[2].args = [
>   e.args[2].args[1],
>   :(@_NEW_ENV_MACRO_),
>   :(_ENV_REPLACE!_(
>   $(Expr(:block,
>  e.args[2].args[2:end]...,
>   :(@_REMOVE_ENV_MACRO_)]
>
>   # ignore line numbers
> elseif e.head != :line
> # for each sentence
> for i in 1:length(e.args)
>   # replace symbols with their dict scoped version
>   if typeof(e.args[i]) == Symbol
>
> #avoid types
> if !(e.args[i] in _TYPES_)
>   e.args[i] = :($_ENV_[$(string(e.args[i]))])
> end
>
>   # recur into new expressions
>   elseif typeof(e.args[i]) == Expr
> e.args[i] = _ENV_REPLACE!_(e.args[i], _ENV_)
>   end
> end
>   end
>   e
> end
>
> function _LAZY_(e::Expr)
>   _LAZY_(e, _ENV_)
> end
>
> macro _LAZY_EVAL_(_Lazy_)
>   esc(quote
> eval(_ENV_REPLACE_($_Lazy_))
>   end)
> end
>
>
> @_LAZY_EVAL_(_LAZY_(e))
>
> eval(_ENV_)["a"]
> eval(_ENV_)["b"]
> eval(_ENV_)["test"]() ## ERROR HERE
>
>
>
>
>
> On Wednesday, July 22, 2015 at 11:30:10 AM UTC+8, Brandon Taylor wrote:
>>
>> More to do:
>> Expressions would also have to be escaped from quoting.
>> If we can't scope types within dicts, it might be necessary to have
>> special markers for types so they can avoid being scoped.
>> I don't think that macros will be necessary anymore
>>
>>
>> On Wednesday, July 22, 2015 at 11:14:38 AM UTC+8, Brandon Taylor wrote:
>>>
>>> Ok so I've got a good start. I bet John Myles White didn't think I could
>>> get this far. Anyway, I'm gettin

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Ok, made some progress. Still having trouble with a _ENV_ not defined 
error: how could it not be defined if it's a global variable??? Figuring 
out argument passing is going to be tricky. Is this kind of system feasible 
or is it going to cause huge slowdowns?

using DataFrames

import Base.convert

_TYPES_ = Symbol[]

type _LAZY_
  _E_::Expr
  _ENV_::Symbol
end

# modify dicts such that if a key is not found, search the parent
function Base.getindex{K,V}(h::Dict{K,V}, key)
  index = Base.ht_keyindex(h, key)
  if index < 0
if :_parent in keys(h)
  Base.getindex(eval(h[:_parent]), key)
else
  throw(KeyError(key))
end
  else
h.vals[index]::V
  end
end

# allow inheritance from modules
function convert(::Type{Dict}, m::Module)
  dict = Dict()
  for name in names(m)
dict[name] = eval( :(Base.$name) )
# add types to type list
if typeof(dict[name]) <: DataType
  push!(_TYPES_, name)
end
  end
  dict
end

# allow inheritance from DataFrames
function convert(::Type{Dict}, d::DataFrame)
dict = Dict()
  for name in names(d)
dict[name] = d[name]
# add types to type list
if typeof(dict[name]) <: DataType
  push!(_TYPES_, name)
end
  end
  dict
end

# establish the base environment, save it as global, and point it to an 
empty dictionary
macro _ENV_MACRO_()
  esc(quote
_ENV_ = gensym()
_GLOBAL_ = _ENV_
eval(quote
   $_ENV_ = Dict()
 end)
  end)
end

# establish a new environment and point it towards a dict daughter of the 
old environment
macro _NEW_ENV_MACRO_()
  esc(quote
_NEW_ENV_ = gensym()
eval(quote
   $_NEW_ENV_ = {:_parent => $(Expr(:quote, _ENV_)) }
 end)
_ENV_ = _NEW_ENV_
  end)
end

# establish a new environment and point it towards a dict daughter of the 
old environment with dict contents
macro _ADD_ENV_MACRO_(dict)
  esc(quote
_DICT_ = $dict
_ADD_ENV_ = gensym()
eval(quote
   $_ADD_ENV_ = convert(Dict, $_DICT_)
   $_ADD_ENV_[:_parent] = $(Expr(:quote, _ENV_))
 end)
_ENV_ = _ADD_ENV_
  end)
end

# jump back in time to the previous generation
macro _REMOVE_ENV_MACRO_()
  esc(quote
_ENV_ = eval(_ENV_)[:_parent]
  end)
end


# new types will have to be included in a module at the beginning of code
# that module will need to be converted to a Dict along with base
# a namespace will need to be created such that module dicts inherit from 
each other, with base at the top

@_ENV_MACRO_()
@_ADD_ENV_MACRO_(Base)
@_NEW_ENV_MACRO_

# test expression
e =
  quote
a = 1
b = 2
test = function()
  b = a
end
  end


# reformat code to use dict scoping
function _ENV_REPLACE_(_Lazy_::_LAZY_)
  
  e = copy(_Lazy_._E_)
  _ENV_ = _Lazy_._ENV_

  # expressions wrapped in _esc will be left alone
  if length(e.args) > 0
if (e.head == :call) & (e.args[1] == :_esc)
  return e.args[2]
end
  end

  # set a new scope for a new function. This will also have to be done with 
for loops, modules, etc.
  if (e.head == :function)
# insert a new scope definition into the function definition
e.args[2].args = [
  e.args[2].args[1],
  :(@_NEW_ENV_MACRO_),
  :(_ENV_REPLACE!_(
  $(Expr(:block,
 e.args[2].args[2:end]...,
  :(@_REMOVE_ENV_MACRO_)]

  # ignore line numbers
elseif e.head != :line
# for each sentence
for i in 1:length(e.args)
  # replace symbols with their dict scoped version
  if typeof(e.args[i]) == Symbol

#avoid types
if !(e.args[i] in _TYPES_)
  e.args[i] = :($_ENV_[$(string(e.args[i]))])
end

  # recur into new expressions
  elseif typeof(e.args[i]) == Expr
e.args[i] = _ENV_REPLACE!_(e.args[i], _ENV_)
  end
end
  end
  e
end

function _LAZY_(e::Expr)
  _LAZY_(e, _ENV_)
end

macro _LAZY_EVAL_(_Lazy_)
  esc(quote
eval(_ENV_REPLACE_($_Lazy_))
  end)
end


@_LAZY_EVAL_(_LAZY_(e))

eval(_ENV_)["a"]
eval(_ENV_)["b"]
eval(_ENV_)["test"]() ## ERROR HERE





On Wednesday, July 22, 2015 at 11:30:10 AM UTC+8, Brandon Taylor wrote:
>
> More to do:
> Expressions would also have to be escaped from quoting. 
> If we can't scope types within dicts, it might be necessary to have 
> special markers for types so they can avoid being scoped.
> I don't think that macros will be necessary anymore
>
>
> On Wednesday, July 22, 2015 at 11:14:38 AM UTC+8, Brandon Taylor wrote:
>>
>> Ok so I've got a good start. I bet John Myles White didn't think I could 
>> get this far. Anyway, I'm getting caught up in defining my own escape 
>> function. I'm getting lost in multiple layers of meta.
>>
>> using DataFrames
>>
>> import Base.convert
>>
>> # allow inheritance from modules
>> function convert(::Type{Dict}, m::Module)
>> dict = Dict()
>>   for name in names(m)
>> dict[name] = eval( :(Base.$name) )
>>   end
>>   dict
>> end
>>
>> base_dict = convert(Dict, Base)
>>
>> # allow inheritance from DataFrames
>> function c

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

More to do:
Expressions would also have to be escaped from quoting. 
If we can't scope types within dicts, it might be necessary to have special 
markers for types so they can avoid being scoped.
I don't think that macros will be necessary anymore


On Wednesday, July 22, 2015 at 11:14:38 AM UTC+8, Brandon Taylor wrote:
>
> Ok so I've got a good start. I bet John Myles White didn't think I could 
> get this far. Anyway, I'm getting caught up in defining my own escape 
> function. I'm getting lost in multiple layers of meta.
>
> using DataFrames
>
> import Base.convert
>
> # allow inheritance from modules
> function convert(::Type{Dict}, m::Module)
> dict = Dict()
>   for name in names(m)
> dict[name] = eval( :(Base.$name) )
>   end
>   dict
> end
>
> base_dict = convert(Dict, Base)
>
> # allow inheritance from DataFrames
> function convert(::Type{Dict}, d::DataFrame)
> dict = Dict()
>   for name in names(d)
> dict[name] = d[name]
>   end
>   dict
> end
>
> # modify dicts such that if a key is not found, search the parent
> function Base.getindex{K,V}(h::Dict{K,V}, key)
>   index = Base.ht_keyindex(h, key)
>   if index < 0
> if :_parent in keys(h)
>   Base.getindex(h[:_parent], key)
> else
>   throw(KeyError(key))
> end
>   else
> h.vals[index]::V
>   end
> end
>
> # test expression
> e = 
>   quote
> a = 1
> # anonymous functions required for proper scoping
> test = function()
>   b = a
> end
>   end
>
> # set up the global environment
> _env = gensym()
> eval(:($_env = [:_parent => base_dict] ) )
>
>
> #_new_env = _env
> # this is the code that needs to be escaped
> #eval(:($_new_env = [:_parent => eval(_env)] ) )
>
> # reformat code to use dict scoping
> function env_replace!(e::Expr, 
>   _env::Symbol = _env)
>   
>   # expressions wrapped in _esc will be left alone  
>   if length(e.args) > 0
> if (e.head == :call) & (e.args[1] == :_esc)
>   return e.args[2]
> end
>   end
>   
>   # set a new scope for a new function. This will also have to be done 
> with for loops, modules, etc.
>   if (e.head == :function)
> # insert a new scope definition into the function definition
>   _new_env = gensym()
>   e.args[2].args = [
> e.args[2].args[1],
> :_esc(),  need help here ###
> e.args[2].args[2:end] ]  
>   _env = _new_env
> end
>   
>   # ignore line numbers
>   if e.head != :line 
>
> for i in 1:length(e.args)
>   # replace symbols with their dict scoped version
>   if typeof(e.args[i]) == Symbol
> e.args[i] = :($_env[$(string(e.args[i]))])
> 
>   # recur into new expressions
>   elseif typeof(e.args[i]) == Expr
> e.args[i] = env_replace!(e.args[i], _env)
>   end
> end
>   end
>   e
> end
>
> # here is an eval that allows evaluation within a certain dict scope
> function lazy_eval(e::Expr,
>_env::Symbol = _env)
>   eval(env_replace!(e), _env)
> end
>
>
> ## TO DO
> # fix _esc problem
> # prevent environment symbols from being scoped (perhaps with a special 
> marker)
> # rescope for, while, try, catch, finally, let, and type
> # perhaps use fast anonymous to avoid performance slowdowns?
>
>
> On Tuesday, July 21, 2015 at 10:10:58 AM UTC+8, Brandon Taylor wrote:
>>
>> And there would need to be a special marker for them, such that if I'm in 
>> function f, f[:a] won't get preprocessed as f[:f][:a]
>>
>> On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
>>>
>>> Although that would probably require nested dicts. Each would have a 
>>> parent dict, and if a lookup isn't found in the current dict, the parent 
>>> dict would be searched.
>>>
>>> On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:

 I should be possible to preprocess code such that everything is put 
 into a dict based on the name of enclosing function (and global variables 
 will just go into a dict called global).

 On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
>
> Dicts seem to work pretty well for this kind of thing.
>
> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>>
>> I'm getting a cannot assign variables in other modules error.
>>
>> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>>
>>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
>>>  wrote: 
>>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
>>> possible 
>>> > to hijack this functionality to provide pseudo-environments? That 
>>> is, never 
>>> > referring to anything that is not already in an explicit module? 
>>> And also, 
>>> > have a data-frame simply be a module? 
>>>
>>> I think this would in principle works. A module is basically what 
>>> global scope means so all the performance concern applies. 
>>>
>>> > 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Ok so I've got a good start. I bet John Myles White didn't think I could 
get this far. Anyway, I'm getting caught up in defining my own escape 
function. I'm getting lost in multiple layers of meta.

using DataFrames

import Base.convert

# allow inheritance from modules
function convert(::Type{Dict}, m::Module)
dict = Dict()
  for name in names(m)
dict[name] = eval( :(Base.$name) )
  end
  dict
end

base_dict = convert(Dict, Base)

# allow inheritance from DataFrames
function convert(::Type{Dict}, d::DataFrame)
dict = Dict()
  for name in names(d)
dict[name] = d[name]
  end
  dict
end

# modify dicts such that if a key is not found, search the parent
function Base.getindex{K,V}(h::Dict{K,V}, key)
  index = Base.ht_keyindex(h, key)
  if index < 0
if :_parent in keys(h)
  Base.getindex(h[:_parent], key)
else
  throw(KeyError(key))
end
  else
h.vals[index]::V
  end
end

# test expression
e = 
  quote
a = 1
# anonymous functions required for proper scoping
test = function()
  b = a
end
  end

# set up the global environment
_env = gensym()
eval(:($_env = [:_parent => base_dict] ) )


#_new_env = _env
# this is the code that needs to be escaped
#eval(:($_new_env = [:_parent => eval(_env)] ) )

# reformat code to use dict scoping
function env_replace!(e::Expr, 
  _env::Symbol = _env)
  
  # expressions wrapped in _esc will be left alone  
  if length(e.args) > 0
if (e.head == :call) & (e.args[1] == :_esc)
  return e.args[2]
end
  end
  
  # set a new scope for a new function. This will also have to be done with 
for loops, modules, etc.
  if (e.head == :function)
# insert a new scope definition into the function definition
  _new_env = gensym()
  e.args[2].args = [
e.args[2].args[1],
:_esc(),  need help here ###
e.args[2].args[2:end] ]  
  _env = _new_env
end
  
  # ignore line numbers
  if e.head != :line 

for i in 1:length(e.args)
  # replace symbols with their dict scoped version
  if typeof(e.args[i]) == Symbol
e.args[i] = :($_env[$(string(e.args[i]))])

  # recur into new expressions
  elseif typeof(e.args[i]) == Expr
e.args[i] = env_replace!(e.args[i], _env)
  end
end
  end
  e
end

# here is an eval that allows evaluation within a certain dict scope
function lazy_eval(e::Expr,
   _env::Symbol = _env)
  eval(env_replace!(e), _env)
end


## TO DO
# fix _esc problem
# prevent environment symbols from being scoped (perhaps with a special 
marker)
# rescope for, while, try, catch, finally, let, and type
# perhaps use fast anonymous to avoid performance slowdowns?


On Tuesday, July 21, 2015 at 10:10:58 AM UTC+8, Brandon Taylor wrote:
>
> And there would need to be a special marker for them, such that if I'm in 
> function f, f[:a] won't get preprocessed as f[:f][:a]
>
> On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
>>
>> Although that would probably require nested dicts. Each would have a 
>> parent dict, and if a lookup isn't found in the current dict, the parent 
>> dict would be searched.
>>
>> On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
>>>
>>> I should be possible to preprocess code such that everything is put into 
>>> a dict based on the name of enclosing function (and global variables will 
>>> just go into a dict called global).
>>>
>>> On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:

 Dicts seem to work pretty well for this kind of thing.

 On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>
> I'm getting a cannot assign variables in other modules error.
>
> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>
>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
>>  wrote: 
>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
>> possible 
>> > to hijack this functionality to provide pseudo-environments? That 
>> is, never 
>> > referring to anything that is not already in an explicit module? 
>> And also, 
>> > have a data-frame simply be a module? 
>>
>> I think this would in principle works. A module is basically what 
>> global scope means so all the performance concern applies. 
>>
>> > 
>> > 
>> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor 
>> wrote: 
>> >> 
>> >> I don't know if you came across the vignette? 
>> >> 
>> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html 
>> ? 
>> >> dplyr uses lazyeval extensively, see 
>> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . 
>> The cool 
>> >> thing about being able to incorporate this kind of thing in Julia 
>> would be 
>> >> being able to use the self-reflection capabilities. 
>> >> 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

And there would need to be a special marker for them, such that if I'm in 
function f, f[:a] won't get preprocessed as f[:f][:a]

On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
>
> Although that would probably require nested dicts. Each would have a 
> parent dict, and if a lookup isn't found in the current dict, the parent 
> dict would be searched.
>
> On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
>>
>> I should be possible to preprocess code such that everything is put into 
>> a dict based on the name of enclosing function (and global variables will 
>> just go into a dict called global).
>>
>> On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
>>>
>>> Dicts seem to work pretty well for this kind of thing.
>>>
>>> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:

 I'm getting a cannot assign variables in other modules error.

 On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>
> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
>  wrote: 
> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
> possible 
> > to hijack this functionality to provide pseudo-environments? That 
> is, never 
> > referring to anything that is not already in an explicit module? And 
> also, 
> > have a data-frame simply be a module? 
>
> I think this would in principle works. A module is basically what 
> global scope means so all the performance concern applies. 
>
> > 
> > 
> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote: 
> >> 
> >> I don't know if you came across the vignette? 
> >> 
> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html 
> ? 
> >> dplyr uses lazyeval extensively, see 
> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . 
> The cool 
> >> thing about being able to incorporate this kind of thing in Julia 
> would be 
> >> being able to use the self-reflection capabilities. 
> >> 
> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean 
> wrote: 
> >>> 
> >>> 
> >>> 
> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor 
> wrote: 
>  
>  To walk back in time, you could say something like: compile this 
> like 
>  this was is in line 8. Or compile this like this was in line 5. 
> It seems 
>  like Julia already has some of this functionality in macros. 
> Internal 
>  variables are compiled as if they were in local scope. But 
> escaped 
>  expressions are compiled as if they were in global scope. 
> >>> 
> >>> 
> >>> Could you provide context or a real-world use? I've looked at the 
> >>> lazyeval package, and I'm not entirely sure what it does. Does it 
> provide 
> >>> lazy evaluation for R? That's easy to achieve in Julia (well, 
> sorta). 
> >>> Instead of 
> >>> 
> >>> d = determinant(matrix) 
> >>>  
> >>> u = 2 * d 
> >>> 
> >>> you can write 
> >>> 
> >>> d = ()->determinant(matrix) 
> >>>  
> >>> u = 2 * d() # determinant is evaluated on use, in the context 
> where it 
> >>> was originally defined 
> >>> 
> >>> With macros this can turn into 
> >>> 
> >>> d = lazy(determinant(matrix)) 
> >>> 
> >>> which looks nicer (and also can avoid computing the determinant 
> twice if 
> >>> d() is called twice). 
> >>> 
> >>> Cédric 
> >>> 
>  
>  
>  On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean 
> wrote: 
> > 
> > 
> > 
> > On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor 
> wrote: 
> >> 
> >> Ok, here's where I'm getting hung up. You said that the 
> compiler 
> >> figures out the creation/lifetime of all variables at compile 
> time. So does 
> >> that mean there's a list like: 
> >> 
> >> a maps to location 0 and exists from line 3 to line 9 
> >> b maps to location 1 and exists from line 7 to line 9 
> >> a maps to location 10 and exists from line 7 to 9? 
> >> 
> >> and that to map variables to locations on any particular line, 
> the 
> >> compiler works its way up the list, 
> > 
> > 
> > Yes, more or less. 
> > 
> >> 
> >> 
> >> This is perhaps even more helpful than the environment. The 
> >> environment is immediately and completely determinable at any 
> point in the 
> >> program. This could make it possible to walk back in time even 
> within the 
> >> same scope. 
> > 
> > 
> > Could you expand on what you're thinking of? 
> > 
> >

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Although that would probably require nested dicts. Each would have a parent 
dict, and if a lookup isn't found in the current dict, the parent dict 
would be searched.

On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
>
> I should be possible to preprocess code such that everything is put into a 
> dict based on the name of enclosing function (and global variables will 
> just go into a dict called global).
>
> On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
>>
>> Dicts seem to work pretty well for this kind of thing.
>>
>> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>>>
>>> I'm getting a cannot assign variables in other modules error.
>>>
>>> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:

 On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
  wrote: 
 > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
 possible 
 > to hijack this functionality to provide pseudo-environments? That is, 
 never 
 > referring to anything that is not already in an explicit module? And 
 also, 
 > have a data-frame simply be a module? 

 I think this would in principle works. A module is basically what 
 global scope means so all the performance concern applies. 

 > 
 > 
 > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote: 
 >> 
 >> I don't know if you came across the vignette? 
 >> 
 http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html 
 ? 
 >> dplyr uses lazyeval extensively, see 
 >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . 
 The cool 
 >> thing about being able to incorporate this kind of thing in Julia 
 would be 
 >> being able to use the self-reflection capabilities. 
 >> 
 >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote: 
 >>> 
 >>> 
 >>> 
 >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor 
 wrote: 
  
  To walk back in time, you could say something like: compile this 
 like 
  this was is in line 8. Or compile this like this was in line 5. It 
 seems 
  like Julia already has some of this functionality in macros. 
 Internal 
  variables are compiled as if they were in local scope. But escaped 
  expressions are compiled as if they were in global scope. 
 >>> 
 >>> 
 >>> Could you provide context or a real-world use? I've looked at the 
 >>> lazyeval package, and I'm not entirely sure what it does. Does it 
 provide 
 >>> lazy evaluation for R? That's easy to achieve in Julia (well, 
 sorta). 
 >>> Instead of 
 >>> 
 >>> d = determinant(matrix) 
 >>>  
 >>> u = 2 * d 
 >>> 
 >>> you can write 
 >>> 
 >>> d = ()->determinant(matrix) 
 >>>  
 >>> u = 2 * d() # determinant is evaluated on use, in the context where 
 it 
 >>> was originally defined 
 >>> 
 >>> With macros this can turn into 
 >>> 
 >>> d = lazy(determinant(matrix)) 
 >>> 
 >>> which looks nicer (and also can avoid computing the determinant 
 twice if 
 >>> d() is called twice). 
 >>> 
 >>> Cédric 
 >>> 
  
  
  On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean 
 wrote: 
 > 
 > 
 > 
 > On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor 
 wrote: 
 >> 
 >> Ok, here's where I'm getting hung up. You said that the compiler 
 >> figures out the creation/lifetime of all variables at compile 
 time. So does 
 >> that mean there's a list like: 
 >> 
 >> a maps to location 0 and exists from line 3 to line 9 
 >> b maps to location 1 and exists from line 7 to line 9 
 >> a maps to location 10 and exists from line 7 to 9? 
 >> 
 >> and that to map variables to locations on any particular line, 
 the 
 >> compiler works its way up the list, 
 > 
 > 
 > Yes, more or less. 
 > 
 >> 
 >> 
 >> This is perhaps even more helpful than the environment. The 
 >> environment is immediately and completely determinable at any 
 point in the 
 >> program. This could make it possible to walk back in time even 
 within the 
 >> same scope. 
 > 
 > 
 > Could you expand on what you're thinking of? 
 > 
 > This kind of compile-time environment could conceivably be 
 exposed to 
 > macros. Common Lisp had proposals along that line 
 > (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) 
 but as far as 
 > I can tell, it was too complicated and not useful enough, so it 
 was 
 > axed/neutered at some point i

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

I should be possible to preprocess code such that everything is put into a 
dict based on the name of enclosing function (and global variables will 
just go into a dict called global).

On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
>
> Dicts seem to work pretty well for this kind of thing.
>
> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>>
>> I'm getting a cannot assign variables in other modules error.
>>
>> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>>
>>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
>>>  wrote: 
>>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
>>> possible 
>>> > to hijack this functionality to provide pseudo-environments? That is, 
>>> never 
>>> > referring to anything that is not already in an explicit module? And 
>>> also, 
>>> > have a data-frame simply be a module? 
>>>
>>> I think this would in principle works. A module is basically what 
>>> global scope means so all the performance concern applies. 
>>>
>>> > 
>>> > 
>>> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote: 
>>> >> 
>>> >> I don't know if you came across the vignette? 
>>> >> 
>>> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html 
>>> ? 
>>> >> dplyr uses lazyeval extensively, see 
>>> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . 
>>> The cool 
>>> >> thing about being able to incorporate this kind of thing in Julia 
>>> would be 
>>> >> being able to use the self-reflection capabilities. 
>>> >> 
>>> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote: 
>>> >>> 
>>> >>> 
>>> >>> 
>>> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor 
>>> wrote: 
>>>  
>>>  To walk back in time, you could say something like: compile this 
>>> like 
>>>  this was is in line 8. Or compile this like this was in line 5. It 
>>> seems 
>>>  like Julia already has some of this functionality in macros. 
>>> Internal 
>>>  variables are compiled as if they were in local scope. But escaped 
>>>  expressions are compiled as if they were in global scope. 
>>> >>> 
>>> >>> 
>>> >>> Could you provide context or a real-world use? I've looked at the 
>>> >>> lazyeval package, and I'm not entirely sure what it does. Does it 
>>> provide 
>>> >>> lazy evaluation for R? That's easy to achieve in Julia (well, 
>>> sorta). 
>>> >>> Instead of 
>>> >>> 
>>> >>> d = determinant(matrix) 
>>> >>>  
>>> >>> u = 2 * d 
>>> >>> 
>>> >>> you can write 
>>> >>> 
>>> >>> d = ()->determinant(matrix) 
>>> >>>  
>>> >>> u = 2 * d() # determinant is evaluated on use, in the context where 
>>> it 
>>> >>> was originally defined 
>>> >>> 
>>> >>> With macros this can turn into 
>>> >>> 
>>> >>> d = lazy(determinant(matrix)) 
>>> >>> 
>>> >>> which looks nicer (and also can avoid computing the determinant 
>>> twice if 
>>> >>> d() is called twice). 
>>> >>> 
>>> >>> Cédric 
>>> >>> 
>>>  
>>>  
>>>  On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean 
>>> wrote: 
>>> > 
>>> > 
>>> > 
>>> > On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor 
>>> wrote: 
>>> >> 
>>> >> Ok, here's where I'm getting hung up. You said that the compiler 
>>> >> figures out the creation/lifetime of all variables at compile 
>>> time. So does 
>>> >> that mean there's a list like: 
>>> >> 
>>> >> a maps to location 0 and exists from line 3 to line 9 
>>> >> b maps to location 1 and exists from line 7 to line 9 
>>> >> a maps to location 10 and exists from line 7 to 9? 
>>> >> 
>>> >> and that to map variables to locations on any particular line, 
>>> the 
>>> >> compiler works its way up the list, 
>>> > 
>>> > 
>>> > Yes, more or less. 
>>> > 
>>> >> 
>>> >> 
>>> >> This is perhaps even more helpful than the environment. The 
>>> >> environment is immediately and completely determinable at any 
>>> point in the 
>>> >> program. This could make it possible to walk back in time even 
>>> within the 
>>> >> same scope. 
>>> > 
>>> > 
>>> > Could you expand on what you're thinking of? 
>>> > 
>>> > This kind of compile-time environment could conceivably be exposed 
>>> to 
>>> > macros. Common Lisp had proposals along that line 
>>> > (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but 
>>> as far as 
>>> > I can tell, it was too complicated and not useful enough, so it 
>>> was 
>>> > axed/neutered at some point in the standardization process. 
>>> > 
>>> > > Hadley Wickham's lazyeval package in R is pretty cool in that 
>>> you can 
>>> > > attach an environment to an expression, pass it in and out of 
>>> functions with 
>>> > > various modifications, and then evaluate the expression within 
>>> the original 
>>> > > environment 
>>> > 
>>> > I don't know

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Mon, Jul 20, 2015 at 9:41 PM, Brandon Taylor
 wrote:
> Dicts seem to work pretty well for this kind of thing.
>

For storage, maybe.

>
> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>>
>> I'm getting a cannot assign variables in other modules error.

You cannot assign to module. You need `eval`

>>
>> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>>
>>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor
>>>  wrote:
>>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be
>>> > possible
>>> > to hijack this functionality to provide pseudo-environments? That is,
>>> > never
>>> > referring to anything that is not already in an explicit module? And
>>> > also,
>>> > have a data-frame simply be a module?
>>>
>>> I think this would in principle works. A module is basically what
>>> global scope means so all the performance concern applies.
>>>
>>> >
>>> >
>>> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote:
>>> >>
>>> >> I don't know if you came across the vignette?
>>> >>
>>> >> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html ?
>>> >> dplyr uses lazyeval extensively, see
>>> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . The
>>> >> cool
>>> >> thing about being able to incorporate this kind of thing in Julia
>>> >> would be
>>> >> being able to use the self-reflection capabilities.
>>> >>
>>> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>>> 
>>>  To walk back in time, you could say something like: compile this
>>>  like
>>>  this was is in line 8. Or compile this like this was in line 5. It
>>>  seems
>>>  like Julia already has some of this functionality in macros.
>>>  Internal
>>>  variables are compiled as if they were in local scope. But escaped
>>>  expressions are compiled as if they were in global scope.
>>> >>>
>>> >>>
>>> >>> Could you provide context or a real-world use? I've looked at the
>>> >>> lazyeval package, and I'm not entirely sure what it does. Does it
>>> >>> provide
>>> >>> lazy evaluation for R? That's easy to achieve in Julia (well, sorta).
>>> >>> Instead of
>>> >>>
>>> >>> d = determinant(matrix)
>>> >>> 
>>> >>> u = 2 * d
>>> >>>
>>> >>> you can write
>>> >>>
>>> >>> d = ()->determinant(matrix)
>>> >>> 
>>> >>> u = 2 * d() # determinant is evaluated on use, in the context where
>>> >>> it
>>> >>> was originally defined
>>> >>>
>>> >>> With macros this can turn into
>>> >>>
>>> >>> d = lazy(determinant(matrix))
>>> >>>
>>> >>> which looks nicer (and also can avoid computing the determinant twice
>>> >>> if
>>> >>> d() is called twice).
>>> >>>
>>> >>> Cédric
>>> >>>
>>> 
>>> 
>>>  On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>>> >
>>> >
>>> >
>>> > On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor
>>> > wrote:
>>> >>
>>> >> Ok, here's where I'm getting hung up. You said that the compiler
>>> >> figures out the creation/lifetime of all variables at compile
>>> >> time. So does
>>> >> that mean there's a list like:
>>> >>
>>> >> a maps to location 0 and exists from line 3 to line 9
>>> >> b maps to location 1 and exists from line 7 to line 9
>>> >> a maps to location 10 and exists from line 7 to 9?
>>> >>
>>> >> and that to map variables to locations on any particular line, the
>>> >> compiler works its way up the list,
>>> >
>>> >
>>> > Yes, more or less.
>>> >
>>> >>
>>> >>
>>> >> This is perhaps even more helpful than the environment. The
>>> >> environment is immediately and completely determinable at any
>>> >> point in the
>>> >> program. This could make it possible to walk back in time even
>>> >> within the
>>> >> same scope.
>>> >
>>> >
>>> > Could you expand on what you're thinking of?
>>> >
>>> > This kind of compile-time environment could conceivably be exposed
>>> > to
>>> > macros. Common Lisp had proposals along that line
>>> > (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but
>>> > as far as
>>> > I can tell, it was too complicated and not useful enough, so it was
>>> > axed/neutered at some point in the standardization process.
>>> >
>>> > > Hadley Wickham's lazyeval package in R is pretty cool in that you
>>> > > can
>>> > > attach an environment to an expression, pass it in and out of
>>> > > functions with
>>> > > various modifications, and then evaluate the expression within
>>> > > the original
>>> > > environment
>>> >
>>> > I don't know about R, but to me that sounds entirely doable with
>>> > closures (and macros will give you a nice syntax for it)
>>> >
>>> >>
>>> >>
>>> >> On Wednesday,

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Dicts seem to work pretty well for this kind of thing.

On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>
> I'm getting a cannot assign variables in other modules error.
>
> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>
>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
>>  wrote: 
>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
>> possible 
>> > to hijack this functionality to provide pseudo-environments? That is, 
>> never 
>> > referring to anything that is not already in an explicit module? And 
>> also, 
>> > have a data-frame simply be a module? 
>>
>> I think this would in principle works. A module is basically what 
>> global scope means so all the performance concern applies. 
>>
>> > 
>> > 
>> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote: 
>> >> 
>> >> I don't know if you came across the vignette? 
>> >> 
>> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html 
>> ? 
>> >> dplyr uses lazyeval extensively, see 
>> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . The 
>> cool 
>> >> thing about being able to incorporate this kind of thing in Julia 
>> would be 
>> >> being able to use the self-reflection capabilities. 
>> >> 
>> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote: 
>> >>> 
>> >>> 
>> >>> 
>> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote: 
>>  
>>  To walk back in time, you could say something like: compile this 
>> like 
>>  this was is in line 8. Or compile this like this was in line 5. It 
>> seems 
>>  like Julia already has some of this functionality in macros. 
>> Internal 
>>  variables are compiled as if they were in local scope. But escaped 
>>  expressions are compiled as if they were in global scope. 
>> >>> 
>> >>> 
>> >>> Could you provide context or a real-world use? I've looked at the 
>> >>> lazyeval package, and I'm not entirely sure what it does. Does it 
>> provide 
>> >>> lazy evaluation for R? That's easy to achieve in Julia (well, sorta). 
>> >>> Instead of 
>> >>> 
>> >>> d = determinant(matrix) 
>> >>>  
>> >>> u = 2 * d 
>> >>> 
>> >>> you can write 
>> >>> 
>> >>> d = ()->determinant(matrix) 
>> >>>  
>> >>> u = 2 * d() # determinant is evaluated on use, in the context where 
>> it 
>> >>> was originally defined 
>> >>> 
>> >>> With macros this can turn into 
>> >>> 
>> >>> d = lazy(determinant(matrix)) 
>> >>> 
>> >>> which looks nicer (and also can avoid computing the determinant twice 
>> if 
>> >>> d() is called twice). 
>> >>> 
>> >>> Cédric 
>> >>> 
>>  
>>  
>>  On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote: 
>> > 
>> > 
>> > 
>> > On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor 
>> wrote: 
>> >> 
>> >> Ok, here's where I'm getting hung up. You said that the compiler 
>> >> figures out the creation/lifetime of all variables at compile 
>> time. So does 
>> >> that mean there's a list like: 
>> >> 
>> >> a maps to location 0 and exists from line 3 to line 9 
>> >> b maps to location 1 and exists from line 7 to line 9 
>> >> a maps to location 10 and exists from line 7 to 9? 
>> >> 
>> >> and that to map variables to locations on any particular line, the 
>> >> compiler works its way up the list, 
>> > 
>> > 
>> > Yes, more or less. 
>> > 
>> >> 
>> >> 
>> >> This is perhaps even more helpful than the environment. The 
>> >> environment is immediately and completely determinable at any 
>> point in the 
>> >> program. This could make it possible to walk back in time even 
>> within the 
>> >> same scope. 
>> > 
>> > 
>> > Could you expand on what you're thinking of? 
>> > 
>> > This kind of compile-time environment could conceivably be exposed 
>> to 
>> > macros. Common Lisp had proposals along that line 
>> > (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but 
>> as far as 
>> > I can tell, it was too complicated and not useful enough, so it was 
>> > axed/neutered at some point in the standardization process. 
>> > 
>> > > Hadley Wickham's lazyeval package in R is pretty cool in that you 
>> can 
>> > > attach an environment to an expression, pass it in and out of 
>> functions with 
>> > > various modifications, and then evaluate the expression within 
>> the original 
>> > > environment 
>> > 
>> > I don't know about R, but to me that sounds entirely doable with 
>> > closures (and macros will give you a nice syntax for it) 
>> > 
>> >> 
>> >> 
>> >> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote: 
>> >>> 
>> >>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  
>> wrote: 
>> >>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>> >>> >  wrote: 
>> >>> >> Hmm, maybe I'

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

I'm getting a cannot assign variables in other modules error.

On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>
> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor 
> > wrote: 
> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be 
> possible 
> > to hijack this functionality to provide pseudo-environments? That is, 
> never 
> > referring to anything that is not already in an explicit module? And 
> also, 
> > have a data-frame simply be a module? 
>
> I think this would in principle works. A module is basically what 
> global scope means so all the performance concern applies. 
>
> > 
> > 
> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote: 
> >> 
> >> I don't know if you came across the vignette? 
> >> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html 
> ? 
> >> dplyr uses lazyeval extensively, see 
> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . The 
> cool 
> >> thing about being able to incorporate this kind of thing in Julia would 
> be 
> >> being able to use the self-reflection capabilities. 
> >> 
> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote: 
> >>> 
> >>> 
> >>> 
> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote: 
>  
>  To walk back in time, you could say something like: compile this like 
>  this was is in line 8. Or compile this like this was in line 5. It 
> seems 
>  like Julia already has some of this functionality in macros. Internal 
>  variables are compiled as if they were in local scope. But escaped 
>  expressions are compiled as if they were in global scope. 
> >>> 
> >>> 
> >>> Could you provide context or a real-world use? I've looked at the 
> >>> lazyeval package, and I'm not entirely sure what it does. Does it 
> provide 
> >>> lazy evaluation for R? That's easy to achieve in Julia (well, sorta). 
> >>> Instead of 
> >>> 
> >>> d = determinant(matrix) 
> >>>  
> >>> u = 2 * d 
> >>> 
> >>> you can write 
> >>> 
> >>> d = ()->determinant(matrix) 
> >>>  
> >>> u = 2 * d() # determinant is evaluated on use, in the context where it 
> >>> was originally defined 
> >>> 
> >>> With macros this can turn into 
> >>> 
> >>> d = lazy(determinant(matrix)) 
> >>> 
> >>> which looks nicer (and also can avoid computing the determinant twice 
> if 
> >>> d() is called twice). 
> >>> 
> >>> Cédric 
> >>> 
>  
>  
>  On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote: 
> > 
> > 
> > 
> > On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote: 
> >> 
> >> Ok, here's where I'm getting hung up. You said that the compiler 
> >> figures out the creation/lifetime of all variables at compile time. 
> So does 
> >> that mean there's a list like: 
> >> 
> >> a maps to location 0 and exists from line 3 to line 9 
> >> b maps to location 1 and exists from line 7 to line 9 
> >> a maps to location 10 and exists from line 7 to 9? 
> >> 
> >> and that to map variables to locations on any particular line, the 
> >> compiler works its way up the list, 
> > 
> > 
> > Yes, more or less. 
> > 
> >> 
> >> 
> >> This is perhaps even more helpful than the environment. The 
> >> environment is immediately and completely determinable at any point 
> in the 
> >> program. This could make it possible to walk back in time even 
> within the 
> >> same scope. 
> > 
> > 
> > Could you expand on what you're thinking of? 
> > 
> > This kind of compile-time environment could conceivably be exposed 
> to 
> > macros. Common Lisp had proposals along that line 
> > (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but 
> as far as 
> > I can tell, it was too complicated and not useful enough, so it was 
> > axed/neutered at some point in the standardization process. 
> > 
> > > Hadley Wickham's lazyeval package in R is pretty cool in that you 
> can 
> > > attach an environment to an expression, pass it in and out of 
> functions with 
> > > various modifications, and then evaluate the expression within the 
> original 
> > > environment 
> > 
> > I don't know about R, but to me that sounds entirely doable with 
> > closures (and macros will give you a nice syntax for it) 
> > 
> >> 
> >> 
> >> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote: 
> >>> 
> >>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  
> wrote: 
> >>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
> >>> >  wrote: 
> >>> >> Hmm, maybe I'm confused about compilation vs interpretation. 
> Let 
> >>> >> me 
> >>> >> rephrase. Regardless of a how or when statement is evaluated, 
> it 
> >>> >> must have 
> >>> >> access at least to its parent environments to successfully 
> resolve 
> >>> >> a symbo

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor
 wrote:
> Ok, a thought, Julia has an inbuilt idea of a module. Would it be possible
> to hijack this functionality to provide pseudo-environments? That is, never
> referring to anything that is not already in an explicit module? And also,
> have a data-frame simply be a module?

I think this would in principle works. A module is basically what
global scope means so all the performance concern applies.

>
>
> On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote:
>>
>> I don't know if you came across the vignette?
>> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html ?
>> dplyr uses lazyeval extensively, see
>> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . The cool
>> thing about being able to incorporate this kind of thing in Julia would be
>> being able to use the self-reflection capabilities.
>>
>> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote:
>>>
>>>
>>>
>>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:

 To walk back in time, you could say something like: compile this like
 this was is in line 8. Or compile this like this was in line 5. It seems
 like Julia already has some of this functionality in macros. Internal
 variables are compiled as if they were in local scope. But escaped
 expressions are compiled as if they were in global scope.
>>>
>>>
>>> Could you provide context or a real-world use? I've looked at the
>>> lazyeval package, and I'm not entirely sure what it does. Does it provide
>>> lazy evaluation for R? That's easy to achieve in Julia (well, sorta).
>>> Instead of
>>>
>>> d = determinant(matrix)
>>> 
>>> u = 2 * d
>>>
>>> you can write
>>>
>>> d = ()->determinant(matrix)
>>> 
>>> u = 2 * d() # determinant is evaluated on use, in the context where it
>>> was originally defined
>>>
>>> With macros this can turn into
>>>
>>> d = lazy(determinant(matrix))
>>>
>>> which looks nicer (and also can avoid computing the determinant twice if
>>> d() is called twice).
>>>
>>> Cédric
>>>


 On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>
>
>
> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>>
>> Ok, here's where I'm getting hung up. You said that the compiler
>> figures out the creation/lifetime of all variables at compile time. So 
>> does
>> that mean there's a list like:
>>
>> a maps to location 0 and exists from line 3 to line 9
>> b maps to location 1 and exists from line 7 to line 9
>> a maps to location 10 and exists from line 7 to 9?
>>
>> and that to map variables to locations on any particular line, the
>> compiler works its way up the list,
>
>
> Yes, more or less.
>
>>
>>
>> This is perhaps even more helpful than the environment. The
>> environment is immediately and completely determinable at any point in 
>> the
>> program. This could make it possible to walk back in time even within the
>> same scope.
>
>
> Could you expand on what you're thinking of?
>
> This kind of compile-time environment could conceivably be exposed to
> macros. Common Lisp had proposals along that line
> (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far 
> as
> I can tell, it was too complicated and not useful enough, so it was
> axed/neutered at some point in the standardization process.
>
> > Hadley Wickham's lazyeval package in R is pretty cool in that you can
> > attach an environment to an expression, pass it in and out of functions 
> > with
> > various modifications, and then evaluate the expression within the 
> > original
> > environment
>
> I don't know about R, but to me that sounds entirely doable with
> closures (and macros will give you a nice syntax for it)
>
>>
>>
>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>>
>>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote:
>>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
>>> >  wrote:
>>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let
>>> >> me
>>> >> rephrase. Regardless of a how or when statement is evaluated, it
>>> >> must have
>>> >> access at least to its parent environments to successfully resolve
>>> >> a symbol.
>>>
>>> AFAIK, the only scope you can dynamically add variable to is the
>>> global scope. (This can be done with the `global` keyword or `eval`
>>> etc). The compiler figure out the creation/lifetime of all local
>>> variables (at compile time). Therefore, to access a variable in the
>>> parent scope:
>>>
>>> 1. If it's a global, then it need a runtime lookup/binding (the
>>> reason
>>> global are slow)
>>> 2. If it's in a parent non-global scope, t

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Ok, a thought, Julia has an inbuilt idea of a module. Would it be possible 
to hijack this functionality to provide pseudo-environments? That is, never 
referring to anything that is not already in an explicit module? And also, 
have a data-frame simply be a module?

On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor wrote:
>
> I don't know if you came across the vignette? 
> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html ? 
> dplyr uses lazyeval extensively, see 
> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . The 
> cool thing about being able to incorporate this kind of thing in Julia 
> would be being able to use the self-reflection capabilities.
>
> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote:
>>
>>
>>
>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>>>
>>> To walk back in time, you could say something like: compile this like 
>>> this was is in line 8. Or compile this like this was in line 5. It seems 
>>> like Julia already has some of this functionality in macros. Internal 
>>> variables are compiled as if they were in local scope. But escaped 
>>> expressions are compiled as if they were in global scope. 
>>>
>>
>> Could you provide context or a real-world use? I've looked at the 
>> lazyeval package, and I'm not entirely sure what it does. Does it provide 
>> lazy evaluation for R? That's easy to achieve in Julia (well, sorta). 
>> Instead of 
>>
>> d = determinant(matrix)
>> 
>> u = 2 * d
>>
>> you can write
>>
>> d = ()->determinant(matrix)
>> 
>> u = 2 * d() # determinant is evaluated on use, in the context where it 
>> was originally defined
>>
>> With macros this can turn into
>>
>> d = lazy(determinant(matrix))
>>
>> which looks nicer (and also can avoid computing the determinant twice if 
>> d() is called twice).
>>
>> Cédric
>>  
>>
>>>
>>> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:



 On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>
> Ok, here's where I'm getting hung up. You said that the compiler 
> figures out the creation/lifetime of all variables at compile time. So 
> does 
> that mean there's a list like:
>
> a maps to location 0 and exists from line 3 to line 9
> b maps to location 1 and exists from line 7 to line 9
> a maps to location 10 and exists from line 7 to 9?
>
> and that to map variables to locations on any particular line, the 
> compiler works its way up the list, 
>

 Yes, more or less.
  

>
> This is perhaps even more helpful than the environment. The 
> environment is immediately and completely determinable at any point in 
> the 
> program. This could make it possible to walk back in time even within the 
> same scope.
>

 Could you expand on what you're thinking of?

 This kind of compile-time environment could conceivably be exposed to 
 macros. Common Lisp had proposals along that line (
 https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as 
 far as I can tell, it was too complicated and not useful enough, so it was 
 axed/neutered at some point in the standardization process.
  
 > Hadley Wickham's lazyeval package in R is pretty cool in that you can 
 attach an environment to an expression, pass it in and out of functions 
 with various modifications, and then evaluate the expression within the 
 original environment

 I don't know about R, but to me that sounds entirely doable with 
 closures (and macros will give you a nice syntax for it)


>
> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>
>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>> >  wrote: 
>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let 
>> me 
>> >> rephrase. Regardless of a how or when statement is evaluated, it 
>> must have 
>> >> access at least to its parent environments to successfully resolve 
>> a symbol. 
>>
>> AFAIK, the only scope you can dynamically add variable to is the 
>> global scope. (This can be done with the `global` keyword or `eval` 
>> etc). The compiler figure out the creation/lifetime of all local 
>> variables (at compile time). Therefore, to access a variable in the 
>> parent scope: 
>>
>> 1. If it's a global, then it need a runtime lookup/binding (the 
>> reason 
>> global are slow) 
>> 2. If it's in a parent non-global scope, the compiler can figure out 
>> how to bind/access it at compile time and no extra (lookup) code at 
>> runtime is necessary. 
>>
>> >> 
>> > 
>> > A julia local variable is basically a variable in C. There's a 
>> table 
>> > at compile time to map between symbols 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

I don't know if you came across the vignette? 
http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html ? 
dplyr uses lazyeval extensively, 
see http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html . The 
cool thing about being able to incorporate this kind of thing in Julia 
would be being able to use the self-reflection capabilities.

On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean wrote:
>
>
>
> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>>
>> To walk back in time, you could say something like: compile this like 
>> this was is in line 8. Or compile this like this was in line 5. It seems 
>> like Julia already has some of this functionality in macros. Internal 
>> variables are compiled as if they were in local scope. But escaped 
>> expressions are compiled as if they were in global scope. 
>>
>
> Could you provide context or a real-world use? I've looked at the lazyeval 
> package, and I'm not entirely sure what it does. Does it provide lazy 
> evaluation for R? That's easy to achieve in Julia (well, sorta). Instead of 
>
> d = determinant(matrix)
> 
> u = 2 * d
>
> you can write
>
> d = ()->determinant(matrix)
> 
> u = 2 * d() # determinant is evaluated on use, in the context where it 
> was originally defined
>
> With macros this can turn into
>
> d = lazy(determinant(matrix))
>
> which looks nicer (and also can avoid computing the determinant twice if 
> d() is called twice).
>
> Cédric
>  
>
>>
>> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>>>
>>>
>>>
>>> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:

 Ok, here's where I'm getting hung up. You said that the compiler 
 figures out the creation/lifetime of all variables at compile time. So 
 does 
 that mean there's a list like:

 a maps to location 0 and exists from line 3 to line 9
 b maps to location 1 and exists from line 7 to line 9
 a maps to location 10 and exists from line 7 to 9?

 and that to map variables to locations on any particular line, the 
 compiler works its way up the list, 

>>>
>>> Yes, more or less.
>>>  
>>>

 This is perhaps even more helpful than the environment. The environment 
 is immediately and completely determinable at any point in the program. 
 This could make it possible to walk back in time even within the same 
 scope.

>>>
>>> Could you expand on what you're thinking of?
>>>
>>> This kind of compile-time environment could conceivably be exposed to 
>>> macros. Common Lisp had proposals along that line (
>>> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far 
>>> as I can tell, it was too complicated and not useful enough, so it was 
>>> axed/neutered at some point in the standardization process.
>>>  
>>> > Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>>> attach an environment to an expression, pass it in and out of functions 
>>> with various modifications, and then evaluate the expression within the 
>>> original environment
>>>
>>> I don't know about R, but to me that sounds entirely doable with 
>>> closures (and macros will give you a nice syntax for it)
>>>
>>>

 On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>
> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
> >  wrote: 
> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
> >> rephrase. Regardless of a how or when statement is evaluated, it 
> must have 
> >> access at least to its parent environments to successfully resolve 
> a symbol. 
>
> AFAIK, the only scope you can dynamically add variable to is the 
> global scope. (This can be done with the `global` keyword or `eval` 
> etc). The compiler figure out the creation/lifetime of all local 
> variables (at compile time). Therefore, to access a variable in the 
> parent scope: 
>
> 1. If it's a global, then it need a runtime lookup/binding (the reason 
> global are slow) 
> 2. If it's in a parent non-global scope, the compiler can figure out 
> how to bind/access it at compile time and no extra (lookup) code at 
> runtime is necessary. 
>
> >> 
> > 
> > A julia local variable is basically a variable in C. There's a table 
> > at compile time to map between symbols and stack slots (or whereever 
> > they are stored) but such a map does not exist at runtime anymore 
> > (except for debugging). 
> > 
> >> 
> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor 
> wrote: 
> >>> 
> >>> They must exist at runtime and at local scope. Evaluating a symbol 
> is 
> >>> impossible without a pool of defined symbols in various scopes to 
> match it 
> >>> to. Unless I'm missing something? 
> >>> 
> >>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Cedric St-Jean]

On Friday, July 10, 2015 at 5:51:05 AM UTC-4, Fabian Gans wrote:
>
> Hi Cedric, 
>
> a macroexpand shortcut for the REPL shouldn't be a big problem, I just 
> tried to prototype one and it seems to work fine: 
> https://github.com/meggart/julia/commit/73c96897572cedf10f74fd09907232c706bbdf48
>
> Is there any special functionality/problems which I don't see that should 
> be considered?
>

Cool, thanks! I don't build from source, but I look forward to that support 
in the future.

You can see how SLIME (the Common Lisp emacs support library) does it here 
.
 
I use mostly IJulia these days, and I don't see a clean mechanism for 
integrating it in there, but Jupyter is moving at a steady pace, so maybe 
in a few releases.

The main use for Emacs' macroexpand is to understand how a complex macro 
works in a given piece of code, and to debug new macros. The ability to 
selectively macroexpand and contract specific portions of the tree is 
important.

Best,

Cédric
 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Cedric St-Jean]

On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>
> To walk back in time, you could say something like: compile this like this 
> was is in line 8. Or compile this like this was in line 5. It seems like 
> Julia already has some of this functionality in macros. Internal variables 
> are compiled as if they were in local scope. But escaped expressions are 
> compiled as if they were in global scope. 
>

Could you provide context or a real-world use? I've looked at the lazyeval 
package, and I'm not entirely sure what it does. Does it provide lazy 
evaluation for R? That's easy to achieve in Julia (well, sorta). Instead of 

d = determinant(matrix)

u = 2 * d

you can write

d = ()->determinant(matrix)

u = 2 * d() # determinant is evaluated on use, in the context where it was 
originally defined

With macros this can turn into

d = lazy(determinant(matrix))

which looks nicer (and also can avoid computing the determinant twice if 
d() is called twice).

Cédric
 

>
> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>>
>>
>>
>> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>>>
>>> Ok, here's where I'm getting hung up. You said that the compiler figures 
>>> out the creation/lifetime of all variables at compile time. So does that 
>>> mean there's a list like:
>>>
>>> a maps to location 0 and exists from line 3 to line 9
>>> b maps to location 1 and exists from line 7 to line 9
>>> a maps to location 10 and exists from line 7 to 9?
>>>
>>> and that to map variables to locations on any particular line, the 
>>> compiler works its way up the list, 
>>>
>>
>> Yes, more or less.
>>  
>>
>>>
>>> This is perhaps even more helpful than the environment. The environment 
>>> is immediately and completely determinable at any point in the program. 
>>> This could make it possible to walk back in time even within the same scope.
>>>
>>
>> Could you expand on what you're thinking of?
>>
>> This kind of compile-time environment could conceivably be exposed to 
>> macros. Common Lisp had proposals along that line (
>> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far 
>> as I can tell, it was too complicated and not useful enough, so it was 
>> axed/neutered at some point in the standardization process.
>>  
>> > Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>> attach an environment to an expression, pass it in and out of functions 
>> with various modifications, and then evaluate the expression within the 
>> original environment
>>
>> I don't know about R, but to me that sounds entirely doable with closures 
>> (and macros will give you a nice syntax for it)
>>
>>
>>>
>>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:

 On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
 > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
 >  wrote: 
 >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
 >> rephrase. Regardless of a how or when statement is evaluated, it 
 must have 
 >> access at least to its parent environments to successfully resolve a 
 symbol. 

 AFAIK, the only scope you can dynamically add variable to is the 
 global scope. (This can be done with the `global` keyword or `eval` 
 etc). The compiler figure out the creation/lifetime of all local 
 variables (at compile time). Therefore, to access a variable in the 
 parent scope: 

 1. If it's a global, then it need a runtime lookup/binding (the reason 
 global are slow) 
 2. If it's in a parent non-global scope, the compiler can figure out 
 how to bind/access it at compile time and no extra (lookup) code at 
 runtime is necessary. 

 >> 
 > 
 > A julia local variable is basically a variable in C. There's a table 
 > at compile time to map between symbols and stack slots (or whereever 
 > they are stored) but such a map does not exist at runtime anymore 
 > (except for debugging). 
 > 
 >> 
 >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor 
 wrote: 
 >>> 
 >>> They must exist at runtime and at local scope. Evaluating a symbol 
 is 
 >>> impossible without a pool of defined symbols in various scopes to 
 match it 
 >>> to. Unless I'm missing something? 
 >>> 
 >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
  
  There are global symbol tables for static analysis / reflection, 
 but they 
  do not exist at runtime or for the local scope. 
  
  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor <
 brandon@gmail.com> 
  wrote: 
 > 
 > Surely environments already exist somewhere inside Julia? How 
 else could 
 > you keep track of scope? It would be simply a matter of granting 
 users 
 > access to them. Symbol tables in a mutable 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Isaiah Norton]

Right now the best resources to learn how the compiler works are the
developer docs and Jeff's talk at JuliaCon last year.

http://docs.julialang.org/en/latest/#developer-documentation
https://www.youtube.com/watch?v=osdeT-tWjzk

 (there were several more compiler-related talks this year; hopefully the
recordings will be posted soon)

On Fri, Jul 10, 2015 at 10:07 AM, Brandon Taylor <
brandon.taylor...@gmail.com> wrote:

> Hmm, so the reason that macros are able to function is that they occur
> before type computation? One solution might be to make scope annotations
> that override defaults. Then, when eval is called, it optionally attaches
> scope annotations to a given scope. Note that scope annotations should be
> able to refer not only to regular scopes but also indexed items like dicts
> and dataframes. Wait, how does eval handle type computation if it occurs
> after type computation?
>
>
> On Friday, July 10, 2015 at 5:51:05 AM UTC-4, Fabian Gans wrote:
>>
>> Hi Cedric,
>>
>> a macroexpand shortcut for the REPL shouldn't be a big problem, I just
>> tried to prototype one and it seems to work fine:
>> https://github.com/meggart/julia/commit/73c96897572cedf10f74fd09907232c706bbdf48
>>
>> Is there any special functionality/problems which I don't see that should
>> be considered?
>>
>>
>>
>>
>>

Re: [julia-users] Re: Environment reification and lazy evaluation

[David Gold]

>
> Hmm, so the reason that macros are able to function is that they occur 
> before type computation?


I don't know how best to respond to this question, but I will throw out 
that staged functions are expanded after type information is known to the 
compiler. 

On Friday, July 10, 2015 at 10:07:23 AM UTC-4, Brandon Taylor wrote:
>
> Hmm, so the reason that macros are able to function is that they occur 
> before type computation? One solution might be to make scope annotations 
> that override defaults. Then, when eval is called, it optionally attaches 
> scope annotations to a given scope. Note that scope annotations should be 
> able to refer not only to regular scopes but also indexed items like dicts 
> and dataframes. Wait, how does eval handle type computation if it occurs 
> after type computation?
>
> On Friday, July 10, 2015 at 5:51:05 AM UTC-4, Fabian Gans wrote:
>>
>> Hi Cedric, 
>>
>> a macroexpand shortcut for the REPL shouldn't be a big problem, I just 
>> tried to prototype one and it seems to work fine: 
>> https://github.com/meggart/julia/commit/73c96897572cedf10f74fd09907232c706bbdf48
>>
>> Is there any special functionality/problems which I don't see that should 
>> be considered?
>>
>>
>>
>>
>>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Hmm, so the reason that macros are able to function is that they occur 
before type computation? One solution might be to make scope annotations 
that override defaults. Then, when eval is called, it optionally attaches 
scope annotations to a given scope. Note that scope annotations should be 
able to refer not only to regular scopes but also indexed items like dicts 
and dataframes. Wait, how does eval handle type computation if it occurs 
after type computation?

On Friday, July 10, 2015 at 5:51:05 AM UTC-4, Fabian Gans wrote:
>
> Hi Cedric, 
>
> a macroexpand shortcut for the REPL shouldn't be a big problem, I just 
> tried to prototype one and it seems to work fine: 
> https://github.com/meggart/julia/commit/73c96897572cedf10f74fd09907232c706bbdf48
>
> Is there any special functionality/problems which I don't see that should 
> be considered?
>
>
>
>
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Fabian Gans]

Hi Cedric, 

a macroexpand shortcut for the REPL shouldn't be a big problem, I just 
tried to prototype one and it seems to work fine: 
https://github.com/meggart/julia/commit/73c96897572cedf10f74fd09907232c706bbdf48

Is there any special functionality/problems which I don't see that should 
be considered?

Re: [julia-users] Re: Environment reification and lazy evaluation

[Jameson Nash]

gc optimizations aren't really the critical issue (although it is a
possible side-benefit). the big advantages are enforced type computability
(they cannot be accessed in ways that are invisible to inference) and
semantic purity (the variables are exactly those you see in the program,
plus any spliced in from a macro).

it's not quite right to consider this a compiler phase either, since the
identification of a variable's scope is actually done at parse time (just
after splicing in the return values from any macro).

i agree that macros are confusing when encountering them for the first
time. it sometimes helps to view them as functions that take code syntax as
arguments and return modified code syntax. that return value is then
spliced into the original code exactly as if it was the expression that you
typed originally. it is also sometimes also helpful to note that this code
transformation happens very early, just after parsing the code, and long
before any variables exist or any code has been executed.


> On Friday, July 10, 2015 at 4:38:55 AM UTC+2, Brandon Taylor wrote:
>>
>> Is the issue garbage collection? That because by choosing variables
>> dynamically from a symbol table, you don't know which variables are going
>> to be used, so you don't know which data can be deleted early?
>>
>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>>>
>>> To walk back in time, you could say something like: compile this like
>>> this was is in line 8. Or compile this like this was in line 5. It seems
>>> like Julia already has some of this functionality in macros. Internal
>>> variables are compiled as if they were in local scope. But escaped
>>> expressions are compiled as if they were in global scope. Or maybe not,
>>> macros still confuse me.
>>>
>>> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:



 On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>
> Ok, here's where I'm getting hung up. You said that the compiler
> figures out the creation/lifetime of all variables at compile time. So 
> does
> that mean there's a list like:
>
> a maps to location 0 and exists from line 3 to line 9
> b maps to location 1 and exists from line 7 to line 9
> a maps to location 10 and exists from line 7 to 9?
>
> and that to map variables to locations on any particular line, the
> compiler works its way up the list,
>

 Yes, more or less.


>
> This is perhaps even more helpful than the environment. The
> environment is immediately and completely determinable at any point in the
> program. This could make it possible to walk back in time even within the
> same scope.
>

 Could you expand on what you're thinking of?

 This kind of compile-time environment could conceivably be exposed to
 macros. Common Lisp had proposals along that line (
 https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as
 far as I can tell, it was too complicated and not useful enough, so it was
 axed/neutered at some point in the standardization process.

 > Hadley Wickham's lazyeval package in R is pretty cool in that you can
 attach an environment to an expression, pass it in and out of functions
 with various modifications, and then evaluate the expression within the
 original environment

 I don't know about R, but to me that sounds entirely doable with
 closures (and macros will give you a nice syntax for it)


>
> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>
>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote:
>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
>> >  wrote:
>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let
>> me
>> >> rephrase. Regardless of a how or when statement is evaluated, it
>> must have
>> >> access at least to its parent environments to successfully resolve
>> a symbol.
>>
>> AFAIK, the only scope you can dynamically add variable to is the
>> global scope. (This can be done with the `global` keyword or `eval`
>> etc). The compiler figure out the creation/lifetime of all local
>> variables (at compile time). Therefore, to access a variable in the
>> parent scope:
>>
>> 1. If it's a global, then it need a runtime lookup/binding (the
>> reason
>> global are slow)
>> 2. If it's in a parent non-global scope, the compiler can figure out
>> how to bind/access it at compile time and no extra (lookup) code at
>> runtime is necessary.
>>
>> >>
>> >
>> > A julia local variable is basically a variable in C. There's a
>> table
>> > at compile time to map between symbols and stack slots (or
>> whereever
>> > they are stored) but such a map does not exist at runtime anymore
>> > (except for debugging).
>> 

Re: [julia-users] Re: Environment reification and lazy evaluation

[John Myles White]

Instead of speculating about how Julia's compiler handles scope, perhaps it 
would be more fruitful to read the source code and learn how things 
currently work?

Better still: why not spend some time trying to implement your feature 
request? If you can manage to implement scope reification without any 
performance degradations for any existing code, I think your work will be 
extremely well received

In general, a working prototype is a sine qua non for this kind of sweeping 
redesign to be given serious consideration.

 -- John

On Friday, July 10, 2015 at 4:38:55 AM UTC+2, Brandon Taylor wrote:
>
> Is the issue garbage collection? That because by choosing variables 
> dynamically from a symbol table, you don't know which variables are going 
> to be used, so you don't know which data can be deleted early?
>
> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>>
>> To walk back in time, you could say something like: compile this like 
>> this was is in line 8. Or compile this like this was in line 5. It seems 
>> like Julia already has some of this functionality in macros. Internal 
>> variables are compiled as if they were in local scope. But escaped 
>> expressions are compiled as if they were in global scope. Or maybe not, 
>> macros still confuse me.
>>
>> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>>>
>>>
>>>
>>> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:

 Ok, here's where I'm getting hung up. You said that the compiler 
 figures out the creation/lifetime of all variables at compile time. So 
 does 
 that mean there's a list like:

 a maps to location 0 and exists from line 3 to line 9
 b maps to location 1 and exists from line 7 to line 9
 a maps to location 10 and exists from line 7 to 9?

 and that to map variables to locations on any particular line, the 
 compiler works its way up the list, 

>>>
>>> Yes, more or less.
>>>  
>>>

 This is perhaps even more helpful than the environment. The environment 
 is immediately and completely determinable at any point in the program. 
 This could make it possible to walk back in time even within the same 
 scope.

>>>
>>> Could you expand on what you're thinking of?
>>>
>>> This kind of compile-time environment could conceivably be exposed to 
>>> macros. Common Lisp had proposals along that line (
>>> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far 
>>> as I can tell, it was too complicated and not useful enough, so it was 
>>> axed/neutered at some point in the standardization process.
>>>  
>>> > Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>>> attach an environment to an expression, pass it in and out of functions 
>>> with various modifications, and then evaluate the expression within the 
>>> original environment
>>>
>>> I don't know about R, but to me that sounds entirely doable with 
>>> closures (and macros will give you a nice syntax for it)
>>>
>>>

 On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>
> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
> >  wrote: 
> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
> >> rephrase. Regardless of a how or when statement is evaluated, it 
> must have 
> >> access at least to its parent environments to successfully resolve 
> a symbol. 
>
> AFAIK, the only scope you can dynamically add variable to is the 
> global scope. (This can be done with the `global` keyword or `eval` 
> etc). The compiler figure out the creation/lifetime of all local 
> variables (at compile time). Therefore, to access a variable in the 
> parent scope: 
>
> 1. If it's a global, then it need a runtime lookup/binding (the reason 
> global are slow) 
> 2. If it's in a parent non-global scope, the compiler can figure out 
> how to bind/access it at compile time and no extra (lookup) code at 
> runtime is necessary. 
>
> >> 
> > 
> > A julia local variable is basically a variable in C. There's a table 
> > at compile time to map between symbols and stack slots (or whereever 
> > they are stored) but such a map does not exist at runtime anymore 
> > (except for debugging). 
> > 
> >> 
> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor 
> wrote: 
> >>> 
> >>> They must exist at runtime and at local scope. Evaluating a symbol 
> is 
> >>> impossible without a pool of defined symbols in various scopes to 
> match it 
> >>> to. Unless I'm missing something? 
> >>> 
> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>  
>  There are global symbol tables for static analysis / reflection, 
> but they 
>  do not exist at run

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Is the issue garbage collection? That because by choosing variables 
dynamically from a symbol table, you don't know which variables are going 
to be used, so you don't know which data can be deleted early?

On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor wrote:
>
> To walk back in time, you could say something like: compile this like this 
> was is in line 8. Or compile this like this was in line 5. It seems like 
> Julia already has some of this functionality in macros. Internal variables 
> are compiled as if they were in local scope. But escaped expressions are 
> compiled as if they were in global scope. Or maybe not, macros still 
> confuse me.
>
> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>>
>>
>>
>> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>>>
>>> Ok, here's where I'm getting hung up. You said that the compiler figures 
>>> out the creation/lifetime of all variables at compile time. So does that 
>>> mean there's a list like:
>>>
>>> a maps to location 0 and exists from line 3 to line 9
>>> b maps to location 1 and exists from line 7 to line 9
>>> a maps to location 10 and exists from line 7 to 9?
>>>
>>> and that to map variables to locations on any particular line, the 
>>> compiler works its way up the list, 
>>>
>>
>> Yes, more or less.
>>  
>>
>>>
>>> This is perhaps even more helpful than the environment. The environment 
>>> is immediately and completely determinable at any point in the program. 
>>> This could make it possible to walk back in time even within the same scope.
>>>
>>
>> Could you expand on what you're thinking of?
>>
>> This kind of compile-time environment could conceivably be exposed to 
>> macros. Common Lisp had proposals along that line (
>> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far 
>> as I can tell, it was too complicated and not useful enough, so it was 
>> axed/neutered at some point in the standardization process.
>>  
>> > Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>> attach an environment to an expression, pass it in and out of functions 
>> with various modifications, and then evaluate the expression within the 
>> original environment
>>
>> I don't know about R, but to me that sounds entirely doable with closures 
>> (and macros will give you a nice syntax for it)
>>
>>
>>>
>>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:

 On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
 > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
 >  wrote: 
 >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
 >> rephrase. Regardless of a how or when statement is evaluated, it 
 must have 
 >> access at least to its parent environments to successfully resolve a 
 symbol. 

 AFAIK, the only scope you can dynamically add variable to is the 
 global scope. (This can be done with the `global` keyword or `eval` 
 etc). The compiler figure out the creation/lifetime of all local 
 variables (at compile time). Therefore, to access a variable in the 
 parent scope: 

 1. If it's a global, then it need a runtime lookup/binding (the reason 
 global are slow) 
 2. If it's in a parent non-global scope, the compiler can figure out 
 how to bind/access it at compile time and no extra (lookup) code at 
 runtime is necessary. 

 >> 
 > 
 > A julia local variable is basically a variable in C. There's a table 
 > at compile time to map between symbols and stack slots (or whereever 
 > they are stored) but such a map does not exist at runtime anymore 
 > (except for debugging). 
 > 
 >> 
 >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor 
 wrote: 
 >>> 
 >>> They must exist at runtime and at local scope. Evaluating a symbol 
 is 
 >>> impossible without a pool of defined symbols in various scopes to 
 match it 
 >>> to. Unless I'm missing something? 
 >>> 
 >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
  
  There are global symbol tables for static analysis / reflection, 
 but they 
  do not exist at runtime or for the local scope. 
  
  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor <
 brandon@gmail.com> 
  wrote: 
 > 
 > Surely environments already exist somewhere inside Julia? How 
 else could 
 > you keep track of scope? It would be simply a matter of granting 
 users 
 > access to them. Symbol tables in a mutable language are by 
 default mutable. 
 > It would certainly be possible only give users access to 
 immutable 
 > reifications (which could solve a bunch of problems as is). 
 However, it 
 > seems natural to match mutable symbol tables with mutable 
 reifications, and 
 > immutable symbo

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

To walk back in time, you could say something like: compile this like this 
was is in line 8. Or compile this like this was in line 5. It seems like 
Julia already has some of this functionality in macros. Internal variables 
are compiled as if they were in local scope. But escaped expressions are 
compiled as if they were in global scope. Or maybe not, macros still 
confuse me.

On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean wrote:
>
>
>
> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>>
>> Ok, here's where I'm getting hung up. You said that the compiler figures 
>> out the creation/lifetime of all variables at compile time. So does that 
>> mean there's a list like:
>>
>> a maps to location 0 and exists from line 3 to line 9
>> b maps to location 1 and exists from line 7 to line 9
>> a maps to location 10 and exists from line 7 to 9?
>>
>> and that to map variables to locations on any particular line, the 
>> compiler works its way up the list, 
>>
>
> Yes, more or less.
>  
>
>>
>> This is perhaps even more helpful than the environment. The environment 
>> is immediately and completely determinable at any point in the program. 
>> This could make it possible to walk back in time even within the same scope.
>>
>
> Could you expand on what you're thinking of?
>
> This kind of compile-time environment could conceivably be exposed to 
> macros. Common Lisp had proposals along that line (
> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far 
> as I can tell, it was too complicated and not useful enough, so it was 
> axed/neutered at some point in the standardization process.
>  
> > Hadley Wickham's lazyeval package in R is pretty cool in that you can 
> attach an environment to an expression, pass it in and out of functions 
> with various modifications, and then evaluate the expression within the 
> original environment
>
> I don't know about R, but to me that sounds entirely doable with closures 
> (and macros will give you a nice syntax for it)
>
>
>>
>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>>
>>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>>> >  wrote: 
>>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
>>> >> rephrase. Regardless of a how or when statement is evaluated, it must 
>>> have 
>>> >> access at least to its parent environments to successfully resolve a 
>>> symbol. 
>>>
>>> AFAIK, the only scope you can dynamically add variable to is the 
>>> global scope. (This can be done with the `global` keyword or `eval` 
>>> etc). The compiler figure out the creation/lifetime of all local 
>>> variables (at compile time). Therefore, to access a variable in the 
>>> parent scope: 
>>>
>>> 1. If it's a global, then it need a runtime lookup/binding (the reason 
>>> global are slow) 
>>> 2. If it's in a parent non-global scope, the compiler can figure out 
>>> how to bind/access it at compile time and no extra (lookup) code at 
>>> runtime is necessary. 
>>>
>>> >> 
>>> > 
>>> > A julia local variable is basically a variable in C. There's a table 
>>> > at compile time to map between symbols and stack slots (or whereever 
>>> > they are stored) but such a map does not exist at runtime anymore 
>>> > (except for debugging). 
>>> > 
>>> >> 
>>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
>>> >>> 
>>> >>> They must exist at runtime and at local scope. Evaluating a symbol 
>>> is 
>>> >>> impossible without a pool of defined symbols in various scopes to 
>>> match it 
>>> >>> to. Unless I'm missing something? 
>>> >>> 
>>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>>>  
>>>  There are global symbol tables for static analysis / reflection, 
>>> but they 
>>>  do not exist at runtime or for the local scope. 
>>>  
>>>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor <
>>> brandon@gmail.com> 
>>>  wrote: 
>>> > 
>>> > Surely environments already exist somewhere inside Julia? How else 
>>> could 
>>> > you keep track of scope? It would be simply a matter of granting 
>>> users 
>>> > access to them. Symbol tables in a mutable language are by default 
>>> mutable. 
>>> > It would certainly be possible only give users access to immutable 
>>> > reifications (which could solve a bunch of problems as is). 
>>> However, it 
>>> > seems natural to match mutable symbol tables with mutable 
>>> reifications, and 
>>> > immutable symbol tables with immutable reifications. 
>>> > 
>>> > 
>>> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
>>> wrote: 
>>> >> 
>>> >> I'm not sure I understand... 
>>> >> 
>>> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
>>> wrote: 
>>> >>> 
>>> >>> Reified scope makes static analysis much too hard. Take any 
>>> criticism 
>>> >>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Cedric St-Jean]

On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor wrote:
>
> Ok, here's where I'm getting hung up. You said that the compiler figures 
> out the creation/lifetime of all variables at compile time. So does that 
> mean there's a list like:
>
> a maps to location 0 and exists from line 3 to line 9
> b maps to location 1 and exists from line 7 to line 9
> a maps to location 10 and exists from line 7 to 9?
>
> and that to map variables to locations on any particular line, the 
> compiler works its way up the list, 
>

Yes, more or less.
 

>
> This is perhaps even more helpful than the environment. The environment is 
> immediately and completely determinable at any point in the program. This 
> could make it possible to walk back in time even within the same scope.
>

Could you expand on what you're thinking of?

This kind of compile-time environment could conceivably be exposed to 
macros. Common Lisp had proposals along that line (
https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html) but as far as 
I can tell, it was too complicated and not useful enough, so it was 
axed/neutered at some point in the standardization process.
 
> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
attach an environment to an expression, pass it in and out of functions 
with various modifications, and then evaluate the expression within the 
original environment

I don't know about R, but to me that sounds entirely doable with closures 
(and macros will give you a nice syntax for it)


>
> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>
>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>> >  wrote: 
>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
>> >> rephrase. Regardless of a how or when statement is evaluated, it must 
>> have 
>> >> access at least to its parent environments to successfully resolve a 
>> symbol. 
>>
>> AFAIK, the only scope you can dynamically add variable to is the 
>> global scope. (This can be done with the `global` keyword or `eval` 
>> etc). The compiler figure out the creation/lifetime of all local 
>> variables (at compile time). Therefore, to access a variable in the 
>> parent scope: 
>>
>> 1. If it's a global, then it need a runtime lookup/binding (the reason 
>> global are slow) 
>> 2. If it's in a parent non-global scope, the compiler can figure out 
>> how to bind/access it at compile time and no extra (lookup) code at 
>> runtime is necessary. 
>>
>> >> 
>> > 
>> > A julia local variable is basically a variable in C. There's a table 
>> > at compile time to map between symbols and stack slots (or whereever 
>> > they are stored) but such a map does not exist at runtime anymore 
>> > (except for debugging). 
>> > 
>> >> 
>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
>> >>> 
>> >>> They must exist at runtime and at local scope. Evaluating a symbol is 
>> >>> impossible without a pool of defined symbols in various scopes to 
>> match it 
>> >>> to. Unless I'm missing something? 
>> >>> 
>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>>  
>>  There are global symbol tables for static analysis / reflection, but 
>> they 
>>  do not exist at runtime or for the local scope. 
>>  
>>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor  
>>
>>  wrote: 
>> > 
>> > Surely environments already exist somewhere inside Julia? How else 
>> could 
>> > you keep track of scope? It would be simply a matter of granting 
>> users 
>> > access to them. Symbol tables in a mutable language are by default 
>> mutable. 
>> > It would certainly be possible only give users access to immutable 
>> > reifications (which could solve a bunch of problems as is). 
>> However, it 
>> > seems natural to match mutable symbol tables with mutable 
>> reifications, and 
>> > immutable symbol tables with immutable reifications. 
>> > 
>> > 
>> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
>> wrote: 
>> >> 
>> >> I'm not sure I understand... 
>> >> 
>> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
>> wrote: 
>> >>> 
>> >>> Reified scope makes static analysis much too hard. Take any 
>> criticism 
>> >>> of mutable state: they all apply to globally mutable symbol 
>> tables. 
>> >>> 
>> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
>> Bouchet-Valat 
>> >>> wrote: 
>>  
>>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a 
>> écrit : 
>>  > All functions. 
>>  Well, I don't know of any language which doesn't have scoping 
>>  rules... 
>>  
>>  Anyway, I didn't say scoping rules are necessarily confusing, I 
>> was 
>>  only referring to R formulas. But according to the examples you 
>>  posted, 
>> 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Unless it did, in which case, I'm just uneducated.

On Thursday, July 9, 2015 at 6:31:03 PM UTC-4, Brandon Taylor wrote:
>
> Ok, that was an interesting article, but it didn't really answer my 
> question.
>
> On Thursday, July 9, 2015 at 4:20:45 PM UTC-4, Isaiah wrote:
>>
>> https://en.wikipedia.org/wiki/Variable_(computer_science)#Scope_and_extent
>>
>> On Thu, Jul 9, 2015 at 4:14 PM, Brandon Taylor  
>> wrote:
>>
>>> Ok, here's where I'm getting hung up. You said that the compiler figures 
>>> out the creation/lifetime of all variables at compile time. So does that 
>>> mean there's a list like:
>>>
>>> a maps to location 0 and exists from line 3 to line 9
>>> b maps to location 1 and exists from line 7 to line 9
>>> a maps to location 10 and exists from line 7 to 9?
>>>
>>> and that to map variables to locations on any particular line, the 
>>> compiler works its way up the list, 
>>>
>>> This is perhaps even more helpful than the environment. The environment 
>>> is immediately and completely determinable at any point in the program. 
>>> This could make it possible to walk back in time even within the same scope.
>>>
>>>
>>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:

 On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
 > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
 >  wrote: 
 >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
 >> rephrase. Regardless of a how or when statement is evaluated, it 
 must have 
 >> access at least to its parent environments to successfully resolve a 
 symbol. 

 AFAIK, the only scope you can dynamically add variable to is the 
 global scope. (This can be done with the `global` keyword or `eval` 
 etc). The compiler figure out the creation/lifetime of all local 
 variables (at compile time). Therefore, to access a variable in the 
 parent scope: 

 1. If it's a global, then it need a runtime lookup/binding (the reason 
 global are slow) 
 2. If it's in a parent non-global scope, the compiler can figure out 
 how to bind/access it at compile time and no extra (lookup) code at 
 runtime is necessary. 

 >> 
 > 
 > A julia local variable is basically a variable in C. There's a table 
 > at compile time to map between symbols and stack slots (or whereever 
 > they are stored) but such a map does not exist at runtime anymore 
 > (except for debugging). 
 > 
 >> 
 >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor 
 wrote: 
 >>> 
 >>> They must exist at runtime and at local scope. Evaluating a symbol 
 is 
 >>> impossible without a pool of defined symbols in various scopes to 
 match it 
 >>> to. Unless I'm missing something? 
 >>> 
 >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
  
  There are global symbol tables for static analysis / reflection, 
 but they 
  do not exist at runtime or for the local scope. 
  
  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor <
 brandon@gmail.com> 
  wrote: 
 > 
 > Surely environments already exist somewhere inside Julia? How 
 else could 
 > you keep track of scope? It would be simply a matter of granting 
 users 
 > access to them. Symbol tables in a mutable language are by 
 default mutable. 
 > It would certainly be possible only give users access to 
 immutable 
 > reifications (which could solve a bunch of problems as is). 
 However, it 
 > seems natural to match mutable symbol tables with mutable 
 reifications, and 
 > immutable symbol tables with immutable reifications. 
 > 
 > 
 > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
 wrote: 
 >> 
 >> I'm not sure I understand... 
 >> 
 >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
 wrote: 
 >>> 
 >>> Reified scope makes static analysis much too hard. Take any 
 criticism 
 >>> of mutable state: they all apply to globally mutable symbol 
 tables. 
 >>> 
 >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
 Bouchet-Valat 
 >>> wrote: 
  
  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a 
 écrit : 
  > All functions. 
  Well, I don't know of any language which doesn't have scoping 
  rules... 
  
  Anyway, I didn't say scoping rules are necessarily confusing, 
 I was 
  only referring to R formulas. But according to the examples 
 you 
  posted, 
  your question appears to be different. 
  
  
  Regards 
  
  > On Wednesday, July 8, 2015 a

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Ok, that was an interesting article, but it didn't really answer my 
question.

On Thursday, July 9, 2015 at 4:20:45 PM UTC-4, Isaiah wrote:
>
> https://en.wikipedia.org/wiki/Variable_(computer_science)#Scope_and_extent
>
> On Thu, Jul 9, 2015 at 4:14 PM, Brandon Taylor  > wrote:
>
>> Ok, here's where I'm getting hung up. You said that the compiler figures 
>> out the creation/lifetime of all variables at compile time. So does that 
>> mean there's a list like:
>>
>> a maps to location 0 and exists from line 3 to line 9
>> b maps to location 1 and exists from line 7 to line 9
>> a maps to location 10 and exists from line 7 to 9?
>>
>> and that to map variables to locations on any particular line, the 
>> compiler works its way up the list, 
>>
>> This is perhaps even more helpful than the environment. The environment 
>> is immediately and completely determinable at any point in the program. 
>> This could make it possible to walk back in time even within the same scope.
>>
>>
>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>>
>>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>>> >  wrote: 
>>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
>>> >> rephrase. Regardless of a how or when statement is evaluated, it must 
>>> have 
>>> >> access at least to its parent environments to successfully resolve a 
>>> symbol. 
>>>
>>> AFAIK, the only scope you can dynamically add variable to is the 
>>> global scope. (This can be done with the `global` keyword or `eval` 
>>> etc). The compiler figure out the creation/lifetime of all local 
>>> variables (at compile time). Therefore, to access a variable in the 
>>> parent scope: 
>>>
>>> 1. If it's a global, then it need a runtime lookup/binding (the reason 
>>> global are slow) 
>>> 2. If it's in a parent non-global scope, the compiler can figure out 
>>> how to bind/access it at compile time and no extra (lookup) code at 
>>> runtime is necessary. 
>>>
>>> >> 
>>> > 
>>> > A julia local variable is basically a variable in C. There's a table 
>>> > at compile time to map between symbols and stack slots (or whereever 
>>> > they are stored) but such a map does not exist at runtime anymore 
>>> > (except for debugging). 
>>> > 
>>> >> 
>>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
>>> >>> 
>>> >>> They must exist at runtime and at local scope. Evaluating a symbol 
>>> is 
>>> >>> impossible without a pool of defined symbols in various scopes to 
>>> match it 
>>> >>> to. Unless I'm missing something? 
>>> >>> 
>>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>>>  
>>>  There are global symbol tables for static analysis / reflection, 
>>> but they 
>>>  do not exist at runtime or for the local scope. 
>>>  
>>>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor <
>>> brandon@gmail.com> 
>>>  wrote: 
>>> > 
>>> > Surely environments already exist somewhere inside Julia? How else 
>>> could 
>>> > you keep track of scope? It would be simply a matter of granting 
>>> users 
>>> > access to them. Symbol tables in a mutable language are by default 
>>> mutable. 
>>> > It would certainly be possible only give users access to immutable 
>>> > reifications (which could solve a bunch of problems as is). 
>>> However, it 
>>> > seems natural to match mutable symbol tables with mutable 
>>> reifications, and 
>>> > immutable symbol tables with immutable reifications. 
>>> > 
>>> > 
>>> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
>>> wrote: 
>>> >> 
>>> >> I'm not sure I understand... 
>>> >> 
>>> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
>>> wrote: 
>>> >>> 
>>> >>> Reified scope makes static analysis much too hard. Take any 
>>> criticism 
>>> >>> of mutable state: they all apply to globally mutable symbol 
>>> tables. 
>>> >>> 
>>> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
>>> Bouchet-Valat 
>>> >>> wrote: 
>>>  
>>>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a 
>>> écrit : 
>>>  > All functions. 
>>>  Well, I don't know of any language which doesn't have scoping 
>>>  rules... 
>>>  
>>>  Anyway, I didn't say scoping rules are necessarily confusing, I 
>>> was 
>>>  only referring to R formulas. But according to the examples you 
>>>  posted, 
>>>  your question appears to be different. 
>>>  
>>>  
>>>  Regards 
>>>  
>>>  > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan 
>>> Bouchet-Valat 
>>>  > wrote: 
>>>  > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a 
>>> écrit 
>>>  > > : 
>>>  > > 
>>>  > > > If scoping rules are too complicated and cause confu

Re: [julia-users] Re: Environment reification and lazy evaluation

[Isaiah Norton]

https://en.wikipedia.org/wiki/Variable_(computer_science)#Scope_and_extent

On Thu, Jul 9, 2015 at 4:14 PM, Brandon Taylor 
wrote:

> Ok, here's where I'm getting hung up. You said that the compiler figures
> out the creation/lifetime of all variables at compile time. So does that
> mean there's a list like:
>
> a maps to location 0 and exists from line 3 to line 9
> b maps to location 1 and exists from line 7 to line 9
> a maps to location 10 and exists from line 7 to 9?
>
> and that to map variables to locations on any particular line, the
> compiler works its way up the list,
>
> This is perhaps even more helpful than the environment. The environment is
> immediately and completely determinable at any point in the program. This
> could make it possible to walk back in time even within the same scope.
>
>
> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>
>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote:
>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
>> >  wrote:
>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me
>> >> rephrase. Regardless of a how or when statement is evaluated, it must
>> have
>> >> access at least to its parent environments to successfully resolve a
>> symbol.
>>
>> AFAIK, the only scope you can dynamically add variable to is the
>> global scope. (This can be done with the `global` keyword or `eval`
>> etc). The compiler figure out the creation/lifetime of all local
>> variables (at compile time). Therefore, to access a variable in the
>> parent scope:
>>
>> 1. If it's a global, then it need a runtime lookup/binding (the reason
>> global are slow)
>> 2. If it's in a parent non-global scope, the compiler can figure out
>> how to bind/access it at compile time and no extra (lookup) code at
>> runtime is necessary.
>>
>> >>
>> >
>> > A julia local variable is basically a variable in C. There's a table
>> > at compile time to map between symbols and stack slots (or whereever
>> > they are stored) but such a map does not exist at runtime anymore
>> > (except for debugging).
>> >
>> >>
>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote:
>> >>>
>> >>> They must exist at runtime and at local scope. Evaluating a symbol is
>> >>> impossible without a pool of defined symbols in various scopes to
>> match it
>> >>> to. Unless I'm missing something?
>> >>>
>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote:
>> 
>>  There are global symbol tables for static analysis / reflection, but
>> they
>>  do not exist at runtime or for the local scope.
>> 
>>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor 
>>
>>  wrote:
>> >
>> > Surely environments already exist somewhere inside Julia? How else
>> could
>> > you keep track of scope? It would be simply a matter of granting
>> users
>> > access to them. Symbol tables in a mutable language are by default
>> mutable.
>> > It would certainly be possible only give users access to immutable
>> > reifications (which could solve a bunch of problems as is).
>> However, it
>> > seems natural to match mutable symbol tables with mutable
>> reifications, and
>> > immutable symbol tables with immutable reifications.
>> >
>> >
>> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor
>> wrote:
>> >>
>> >> I'm not sure I understand...
>> >>
>> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White
>> wrote:
>> >>>
>> >>> Reified scope makes static analysis much too hard. Take any
>> criticism
>> >>> of mutable state: they all apply to globally mutable symbol
>> tables.
>> >>>
>> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan
>> Bouchet-Valat
>> >>> wrote:
>> 
>>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a
>> écrit :
>>  > All functions.
>>  Well, I don't know of any language which doesn't have scoping
>>  rules...
>> 
>>  Anyway, I didn't say scoping rules are necessarily confusing, I
>> was
>>  only referring to R formulas. But according to the examples you
>>  posted,
>>  your question appears to be different.
>> 
>> 
>>  Regards
>> 
>>  > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan
>> Bouchet-Valat
>>  > wrote:
>>  > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a
>> écrit
>>  > > :
>>  > >
>>  > > > If scoping rules are too complicated and cause confusion,
>> why
>>  > > > are
>>  > >
>>  > > > they built into the base implementation of function?
>>  > > What do you mean? Which function?
>>  > >
>>  > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan
>> Bouchet
>>  > > -Valat
>>  > > > wrote:
>>  > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon
>> Taylor

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Ok, here's where I'm getting hung up. You said that the compiler figures 
out the creation/lifetime of all variables at compile time. So does that 
mean there's a list like:

a maps to location 0 and exists from line 3 to line 9
b maps to location 1 and exists from line 7 to line 9
a maps to location 10 and exists from line 7 to 9?

and that to map variables to locations on any particular line, the compiler 
works its way up the list, 

This is perhaps even more helpful than the environment. The environment is 
immediately and completely determinable at any point in the program. This 
could make it possible to walk back in time even within the same scope.


On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>
> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu > 
> wrote: 
> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
> > > wrote: 
> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
> >> rephrase. Regardless of a how or when statement is evaluated, it must 
> have 
> >> access at least to its parent environments to successfully resolve a 
> symbol. 
>
> AFAIK, the only scope you can dynamically add variable to is the 
> global scope. (This can be done with the `global` keyword or `eval` 
> etc). The compiler figure out the creation/lifetime of all local 
> variables (at compile time). Therefore, to access a variable in the 
> parent scope: 
>
> 1. If it's a global, then it need a runtime lookup/binding (the reason 
> global are slow) 
> 2. If it's in a parent non-global scope, the compiler can figure out 
> how to bind/access it at compile time and no extra (lookup) code at 
> runtime is necessary. 
>
> >> 
> > 
> > A julia local variable is basically a variable in C. There's a table 
> > at compile time to map between symbols and stack slots (or whereever 
> > they are stored) but such a map does not exist at runtime anymore 
> > (except for debugging). 
> > 
> >> 
> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
> >>> 
> >>> They must exist at runtime and at local scope. Evaluating a symbol is 
> >>> impossible without a pool of defined symbols in various scopes to 
> match it 
> >>> to. Unless I'm missing something? 
> >>> 
> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>  
>  There are global symbol tables for static analysis / reflection, but 
> they 
>  do not exist at runtime or for the local scope. 
>  
>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor  
>
>  wrote: 
> > 
> > Surely environments already exist somewhere inside Julia? How else 
> could 
> > you keep track of scope? It would be simply a matter of granting 
> users 
> > access to them. Symbol tables in a mutable language are by default 
> mutable. 
> > It would certainly be possible only give users access to immutable 
> > reifications (which could solve a bunch of problems as is). However, 
> it 
> > seems natural to match mutable symbol tables with mutable 
> reifications, and 
> > immutable symbol tables with immutable reifications. 
> > 
> > 
> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
> wrote: 
> >> 
> >> I'm not sure I understand... 
> >> 
> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
> wrote: 
> >>> 
> >>> Reified scope makes static analysis much too hard. Take any 
> criticism 
> >>> of mutable state: they all apply to globally mutable symbol 
> tables. 
> >>> 
> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
> Bouchet-Valat 
> >>> wrote: 
>  
>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit 
> : 
>  > All functions. 
>  Well, I don't know of any language which doesn't have scoping 
>  rules... 
>  
>  Anyway, I didn't say scoping rules are necessarily confusing, I 
> was 
>  only referring to R formulas. But according to the examples you 
>  posted, 
>  your question appears to be different. 
>  
>  
>  Regards 
>  
>  > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan 
> Bouchet-Valat 
>  > wrote: 
>  > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a 
> écrit 
>  > > : 
>  > > 
>  > > > If scoping rules are too complicated and cause confusion, 
> why 
>  > > > are 
>  > > 
>  > > > they built into the base implementation of function? 
>  > > What do you mean? Which function? 
>  > > 
>  > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan 
> Bouchet 
>  > > -Valat 
>  > > > wrote: 
>  > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor 
> a 
>  > > écrit : 
>  > > > > 
>  > > > > > I was aware of those packages (though I hadn't read the 
>  > > > > discussions

Re: [julia-users] Re: Environment reification and lazy evaluation

[Cedric St-Jean]

On Thursday, July 9, 2015 at 10:25:24 AM UTC-4, Yichao Yu wrote:

> Julia already have `macroexpand` so I guess you are talking about 
> editor/REPL feature? 
>

Yeah. Emacs' macroexpand makes a big difference in usability for macros 
IMO, otherwise one has to repeatedly cut and paste code in the REPL, which 
is no fun at all. 
 

>
> > 
> > 
> > 
> > 
> > On Wednesday, July 8, 2015 at 9:51:16 PM UTC-4, Brandon Taylor wrote: 
> >> 
> >> Edit: Hmm, so that means that any implementation of environments would 
> >> have to be handled in two separate ways: one by the compiler for 
> non-global 
> >> scope, and one in run time for the special global system?] 
> >> 
> >> 
> >> On Wednesday, July 8, 2015 at 9:49:50 PM UTC-4, Brandon Taylor wrote: 
> >>> 
> >>> Hmm, so that means that any implementation of environments would have 
> to 
> >>> be handled in to separate ways: one by the compiler for non-global 
> scope, 
> >>> and one in run time using a second for the special global system?] 
> >>> 
> >>> 
> >>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote: 
>  
>  On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>  > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>  >  wrote: 
>  >> Hmm, maybe I'm confused about compilation vs interpretation. Let 
> me 
>  >> rephrase. Regardless of a how or when statement is evaluated, it 
> must 
>  >> have 
>  >> access at least to its parent environments to successfully resolve 
> a 
>  >> symbol. 
>  
>  AFAIK, the only scope you can dynamically add variable to is the 
>  global scope. (This can be done with the `global` keyword or `eval` 
>  etc). The compiler figure out the creation/lifetime of all local 
>  variables (at compile time). Therefore, to access a variable in the 
>  parent scope: 
>  
>  1. If it's a global, then it need a runtime lookup/binding (the 
> reason 
>  global are slow) 
>  2. If it's in a parent non-global scope, the compiler can figure out 
>  how to bind/access it at compile time and no extra (lookup) code at 
>  runtime is necessary. 
>  
>  >> 
>  > 
>  > A julia local variable is basically a variable in C. There's a 
> table 
>  > at compile time to map between symbols and stack slots (or 
> whereever 
>  > they are stored) but such a map does not exist at runtime anymore 
>  > (except for debugging). 
>  > 
>  >> 
>  >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor 
> wrote: 
>  >>> 
>  >>> They must exist at runtime and at local scope. Evaluating a 
> symbol 
>  >>> is 
>  >>> impossible without a pool of defined symbols in various scopes to 
>  >>> match it 
>  >>> to. Unless I'm missing something? 
>  >>> 
>  >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>   
>   There are global symbol tables for static analysis / reflection, 
>   but they 
>   do not exist at runtime or for the local scope. 
>   
>   On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor 
>    
>   wrote: 
>  > 
>  > Surely environments already exist somewhere inside Julia? How 
> else 
>  > could 
>  > you keep track of scope? It would be simply a matter of 
> granting 
>  > users 
>  > access to them. Symbol tables in a mutable language are by 
> default 
>  > mutable. 
>  > It would certainly be possible only give users access to 
> immutable 
>  > reifications (which could solve a bunch of problems as is). 
>  > However, it 
>  > seems natural to match mutable symbol tables with mutable 
>  > reifications, and 
>  > immutable symbol tables with immutable reifications. 
>  > 
>  > 
>  > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
>  > wrote: 
>  >> 
>  >> I'm not sure I understand... 
>  >> 
>  >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles 
> White 
>  >> wrote: 
>  >>> 
>  >>> Reified scope makes static analysis much too hard. Take any 
>  >>> criticism 
>  >>> of mutable state: they all apply to globally mutable symbol 
>  >>> tables. 
>  >>> 
>  >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
>  >>> Bouchet-Valat 
>  >>> wrote: 
>   
>   Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a 
>   écrit : 
>   > All functions. 
>   Well, I don't know of any language which doesn't have 
> scoping 
>   rules... 
>   
>   Anyway, I didn't say scoping rules are necessarily 
> confusing, I 
>   was 
>   only referring to R formulas. But according to the examples 
> you 
>   posted

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Thu, Jul 9, 2015 at 9:58 AM, Cedric St-Jean  wrote:
> Environments are very natural in interpreted languages, but I'm not aware of
> any fully-compiled language that supports them because they'd be a major
> headache. I agree that macro behavior can be hard to predict. In Common Lisp
> with Emacs, one can macroexpand any expression anywhere with one keypress,
> and it is incredibly useful for figuring things out. I'm sure that Julia
> will eventually get that functionality, it's several orders of magnitude
> easier than supporting environments.

Julia already have `macroexpand` so I guess you are talking about
editor/REPL feature?

>
>
>
>
> On Wednesday, July 8, 2015 at 9:51:16 PM UTC-4, Brandon Taylor wrote:
>>
>> Edit: Hmm, so that means that any implementation of environments would
>> have to be handled in two separate ways: one by the compiler for non-global
>> scope, and one in run time for the special global system?]
>>
>>
>> On Wednesday, July 8, 2015 at 9:49:50 PM UTC-4, Brandon Taylor wrote:
>>>
>>> Hmm, so that means that any implementation of environments would have to
>>> be handled in to separate ways: one by the compiler for non-global scope,
>>> and one in run time using a second for the special global system?]
>>>
>>>
>>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:

 On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote:
 > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
 >  wrote:
 >> Hmm, maybe I'm confused about compilation vs interpretation. Let me
 >> rephrase. Regardless of a how or when statement is evaluated, it must
 >> have
 >> access at least to its parent environments to successfully resolve a
 >> symbol.

 AFAIK, the only scope you can dynamically add variable to is the
 global scope. (This can be done with the `global` keyword or `eval`
 etc). The compiler figure out the creation/lifetime of all local
 variables (at compile time). Therefore, to access a variable in the
 parent scope:

 1. If it's a global, then it need a runtime lookup/binding (the reason
 global are slow)
 2. If it's in a parent non-global scope, the compiler can figure out
 how to bind/access it at compile time and no extra (lookup) code at
 runtime is necessary.

 >>
 >
 > A julia local variable is basically a variable in C. There's a table
 > at compile time to map between symbols and stack slots (or whereever
 > they are stored) but such a map does not exist at runtime anymore
 > (except for debugging).
 >
 >>
 >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote:
 >>>
 >>> They must exist at runtime and at local scope. Evaluating a symbol
 >>> is
 >>> impossible without a pool of defined symbols in various scopes to
 >>> match it
 >>> to. Unless I'm missing something?
 >>>
 >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote:
 
  There are global symbol tables for static analysis / reflection,
  but they
  do not exist at runtime or for the local scope.
 
  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor
  
  wrote:
 >
 > Surely environments already exist somewhere inside Julia? How else
 > could
 > you keep track of scope? It would be simply a matter of granting
 > users
 > access to them. Symbol tables in a mutable language are by default
 > mutable.
 > It would certainly be possible only give users access to immutable
 > reifications (which could solve a bunch of problems as is).
 > However, it
 > seems natural to match mutable symbol tables with mutable
 > reifications, and
 > immutable symbol tables with immutable reifications.
 >
 >
 > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor
 > wrote:
 >>
 >> I'm not sure I understand...
 >>
 >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White
 >> wrote:
 >>>
 >>> Reified scope makes static analysis much too hard. Take any
 >>> criticism
 >>> of mutable state: they all apply to globally mutable symbol
 >>> tables.
 >>>
 >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan
 >>> Bouchet-Valat
 >>> wrote:
 
  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a
  écrit :
  > All functions.
  Well, I don't know of any language which doesn't have scoping
  rules...
 
  Anyway, I didn't say scoping rules are necessarily confusing, I
  was
  only referring to R formulas. But according to the examples you
  posted,
  your question appears to be different.
 
>>>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Cedric St-Jean]

Environments are very natural in interpreted languages, but I'm not aware 
of any fully-compiled language that supports them because they'd be a major 
headache. I agree that macro behavior can be hard to predict. In Common 
Lisp with Emacs, one can macroexpand any expression anywhere with one 
keypress, and it is incredibly useful for figuring things out. I'm sure 
that Julia will eventually get that functionality, it's several orders of 
magnitude easier than supporting environments.



On Wednesday, July 8, 2015 at 9:51:16 PM UTC-4, Brandon Taylor wrote:
>
> Edit: Hmm, so that means that any implementation of environments would 
> have to be handled in two separate ways: one by the compiler for non-global 
> scope, and one in run time for the special global system?]
>
>
> On Wednesday, July 8, 2015 at 9:49:50 PM UTC-4, Brandon Taylor wrote:
>>
>> Hmm, so that means that any implementation of environments would have to 
>> be handled in to separate ways: one by the compiler for non-global scope, 
>> and one in run time using a second for the special global system?]
>>
>>
>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>>
>>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>>> >  wrote: 
>>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
>>> >> rephrase. Regardless of a how or when statement is evaluated, it must 
>>> have 
>>> >> access at least to its parent environments to successfully resolve a 
>>> symbol. 
>>>
>>> AFAIK, the only scope you can dynamically add variable to is the 
>>> global scope. (This can be done with the `global` keyword or `eval` 
>>> etc). The compiler figure out the creation/lifetime of all local 
>>> variables (at compile time). Therefore, to access a variable in the 
>>> parent scope: 
>>>
>>> 1. If it's a global, then it need a runtime lookup/binding (the reason 
>>> global are slow) 
>>> 2. If it's in a parent non-global scope, the compiler can figure out 
>>> how to bind/access it at compile time and no extra (lookup) code at 
>>> runtime is necessary. 
>>>
>>> >> 
>>> > 
>>> > A julia local variable is basically a variable in C. There's a table 
>>> > at compile time to map between symbols and stack slots (or whereever 
>>> > they are stored) but such a map does not exist at runtime anymore 
>>> > (except for debugging). 
>>> > 
>>> >> 
>>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
>>> >>> 
>>> >>> They must exist at runtime and at local scope. Evaluating a symbol 
>>> is 
>>> >>> impossible without a pool of defined symbols in various scopes to 
>>> match it 
>>> >>> to. Unless I'm missing something? 
>>> >>> 
>>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>>>  
>>>  There are global symbol tables for static analysis / reflection, 
>>> but they 
>>>  do not exist at runtime or for the local scope. 
>>>  
>>>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor <
>>> brandon@gmail.com> 
>>>  wrote: 
>>> > 
>>> > Surely environments already exist somewhere inside Julia? How else 
>>> could 
>>> > you keep track of scope? It would be simply a matter of granting 
>>> users 
>>> > access to them. Symbol tables in a mutable language are by default 
>>> mutable. 
>>> > It would certainly be possible only give users access to immutable 
>>> > reifications (which could solve a bunch of problems as is). 
>>> However, it 
>>> > seems natural to match mutable symbol tables with mutable 
>>> reifications, and 
>>> > immutable symbol tables with immutable reifications. 
>>> > 
>>> > 
>>> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
>>> wrote: 
>>> >> 
>>> >> I'm not sure I understand... 
>>> >> 
>>> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
>>> wrote: 
>>> >>> 
>>> >>> Reified scope makes static analysis much too hard. Take any 
>>> criticism 
>>> >>> of mutable state: they all apply to globally mutable symbol 
>>> tables. 
>>> >>> 
>>> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
>>> Bouchet-Valat 
>>> >>> wrote: 
>>>  
>>>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a 
>>> écrit : 
>>>  > All functions. 
>>>  Well, I don't know of any language which doesn't have scoping 
>>>  rules... 
>>>  
>>>  Anyway, I didn't say scoping rules are necessarily confusing, I 
>>> was 
>>>  only referring to R formulas. But according to the examples you 
>>>  posted, 
>>>  your question appears to be different. 
>>>  
>>>  
>>>  Regards 
>>>  
>>>  > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan 
>>> Bouchet-Valat 
>>>  > wrote: 
>>>  > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a 
>>> écrit 
>>> >>>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Edit: Hmm, so that means that any implementation of environments would have 
to be handled in two separate ways: one by the compiler for non-global 
scope, and one in run time for the special global system?]


On Wednesday, July 8, 2015 at 9:49:50 PM UTC-4, Brandon Taylor wrote:
>
> Hmm, so that means that any implementation of environments would have to 
> be handled in to separate ways: one by the compiler for non-global scope, 
> and one in run time using a second for the special global system?]
>
>
> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>>
>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote: 
>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
>> >  wrote: 
>> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
>> >> rephrase. Regardless of a how or when statement is evaluated, it must 
>> have 
>> >> access at least to its parent environments to successfully resolve a 
>> symbol. 
>>
>> AFAIK, the only scope you can dynamically add variable to is the 
>> global scope. (This can be done with the `global` keyword or `eval` 
>> etc). The compiler figure out the creation/lifetime of all local 
>> variables (at compile time). Therefore, to access a variable in the 
>> parent scope: 
>>
>> 1. If it's a global, then it need a runtime lookup/binding (the reason 
>> global are slow) 
>> 2. If it's in a parent non-global scope, the compiler can figure out 
>> how to bind/access it at compile time and no extra (lookup) code at 
>> runtime is necessary. 
>>
>> >> 
>> > 
>> > A julia local variable is basically a variable in C. There's a table 
>> > at compile time to map between symbols and stack slots (or whereever 
>> > they are stored) but such a map does not exist at runtime anymore 
>> > (except for debugging). 
>> > 
>> >> 
>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
>> >>> 
>> >>> They must exist at runtime and at local scope. Evaluating a symbol is 
>> >>> impossible without a pool of defined symbols in various scopes to 
>> match it 
>> >>> to. Unless I'm missing something? 
>> >>> 
>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>>  
>>  There are global symbol tables for static analysis / reflection, but 
>> they 
>>  do not exist at runtime or for the local scope. 
>>  
>>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor  
>>
>>  wrote: 
>> > 
>> > Surely environments already exist somewhere inside Julia? How else 
>> could 
>> > you keep track of scope? It would be simply a matter of granting 
>> users 
>> > access to them. Symbol tables in a mutable language are by default 
>> mutable. 
>> > It would certainly be possible only give users access to immutable 
>> > reifications (which could solve a bunch of problems as is). 
>> However, it 
>> > seems natural to match mutable symbol tables with mutable 
>> reifications, and 
>> > immutable symbol tables with immutable reifications. 
>> > 
>> > 
>> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
>> wrote: 
>> >> 
>> >> I'm not sure I understand... 
>> >> 
>> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
>> wrote: 
>> >>> 
>> >>> Reified scope makes static analysis much too hard. Take any 
>> criticism 
>> >>> of mutable state: they all apply to globally mutable symbol 
>> tables. 
>> >>> 
>> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
>> Bouchet-Valat 
>> >>> wrote: 
>>  
>>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a 
>> écrit : 
>>  > All functions. 
>>  Well, I don't know of any language which doesn't have scoping 
>>  rules... 
>>  
>>  Anyway, I didn't say scoping rules are necessarily confusing, I 
>> was 
>>  only referring to R formulas. But according to the examples you 
>>  posted, 
>>  your question appears to be different. 
>>  
>>  
>>  Regards 
>>  
>>  > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan 
>> Bouchet-Valat 
>>  > wrote: 
>>  > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a 
>> écrit 
>>  > > : 
>>  > > 
>>  > > > If scoping rules are too complicated and cause confusion, 
>> why 
>>  > > > are 
>>  > > 
>>  > > > they built into the base implementation of function? 
>>  > > What do you mean? Which function? 
>>  > > 
>>  > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan 
>> Bouchet 
>>  > > -Valat 
>>  > > > wrote: 
>>  > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon 
>> Taylor a 
>>  > > écrit : 
>>  > > > > 
>>  > > > > > I was aware of those packages (though I hadn't read 
>> the 
>>  > > > > discussions 
>>  > > > > > referenced). Mac

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Hmm, so that means that any implementation of environments would have to be 
handled in to separate ways: one by the compiler for non-global scope, and 
one in run time using a second for the special global system?]


On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu wrote:
>
> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu > 
> wrote: 
> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor 
> > > wrote: 
> >> Hmm, maybe I'm confused about compilation vs interpretation. Let me 
> >> rephrase. Regardless of a how or when statement is evaluated, it must 
> have 
> >> access at least to its parent environments to successfully resolve a 
> symbol. 
>
> AFAIK, the only scope you can dynamically add variable to is the 
> global scope. (This can be done with the `global` keyword or `eval` 
> etc). The compiler figure out the creation/lifetime of all local 
> variables (at compile time). Therefore, to access a variable in the 
> parent scope: 
>
> 1. If it's a global, then it need a runtime lookup/binding (the reason 
> global are slow) 
> 2. If it's in a parent non-global scope, the compiler can figure out 
> how to bind/access it at compile time and no extra (lookup) code at 
> runtime is necessary. 
>
> >> 
> > 
> > A julia local variable is basically a variable in C. There's a table 
> > at compile time to map between symbols and stack slots (or whereever 
> > they are stored) but such a map does not exist at runtime anymore 
> > (except for debugging). 
> > 
> >> 
> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote: 
> >>> 
> >>> They must exist at runtime and at local scope. Evaluating a symbol is 
> >>> impossible without a pool of defined symbols in various scopes to 
> match it 
> >>> to. Unless I'm missing something? 
> >>> 
> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote: 
>  
>  There are global symbol tables for static analysis / reflection, but 
> they 
>  do not exist at runtime or for the local scope. 
>  
>  On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor  
>
>  wrote: 
> > 
> > Surely environments already exist somewhere inside Julia? How else 
> could 
> > you keep track of scope? It would be simply a matter of granting 
> users 
> > access to them. Symbol tables in a mutable language are by default 
> mutable. 
> > It would certainly be possible only give users access to immutable 
> > reifications (which could solve a bunch of problems as is). However, 
> it 
> > seems natural to match mutable symbol tables with mutable 
> reifications, and 
> > immutable symbol tables with immutable reifications. 
> > 
> > 
> > On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor 
> wrote: 
> >> 
> >> I'm not sure I understand... 
> >> 
> >> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White 
> wrote: 
> >>> 
> >>> Reified scope makes static analysis much too hard. Take any 
> criticism 
> >>> of mutable state: they all apply to globally mutable symbol 
> tables. 
> >>> 
> >>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan 
> Bouchet-Valat 
> >>> wrote: 
>  
>  Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit 
> : 
>  > All functions. 
>  Well, I don't know of any language which doesn't have scoping 
>  rules... 
>  
>  Anyway, I didn't say scoping rules are necessarily confusing, I 
> was 
>  only referring to R formulas. But according to the examples you 
>  posted, 
>  your question appears to be different. 
>  
>  
>  Regards 
>  
>  > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan 
> Bouchet-Valat 
>  > wrote: 
>  > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a 
> écrit 
>  > > : 
>  > > 
>  > > > If scoping rules are too complicated and cause confusion, 
> why 
>  > > > are 
>  > > 
>  > > > they built into the base implementation of function? 
>  > > What do you mean? Which function? 
>  > > 
>  > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan 
> Bouchet 
>  > > -Valat 
>  > > > wrote: 
>  > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor 
> a 
>  > > écrit : 
>  > > > > 
>  > > > > > I was aware of those packages (though I hadn't read the 
>  > > > > discussions 
>  > > > > > referenced). Macros are great but they are incredibly 
>  > > difficult 
>  > > > > to 
>  > > > > > reason with concerning issues of scope (at least for 
> me). 
>  > > > > Deifying 
>  > > > > > environments could solve all of these issues (and so 
> much 
>  > > more) 
>  > > > > in 
>  > > > > > one fell swoop. 
>  > > > > On the contrary, I think

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu  wrote:
> On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
>  wrote:
>> Hmm, maybe I'm confused about compilation vs interpretation. Let me
>> rephrase. Regardless of a how or when statement is evaluated, it must have
>> access at least to its parent environments to successfully resolve a symbol.

AFAIK, the only scope you can dynamically add variable to is the
global scope. (This can be done with the `global` keyword or `eval`
etc). The compiler figure out the creation/lifetime of all local
variables (at compile time). Therefore, to access a variable in the
parent scope:

1. If it's a global, then it need a runtime lookup/binding (the reason
global are slow)
2. If it's in a parent non-global scope, the compiler can figure out
how to bind/access it at compile time and no extra (lookup) code at
runtime is necessary.

>>
>
> A julia local variable is basically a variable in C. There's a table
> at compile time to map between symbols and stack slots (or whereever
> they are stored) but such a map does not exist at runtime anymore
> (except for debugging).
>
>>
>> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote:
>>>
>>> They must exist at runtime and at local scope. Evaluating a symbol is
>>> impossible without a pool of defined symbols in various scopes to match it
>>> to. Unless I'm missing something?
>>>
>>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote:

 There are global symbol tables for static analysis / reflection, but they
 do not exist at runtime or for the local scope.

 On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor 
 wrote:
>
> Surely environments already exist somewhere inside Julia? How else could
> you keep track of scope? It would be simply a matter of granting users
> access to them. Symbol tables in a mutable language are by default 
> mutable.
> It would certainly be possible only give users access to immutable
> reifications (which could solve a bunch of problems as is). However, it
> seems natural to match mutable symbol tables with mutable reifications, 
> and
> immutable symbol tables with immutable reifications.
>
>
> On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor wrote:
>>
>> I'm not sure I understand...
>>
>> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:
>>>
>>> Reified scope makes static analysis much too hard. Take any criticism
>>> of mutable state: they all apply to globally mutable symbol tables.
>>>
>>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat
>>> wrote:

 Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit :
 > All functions.
 Well, I don't know of any language which doesn't have scoping
 rules...

 Anyway, I didn't say scoping rules are necessarily confusing, I was
 only referring to R formulas. But according to the examples you
 posted,
 your question appears to be different.


 Regards

 > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat
 > wrote:
 > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit
 > > :
 > >
 > > > If scoping rules are too complicated and cause confusion, why
 > > > are
 > >
 > > > they built into the base implementation of function?
 > > What do you mean? Which function?
 > >
 > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet
 > > -Valat
 > > > wrote:
 > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a
 > > écrit :
 > > > >
 > > > > > I was aware of those packages (though I hadn't read the
 > > > > discussions
 > > > > > referenced). Macros are great but they are incredibly
 > > difficult
 > > > > to
 > > > > > reason with concerning issues of scope (at least for me).
 > > > > Deifying
 > > > > > environments could solve all of these issues (and so much
 > > more)
 > > > > in
 > > > > > one fell swoop.
 > > > > On the contrary, I think well-designed macros can be much
 > > easier to
 > > > >
 > > > > think about than environments in R. If the macro takes a
 > > DataFrame
 > > > > object and an expression, there's no ambiguity about what the
 > > scope
 > > > > is.
 > > > > This is even better if variables that should be found in the
 > > data
 > > > > frame
 > > > > are passed as symbols, like :var, while standard variables
 > > > > are
 > > > > specified as usual.
 > > > >
 > > > > On the other hand, I find R formulas too flexible and complex
 > > 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Yichao Yu]

On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
 wrote:
> Hmm, maybe I'm confused about compilation vs interpretation. Let me
> rephrase. Regardless of a how or when statement is evaluated, it must have
> access at least to its parent environments to successfully resolve a symbol.
>

A julia local variable is basically a variable in C. There's a table
at compile time to map between symbols and stack slots (or whereever
they are stored) but such a map does not exist at runtime anymore
(except for debugging).

>
> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote:
>>
>> They must exist at runtime and at local scope. Evaluating a symbol is
>> impossible without a pool of defined symbols in various scopes to match it
>> to. Unless I'm missing something?
>>
>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote:
>>>
>>> There are global symbol tables for static analysis / reflection, but they
>>> do not exist at runtime or for the local scope.
>>>
>>> On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor 
>>> wrote:

 Surely environments already exist somewhere inside Julia? How else could
 you keep track of scope? It would be simply a matter of granting users
 access to them. Symbol tables in a mutable language are by default mutable.
 It would certainly be possible only give users access to immutable
 reifications (which could solve a bunch of problems as is). However, it
 seems natural to match mutable symbol tables with mutable reifications, and
 immutable symbol tables with immutable reifications.


 On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor wrote:
>
> I'm not sure I understand...
>
> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:
>>
>> Reified scope makes static analysis much too hard. Take any criticism
>> of mutable state: they all apply to globally mutable symbol tables.
>>
>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat
>> wrote:
>>>
>>> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit :
>>> > All functions.
>>> Well, I don't know of any language which doesn't have scoping
>>> rules...
>>>
>>> Anyway, I didn't say scoping rules are necessarily confusing, I was
>>> only referring to R formulas. But according to the examples you
>>> posted,
>>> your question appears to be different.
>>>
>>>
>>> Regards
>>>
>>> > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat
>>> > wrote:
>>> > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit
>>> > > :
>>> > >
>>> > > > If scoping rules are too complicated and cause confusion, why
>>> > > > are
>>> > >
>>> > > > they built into the base implementation of function?
>>> > > What do you mean? Which function?
>>> > >
>>> > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet
>>> > > -Valat
>>> > > > wrote:
>>> > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a
>>> > > écrit :
>>> > > > >
>>> > > > > > I was aware of those packages (though I hadn't read the
>>> > > > > discussions
>>> > > > > > referenced). Macros are great but they are incredibly
>>> > > difficult
>>> > > > > to
>>> > > > > > reason with concerning issues of scope (at least for me).
>>> > > > > Deifying
>>> > > > > > environments could solve all of these issues (and so much
>>> > > more)
>>> > > > > in
>>> > > > > > one fell swoop.
>>> > > > > On the contrary, I think well-designed macros can be much
>>> > > easier to
>>> > > > >
>>> > > > > think about than environments in R. If the macro takes a
>>> > > DataFrame
>>> > > > > object and an expression, there's no ambiguity about what the
>>> > > scope
>>> > > > > is.
>>> > > > > This is even better if variables that should be found in the
>>> > > data
>>> > > > > frame
>>> > > > > are passed as symbols, like :var, while standard variables
>>> > > > > are
>>> > > > > specified as usual.
>>> > > > >
>>> > > > > On the other hand, I find R formulas too flexible and complex
>>> > > to
>>> > > > > reason
>>> > > > > about. You never know whether an object will be found in the
>>> > > > > formula's
>>> > > > > environment, in one of the parent environments of the
>>> > > > > function/package
>>> > > > > you called, in your function, or in the global environment.
>>> > > > >
>>> > > > >
>>> > > > > Regards
>>> > > > >
>>> > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold
>>> > > wrote:
>>> > > > >
>>> > > > > > > Some of these issues have been thought about fairly
>>> > > extensively
>>> > > > > by
>>> > > > > > > the stats community in particular, precisely on account
>>> > > > > > > of

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Hmm, maybe I'm confused about compilation vs interpretation. Let me 
rephrase. Regardless of a how or when statement is evaluated, it must have 
access at least to its parent environments to successfully resolve a symbol.

On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon Taylor wrote:
>
> They must exist at runtime and at local scope. Evaluating a symbol is 
> impossible without a pool of defined symbols in various scopes to match it 
> to. Unless I'm missing something?
>
> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote:
>>
>> There are global symbol tables for static analysis / reflection, but they 
>> do not exist at runtime or for the local scope.
>>
>> On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor  
>> wrote:
>>
>>> Surely environments already exist somewhere inside Julia? How else could 
>>> you keep track of scope? It would be simply a matter of granting users 
>>> access to them. Symbol tables in a mutable language are by default mutable. 
>>> It would certainly be possible only give users access to immutable 
>>> reifications (which could solve a bunch of problems as is). However, it 
>>> seems natural to match mutable symbol tables with mutable reifications, and 
>>> immutable symbol tables with immutable reifications.
>>>
>>>
>>> On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor wrote:

 I'm not sure I understand...

 On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:
>
> Reified scope makes static analysis much too hard. Take any criticism 
> of mutable state: they all apply to globally mutable symbol tables.
>
> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat 
> wrote:
>>
>> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit : 
>> > All functions. 
>> Well, I don't know of any language which doesn't have scoping 
>> rules... 
>>
>> Anyway, I didn't say scoping rules are necessarily confusing, I was 
>> only referring to R formulas. But according to the examples you 
>> posted, 
>> your question appears to be different. 
>>
>>
>> Regards 
>>
>> > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat 
>> > wrote: 
>> > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit 
>> : 
>> > > 
>> > > > If scoping rules are too complicated and cause confusion, why 
>> are 
>> > > 
>> > > > they built into the base implementation of function? 
>> > > What do you mean? Which function? 
>> > > 
>> > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet 
>> > > -Valat 
>> > > > wrote: 
>> > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a 
>> > > écrit : 
>> > > > > 
>> > > > > > I was aware of those packages (though I hadn't read the 
>> > > > > discussions 
>> > > > > > referenced). Macros are great but they are incredibly 
>> > > difficult 
>> > > > > to 
>> > > > > > reason with concerning issues of scope (at least for me). 
>> > > > > Deifying 
>> > > > > > environments could solve all of these issues (and so much 
>> > > more) 
>> > > > > in 
>> > > > > > one fell swoop. 
>> > > > > On the contrary, I think well-designed macros can be much 
>> > > easier to 
>> > > > > 
>> > > > > think about than environments in R. If the macro takes a 
>> > > DataFrame 
>> > > > > object and an expression, there's no ambiguity about what the 
>> > > scope 
>> > > > > is. 
>> > > > > This is even better if variables that should be found in the 
>> > > data 
>> > > > > frame 
>> > > > > are passed as symbols, like :var, while standard variables 
>> are 
>> > > > > specified as usual. 
>> > > > > 
>> > > > > On the other hand, I find R formulas too flexible and complex 
>> > > to 
>> > > > > reason 
>> > > > > about. You never know whether an object will be found in the 
>> > > > > formula's 
>> > > > > environment, in one of the parent environments of the 
>> > > > > function/package 
>> > > > > you called, in your function, or in the global environment. 
>> > > > > 
>> > > > > 
>> > > > > Regards 
>> > > > > 
>> > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold 
>> > > wrote: 
>> > > > > 
>> > > > > > > Some of these issues have been thought about fairly 
>> > > extensively 
>> > > > > by 
>> > > > > > > the stats community in particular, precisely on account 
>> of 
>> > > the 
>> > > > > use 
>> > > > > > > cases you cite: 
>> > > > > > > 
>> > > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
>> > > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
>> > > > > > > 
>> > > > > > > I think that the matter is still very much an open 
>> > > question.

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

They must exist at runtime and at local scope. Evaluating a symbol is 
impossible without a pool of defined symbols in various scopes to match it 
to. Unless I'm missing something?

On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson wrote:
>
> There are global symbol tables for static analysis / reflection, but they 
> do not exist at runtime or for the local scope.
>
> On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor  > wrote:
>
>> Surely environments already exist somewhere inside Julia? How else could 
>> you keep track of scope? It would be simply a matter of granting users 
>> access to them. Symbol tables in a mutable language are by default mutable. 
>> It would certainly be possible only give users access to immutable 
>> reifications (which could solve a bunch of problems as is). However, it 
>> seems natural to match mutable symbol tables with mutable reifications, and 
>> immutable symbol tables with immutable reifications.
>>
>>
>> On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor wrote:
>>>
>>> I'm not sure I understand...
>>>
>>> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:

 Reified scope makes static analysis much too hard. Take any criticism 
 of mutable state: they all apply to globally mutable symbol tables.

 On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat 
 wrote:
>
> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit : 
> > All functions. 
> Well, I don't know of any language which doesn't have scoping rules... 
>
> Anyway, I didn't say scoping rules are necessarily confusing, I was 
> only referring to R formulas. But according to the examples you 
> posted, 
> your question appears to be different. 
>
>
> Regards 
>
> > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat 
> > wrote: 
> > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit 
> : 
> > > 
> > > > If scoping rules are too complicated and cause confusion, why 
> are 
> > > 
> > > > they built into the base implementation of function? 
> > > What do you mean? Which function? 
> > > 
> > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet 
> > > -Valat 
> > > > wrote: 
> > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a 
> > > écrit : 
> > > > > 
> > > > > > I was aware of those packages (though I hadn't read the 
> > > > > discussions 
> > > > > > referenced). Macros are great but they are incredibly 
> > > difficult 
> > > > > to 
> > > > > > reason with concerning issues of scope (at least for me). 
> > > > > Deifying 
> > > > > > environments could solve all of these issues (and so much 
> > > more) 
> > > > > in 
> > > > > > one fell swoop. 
> > > > > On the contrary, I think well-designed macros can be much 
> > > easier to 
> > > > > 
> > > > > think about than environments in R. If the macro takes a 
> > > DataFrame 
> > > > > object and an expression, there's no ambiguity about what the 
> > > scope 
> > > > > is. 
> > > > > This is even better if variables that should be found in the 
> > > data 
> > > > > frame 
> > > > > are passed as symbols, like :var, while standard variables are 
> > > > > specified as usual. 
> > > > > 
> > > > > On the other hand, I find R formulas too flexible and complex 
> > > to 
> > > > > reason 
> > > > > about. You never know whether an object will be found in the 
> > > > > formula's 
> > > > > environment, in one of the parent environments of the 
> > > > > function/package 
> > > > > you called, in your function, or in the global environment. 
> > > > > 
> > > > > 
> > > > > Regards 
> > > > > 
> > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold 
> > > wrote: 
> > > > > 
> > > > > > > Some of these issues have been thought about fairly 
> > > extensively 
> > > > > by 
> > > > > > > the stats community in particular, precisely on account of 
> > > the 
> > > > > use 
> > > > > > > cases you cite: 
> > > > > > > 
> > > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
> > > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
> > > > > > > 
> > > > > > > I think that the matter is still very much an open 
> > > question. I 
> > > > > have 
> > > > > > > no sense that anything is going to be added to Base Julia 
> > > > > itself. 
> > > > > > > Currently, the best way (that I know of, anyway) to 
> achieve 
> > > the 
> > > > > 
> > > > > > > delayed evaluation effect is via the use of macros. See 
> for 
> > > 
> > > > > > > instance: 
> > > > > > > 
> > > > > > > https://github.com/JuliaStats/DataFr

Re: [julia-users] Re: Environment reification and lazy evaluation

[Jameson Nash]

There are global symbol tables for static analysis / reflection, but they
do not exist at runtime or for the local scope.

On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor 
wrote:

> Surely environments already exist somewhere inside Julia? How else could
> you keep track of scope? It would be simply a matter of granting users
> access to them. Symbol tables in a mutable language are by default mutable.
> It would certainly be possible only give users access to immutable
> reifications (which could solve a bunch of problems as is). However, it
> seems natural to match mutable symbol tables with mutable reifications, and
> immutable symbol tables with immutable reifications.
>
>
> On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor wrote:
>>
>> I'm not sure I understand...
>>
>> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:
>>>
>>> Reified scope makes static analysis much too hard. Take any criticism of
>>> mutable state: they all apply to globally mutable symbol tables.
>>>
>>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat
>>> wrote:

 Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit :
 > All functions.
 Well, I don't know of any language which doesn't have scoping rules...

 Anyway, I didn't say scoping rules are necessarily confusing, I was
 only referring to R formulas. But according to the examples you posted,
 your question appears to be different.


 Regards

 > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat
 > wrote:
 > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit :
 > >
 > > > If scoping rules are too complicated and cause confusion, why are
 > >
 > > > they built into the base implementation of function?
 > > What do you mean? Which function?
 > >
 > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet
 > > -Valat
 > > > wrote:
 > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a
 > > écrit :
 > > > >
 > > > > > I was aware of those packages (though I hadn't read the
 > > > > discussions
 > > > > > referenced). Macros are great but they are incredibly
 > > difficult
 > > > > to
 > > > > > reason with concerning issues of scope (at least for me).
 > > > > Deifying
 > > > > > environments could solve all of these issues (and so much
 > > more)
 > > > > in
 > > > > > one fell swoop.
 > > > > On the contrary, I think well-designed macros can be much
 > > easier to
 > > > >
 > > > > think about than environments in R. If the macro takes a
 > > DataFrame
 > > > > object and an expression, there's no ambiguity about what the
 > > scope
 > > > > is.
 > > > > This is even better if variables that should be found in the
 > > data
 > > > > frame
 > > > > are passed as symbols, like :var, while standard variables are
 > > > > specified as usual.
 > > > >
 > > > > On the other hand, I find R formulas too flexible and complex
 > > to
 > > > > reason
 > > > > about. You never know whether an object will be found in the
 > > > > formula's
 > > > > environment, in one of the parent environments of the
 > > > > function/package
 > > > > you called, in your function, or in the global environment.
 > > > >
 > > > >
 > > > > Regards
 > > > >
 > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold
 > > wrote:
 > > > >
 > > > > > > Some of these issues have been thought about fairly
 > > extensively
 > > > > by
 > > > > > > the stats community in particular, precisely on account of
 > > the
 > > > > use
 > > > > > > cases you cite:
 > > > > > >
 > > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472
 > > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504
 > > > > > >
 > > > > > > I think that the matter is still very much an open
 > > question. I
 > > > > have
 > > > > > > no sense that anything is going to be added to Base Julia
 > > > > itself.
 > > > > > > Currently, the best way (that I know of, anyway) to achieve
 > > the
 > > > >
 > > > > > > delayed evaluation effect is via the use of macros. See for
 > >
 > > > > > > instance:
 > > > > > >
 > > > > > > https://github.com/JuliaStats/DataFramesMeta.jl
 > > > > > > https://github.com/one-more-minute/Lazy.jl
 > > > > > >
 > > > > > > I'm hope somebody else will be able to pop in an give a
 > > more
 > > > > > > thorough answer, but the above may at least be a place to
 > > > > start.
 > > > > > >
 > > > > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon
 > > Taylor
 > > > > > > wrote:
 > > > > > > > Hadley Wickham's lazyeval package in R is pretty cool in
 > > that
 >

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Surely environments already exist somewhere inside Julia? How else could 
you keep track of scope? It would be simply a matter of granting users 
access to them. Symbol tables in a mutable language are by default mutable. 
It would certainly be possible only give users access to immutable 
reifications (which could solve a bunch of problems as is). However, it 
seems natural to match mutable symbol tables with mutable reifications, and 
immutable symbol tables with immutable reifications.

On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon Taylor wrote:
>
> I'm not sure I understand...
>
> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:
>>
>> Reified scope makes static analysis much too hard. Take any criticism of 
>> mutable state: they all apply to globally mutable symbol tables.
>>
>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat 
>> wrote:
>>>
>>> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit : 
>>> > All functions. 
>>> Well, I don't know of any language which doesn't have scoping rules... 
>>>
>>> Anyway, I didn't say scoping rules are necessarily confusing, I was 
>>> only referring to R formulas. But according to the examples you posted, 
>>> your question appears to be different. 
>>>
>>>
>>> Regards 
>>>
>>> > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat 
>>> > wrote: 
>>> > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit : 
>>> > > 
>>> > > > If scoping rules are too complicated and cause confusion, why are 
>>> > > 
>>> > > > they built into the base implementation of function? 
>>> > > What do you mean? Which function? 
>>> > > 
>>> > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet 
>>> > > -Valat 
>>> > > > wrote: 
>>> > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a 
>>> > > écrit : 
>>> > > > > 
>>> > > > > > I was aware of those packages (though I hadn't read the 
>>> > > > > discussions 
>>> > > > > > referenced). Macros are great but they are incredibly 
>>> > > difficult 
>>> > > > > to 
>>> > > > > > reason with concerning issues of scope (at least for me). 
>>> > > > > Deifying 
>>> > > > > > environments could solve all of these issues (and so much 
>>> > > more) 
>>> > > > > in 
>>> > > > > > one fell swoop. 
>>> > > > > On the contrary, I think well-designed macros can be much 
>>> > > easier to 
>>> > > > > 
>>> > > > > think about than environments in R. If the macro takes a 
>>> > > DataFrame 
>>> > > > > object and an expression, there's no ambiguity about what the 
>>> > > scope 
>>> > > > > is. 
>>> > > > > This is even better if variables that should be found in the 
>>> > > data 
>>> > > > > frame 
>>> > > > > are passed as symbols, like :var, while standard variables are 
>>> > > > > specified as usual. 
>>> > > > > 
>>> > > > > On the other hand, I find R formulas too flexible and complex 
>>> > > to 
>>> > > > > reason 
>>> > > > > about. You never know whether an object will be found in the 
>>> > > > > formula's 
>>> > > > > environment, in one of the parent environments of the 
>>> > > > > function/package 
>>> > > > > you called, in your function, or in the global environment. 
>>> > > > > 
>>> > > > > 
>>> > > > > Regards 
>>> > > > > 
>>> > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold 
>>> > > wrote: 
>>> > > > > 
>>> > > > > > > Some of these issues have been thought about fairly 
>>> > > extensively 
>>> > > > > by 
>>> > > > > > > the stats community in particular, precisely on account of 
>>> > > the 
>>> > > > > use 
>>> > > > > > > cases you cite: 
>>> > > > > > > 
>>> > > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
>>> > > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
>>> > > > > > > 
>>> > > > > > > I think that the matter is still very much an open 
>>> > > question. I 
>>> > > > > have 
>>> > > > > > > no sense that anything is going to be added to Base Julia 
>>> > > > > itself. 
>>> > > > > > > Currently, the best way (that I know of, anyway) to achieve 
>>> > > the 
>>> > > > > 
>>> > > > > > > delayed evaluation effect is via the use of macros. See for 
>>> > > 
>>> > > > > > > instance: 
>>> > > > > > > 
>>> > > > > > > https://github.com/JuliaStats/DataFramesMeta.jl 
>>> > > > > > > https://github.com/one-more-minute/Lazy.jl 
>>> > > > > > > 
>>> > > > > > > I'm hope somebody else will be able to pop in an give a 
>>> > > more 
>>> > > > > > > thorough answer, but the above may at least be a place to 
>>> > > > > start. 
>>> > > > > > > 
>>> > > > > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon 
>>> > > Taylor 
>>> > > > > > > wrote: 
>>> > > > > > > > Hadley Wickham's lazyeval package in R is pretty cool in 
>>> > > that 
>>> > > > > you 
>>> > > > > > > > can attach an environment to an expression, pass it in 
>>> > > and 
>>> > > > > out of 
>>> > > > > > > > functions with various modifications, and then evaluate 
>>> > > the

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

I'm not sure I understand...

On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John Myles White wrote:
>
> Reified scope makes static analysis much too hard. Take any criticism of 
> mutable state: they all apply to globally mutable symbol tables.
>
> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat wrote:
>>
>> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit : 
>> > All functions. 
>> Well, I don't know of any language which doesn't have scoping rules... 
>>
>> Anyway, I didn't say scoping rules are necessarily confusing, I was 
>> only referring to R formulas. But according to the examples you posted, 
>> your question appears to be different. 
>>
>>
>> Regards 
>>
>> > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat 
>> > wrote: 
>> > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit : 
>> > > 
>> > > > If scoping rules are too complicated and cause confusion, why are 
>> > > 
>> > > > they built into the base implementation of function? 
>> > > What do you mean? Which function? 
>> > > 
>> > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet 
>> > > -Valat 
>> > > > wrote: 
>> > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a 
>> > > écrit : 
>> > > > > 
>> > > > > > I was aware of those packages (though I hadn't read the 
>> > > > > discussions 
>> > > > > > referenced). Macros are great but they are incredibly 
>> > > difficult 
>> > > > > to 
>> > > > > > reason with concerning issues of scope (at least for me). 
>> > > > > Deifying 
>> > > > > > environments could solve all of these issues (and so much 
>> > > more) 
>> > > > > in 
>> > > > > > one fell swoop. 
>> > > > > On the contrary, I think well-designed macros can be much 
>> > > easier to 
>> > > > > 
>> > > > > think about than environments in R. If the macro takes a 
>> > > DataFrame 
>> > > > > object and an expression, there's no ambiguity about what the 
>> > > scope 
>> > > > > is. 
>> > > > > This is even better if variables that should be found in the 
>> > > data 
>> > > > > frame 
>> > > > > are passed as symbols, like :var, while standard variables are 
>> > > > > specified as usual. 
>> > > > > 
>> > > > > On the other hand, I find R formulas too flexible and complex 
>> > > to 
>> > > > > reason 
>> > > > > about. You never know whether an object will be found in the 
>> > > > > formula's 
>> > > > > environment, in one of the parent environments of the 
>> > > > > function/package 
>> > > > > you called, in your function, or in the global environment. 
>> > > > > 
>> > > > > 
>> > > > > Regards 
>> > > > > 
>> > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold 
>> > > wrote: 
>> > > > > 
>> > > > > > > Some of these issues have been thought about fairly 
>> > > extensively 
>> > > > > by 
>> > > > > > > the stats community in particular, precisely on account of 
>> > > the 
>> > > > > use 
>> > > > > > > cases you cite: 
>> > > > > > > 
>> > > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
>> > > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
>> > > > > > > 
>> > > > > > > I think that the matter is still very much an open 
>> > > question. I 
>> > > > > have 
>> > > > > > > no sense that anything is going to be added to Base Julia 
>> > > > > itself. 
>> > > > > > > Currently, the best way (that I know of, anyway) to achieve 
>> > > the 
>> > > > > 
>> > > > > > > delayed evaluation effect is via the use of macros. See for 
>> > > 
>> > > > > > > instance: 
>> > > > > > > 
>> > > > > > > https://github.com/JuliaStats/DataFramesMeta.jl 
>> > > > > > > https://github.com/one-more-minute/Lazy.jl 
>> > > > > > > 
>> > > > > > > I'm hope somebody else will be able to pop in an give a 
>> > > more 
>> > > > > > > thorough answer, but the above may at least be a place to 
>> > > > > start. 
>> > > > > > > 
>> > > > > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon 
>> > > Taylor 
>> > > > > > > wrote: 
>> > > > > > > > Hadley Wickham's lazyeval package in R is pretty cool in 
>> > > that 
>> > > > > you 
>> > > > > > > > can attach an environment to an expression, pass it in 
>> > > and 
>> > > > > out of 
>> > > > > > > > functions with various modifications, and then evaluate 
>> > > the 
>> > > > > > > > expression within the original environment (or any other 
>> > > > > > > > environment that you choose). R in general has the 
>> > > functions 
>> > > > > like 
>> > > > > > > > list2env and list(environment()) that allow one to 
>> > > convert an 
>> > > > > 
>> > > > > > > > environment into a list and back again (list being the R 
>> > > > > > > > equivalent of a Dict). Are there any plans to add these 
>> > > kind 
>> > > > > of 
>> > > > > > > > features to Julia? 
>> > > > > > > > 
>>
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[John Myles White]

Reified scope makes static analysis much too hard. Take any criticism of 
mutable state: they all apply to globally mutable symbol tables.

On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2, Milan Bouchet-Valat wrote:
>
> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit : 
> > All functions. 
> Well, I don't know of any language which doesn't have scoping rules... 
>
> Anyway, I didn't say scoping rules are necessarily confusing, I was 
> only referring to R formulas. But according to the examples you posted, 
> your question appears to be different. 
>
>
> Regards 
>
> > On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat 
> > wrote: 
> > > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit : 
> > > 
> > > > If scoping rules are too complicated and cause confusion, why are 
> > > 
> > > > they built into the base implementation of function? 
> > > What do you mean? Which function? 
> > > 
> > > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet 
> > > -Valat 
> > > > wrote: 
> > > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a 
> > > écrit : 
> > > > > 
> > > > > > I was aware of those packages (though I hadn't read the 
> > > > > discussions 
> > > > > > referenced). Macros are great but they are incredibly 
> > > difficult 
> > > > > to 
> > > > > > reason with concerning issues of scope (at least for me). 
> > > > > Deifying 
> > > > > > environments could solve all of these issues (and so much 
> > > more) 
> > > > > in 
> > > > > > one fell swoop. 
> > > > > On the contrary, I think well-designed macros can be much 
> > > easier to 
> > > > > 
> > > > > think about than environments in R. If the macro takes a 
> > > DataFrame 
> > > > > object and an expression, there's no ambiguity about what the 
> > > scope 
> > > > > is. 
> > > > > This is even better if variables that should be found in the 
> > > data 
> > > > > frame 
> > > > > are passed as symbols, like :var, while standard variables are 
> > > > > specified as usual. 
> > > > > 
> > > > > On the other hand, I find R formulas too flexible and complex 
> > > to 
> > > > > reason 
> > > > > about. You never know whether an object will be found in the 
> > > > > formula's 
> > > > > environment, in one of the parent environments of the 
> > > > > function/package 
> > > > > you called, in your function, or in the global environment. 
> > > > > 
> > > > > 
> > > > > Regards 
> > > > > 
> > > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold 
> > > wrote: 
> > > > > 
> > > > > > > Some of these issues have been thought about fairly 
> > > extensively 
> > > > > by 
> > > > > > > the stats community in particular, precisely on account of 
> > > the 
> > > > > use 
> > > > > > > cases you cite: 
> > > > > > > 
> > > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
> > > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
> > > > > > > 
> > > > > > > I think that the matter is still very much an open 
> > > question. I 
> > > > > have 
> > > > > > > no sense that anything is going to be added to Base Julia 
> > > > > itself. 
> > > > > > > Currently, the best way (that I know of, anyway) to achieve 
> > > the 
> > > > > 
> > > > > > > delayed evaluation effect is via the use of macros. See for 
> > > 
> > > > > > > instance: 
> > > > > > > 
> > > > > > > https://github.com/JuliaStats/DataFramesMeta.jl 
> > > > > > > https://github.com/one-more-minute/Lazy.jl 
> > > > > > > 
> > > > > > > I'm hope somebody else will be able to pop in an give a 
> > > more 
> > > > > > > thorough answer, but the above may at least be a place to 
> > > > > start. 
> > > > > > > 
> > > > > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon 
> > > Taylor 
> > > > > > > wrote: 
> > > > > > > > Hadley Wickham's lazyeval package in R is pretty cool in 
> > > that 
> > > > > you 
> > > > > > > > can attach an environment to an expression, pass it in 
> > > and 
> > > > > out of 
> > > > > > > > functions with various modifications, and then evaluate 
> > > the 
> > > > > > > > expression within the original environment (or any other 
> > > > > > > > environment that you choose). R in general has the 
> > > functions 
> > > > > like 
> > > > > > > > list2env and list(environment()) that allow one to 
> > > convert an 
> > > > > 
> > > > > > > > environment into a list and back again (list being the R 
> > > > > > > > equivalent of a Dict). Are there any plans to add these 
> > > kind 
> > > > > of 
> > > > > > > > features to Julia? 
> > > > > > > > 
>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon Taylor a écrit :
> All functions.
Well, I don't know of any language which doesn't have scoping rules...

Anyway, I didn't say scoping rules are necessarily confusing, I was
only referring to R formulas. But according to the examples you posted,
your question appears to be different.


Regards

> On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat 
> wrote:
> > Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit : 
> > 
> > > If scoping rules are too complicated and cause confusion, why are 
> > 
> > > they built into the base implementation of function? 
> > What do you mean? Which function? 
> > 
> > > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet
> > -Valat 
> > > wrote: 
> > > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a 
> > écrit : 
> > > > 
> > > > > I was aware of those packages (though I hadn't read the 
> > > > discussions 
> > > > > referenced). Macros are great but they are incredibly 
> > difficult 
> > > > to 
> > > > > reason with concerning issues of scope (at least for me). 
> > > > Deifying 
> > > > > environments could solve all of these issues (and so much 
> > more) 
> > > > in 
> > > > > one fell swoop. 
> > > > On the contrary, I think well-designed macros can be much 
> > easier to 
> > > > 
> > > > think about than environments in R. If the macro takes a 
> > DataFrame 
> > > > object and an expression, there's no ambiguity about what the 
> > scope 
> > > > is. 
> > > > This is even better if variables that should be found in the 
> > data 
> > > > frame 
> > > > are passed as symbols, like :var, while standard variables are 
> > > > specified as usual. 
> > > > 
> > > > On the other hand, I find R formulas too flexible and complex 
> > to 
> > > > reason 
> > > > about. You never know whether an object will be found in the 
> > > > formula's 
> > > > environment, in one of the parent environments of the 
> > > > function/package 
> > > > you called, in your function, or in the global environment. 
> > > > 
> > > > 
> > > > Regards 
> > > > 
> > > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold 
> > wrote: 
> > > > 
> > > > > > Some of these issues have been thought about fairly 
> > extensively 
> > > > by 
> > > > > > the stats community in particular, precisely on account of 
> > the 
> > > > use 
> > > > > > cases you cite: 
> > > > > > 
> > > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
> > > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
> > > > > > 
> > > > > > I think that the matter is still very much an open 
> > question. I 
> > > > have 
> > > > > > no sense that anything is going to be added to Base Julia 
> > > > itself. 
> > > > > > Currently, the best way (that I know of, anyway) to achieve 
> > the 
> > > > 
> > > > > > delayed evaluation effect is via the use of macros. See for 
> > 
> > > > > > instance: 
> > > > > > 
> > > > > > https://github.com/JuliaStats/DataFramesMeta.jl 
> > > > > > https://github.com/one-more-minute/Lazy.jl 
> > > > > > 
> > > > > > I'm hope somebody else will be able to pop in an give a 
> > more 
> > > > > > thorough answer, but the above may at least be a place to 
> > > > start. 
> > > > > > 
> > > > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon 
> > Taylor 
> > > > > > wrote: 
> > > > > > > Hadley Wickham's lazyeval package in R is pretty cool in 
> > that 
> > > > you 
> > > > > > > can attach an environment to an expression, pass it in 
> > and 
> > > > out of 
> > > > > > > functions with various modifications, and then evaluate 
> > the 
> > > > > > > expression within the original environment (or any other 
> > > > > > > environment that you choose). R in general has the 
> > functions 
> > > > like 
> > > > > > > list2env and list(environment()) that allow one to 
> > convert an 
> > > > 
> > > > > > > environment into a list and back again (list being the R 
> > > > > > > equivalent of a Dict). Are there any plans to add these 
> > kind 
> > > > of 
> > > > > > > features to Julia? 
> > > > > > > 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

All functions.

On Wednesday, July 8, 2015 at 4:18:09 PM UTC-4, Milan Bouchet-Valat wrote:
>
> Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit : 
> > If scoping rules are too complicated and cause confusion, why are 
> > they built into the base implementation of function? 
> What do you mean? Which function? 
>
> > On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet-Valat 
> > wrote: 
> > > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a écrit : 
> > > 
> > > > I was aware of those packages (though I hadn't read the 
> > > discussions 
> > > > referenced). Macros are great but they are incredibly difficult 
> > > to 
> > > > reason with concerning issues of scope (at least for me). 
> > > Deifying 
> > > > environments could solve all of these issues (and so much more) 
> > > in 
> > > > one fell swoop. 
> > > On the contrary, I think well-designed macros can be much easier to 
> > > 
> > > think about than environments in R. If the macro takes a DataFrame 
> > > object and an expression, there's no ambiguity about what the scope 
> > > is. 
> > > This is even better if variables that should be found in the data 
> > > frame 
> > > are passed as symbols, like :var, while standard variables are 
> > > specified as usual. 
> > > 
> > > On the other hand, I find R formulas too flexible and complex to 
> > > reason 
> > > about. You never know whether an object will be found in the 
> > > formula's 
> > > environment, in one of the parent environments of the 
> > > function/package 
> > > you called, in your function, or in the global environment. 
> > > 
> > > 
> > > Regards 
> > > 
> > > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote: 
> > > 
> > > > > Some of these issues have been thought about fairly extensively 
> > > by 
> > > > > the stats community in particular, precisely on account of the 
> > > use 
> > > > > cases you cite: 
> > > > > 
> > > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
> > > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
> > > > > 
> > > > > I think that the matter is still very much an open question. I 
> > > have 
> > > > > no sense that anything is going to be added to Base Julia 
> > > itself. 
> > > > > Currently, the best way (that I know of, anyway) to achieve the 
> > > 
> > > > > delayed evaluation effect is via the use of macros. See for 
> > > > > instance: 
> > > > > 
> > > > > https://github.com/JuliaStats/DataFramesMeta.jl 
> > > > > https://github.com/one-more-minute/Lazy.jl 
> > > > > 
> > > > > I'm hope somebody else will be able to pop in an give a more 
> > > > > thorough answer, but the above may at least be a place to 
> > > start. 
> > > > > 
> > > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor 
> > > > > wrote: 
> > > > > > Hadley Wickham's lazyeval package in R is pretty cool in that 
> > > you 
> > > > > > can attach an environment to an expression, pass it in and 
> > > out of 
> > > > > > functions with various modifications, and then evaluate the 
> > > > > > expression within the original environment (or any other 
> > > > > > environment that you choose). R in general has the functions 
> > > like 
> > > > > > list2env and list(environment()) that allow one to convert an 
> > > 
> > > > > > environment into a list and back again (list being the R 
> > > > > > equivalent of a Dict). Are there any plans to add these kind 
> > > of 
> > > > > > features to Julia? 
> > > > > > 
>

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

In fact, if environments could be reifyed, I'd go so far as to say that 
Julia should have a separate syntax for a) evaluating an expression in its 
defining environment and b) evaluating an expression in its evaluation 
environment.

On Wednesday, July 8, 2015 at 3:53:29 PM UTC-4, Brandon Taylor wrote:
>
> For comparison, here is the analogous code from DataFramesMeta.
>
> replace_syms(x, membernames) = x
> function replace_syms(e::Expr, membernames)
> if e.head == :call && length(e.args) == 2 && e.args[1] == :^
> return e.args[2]
> elseif e.head == :. # special case for :a.b
> return Expr(e.head, replace_syms(e.args[1], membernames),
> typeof(e.args[2]) == Expr && e.args[2].head == 
> :quote ? e.args[2] : replace_syms(e.args[2], membernames))
> elseif e.head != :quote
> return Expr(e.head, (isempty(e.args) ? e.args : map(x -> 
> replace_syms(x, membernames), e.args))...)
> else
> if haskey(membernames, e.args[1])
> return membernames[e.args[1]]
> else
> a = gensym()
> membernames[e.args[1]] = a
> return a
> end
> end
> end
>
> function with_helper(d, body)
> membernames = Dict{Symbol, Symbol}()
> body = replace_syms(body, membernames)
> funargs = map(x -> :( getindex($d, $(Meta.quot(x))) ), 
> collect(keys(membernames)))
> funname = gensym()
> return(:( function $funname($(collect(values(membernames))...)) $body 
> end; $funname($(funargs...)) ))
> end
>
> macro with(d, body)
> esc(with_helper(d, body))
> end
>
> On Wednesday, July 8, 2015 at 3:50:12 PM UTC-4, Brandon Taylor wrote:
>>
>> It should be noted that in R, environments are simply pointers. They take 
>> up no memory and shouldn't cause drastic slowdowns (I don't think).
>>
>> On Wednesday, July 8, 2015 at 3:48:08 PM UTC-4, Brandon Taylor wrote:
>>>
>>> For example, here's a code snippet from R:
>>>
>>> function (x, data) eval(x$expr, data, x$env)
>>>
>>> x is an expression which contains an environment attribute. It is 
>>> evaluated first checking the bindings in data (data being a dataframe) and 
>>> then checing the bindings in x's environment.
>>>
>>> On Wednesday, July 8, 2015 at 3:37:04 PM UTC-4, Brandon Taylor wrote:

 *reifying. Deifying environments might not be the best idea.

 On Wednesday, July 8, 2015 at 3:34:53 PM UTC-4, Brandon Taylor wrote:
>
> I was aware of those packages (though I hadn't read the discussions 
> referenced). Macros are great but they are incredibly difficult to reason 
> with concerning issues of scope (at least for me). Deifying environments 
> could solve all of these issues (and so much more) in one fell swoop.
>
> On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:
>>
>> Some of these issues have been thought about fairly extensively by 
>> the stats community in particular, precisely on account of the use cases 
>> you cite:
>>
>> https://github.com/JuliaStats/DataFrames.jl/pull/472
>> https://github.com/JuliaStats/DataFrames.jl/issues/504 
>> 
>>
>> I think that the matter is still very much an open question. I have 
>> no sense that anything is going to be added to Base Julia itself. 
>> Currently, the best way (that I know of, anyway) to achieve the delayed 
>> evaluation effect is via the use of macros. See for instance:
>>
>> https://github.com/JuliaStats/DataFramesMeta.jl
>> https://github.com/one-more-minute/Lazy.jl
>>
>> I'm hope somebody else will be able to pop in an give a more thorough 
>> answer, but the above may at least be a place to start.
>>
>> On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>>>
>>> Hadley Wickham's lazyeval package in R is pretty cool in that you 
>>> can attach an environment to an expression, pass it in and out of 
>>> functions 
>>> with various modifications, and then evaluate the expression within the 
>>> original environment (or any other environment that you choose). R in 
>>> general has the functions like list2env and list(environment()) that 
>>> allow 
>>> one to convert an environment into a list and back again (list being 
>>> the R 
>>> equivalent of a Dict). Are there any plans to add these kind of 
>>> features to 
>>> Julia?
>>>
>>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le mercredi 08 juillet 2015 à 12:57 -0700, Brandon Taylor a écrit :
> If scoping rules are too complicated and cause confusion, why are 
> they built into the base implementation of function?
What do you mean? Which function?

> On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet-Valat 
> wrote:
> > Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a écrit : 
> > 
> > > I was aware of those packages (though I hadn't read the 
> > discussions 
> > > referenced). Macros are great but they are incredibly difficult 
> > to 
> > > reason with concerning issues of scope (at least for me). 
> > Deifying 
> > > environments could solve all of these issues (and so much more) 
> > in 
> > > one fell swoop. 
> > On the contrary, I think well-designed macros can be much easier to 
> > 
> > think about than environments in R. If the macro takes a DataFrame 
> > object and an expression, there's no ambiguity about what the scope 
> > is. 
> > This is even better if variables that should be found in the data 
> > frame 
> > are passed as symbols, like :var, while standard variables are 
> > specified as usual. 
> > 
> > On the other hand, I find R formulas too flexible and complex to 
> > reason 
> > about. You never know whether an object will be found in the 
> > formula's 
> > environment, in one of the parent environments of the 
> > function/package 
> > you called, in your function, or in the global environment. 
> > 
> > 
> > Regards 
> > 
> > > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote: 
> > 
> > > > Some of these issues have been thought about fairly extensively 
> > by 
> > > > the stats community in particular, precisely on account of the 
> > use 
> > > > cases you cite: 
> > > > 
> > > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
> > > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
> > > > 
> > > > I think that the matter is still very much an open question. I 
> > have 
> > > > no sense that anything is going to be added to Base Julia 
> > itself. 
> > > > Currently, the best way (that I know of, anyway) to achieve the 
> > 
> > > > delayed evaluation effect is via the use of macros. See for 
> > > > instance: 
> > > > 
> > > > https://github.com/JuliaStats/DataFramesMeta.jl 
> > > > https://github.com/one-more-minute/Lazy.jl 
> > > > 
> > > > I'm hope somebody else will be able to pop in an give a more 
> > > > thorough answer, but the above may at least be a place to 
> > start. 
> > > > 
> > > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor 
> > > > wrote: 
> > > > > Hadley Wickham's lazyeval package in R is pretty cool in that 
> > you 
> > > > > can attach an environment to an expression, pass it in and 
> > out of 
> > > > > functions with various modifications, and then evaluate the 
> > > > > expression within the original environment (or any other 
> > > > > environment that you choose). R in general has the functions 
> > like 
> > > > > list2env and list(environment()) that allow one to convert an 
> > 
> > > > > environment into a list and back again (list being the R 
> > > > > equivalent of a Dict). Are there any plans to add these kind 
> > of 
> > > > > features to Julia? 
> > > > > 

Re: [julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

If scoping rules are too complicated and cause confusion, why are they 
built into the base implementation of function?

On Wednesday, July 8, 2015 at 3:48:52 PM UTC-4, Milan Bouchet-Valat wrote:
>
> Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a écrit : 
> > I was aware of those packages (though I hadn't read the discussions 
> > referenced). Macros are great but they are incredibly difficult to 
> > reason with concerning issues of scope (at least for me). Deifying 
> > environments could solve all of these issues (and so much more) in 
> > one fell swoop. 
> On the contrary, I think well-designed macros can be much easier to 
> think about than environments in R. If the macro takes a DataFrame 
> object and an expression, there's no ambiguity about what the scope is. 
> This is even better if variables that should be found in the data frame 
> are passed as symbols, like :var, while standard variables are 
> specified as usual. 
>
> On the other hand, I find R formulas too flexible and complex to reason 
> about. You never know whether an object will be found in the formula's 
> environment, in one of the parent environments of the function/package 
> you called, in your function, or in the global environment. 
>
>
> Regards 
>
> > On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote: 
> > > Some of these issues have been thought about fairly extensively by 
> > > the stats community in particular, precisely on account of the use 
> > > cases you cite: 
> > > 
> > > https://github.com/JuliaStats/DataFrames.jl/pull/472 
> > > https://github.com/JuliaStats/DataFrames.jl/issues/504 
> > > 
> > > I think that the matter is still very much an open question. I have 
> > > no sense that anything is going to be added to Base Julia itself. 
> > > Currently, the best way (that I know of, anyway) to achieve the 
> > > delayed evaluation effect is via the use of macros. See for 
> > > instance: 
> > > 
> > > https://github.com/JuliaStats/DataFramesMeta.jl 
> > > https://github.com/one-more-minute/Lazy.jl 
> > > 
> > > I'm hope somebody else will be able to pop in an give a more 
> > > thorough answer, but the above may at least be a place to start. 
> > > 
> > > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor 
> > > wrote: 
> > > > Hadley Wickham's lazyeval package in R is pretty cool in that you 
> > > > can attach an environment to an expression, pass it in and out of 
> > > > functions with various modifications, and then evaluate the 
> > > > expression within the original environment (or any other 
> > > > environment that you choose). R in general has the functions like 
> > > > list2env and list(environment()) that allow one to convert an 
> > > > environment into a list and back again (list being the R 
> > > > equivalent of a Dict). Are there any plans to add these kind of 
> > > > features to Julia? 
> > > > 
>

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

For comparison, here is the analogous code from DataFramesMeta.

replace_syms(x, membernames) = x
function replace_syms(e::Expr, membernames)
if e.head == :call && length(e.args) == 2 && e.args[1] == :^
return e.args[2]
elseif e.head == :. # special case for :a.b
return Expr(e.head, replace_syms(e.args[1], membernames),
typeof(e.args[2]) == Expr && e.args[2].head == 
:quote ? e.args[2] : replace_syms(e.args[2], membernames))
elseif e.head != :quote
return Expr(e.head, (isempty(e.args) ? e.args : map(x -> 
replace_syms(x, membernames), e.args))...)
else
if haskey(membernames, e.args[1])
return membernames[e.args[1]]
else
a = gensym()
membernames[e.args[1]] = a
return a
end
end
end

function with_helper(d, body)
membernames = Dict{Symbol, Symbol}()
body = replace_syms(body, membernames)
funargs = map(x -> :( getindex($d, $(Meta.quot(x))) ), 
collect(keys(membernames)))
funname = gensym()
return(:( function $funname($(collect(values(membernames))...)) $body 
end; $funname($(funargs...)) ))
end

macro with(d, body)
esc(with_helper(d, body))
end

On Wednesday, July 8, 2015 at 3:50:12 PM UTC-4, Brandon Taylor wrote:
>
> It should be noted that in R, environments are simply pointers. They take 
> up no memory and shouldn't cause drastic slowdowns (I don't think).
>
> On Wednesday, July 8, 2015 at 3:48:08 PM UTC-4, Brandon Taylor wrote:
>>
>> For example, here's a code snippet from R:
>>
>> function (x, data) eval(x$expr, data, x$env)
>>
>> x is an expression which contains an environment attribute. It is 
>> evaluated first checking the bindings in data (data being a dataframe) and 
>> then checing the bindings in x's environment.
>>
>> On Wednesday, July 8, 2015 at 3:37:04 PM UTC-4, Brandon Taylor wrote:
>>>
>>> *reifying. Deifying environments might not be the best idea.
>>>
>>> On Wednesday, July 8, 2015 at 3:34:53 PM UTC-4, Brandon Taylor wrote:

 I was aware of those packages (though I hadn't read the discussions 
 referenced). Macros are great but they are incredibly difficult to reason 
 with concerning issues of scope (at least for me). Deifying environments 
 could solve all of these issues (and so much more) in one fell swoop.

 On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:
>
> Some of these issues have been thought about fairly extensively by the 
> stats community in particular, precisely on account of the use cases you 
> cite:
>
> https://github.com/JuliaStats/DataFrames.jl/pull/472
> https://github.com/JuliaStats/DataFrames.jl/issues/504 
> 
>
> I think that the matter is still very much an open question. I have no 
> sense that anything is going to be added to Base Julia itself. Currently, 
> the best way (that I know of, anyway) to achieve the delayed evaluation 
> effect is via the use of macros. See for instance:
>
> https://github.com/JuliaStats/DataFramesMeta.jl
> https://github.com/one-more-minute/Lazy.jl
>
> I'm hope somebody else will be able to pop in an give a more thorough 
> answer, but the above may at least be a place to start.
>
> On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>>
>> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>> attach an environment to an expression, pass it in and out of functions 
>> with various modifications, and then evaluate the expression within the 
>> original environment (or any other environment that you choose). R in 
>> general has the functions like list2env and list(environment()) that 
>> allow 
>> one to convert an environment into a list and back again (list being the 
>> R 
>> equivalent of a Dict). Are there any plans to add these kind of features 
>> to 
>> Julia?
>>
>

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

It should be noted that in R, environments are simply pointers. They take 
up no memory and shouldn't cause drastic slowdowns (I don't think).

On Wednesday, July 8, 2015 at 3:48:08 PM UTC-4, Brandon Taylor wrote:
>
> For example, here's a code snippet from R:
>
> function (x, data) eval(x$expr, data, x$env)
>
> x is an expression which contains an environment attribute. It is 
> evaluated first checking the bindings in data (data being a dataframe) and 
> then checing the bindings in x's environment.
>
> On Wednesday, July 8, 2015 at 3:37:04 PM UTC-4, Brandon Taylor wrote:
>>
>> *reifying. Deifying environments might not be the best idea.
>>
>> On Wednesday, July 8, 2015 at 3:34:53 PM UTC-4, Brandon Taylor wrote:
>>>
>>> I was aware of those packages (though I hadn't read the discussions 
>>> referenced). Macros are great but they are incredibly difficult to reason 
>>> with concerning issues of scope (at least for me). Deifying environments 
>>> could solve all of these issues (and so much more) in one fell swoop.
>>>
>>> On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:

 Some of these issues have been thought about fairly extensively by the 
 stats community in particular, precisely on account of the use cases you 
 cite:

 https://github.com/JuliaStats/DataFrames.jl/pull/472
 https://github.com/JuliaStats/DataFrames.jl/issues/504 
 

 I think that the matter is still very much an open question. I have no 
 sense that anything is going to be added to Base Julia itself. Currently, 
 the best way (that I know of, anyway) to achieve the delayed evaluation 
 effect is via the use of macros. See for instance:

 https://github.com/JuliaStats/DataFramesMeta.jl
 https://github.com/one-more-minute/Lazy.jl

 I'm hope somebody else will be able to pop in an give a more thorough 
 answer, but the above may at least be a place to start.

 On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>
> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
> attach an environment to an expression, pass it in and out of functions 
> with various modifications, and then evaluate the expression within the 
> original environment (or any other environment that you choose). R in 
> general has the functions like list2env and list(environment()) that 
> allow 
> one to convert an environment into a list and back again (list being the 
> R 
> equivalent of a Dict). Are there any plans to add these kind of features 
> to 
> Julia?
>

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

For example, here's a code snippet from R:

function (x, data) eval(x$expr, data, x$env)

x is an expression which contains an environment attribute. It is evaluated 
first checking the bindings in data (data being a dataframe) and then 
checing the bindings in x's environment.

On Wednesday, July 8, 2015 at 3:37:04 PM UTC-4, Brandon Taylor wrote:
>
> *reifying. Deifying environments might not be the best idea.
>
> On Wednesday, July 8, 2015 at 3:34:53 PM UTC-4, Brandon Taylor wrote:
>>
>> I was aware of those packages (though I hadn't read the discussions 
>> referenced). Macros are great but they are incredibly difficult to reason 
>> with concerning issues of scope (at least for me). Deifying environments 
>> could solve all of these issues (and so much more) in one fell swoop.
>>
>> On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:
>>>
>>> Some of these issues have been thought about fairly extensively by the 
>>> stats community in particular, precisely on account of the use cases you 
>>> cite:
>>>
>>> https://github.com/JuliaStats/DataFrames.jl/pull/472
>>> https://github.com/JuliaStats/DataFrames.jl/issues/504 
>>> 
>>>
>>> I think that the matter is still very much an open question. I have no 
>>> sense that anything is going to be added to Base Julia itself. Currently, 
>>> the best way (that I know of, anyway) to achieve the delayed evaluation 
>>> effect is via the use of macros. See for instance:
>>>
>>> https://github.com/JuliaStats/DataFramesMeta.jl
>>> https://github.com/one-more-minute/Lazy.jl
>>>
>>> I'm hope somebody else will be able to pop in an give a more thorough 
>>> answer, but the above may at least be a place to start.
>>>
>>> On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:

 Hadley Wickham's lazyeval package in R is pretty cool in that you can 
 attach an environment to an expression, pass it in and out of functions 
 with various modifications, and then evaluate the expression within the 
 original environment (or any other environment that you choose). R in 
 general has the functions like list2env and list(environment()) that allow 
 one to convert an environment into a list and back again (list being the R 
 equivalent of a Dict). Are there any plans to add these kind of features 
 to 
 Julia?

>>>

Re: [julia-users] Re: Environment reification and lazy evaluation

[Milan Bouchet-Valat]

Le mercredi 08 juillet 2015 à 12:34 -0700, Brandon Taylor a écrit :
> I was aware of those packages (though I hadn't read the discussions 
> referenced). Macros are great but they are incredibly difficult to 
> reason with concerning issues of scope (at least for me). Deifying 
> environments could solve all of these issues (and so much more) in 
> one fell swoop.
On the contrary, I think well-designed macros can be much easier to
think about than environments in R. If the macro takes a DataFrame
object and an expression, there's no ambiguity about what the scope is.
This is even better if variables that should be found in the data frame
are passed as symbols, like :var, while standard variables are
specified as usual.

On the other hand, I find R formulas too flexible and complex to reason
about. You never know whether an object will be found in the formula's
environment, in one of the parent environments of the function/package
you called, in your function, or in the global environment.


Regards

> On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:
> > Some of these issues have been thought about fairly extensively by 
> > the stats community in particular, precisely on account of the use 
> > cases you cite:
> > 
> > https://github.com/JuliaStats/DataFrames.jl/pull/472
> > https://github.com/JuliaStats/DataFrames.jl/issues/504
> > 
> > I think that the matter is still very much an open question. I have 
> > no sense that anything is going to be added to Base Julia itself. 
> > Currently, the best way (that I know of, anyway) to achieve the 
> > delayed evaluation effect is via the use of macros. See for 
> > instance:
> > 
> > https://github.com/JuliaStats/DataFramesMeta.jl
> > https://github.com/one-more-minute/Lazy.jl
> > 
> > I'm hope somebody else will be able to pop in an give a more 
> > thorough answer, but the above may at least be a place to start.
> > 
> > On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor 
> > wrote:
> > > Hadley Wickham's lazyeval package in R is pretty cool in that you 
> > > can attach an environment to an expression, pass it in and out of 
> > > functions with various modifications, and then evaluate the 
> > > expression within the original environment (or any other 
> > > environment that you choose). R in general has the functions like 
> > > list2env and list(environment()) that allow one to convert an 
> > > environment into a list and back again (list being the R 
> > > equivalent of a Dict). Are there any plans to add these kind of 
> > > features to Julia?
> > > 

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

*reifying. Deifying environments might not be the best idea.

On Wednesday, July 8, 2015 at 3:34:53 PM UTC-4, Brandon Taylor wrote:
>
> I was aware of those packages (though I hadn't read the discussions 
> referenced). Macros are great but they are incredibly difficult to reason 
> with concerning issues of scope (at least for me). Deifying environments 
> could solve all of these issues (and so much more) in one fell swoop.
>
> On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:
>>
>> Some of these issues have been thought about fairly extensively by the 
>> stats community in particular, precisely on account of the use cases you 
>> cite:
>>
>> https://github.com/JuliaStats/DataFrames.jl/pull/472
>> https://github.com/JuliaStats/DataFrames.jl/issues/504 
>> 
>>
>> I think that the matter is still very much an open question. I have no 
>> sense that anything is going to be added to Base Julia itself. Currently, 
>> the best way (that I know of, anyway) to achieve the delayed evaluation 
>> effect is via the use of macros. See for instance:
>>
>> https://github.com/JuliaStats/DataFramesMeta.jl
>> https://github.com/one-more-minute/Lazy.jl
>>
>> I'm hope somebody else will be able to pop in an give a more thorough 
>> answer, but the above may at least be a place to start.
>>
>> On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>>>
>>> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>>> attach an environment to an expression, pass it in and out of functions 
>>> with various modifications, and then evaluate the expression within the 
>>> original environment (or any other environment that you choose). R in 
>>> general has the functions like list2env and list(environment()) that allow 
>>> one to convert an environment into a list and back again (list being the R 
>>> equivalent of a Dict). Are there any plans to add these kind of features to 
>>> Julia?
>>>
>>

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

I was aware of those packages (though I hadn't read the discussions 
referenced). Macros are great but they are incredibly difficult to reason 
with concerning issues of scope (at least for me). Deifying environments 
could solve all of these issues (and so much more) in one fell swoop.

On Wednesday, July 8, 2015 at 3:20:00 PM UTC-4, David Gold wrote:
>
> Some of these issues have been thought about fairly extensively by the 
> stats community in particular, precisely on account of the use cases you 
> cite:
>
> https://github.com/JuliaStats/DataFrames.jl/pull/472
> https://github.com/JuliaStats/DataFrames.jl/issues/504 
> 
>
> I think that the matter is still very much an open question. I have no 
> sense that anything is going to be added to Base Julia itself. Currently, 
> the best way (that I know of, anyway) to achieve the delayed evaluation 
> effect is via the use of macros. See for instance:
>
> https://github.com/JuliaStats/DataFramesMeta.jl
> https://github.com/one-more-minute/Lazy.jl
>
> I'm hope somebody else will be able to pop in an give a more thorough 
> answer, but the above may at least be a place to start.
>
> On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>>
>> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
>> attach an environment to an expression, pass it in and out of functions 
>> with various modifications, and then evaluate the expression within the 
>> original environment (or any other environment that you choose). R in 
>> general has the functions like list2env and list(environment()) that allow 
>> one to convert an environment into a list and back again (list being the R 
>> equivalent of a Dict). Are there any plans to add these kind of features to 
>> Julia?
>>
>

[julia-users] Re: Environment reification and lazy evaluation

[David Gold]

Some of these issues have been thought about fairly extensively by the 
stats community in particular, precisely on account of the use cases you 
cite:

https://github.com/JuliaStats/DataFrames.jl/pull/472
https://github.com/JuliaStats/DataFrames.jl/issues/504

I think that the matter is still very much an open question. I have no 
sense that anything is going to be added to Base Julia itself. Currently, 
the best way (that I know of, anyway) to achieve the delayed evaluation 
effect is via the use of macros. See for instance:

https://github.com/JuliaStats/DataFramesMeta.jl
https://github.com/one-more-minute/Lazy.jl

I'm hope somebody else will be able to pop in an give a more thorough 
answer, but the above may at least be a place to start.

On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>
> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
> attach an environment to an expression, pass it in and out of functions 
> with various modifications, and then evaluate the expression within the 
> original environment (or any other environment that you choose). R in 
> general has the functions like list2env and list(environment()) that allow 
> one to convert an environment into a list and back again (list being the R 
> equivalent of a Dict). Are there any plans to add these kind of features to 
> Julia?
>

[julia-users] Re: Environment reification and lazy evaluation

[Brandon Taylor]

Also a features that allow direct modification of an environment (addition 
or removal of bindings, etc.)

On Wednesday, July 8, 2015 at 2:03:45 PM UTC-4, Brandon Taylor wrote:
>
> Hadley Wickham's lazyeval package in R is pretty cool in that you can 
> attach an environment to an expression, pass it in and out of functions 
> with various modifications, and then evaluate the expression within the 
> original environment (or any other environment that you choose). R in 
> general has the functions like list2env and list(environment()) that allow 
> one to convert an environment into a list and back again (list being the R 
> equivalent of a Dict). Are there any plans to add these kind of features to 
> Julia?
>