Ok so I've got a good start. I bet John Myles White didn't think I could
get this far. Anyway, I'm getting caught up in defining my own escape
function. I'm getting lost in multiple layers of meta.
using DataFrames
import Base.convert
# allow inheritance from modules
function convert(::Type{Dict}, m::Module)
dict = Dict()
for name in names(m)
dict[name] = eval( :(Base.$name) )
end
dict
end
base_dict = convert(Dict, Base)
# allow inheritance from DataFrames
function convert(::Type{Dict}, d::DataFrame)
dict = Dict()
for name in names(d)
dict[name] = d[name]
end
dict
end
# modify dicts such that if a key is not found, search the parent
function Base.getindex{K,V}(h::Dict{K,V}, key)
index = Base.ht_keyindex(h, key)
if index < 0
if :_parent in keys(h)
Base.getindex(h[:_parent], key)
else
throw(KeyError(key))
end
else
h.vals[index]::V
end
end
# test expression
e =
quote
a = 1
# anonymous functions required for proper scoping
test = function()
b = a
end
end
# set up the global environment
_env = gensym()
eval(:($_env = [:_parent => base_dict] ) )
#_new_env = _env
# this is the code that needs to be escaped
#eval(:($_new_env = [:_parent => eval(_env)] ) )
# reformat code to use dict scoping
function env_replace!(e::Expr,
_env::Symbol = _env)
# expressions wrapped in _esc will be left alone
if length(e.args) > 0
if (e.head == :call) & (e.args[1] == :_esc)
return e.args[2]
end
end
# set a new scope for a new function. This will also have to be done with
for loops, modules, etc.
if (e.head == :function)
# insert a new scope definition into the function definition
_new_env = gensym()
e.args[2].args = [
e.args[2].args[1],
:_esc(), #### need help here ###
e.args[2].args[2:end] ]
_env = _new_env
end
# ignore line numbers
if e.head != :line
for i in 1:length(e.args)
# replace symbols with their dict scoped version
if typeof(e.args[i]) == Symbol
e.args[i] = :($_env[$(string(e.args[i]))])
# recur into new expressions
elseif typeof(e.args[i]) == Expr
e.args[i] = env_replace!(e.args[i], _env)
end
end
end
e
end
# here is an eval that allows evaluation within a certain dict scope
function lazy_eval(e::Expr,
_env::Symbol = _env)
eval(env_replace!(e), _env)
end
## TO DO
# fix _esc problem
# prevent environment symbols from being scoped (perhaps with a special
marker)
# rescope for, while, try, catch, finally, let, and type
# perhaps use fast anonymous to avoid performance slowdowns?
On Tuesday, July 21, 2015 at 10:10:58 AM UTC+8, Brandon Taylor wrote:
>
> And there would need to be a special marker for them, such that if I'm in
> function f, f[:a] won't get preprocessed as f[:f][:a]
>
> On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
>>
>> Although that would probably require nested dicts. Each would have a
>> parent dict, and if a lookup isn't found in the current dict, the parent
>> dict would be searched.
>>
>> On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
>>>
>>> I should be possible to preprocess code such that everything is put into
>>> a dict based on the name of enclosing function (and global variables will
>>> just go into a dict called global).
>>>
>>> On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
>>>>
>>>> Dicts seem to work pretty well for this kind of thing.
>>>>
>>>> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>>>>>
>>>>> I'm getting a cannot assign variables in other modules error.
>>>>>
>>>>> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>>>>>
>>>>>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor
>>>>>> <brandon....@gmail.com> wrote:
>>>>>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be
>>>>>> possible
>>>>>> > to hijack this functionality to provide pseudo-environments? That
>>>>>> is, never
>>>>>> > referring to anything that is not already in an explicit module?
>>>>>> And also,
>>>>>> > have a data-frame simply be a module?
>>>>>>
>>>>>> I think this would in principle works. A module is basically what
>>>>>> global scope means so all the performance concern applies.
>>>>>>
>>>>>> >
>>>>>> >
>>>>>> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor
>>>>>> wrote:
>>>>>> >>
>>>>>> >> I don't know if you came across the vignette?
>>>>>> >>
>>>>>> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html
>>>>>> ?
>>>>>> >> dplyr uses lazyeval extensively, see
>>>>>> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html .
>>>>>> The cool
>>>>>> >> thing about being able to incorporate this kind of thing in Julia
>>>>>> would be
>>>>>> >> being able to use the self-reflection capabilities.
>>>>>> >>
>>>>>> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean
>>>>>> wrote:
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor
>>>>>> wrote:
>>>>>> >>>>
>>>>>> >>>> To walk back in time, you could say something like: compile this
>>>>>> like
>>>>>> >>>> this was is in line 8. Or compile this like this was in line 5.
>>>>>> It seems
>>>>>> >>>> like Julia already has some of this functionality in macros.
>>>>>> Internal
>>>>>> >>>> variables are compiled as if they were in local scope. But
>>>>>> escaped
>>>>>> >>>> expressions are compiled as if they were in global scope.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> Could you provide context or a real-world use? I've looked at the
>>>>>> >>> lazyeval package, and I'm not entirely sure what it does. Does it
>>>>>> provide
>>>>>> >>> lazy evaluation for R? That's easy to achieve in Julia (well,
>>>>>> sorta).
>>>>>> >>> Instead of
>>>>>> >>>
>>>>>> >>> d = determinant(matrix)
>>>>>> >>> ....
>>>>>> >>> u = 2 * d
>>>>>> >>>
>>>>>> >>> you can write
>>>>>> >>>
>>>>>> >>> d = ()->determinant(matrix)
>>>>>> >>> ....
>>>>>> >>> u = 2 * d() # determinant is evaluated on use, in the context
>>>>>> where it
>>>>>> >>> was originally defined
>>>>>> >>>
>>>>>> >>> With macros this can turn into
>>>>>> >>>
>>>>>> >>> d = lazy(determinant(matrix))
>>>>>> >>>
>>>>>> >>> which looks nicer (and also can avoid computing the determinant
>>>>>> twice if
>>>>>> >>> d() is called twice).
>>>>>> >>>
>>>>>> >>> Cédric
>>>>>> >>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean
>>>>>> wrote:
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor
>>>>>> wrote:
>>>>>> >>>>>>
>>>>>> >>>>>> Ok, here's where I'm getting hung up. You said that the
>>>>>> compiler
>>>>>> >>>>>> figures out the creation/lifetime of all variables at compile
>>>>>> time. So does
>>>>>> >>>>>> that mean there's a list like:
>>>>>> >>>>>>
>>>>>> >>>>>> a maps to location 0 and exists from line 3 to line 9
>>>>>> >>>>>> b maps to location 1 and exists from line 7 to line 9
>>>>>> >>>>>> a maps to location 10 and exists from line 7 to 9?
>>>>>> >>>>>>
>>>>>> >>>>>> and that to map variables to locations on any particular line,
>>>>>> the
>>>>>> >>>>>> compiler works its way up the list,
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> Yes, more or less.
>>>>>> >>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> This is perhaps even more helpful than the environment. The
>>>>>> >>>>>> environment is immediately and completely determinable at any
>>>>>> point in the
>>>>>> >>>>>> program. This could make it possible to walk back in time even
>>>>>> within the
>>>>>> >>>>>> same scope.
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> Could you expand on what you're thinking of?
>>>>>> >>>>>
>>>>>> >>>>> This kind of compile-time environment could conceivably be
>>>>>> exposed to
>>>>>> >>>>> macros. Common Lisp had proposals along that line
>>>>>> >>>>> (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html)
>>>>>> but as far as
>>>>>> >>>>> I can tell, it was too complicated and not useful enough, so it
>>>>>> was
>>>>>> >>>>> axed/neutered at some point in the standardization process.
>>>>>> >>>>>
>>>>>> >>>>> > Hadley Wickham's lazyeval package in R is pretty cool in that
>>>>>> you can
>>>>>> >>>>> > attach an environment to an expression, pass it in and out of
>>>>>> functions with
>>>>>> >>>>> > various modifications, and then evaluate the expression
>>>>>> within the original
>>>>>> >>>>> > environment
>>>>>> >>>>>
>>>>>> >>>>> I don't know about R, but to me that sounds entirely doable
>>>>>> with
>>>>>> >>>>> closures (and macros will give you a nice syntax for it)
>>>>>> >>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu
>>>>>> wrote:
>>>>>> >>>>>>>
>>>>>> >>>>>>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu <yyc...@gmail.com>
>>>>>> wrote:
>>>>>> >>>>>>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
>>>>>> >>>>>>> > <brandon....@gmail.com> wrote:
>>>>>> >>>>>>> >> Hmm, maybe I'm confused about compilation vs
>>>>>> interpretation. Let
>>>>>> >>>>>>> >> me
>>>>>> >>>>>>> >> rephrase. Regardless of a how or when statement is
>>>>>> evaluated, it
>>>>>> >>>>>>> >> must have
>>>>>> >>>>>>> >> access at least to its parent environments to successfully
>>>>>> resolve
>>>>>> >>>>>>> >> a symbol.
>>>>>> >>>>>>>
>>>>>> >>>>>>> AFAIK, the only scope you can dynamically add variable to is
>>>>>> the
>>>>>> >>>>>>> global scope. (This can be done with the `global` keyword or
>>>>>> `eval`
>>>>>> >>>>>>> etc). The compiler figure out the creation/lifetime of all
>>>>>> local
>>>>>> >>>>>>> variables (at compile time). Therefore, to access a variable
>>>>>> in the
>>>>>> >>>>>>> parent scope:
>>>>>> >>>>>>>
>>>>>> >>>>>>> 1. If it's a global, then it need a runtime lookup/binding
>>>>>> (the
>>>>>> >>>>>>> reason
>>>>>> >>>>>>> global are slow)
>>>>>> >>>>>>> 2. If it's in a parent non-global scope, the compiler can
>>>>>> figure out
>>>>>> >>>>>>> how to bind/access it at compile time and no extra (lookup)
>>>>>> code at
>>>>>> >>>>>>> runtime is necessary.
>>>>>> >>>>>>>
>>>>>> >>>>>>> >>
>>>>>> >>>>>>> >
>>>>>> >>>>>>> > A julia local variable is basically a variable in C.
>>>>>> There's a
>>>>>> >>>>>>> > table
>>>>>> >>>>>>> > at compile time to map between symbols and stack slots (or
>>>>>> >>>>>>> > whereever
>>>>>> >>>>>>> > they are stored) but such a map does not exist at runtime
>>>>>> anymore
>>>>>> >>>>>>> > (except for debugging).
>>>>>> >>>>>>> >
>>>>>> >>>>>>> >>
>>>>>> >>>>>>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon
>>>>>> Taylor
>>>>>> >>>>>>> >> wrote:
>>>>>> >>>>>>> >>>
>>>>>> >>>>>>> >>> They must exist at runtime and at local scope. Evaluating
>>>>>> a
>>>>>> >>>>>>> >>> symbol is
>>>>>> >>>>>>> >>> impossible without a pool of defined symbols in various
>>>>>> scopes to
>>>>>> >>>>>>> >>> match it
>>>>>> >>>>>>> >>> to. Unless I'm missing something?
>>>>>> >>>>>>> >>>
>>>>>> >>>>>>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson
>>>>>> wrote:
>>>>>> >>>>>>> >>>>
>>>>>> >>>>>>> >>>> There are global symbol tables for static analysis /
>>>>>> reflection,
>>>>>> >>>>>>> >>>> but they
>>>>>> >>>>>>> >>>> do not exist at runtime or for the local scope.
>>>>>> >>>>>>> >>>>
>>>>>> >>>>>>> >>>> On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor
>>>>>> >>>>>>> >>>> <brandon....@gmail.com>
>>>>>> >>>>>>> >>>> wrote:
>>>>>> >>>>>>> >>>>>
>>>>>> >>>>>>> >>>>> Surely environments already exist somewhere inside
>>>>>> Julia? How
>>>>>> >>>>>>> >>>>> else could
>>>>>> >>>>>>> >>>>> you keep track of scope? It would be simply a matter of
>>>>>> >>>>>>> >>>>> granting users
>>>>>> >>>>>>> >>>>> access to them. Symbol tables in a mutable language are
>>>>>> by
>>>>>> >>>>>>> >>>>> default mutable.
>>>>>> >>>>>>> >>>>> It would certainly be possible only give users access
>>>>>> to
>>>>>> >>>>>>> >>>>> immutable
>>>>>> >>>>>>> >>>>> reifications (which could solve a bunch of problems as
>>>>>> is).
>>>>>> >>>>>>> >>>>> However, it
>>>>>> >>>>>>> >>>>> seems natural to match mutable symbol tables with
>>>>>> mutable
>>>>>> >>>>>>> >>>>> reifications, and
>>>>>> >>>>>>> >>>>> immutable symbol tables with immutable reifications.
>>>>>> >>>>>>> >>>>>
>>>>>> >>>>>>> >>>>>
>>>>>> >>>>>>> >>>>> On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4, Brandon
>>>>>> Taylor
>>>>>> >>>>>>> >>>>> wrote:
>>>>>> >>>>>>> >>>>>>
>>>>>> >>>>>>> >>>>>> I'm not sure I understand...
>>>>>> >>>>>>> >>>>>>
>>>>>> >>>>>>> >>>>>> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John
>>>>>> Myles
>>>>>> >>>>>>> >>>>>> White wrote:
>>>>>> >>>>>>> >>>>>>>
>>>>>> >>>>>>> >>>>>>> Reified scope makes static analysis much too hard.
>>>>>> Take any
>>>>>> >>>>>>> >>>>>>> criticism
>>>>>> >>>>>>> >>>>>>> of mutable state: they all apply to globally mutable
>>>>>> symbol
>>>>>> >>>>>>> >>>>>>> tables.
>>>>>> >>>>>>> >>>>>>>
>>>>>> >>>>>>> >>>>>>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2,
>>>>>> Milan
>>>>>> >>>>>>> >>>>>>> Bouchet-Valat
>>>>>> >>>>>>> >>>>>>> wrote:
>>>>>> >>>>>>> >>>>>>>>
>>>>>> >>>>>>> >>>>>>>> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon
>>>>>> Taylor a
>>>>>> >>>>>>> >>>>>>>> écrit :
>>>>>> >>>>>>> >>>>>>>> > All functions.
>>>>>> >>>>>>> >>>>>>>> Well, I don't know of any language which doesn't
>>>>>> have
>>>>>> >>>>>>> >>>>>>>> scoping
>>>>>> >>>>>>> >>>>>>>> rules...
>>>>>> >>>>>>> >>>>>>>>
>>>>>> >>>>>>> >>>>>>>> Anyway, I didn't say scoping rules are necessarily
>>>>>> >>>>>>> >>>>>>>> confusing, I was
>>>>>> >>>>>>> >>>>>>>> only referring to R formulas. But according to the
>>>>>> examples
>>>>>> >>>>>>> >>>>>>>> you
>>>>>> >>>>>>> >>>>>>>> posted,
>>>>>> >>>>>>> >>>>>>>> your question appears to be different.
>>>>>> >>>>>>> >>>>>>>>
>>>>>> >>>>>>> >>>>>>>>
>>>>>>
>>>>>
