More to do:
Expressions would also have to be escaped from quoting.
If we can't scope types within dicts, it might be necessary to have special
markers for types so they can avoid being scoped.
I don't think that macros will be necessary anymore
On Wednesday, July 22, 2015 at 11:14:38 AM UTC+8, Brandon Taylor wrote:
>
> Ok so I've got a good start. I bet John Myles White didn't think I could
> get this far. Anyway, I'm getting caught up in defining my own escape
> function. I'm getting lost in multiple layers of meta.
>
> using DataFrames
>
> import Base.convert
>
> # allow inheritance from modules
> function convert(::Type{Dict}, m::Module)
> dict = Dict()
> for name in names(m)
> dict[name] = eval( :(Base.$name) )
> end
> dict
> end
>
> base_dict = convert(Dict, Base)
>
> # allow inheritance from DataFrames
> function convert(::Type{Dict}, d::DataFrame)
> dict = Dict()
> for name in names(d)
> dict[name] = d[name]
> end
> dict
> end
>
> # modify dicts such that if a key is not found, search the parent
> function Base.getindex{K,V}(h::Dict{K,V}, key)
> index = Base.ht_keyindex(h, key)
> if index < 0
> if :_parent in keys(h)
> Base.getindex(h[:_parent], key)
> else
> throw(KeyError(key))
> end
> else
> h.vals[index]::V
> end
> end
>
> # test expression
> e =
> quote
> a = 1
> # anonymous functions required for proper scoping
> test = function()
> b = a
> end
> end
>
> # set up the global environment
> _env = gensym()
> eval(:($_env = [:_parent => base_dict] ) )
>
>
> #_new_env = _env
> # this is the code that needs to be escaped
> #eval(:($_new_env = [:_parent => eval(_env)] ) )
>
> # reformat code to use dict scoping
> function env_replace!(e::Expr,
> _env::Symbol = _env)
>
> # expressions wrapped in _esc will be left alone
> if length(e.args) > 0
> if (e.head == :call) & (e.args[1] == :_esc)
> return e.args[2]
> end
> end
>
> # set a new scope for a new function. This will also have to be done
> with for loops, modules, etc.
> if (e.head == :function)
> # insert a new scope definition into the function definition
> _new_env = gensym()
> e.args[2].args = [
> e.args[2].args[1],
> :_esc(), #### need help here ###
> e.args[2].args[2:end] ]
> _env = _new_env
> end
>
> # ignore line numbers
> if e.head != :line
>
> for i in 1:length(e.args)
> # replace symbols with their dict scoped version
> if typeof(e.args[i]) == Symbol
> e.args[i] = :($_env[$(string(e.args[i]))])
>
> # recur into new expressions
> elseif typeof(e.args[i]) == Expr
> e.args[i] = env_replace!(e.args[i], _env)
> end
> end
> end
> e
> end
>
> # here is an eval that allows evaluation within a certain dict scope
> function lazy_eval(e::Expr,
> _env::Symbol = _env)
> eval(env_replace!(e), _env)
> end
>
>
> ## TO DO
> # fix _esc problem
> # prevent environment symbols from being scoped (perhaps with a special
> marker)
> # rescope for, while, try, catch, finally, let, and type
> # perhaps use fast anonymous to avoid performance slowdowns?
>
>
> On Tuesday, July 21, 2015 at 10:10:58 AM UTC+8, Brandon Taylor wrote:
>>
>> And there would need to be a special marker for them, such that if I'm in
>> function f, f[:a] won't get preprocessed as f[:f][:a]
>>
>> On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
>>>
>>> Although that would probably require nested dicts. Each would have a
>>> parent dict, and if a lookup isn't found in the current dict, the parent
>>> dict would be searched.
>>>
>>> On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
>>>>
>>>> I should be possible to preprocess code such that everything is put
>>>> into a dict based on the name of enclosing function (and global variables
>>>> will just go into a dict called global).
>>>>
>>>> On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
>>>>>
>>>>> Dicts seem to work pretty well for this kind of thing.
>>>>>
>>>>> On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
>>>>>>
>>>>>> I'm getting a cannot assign variables in other modules error.
>>>>>>
>>>>>> On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
>>>>>>>
>>>>>>> On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor
>>>>>>> <brandon....@gmail.com> wrote:
>>>>>>> > Ok, a thought, Julia has an inbuilt idea of a module. Would it be
>>>>>>> possible
>>>>>>> > to hijack this functionality to provide pseudo-environments? That
>>>>>>> is, never
>>>>>>> > referring to anything that is not already in an explicit module?
>>>>>>> And also,
>>>>>>> > have a data-frame simply be a module?
>>>>>>>
>>>>>>> I think this would in principle works. A module is basically what
>>>>>>> global scope means so all the performance concern applies.
>>>>>>>
>>>>>>> >
>>>>>>> >
>>>>>>> > On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> I don't know if you came across the vignette?
>>>>>>> >>
>>>>>>> http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html
>>>>>>> ?
>>>>>>> >> dplyr uses lazyeval extensively, see
>>>>>>> >> http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html
>>>>>>> . The cool
>>>>>>> >> thing about being able to incorporate this kind of thing in Julia
>>>>>>> would be
>>>>>>> >> being able to use the self-reflection capabilities.
>>>>>>> >>
>>>>>>> >> On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean
>>>>>>> wrote:
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor
>>>>>>> wrote:
>>>>>>> >>>>
>>>>>>> >>>> To walk back in time, you could say something like: compile
>>>>>>> this like
>>>>>>> >>>> this was is in line 8. Or compile this like this was in line 5.
>>>>>>> It seems
>>>>>>> >>>> like Julia already has some of this functionality in macros.
>>>>>>> Internal
>>>>>>> >>>> variables are compiled as if they were in local scope. But
>>>>>>> escaped
>>>>>>> >>>> expressions are compiled as if they were in global scope.
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> Could you provide context or a real-world use? I've looked at
>>>>>>> the
>>>>>>> >>> lazyeval package, and I'm not entirely sure what it does. Does
>>>>>>> it provide
>>>>>>> >>> lazy evaluation for R? That's easy to achieve in Julia (well,
>>>>>>> sorta).
>>>>>>> >>> Instead of
>>>>>>> >>>
>>>>>>> >>> d = determinant(matrix)
>>>>>>> >>> ....
>>>>>>> >>> u = 2 * d
>>>>>>> >>>
>>>>>>> >>> you can write
>>>>>>> >>>
>>>>>>> >>> d = ()->determinant(matrix)
>>>>>>> >>> ....
>>>>>>> >>> u = 2 * d() # determinant is evaluated on use, in the context
>>>>>>> where it
>>>>>>> >>> was originally defined
>>>>>>> >>>
>>>>>>> >>> With macros this can turn into
>>>>>>> >>>
>>>>>>> >>> d = lazy(determinant(matrix))
>>>>>>> >>>
>>>>>>> >>> which looks nicer (and also can avoid computing the determinant
>>>>>>> twice if
>>>>>>> >>> d() is called twice).
>>>>>>> >>>
>>>>>>> >>> Cédric
>>>>>>> >>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> On Thursday, July 9, 2015 at 9:11:05 PM UTC-4, Cedric St-Jean
>>>>>>> wrote:
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> On Thursday, July 9, 2015 at 4:14:32 PM UTC-4, Brandon Taylor
>>>>>>> wrote:
>>>>>>> >>>>>>
>>>>>>> >>>>>> Ok, here's where I'm getting hung up. You said that the
>>>>>>> compiler
>>>>>>> >>>>>> figures out the creation/lifetime of all variables at compile
>>>>>>> time. So does
>>>>>>> >>>>>> that mean there's a list like:
>>>>>>> >>>>>>
>>>>>>> >>>>>> a maps to location 0 and exists from line 3 to line 9
>>>>>>> >>>>>> b maps to location 1 and exists from line 7 to line 9
>>>>>>> >>>>>> a maps to location 10 and exists from line 7 to 9?
>>>>>>> >>>>>>
>>>>>>> >>>>>> and that to map variables to locations on any particular
>>>>>>> line, the
>>>>>>> >>>>>> compiler works its way up the list,
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> Yes, more or less.
>>>>>>> >>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>> This is perhaps even more helpful than the environment. The
>>>>>>> >>>>>> environment is immediately and completely determinable at any
>>>>>>> point in the
>>>>>>> >>>>>> program. This could make it possible to walk back in time
>>>>>>> even within the
>>>>>>> >>>>>> same scope.
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> Could you expand on what you're thinking of?
>>>>>>> >>>>>
>>>>>>> >>>>> This kind of compile-time environment could conceivably be
>>>>>>> exposed to
>>>>>>> >>>>> macros. Common Lisp had proposals along that line
>>>>>>> >>>>> (https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node102.html)
>>>>>>> but as far as
>>>>>>> >>>>> I can tell, it was too complicated and not useful enough, so
>>>>>>> it was
>>>>>>> >>>>> axed/neutered at some point in the standardization process.
>>>>>>> >>>>>
>>>>>>> >>>>> > Hadley Wickham's lazyeval package in R is pretty cool in
>>>>>>> that you can
>>>>>>> >>>>> > attach an environment to an expression, pass it in and out
>>>>>>> of functions with
>>>>>>> >>>>> > various modifications, and then evaluate the expression
>>>>>>> within the original
>>>>>>> >>>>> > environment
>>>>>>> >>>>>
>>>>>>> >>>>> I don't know about R, but to me that sounds entirely doable
>>>>>>> with
>>>>>>> >>>>> closures (and macros will give you a nice syntax for it)
>>>>>>> >>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>> On Wednesday, July 8, 2015 at 8:31:44 PM UTC-4, Yichao Yu
>>>>>>> wrote:
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> On Wed, Jul 8, 2015 at 8:23 PM, Yichao Yu <yyc...@gmail.com>
>>>>>>> wrote:
>>>>>>> >>>>>>> > On Wed, Jul 8, 2015 at 7:48 PM, Brandon Taylor
>>>>>>> >>>>>>> > <brandon....@gmail.com> wrote:
>>>>>>> >>>>>>> >> Hmm, maybe I'm confused about compilation vs
>>>>>>> interpretation. Let
>>>>>>> >>>>>>> >> me
>>>>>>> >>>>>>> >> rephrase. Regardless of a how or when statement is
>>>>>>> evaluated, it
>>>>>>> >>>>>>> >> must have
>>>>>>> >>>>>>> >> access at least to its parent environments to
>>>>>>> successfully resolve
>>>>>>> >>>>>>> >> a symbol.
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> AFAIK, the only scope you can dynamically add variable to is
>>>>>>> the
>>>>>>> >>>>>>> global scope. (This can be done with the `global` keyword or
>>>>>>> `eval`
>>>>>>> >>>>>>> etc). The compiler figure out the creation/lifetime of all
>>>>>>> local
>>>>>>> >>>>>>> variables (at compile time). Therefore, to access a variable
>>>>>>> in the
>>>>>>> >>>>>>> parent scope:
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> 1. If it's a global, then it need a runtime lookup/binding
>>>>>>> (the
>>>>>>> >>>>>>> reason
>>>>>>> >>>>>>> global are slow)
>>>>>>> >>>>>>> 2. If it's in a parent non-global scope, the compiler can
>>>>>>> figure out
>>>>>>> >>>>>>> how to bind/access it at compile time and no extra (lookup)
>>>>>>> code at
>>>>>>> >>>>>>> runtime is necessary.
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> >>
>>>>>>> >>>>>>> >
>>>>>>> >>>>>>> > A julia local variable is basically a variable in C.
>>>>>>> There's a
>>>>>>> >>>>>>> > table
>>>>>>> >>>>>>> > at compile time to map between symbols and stack slots (or
>>>>>>> >>>>>>> > whereever
>>>>>>> >>>>>>> > they are stored) but such a map does not exist at runtime
>>>>>>> anymore
>>>>>>> >>>>>>> > (except for debugging).
>>>>>>> >>>>>>> >
>>>>>>> >>>>>>> >>
>>>>>>> >>>>>>> >> On Wednesday, July 8, 2015 at 7:34:09 PM UTC-4, Brandon
>>>>>>> Taylor
>>>>>>> >>>>>>> >> wrote:
>>>>>>> >>>>>>> >>>
>>>>>>> >>>>>>> >>> They must exist at runtime and at local scope.
>>>>>>> Evaluating a
>>>>>>> >>>>>>> >>> symbol is
>>>>>>> >>>>>>> >>> impossible without a pool of defined symbols in various
>>>>>>> scopes to
>>>>>>> >>>>>>> >>> match it
>>>>>>> >>>>>>> >>> to. Unless I'm missing something?
>>>>>>> >>>>>>> >>>
>>>>>>> >>>>>>> >>> On Wednesday, July 8, 2015 at 7:26:27 PM UTC-4, Jameson
>>>>>>> wrote:
>>>>>>> >>>>>>> >>>>
>>>>>>> >>>>>>> >>>> There are global symbol tables for static analysis /
>>>>>>> reflection,
>>>>>>> >>>>>>> >>>> but they
>>>>>>> >>>>>>> >>>> do not exist at runtime or for the local scope.
>>>>>>> >>>>>>> >>>>
>>>>>>> >>>>>>> >>>> On Wed, Jul 8, 2015 at 7:06 PM Brandon Taylor
>>>>>>> >>>>>>> >>>> <brandon....@gmail.com>
>>>>>>> >>>>>>> >>>> wrote:
>>>>>>> >>>>>>> >>>>>
>>>>>>> >>>>>>> >>>>> Surely environments already exist somewhere inside
>>>>>>> Julia? How
>>>>>>> >>>>>>> >>>>> else could
>>>>>>> >>>>>>> >>>>> you keep track of scope? It would be simply a matter
>>>>>>> of
>>>>>>> >>>>>>> >>>>> granting users
>>>>>>> >>>>>>> >>>>> access to them. Symbol tables in a mutable language
>>>>>>> are by
>>>>>>> >>>>>>> >>>>> default mutable.
>>>>>>> >>>>>>> >>>>> It would certainly be possible only give users access
>>>>>>> to
>>>>>>> >>>>>>> >>>>> immutable
>>>>>>> >>>>>>> >>>>> reifications (which could solve a bunch of problems as
>>>>>>> is).
>>>>>>> >>>>>>> >>>>> However, it
>>>>>>> >>>>>>> >>>>> seems natural to match mutable symbol tables with
>>>>>>> mutable
>>>>>>> >>>>>>> >>>>> reifications, and
>>>>>>> >>>>>>> >>>>> immutable symbol tables with immutable reifications.
>>>>>>> >>>>>>> >>>>>
>>>>>>> >>>>>>> >>>>>
>>>>>>> >>>>>>> >>>>> On Wednesday, July 8, 2015 at 6:50:03 PM UTC-4,
>>>>>>> Brandon Taylor
>>>>>>> >>>>>>> >>>>> wrote:
>>>>>>> >>>>>>> >>>>>>
>>>>>>> >>>>>>> >>>>>> I'm not sure I understand...
>>>>>>> >>>>>>> >>>>>>
>>>>>>> >>>>>>> >>>>>> On Wednesday, July 8, 2015 at 6:24:37 PM UTC-4, John
>>>>>>> Myles
>>>>>>> >>>>>>> >>>>>> White wrote:
>>>>>>> >>>>>>> >>>>>>>
>>>>>>> >>>>>>> >>>>>>> Reified scope makes static analysis much too hard.
>>>>>>> Take any
>>>>>>> >>>>>>> >>>>>>> criticism
>>>>>>> >>>>>>> >>>>>>> of mutable state: they all apply to globally mutable
>>>>>>> symbol
>>>>>>> >>>>>>> >>>>>>> tables.
>>>>>>> >>>>>>> >>>>>>>
>>>>>>> >>>>>>> >>>>>>> On Wednesday, July 8, 2015 at 10:26:23 PM UTC+2,
>>>>>>> Milan
>>>>>>> >>>>>>> >>>>>>> Bouchet-Valat
>>>>>>> >>>>>>> >>>>>>> wrote:
>>>>>>> >>>>>>> >>>>>>>>
>>>>>>> >>>>>>> >>>>>>>> Le mercredi 08 juillet 2015 à 13:20 -0700, Brandon
>>>>>>> Taylor a
>>>>>>> >>>>>>> >>>>>>>> écrit :
>>>>>>> >>>>>>> >>>>>>>> > All functions.
>>>>>>> >>>>>>> >>>>>>>> Well, I don't know of any language which doesn't
>>>>>>> have
>>>>>>> >>>>>>> >>>>>>>> scoping
>>>>>>> >>>>>>> >>>>>>>> rules...
>>>>>>> >>>>>>> >>>>>>>>
>>>>>>> >>>>>>> >>>>>>>> Anyway, I didn't say scoping rules are necessarily
>>>>>>> >>>>>>> >>>>>>>> confusing, I was
>>>>>>> >>>>>>> >>>>>>>> only referring to R formulas. But according to the
>>>>>>> examples
>>>>>>> >>>>>>> >>>>>>>> you
>>>>>>> >>>>>>> >>>>>>>> posted,
>>>>>>> >>>>>>> >>>>>>>> your question appears to be different.
>>>>>>> >>>>>>> >>>>>>>>
>>>>>>> >>>>>>> >>>>>>>>
>>>>>>>
>>>>>>
