For what it's worth, I believe this is similar to how Clojure handles this as well.
On Wed, Feb 19, 2014 at 2:43 PM, Toivo Henningsson <[email protected]>wrote: > I must say that I really like the idea of letting symbols keep track of > their context. In > PatternDispatch.jl<https://github.com/toivoh/PatternDispatch.jl>, > the @pattern macro is really just a front end to the actual package > machinery. Since code generation is split across modules, I've had to do > all sorts of shenanigans to get the correct result while avoiding > fragility; mostly resorting to manually quoting in values instead of > leaving any symbols that would be internal, a symptom of which is this > line<https://github.com/toivoh/PatternDispatch.jl/blob/30c8d2cea9bfc181169bd7a35d4385a50264149c/src/Recode.jl#L12> > : > > const qemit!, qcalc!, qfinish!, qArg, qSource, qTupleRef, qCall, qInv,qBinding > , qEgalGuard, qTypeGuard, qgetfield, qtuple, qgetindex = map(quot, (emit!,calc > !, finish!, Arg, Source, TupleRef, Call, Inv, Binding, EgalGuard, > TypeGuard, getfield, tuple, getindex)) > > To me, the natural context of a quote would just be the literal context > where it is found. > > When writing macros, I also find that it quite often is simpler to just > bypass esc altogether by wrapping the whole returned expression in an esc. > There is also the inconvenience that you often need esc in macros, while > eval pukes on it, which makes it hard to share code between expression > generation to be used for one and the other. > > +1 for symbols with context! > > > On Sunday, 9 February 2014 21:41:07 UTC+1, Jeff Bezanson wrote: > >> Hi David, this is a very good and interesting writeup. I will need >> some time to think about it. >> Your design might indeed be better; it is often hard to guess how >> these things will work out in practice. >> >> But at first glance, it does trade manual management of >> internal/external symbols using `esc` for sometimes-manual management >> of contexts. Macro writers would have to deal with things that they >> think of as symbols, but that are not symbols. This is exactly what I >> was trying to avoid. I doubt we're at a global optimum, but this >> tradeoff was chosen carefully. >> >> >> On Sun, Feb 9, 2014 at 3:16 PM, David Moon <[email protected]> wrote: >> > Hygienic macros could be both better and simpler. Julia's hygiene only >> works >> > in simple cases, I think, and requires too much manual intervention. >> This >> > is because it is done in the wrong place, in the output of macro >> expansion, >> > so the macro expander has to guess the context of each symbol. >> Hygiene >> > ought to be done in the input to macro expansion, where the originating >> > context of every symbol is known. The esc function is a dead give-away >> that >> > something is wrong. It could be eliminated, which would make macros >> simpler >> > to define. Errors like the one in the manual's sample definition of >> assert >> > would no longer occur. >> > >> > I believe Julia's macros are taken directly from Scheme. Scheme never >> found >> > a fully satisfactory solution to the hygiene problem, because of the >> > inflexibility of S-expressions. I have been thinking about this issue >> for >> > quite a few years. >> > >> > Fortunately the problem is easily solved in Julia, where the >> representation >> > of expressions is more flexible. The key observation is that there >> should >> > be two kinds of symbols in the expansion of a macro, which I will call >> > "external" and "internal." Internal symbols come from the macro >> definition, >> > external symbols come from the macro call. Informally, external >> symbols are >> > visible to the caller of the macro and internal symbols are not, when >> used >> > as variable names. Other uses for symbols, such as terminals in the >> grammar >> > ("else"), literals (":foo"), and field names (".x") do not distinguish >> > internal from external. >> > >> > More formally, when a variable name is looked up in a scope, an >> internal >> > symbol only matches the same internal symbol. Thus binding an internal >> > symbol does not capture a reference to an external symbol, and vice >> versa. >> > When an internal symbol is not found in the current scope and its >> parents, >> > the external symbol with the same name is looked up in the scope where >> the >> > macro was defined. Thus free references in a macro expansion will have >> the >> > intended meaning, being looked up in the macro definition scope when >> the >> > reference came from the macro, but in the macro call scope when the >> > reference came from the caller. >> > >> > How can this be implemented? External symbols are the plain old >> symbols >> > that already exist. Internal symbols are a new AST type with two >> fields, >> > name and context. The name is the corresponding external symbol. The >> > context remembers the scope where the macro was defined and is a unique >> > object freshly created for each macro call. The quote construct >> converts >> > literal external symbols to internal symbols. Interpolated data and >> literal >> > internal symbols are left alone. The unary colon short form of quote >> is the >> > same. Local variable names can be internal symbols and variable >> binding >> > lookup is adjusted as described above: two internal symbols only match >> if >> > both the names and the contexts are the same. Uses of symbols other >> than as >> > variable names are modified to treat internal symbols the same as >> external. >> > >> > The esc function is no longer needed and should be removed. Any >> expression >> > that originated in the macro call is automatically escaped. Now the >> assert >> > example in the manual actually works. If a macro like the zerox >> example in >> > the manual needs to put an external symbol into the expansion, it just >> uses >> > a plain old symbol: >> > macro zerox() :($(symbol("x")) = 0) end. >> > >> > When a global variable is defined in a module, if the name is an >> internal >> > symbol it is converted to an external symbol so it is generally >> visible. >> > >> > Where does the quote construct get the context when it makes an >> internal >> > symbol? There are several ways it could be done. I prefer for quote >> to use >> > the value of the variable context; if there is no variable with that >> name in >> > scope it is an error. The macro statement implicitly defines the >> external >> > symbol context in the expander function. Users who want to break parts >> of >> > the expander function into separate functions must pass the context >> around >> > explicitly. Users who want to build expressions outside of a macro >> must >> > define context. It may be useful to have a user-callable Context >> > constructor that takes a module as its argument. >> > >> > Because internal symbols are not interned, there may be a speed >> decrease in >> > some cases. However this cost is only incurred at compile time. >> > >> > Macro-defining macros work, provided that when quote sees a literal >> internal >> > symbol it copies it unchanged into the expression being constructed. >> Thus >> > the expansion of a macro defined by a macro-defining macro may contain >> > internal symbols whose context comes from either macro. In the same >> way, >> > recursive or nested macros work, with each internal symbol remembering >> the >> > context where it originated. >> > >> > I have no strong opinion on whether macros are allowed to be defined in >> a >> > non-top-level context. If not, the macro definition scope remembered >> in an >> > internal symbol's context is just a module. >> > >> > A literal symbol is no longer the same thing as construction of an >> > expression consisting of only a literal variable name. The former >> produces >> > an external symbol, the latter produces an internal symbol. One >> approach >> > would be to disallow :x for a literal symbol and require symbol("x") to >> be >> > used, but the verbosity might be unpopular. Another approach would be >> to >> > treat :x as a special case; if you want to produce an internal symbol x >> you >> > must use quote x end or :(x). >> > >> > Incompatible changes here: >> > - Remove esc (or make it a no-op). >> > - quote no longer works if context is not defined. >> > - same for unary colon, unless the argument is just a symbol. >> > >> > Maybe the existing macros are good enough for your purposes, but I >> think the >> > hygiene could work better. What do you think? >> >
