there's a fairly high level description at http://my.safaribooksonline.com/book/programming/clojure/9781935182641/macros/ch08lev1sec5 (needs free trial subscription). i'm not sure it has the details you want, though. andrew
On Thursday, 20 February 2014 07:08:08 UTC-3, Toivo Henningsson wrote: > > I've been perusing clojure.org, but was not able to find any real > documentation on how this works. The best that I found so far was from > http://en.wikipedia.org/wiki/Clojure#Macros: > > Clojure's macro system <http://en.wikipedia.org/wiki/Lisp_macro> is very > similar to that in Common Lisp <http://en.wikipedia.org/wiki/Common_Lisp> > with > the exception that Clojure's version of the > backquote<http://en.wikipedia.org/wiki/Grave_accent#Use_in_programming> > (called > "syntax quote") qualifies symbols with their > namespace<http://en.wikipedia.org/wiki/Namespace>. > This helps prevent unintended name capture, as binding to > namespace-qualified names is forbidden. It is possible to force a capturing > macro expansion, but this must be done explicitly. Clojure also disallows > rebinding global names in other namespaces that have been imported into the > current namespace. > > Does anyone know of where I might read up on how macros and macro hygiene > works in Clojure? > > On Thursday, 20 February 2014 05:57:07 UTC+1, Stefan Karpinski wrote: >> >> For what it's worth, I believe this is similar to how Clojure handles >> this as well. >> >> >> On Wed, Feb 19, 2014 at 2:43 PM, Toivo Henningsson <[email protected]>wrote: >> >>> I must say that I really like the idea of letting symbols keep track of >>> their context. In >>> PatternDispatch.jl<https://github.com/toivoh/PatternDispatch.jl>, >>> the @pattern macro is really just a front end to the actual package >>> machinery. Since code generation is split across modules, I've had to do >>> all sorts of shenanigans to get the correct result while avoiding >>> fragility; mostly resorting to manually quoting in values instead of >>> leaving any symbols that would be internal, a symptom of which is this >>> line<https://github.com/toivoh/PatternDispatch.jl/blob/30c8d2cea9bfc181169bd7a35d4385a50264149c/src/Recode.jl#L12> >>> : >>> >>> const qemit!, qcalc!, qfinish!, qArg, qSource, qTupleRef, qCall, >>> qInv,qBinding >>> , qEgalGuard, qTypeGuard, qgetfield, qtuple, qgetindex = map(quot, (emit >>> !, calc!, finish!, Arg, Source, TupleRef, Call, Inv, Binding, EgalGuard, >>> TypeGuard, getfield, tuple, getindex)) >>> >>> To me, the natural context of a quote would just be the literal context >>> where it is found. >>> >>> When writing macros, I also find that it quite often is simpler to just >>> bypass esc altogether by wrapping the whole returned expression in an esc. >>> There is also the inconvenience that you often need esc in macros, while >>> eval pukes on it, which makes it hard to share code between expression >>> generation to be used for one and the other. >>> >>> +1 for symbols with context! >>> >>> >>> On Sunday, 9 February 2014 21:41:07 UTC+1, Jeff Bezanson wrote: >>> >>>> Hi David, this is a very good and interesting writeup. I will need >>>> some time to think about it. >>>> Your design might indeed be better; it is often hard to guess how >>>> these things will work out in practice. >>>> >>>> But at first glance, it does trade manual management of >>>> internal/external symbols using `esc` for sometimes-manual management >>>> of contexts. Macro writers would have to deal with things that they >>>> think of as symbols, but that are not symbols. This is exactly what I >>>> was trying to avoid. I doubt we're at a global optimum, but this >>>> tradeoff was chosen carefully. >>>> >>>> >>>> On Sun, Feb 9, 2014 at 3:16 PM, David Moon <[email protected]> >>>> wrote: >>>> > Hygienic macros could be both better and simpler. Julia's hygiene >>>> only works >>>> > in simple cases, I think, and requires too much manual intervention. >>>> This >>>> > is because it is done in the wrong place, in the output of macro >>>> expansion, >>>> > so the macro expander has to guess the context of each symbol. >>>> Hygiene >>>> > ought to be done in the input to macro expansion, where the >>>> originating >>>> > context of every symbol is known. The esc function is a dead >>>> give-away that >>>> > something is wrong. It could be eliminated, which would make macros >>>> simpler >>>> > to define. Errors like the one in the manual's sample definition of >>>> assert >>>> > would no longer occur. >>>> > >>>> > I believe Julia's macros are taken directly from Scheme. Scheme >>>> never found >>>> > a fully satisfactory solution to the hygiene problem, because of the >>>> > inflexibility of S-expressions. I have been thinking about this >>>> issue for >>>> > quite a few years. >>>> > >>>> > Fortunately the problem is easily solved in Julia, where the >>>> representation >>>> > of expressions is more flexible. The key observation is that there >>>> should >>>> > be two kinds of symbols in the expansion of a macro, which I will >>>> call >>>> > "external" and "internal." Internal symbols come from the macro >>>> definition, >>>> > external symbols come from the macro call. Informally, external >>>> symbols are >>>> > visible to the caller of the macro and internal symbols are not, when >>>> used >>>> > as variable names. Other uses for symbols, such as terminals in the >>>> grammar >>>> > ("else"), literals (":foo"), and field names (".x") do not >>>> distinguish >>>> > internal from external. >>>> > >>>> > More formally, when a variable name is looked up in a scope, an >>>> internal >>>> > symbol only matches the same internal symbol. Thus binding an >>>> internal >>>> > symbol does not capture a reference to an external symbol, and vice >>>> versa. >>>> > When an internal symbol is not found in the current scope and its >>>> parents, >>>> > the external symbol with the same name is looked up in the scope >>>> where the >>>> > macro was defined. Thus free references in a macro expansion will >>>> have the >>>> > intended meaning, being looked up in the macro definition scope when >>>> the >>>> > reference came from the macro, but in the macro call scope when the >>>> > reference came from the caller. >>>> > >>>> > How can this be implemented? External symbols are the plain old >>>> symbols >>>> > that already exist. Internal symbols are a new AST type with two >>>> fields, >>>> > name and context. The name is the corresponding external symbol. >>>> The >>>> > context remembers the scope where the macro was defined and is a >>>> unique >>>> > object freshly created for each macro call. The quote construct >>>> converts >>>> > literal external symbols to internal symbols. Interpolated data and >>>> literal >>>> > internal symbols are left alone. The unary colon short form of quote >>>> is the >>>> > same. Local variable names can be internal symbols and variable >>>> binding >>>> > lookup is adjusted as described above: two internal symbols only >>>> match if >>>> > both the names and the contexts are the same. Uses of symbols other >>>> than as >>>> > variable names are modified to treat internal symbols the same as >>>> external. >>>> > >>>> > The esc function is no longer needed and should be removed. Any >>>> expression >>>> > that originated in the macro call is automatically escaped. Now the >>>> assert >>>> > example in the manual actually works. If a macro like the zerox >>>> example in >>>> > the manual needs to put an external symbol into the expansion, it >>>> just uses >>>> > a plain old symbol: >>>> > macro zerox() :($(symbol("x")) = 0) end. >>>> > >>>> > When a global variable is defined in a module, if the name is an >>>> internal >>>> > symbol it is converted to an external symbol so it is generally >>>> visible. >>>> > >>>> > Where does the quote construct get the context when it makes an >>>> internal >>>> > symbol? There are several ways it could be done. I prefer for quote >>>> to use >>>> > the value of the variable context; if there is no variable with that >>>> name in >>>> > scope it is an error. The macro statement implicitly defines the >>>> external >>>> > symbol context in the expander function. Users who want to break >>>> parts of >>>> > the expander function into separate functions must pass the context >>>> around >>>> > explicitly. Users who want to build expressions outside of a macro >>>> must >>>> > define context. It may be useful to have a user-callable Context >>>> > constructor that takes a module as its argument. >>>> > >>>> > Because internal symbols are not interned, there may be a speed >>>> decrease in >>>> > some cases. However this cost is only incurred at compile time. >>>> > >>>> > Macro-defining macros work, provided that when quote sees a literal >>>> internal >>>> > symbol it copies it unchanged into the expression being constructed. >>>> Thus >>>> > the expansion of a macro defined by a macro-defining macro may >>>> contain >>>> > internal symbols whose context comes from either macro. In the same >>>> way, >>>> > recursive or nested macros work, with each internal symbol >>>> remembering the >>>> > context where it originated. >>>> > >>>> > I have no strong opinion on whether macros are allowed to be defined >>>> in a >>>> > non-top-level context. If not, the macro definition scope remembered >>>> in an >>>> > internal symbol's context is just a module. >>>> > >>>> > A literal symbol is no longer the same thing as construction of an >>>> > expression consisting of only a literal variable name. The former >>>> produces >>>> > an external symbol, the latter produces an internal symbol. One >>>> approach >>>> > would be to disallow :x for a literal symbol and require symbol("x") >>>> to be >>>> > used, but the verbosity might be unpopular. Another approach would >>>> be to >>>> > treat :x as a special case; if you want to produce an internal symbol >>>> x you >>>> > must use quote x end or :(x). >>>> > >>>> > Incompatible changes here: >>>> > - Remove esc (or make it a no-op). >>>> > - quote no longer works if context is not defined. >>>> > - same for unary colon, unless the argument is just a symbol. >>>> > >>>> > Maybe the existing macros are good enough for your purposes, but I >>>> think the >>>> > hygiene could work better. What do you think? >>>> >>> >>
