Hi, Catonano <[email protected]> writes:
> 2018-05-29 17:01 GMT+02:00 Mark H Weaver <[email protected]>: > > what's the problem with macroexpand-1 and syntax-case ? > > In Guile 1.x, 'macroexpand-1' performed a single macro expansion step at > the top-level form of an expression, using its old non-hygienic macro > expander. There are several problems with trying to provide such an > interface in a Hygienic macro expander, and especially in the > 'syntax-case' expander with its support for 'datum->syntax'. For one > thing, our modern macro expander doesn't even work with the plain > S-expressions which 'macroexpand-1' accepted and produced. It works > with "syntax objects", which effectively annotate every identifier with > extra information needed to determine which binding it references, and > also extra information needed to implement 'datum->syntax'. This in > turn requires detailed knowledge of the lexical environment in which > expansion is taking place, whereas 'macroexpand-1' provides no way for > the user to provide this information. > > Mark > > I have been reading this document about the scheme higienic macros > https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf > > I stopped reading it when I read that the implementation relies on a > previously bootstrapped version of another macro expansion > implementation. That's not an inherent limitation of the 'syntax-case' design. It's merely an unfortunate attribute of the psyntax _implementation_ of 'syntax-case', apparently because they didn't care enough about bootstrapping issues to write psyntax without the benefit of macros. 'syntax-case' could certainly be implemented without using a pre-existing macro expander. > But Racket has some facilities to step and debug macros, as you can > see here https://docs.racket-lang.org/macro-debugger/index.html > > Aren' t Racket macros higienyc ? Yes, of course, and we could certainly implement similar macro stepping facilities in Guile. But that's not what you asked about in your previous message. You asked about 'macroexpand-1', and my answer was specifically about that. I don't see any procedure similar to 'macroexpand-1' in the document you referenced above. > In this question I've been promptly suggested a quick solution to > perform a single macro expansion step > > https://stackoverflow.com/questions/50073207/macro-expansion-in-guile-scheme/50515880#50515880 For posterity, here's the quick solution suggested in the link above: (define-syntax (expand1 stx) (syntax-case stx () [(_expand1 form) (syntax-case #'form () [(id . more) (identifier? #'id) (let ([transformer (syntax-local-value #'id)]) (with-syntax ([expansion (transformer #'form)]) #''expansion))] [_ #''form])])) This is just a toy, and not very useful in practice. Here's the equivalent formulation for Guile: (use-modules (system syntax) (srfi srfi-11)) (define (syntax-local-value id) (let-values (((type value) (syntax-local-binding id))) value)) (define-syntax expand1 (lambda (stx) (syntax-case stx () [(_expand1 form) (syntax-case #'form () [(id . more) (identifier? #'id) (let ([transformer (syntax-local-value #'id)]) (with-syntax ([expansion (transformer #'form)]) #''expansion))] [_ #''form])]))) (I usually prefer to avoid using square brackets in this way, but for sake of comparison, I used them in the definition of 'expand1' above.) Anyway, it works the same way as in Racket for this simple example: scheme@(guile-user)> (expand1 (or 1 2 3)) $2 = (let ((t 1)) (if t t (or 2 3))) So, what's the problem? The first problem is that when quoting the resulting expansion, the binding information associated with identifiers in the syntax objects are lost, so hygiene is lost. For example: scheme@(guile-user)> (expand1 (or 1 2 t)) $3 = (let ((t 1)) (if t t (or 2 t))) Moving on, let's use this to try to investigate how 'define-record-type' works from SRFI-9 in Guile: scheme@(guile-user)> ,use (srfi srfi-9) scheme@(guile-user)> (expand1 (define-record-type <box> (box value) box? (value unbox set-box!))) $4 = (%define-record-type #f (define-record-type <box> (box value) box? (value unbox set-box!)) <box> (box value) box? (value unbox set-box!)) scheme@(guile-user)> (expand1 (%define-record-type #f (define-record-type <box> (box value) box? (value unbox set-box!)) <box> (box value) box? (value unbox set-box!))) While compiling expression: Wrong type to apply: (%define-record-type guile-user) scheme@(guile-user)> So what went wrong here? The problem is that '%define-record-type' is a private macro, used internally within (srfi srfi-9), and therefore not bound in the (guile-user) module where I'm working. If we had been working with syntax objects, each identifier within the expression would have been annotated with the specific binding that it refers to, but as I noted above, that information has been stripped. The awkward error message is because this toy implementation doesn't check if the identifier is a macro or not. One way we could try to improve this is to write 'expandN', which performs N macro expansion steps, keeping them as syntax objects during the intermediate steps: (use-modules (system syntax) (srfi srfi-11)) (define (syntax-local-type id) (let-values (((type value) (syntax-local-binding id))) type)) (define (syntax-local-value id) (let-values (((type value) (syntax-local-binding id))) value)) (define-syntax expandN (lambda (stx) (syntax-case stx () ((_expandN n form) (let ((n (syntax->datum #'n))) (and (number? n) (integer? n))) (let ((n (syntax->datum #'n))) (if (positive? n) (syntax-case #'form () ((id . _) (and (identifier? #'id) (eq? 'macro (syntax-local-type #'id))) (let ((transformer (syntax-local-value #'id))) (with-syntax ((expansion (transformer #'form)) (n-1 (datum->syntax #'id (- n 1)))) #'(expandN n-1 expansion)))) (_ #''form)) #''form)))))) Unfortunately, this is not quite right, because it fails to add "marks" to the identifiers introduced by the macro transformers, and thus is not fully hygienic, and variable capture may occur. However, it is better than what we had before, and good enough to step further into 'define-record-type': --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,pp (expandN 0 (define-record-type <box> (box value) box? (value unbox set-box!))) $2 = (define-record-type <box> (box value) box? (value unbox set-box!)) scheme@(guile-user)> ,pp (expandN 1 (define-record-type <box> (box value) box? (value unbox set-box!))) $3 = (%define-record-type #f (define-record-type <box> (box value) box? (value unbox set-box!)) <box> (box value) box? (value unbox set-box!)) scheme@(guile-user)> ,pp (expandN 2 (define-record-type <box> (box value) box? (value unbox set-box!))) $4 = (begin (define-inlinable (box value) (let ((s (allocate-struct <box> 1))) (struct-set! s 0 value) s)) (define <box> (let ((rtd (make-struct/no-tail record-type-vtable 'pw default-record-printer '<box> '(value)))) (set-struct-vtable-name! rtd '<box>) (struct-set! rtd (+ 2 vtable-offset-user) box) rtd)) (define-inlinable (box? obj) (and (struct? obj) (eq? (struct-vtable obj) <box>))) (define-tagged-inlinable ((%%type <box>) (%%index 0) (%%copier %%<box>-set-fields)) (unbox s) (if (eq? (struct-vtable s) <box>) (struct-ref s 0) (throw-bad-struct s 'unbox))) (define-syntax-rule (%%<box>-set-fields check? s (getter expr) ...) (%%set-fields <box> (unbox) check? s (getter expr) ...)) (define-inlinable (set-box! s val) (if (eq? (struct-vtable s) <box>) (struct-set! s 0 val) (throw-bad-struct s 'set-box!)))) scheme@(guile-user)> --8<---------------cut here---------------end--------------->8--- Unfortunately this is as far as we can go with 'expandN', because it only expands macros at the top-level of the expression. In this case, the top-level expression is a 'begin' form, which is a core form. At this point, a real macro expander descends into the core form and expands subexpressions, but in order to do this properly, it needs to understand the meanings of the core forms that it's descending into. For example, when descending into a 'let' form, it needs to take note of the variables that are bound by the 'let'. For example: scheme@(guile-user)> (expandN 0 (or 1 2 3)) $2 = (or 1 2 3) scheme@(guile-user)> (expandN 1 (or 1 2 3)) $3 = (let ((t 1)) (if t t (or 2 3))) scheme@(guile-user)> (expandN 2 (or 1 2 3)) $4 = (let ((t 1)) (if t t (or 2 3))) The last two outputs are the same because I made 'expandN' just smart enough to notice that 'let' is not a macro, in which case it stops gracefully without triggering an exception. Hopefully this illustrates why the old 'macroexpand-1' procedure from Guile 1.x, which works on plain S-expressions without extra binding information, and which only expands macros at the top level of the expression, cannot be usefully implemented on a modern hygienic macro expander. However, what certainly *could* be done is some kind of interactive tool to incrementally step macro expansions, while keeping track of the syntax objects behind the scenes. To be useful in realistic cases, it would need to understand most if not all of the core forms in Guile. Those core forms are the ones defined using 'global-extend' in psyntax.scm. Regards, Mark
