Thank you very much for the in-depth explanation, and sorry for answering only now. I haven't been able to get around to it until now, and this is exactly what I was looking for. One more question if you feel like answering: where can I learn properly about Scheme macros? Metaprogramming is one of the most powerful features in Lisp, but I haven't come across a really good explanation of all its facets.
On Saturday, 29 September 2018 02:28:37 CET Mark H Weaver wrote: > Hi, > > HiPhish <[email protected]> writes: > > Hello Schemers, > > > > I have written a small macro for writing test specifications: > > (define-syntax test-cases > > > > (syntax-rules () > > > > ((_ title > > > > (given (byte byte* ...)) > > ...) > > > > (begin > > > > (test-begin title) > > (call-with-values (λ () (open-bytevector-output-port)) > > > > (λ (out get-bv) > > > > (pack given out) > > (let ((received (get-bv)) > > > > (expected (u8-list->bytevector '(byte byte* ...)))) > > > > (test-assert (bytevector=? received expected))))) > > > > ... > > (test-end title))))) > > > > The idea is that I can specify a series of test cases where each case > > consists> > > of an object and a sequence of bytes which this object is to be serialized to: > > (test-cases "Single precision floating point numbers" > > > > (+3.1415927410125732 (#xCA #b01000000 #b01001001 #b00001111 > > #b11011011)) > > (-3.1415927410125732 (#xCA #b11000000 #b01001001 #b00001111 > > > > #b11011011))) > > > > This works fine, but sometimes there is a sequence of the same bytes and > > it > > > > would be more readable if I could write something like this: > > ((make-vector 16 0) (#xDC (16 #x00))) > > > > instead of writing out 16 times `#x00`. This would require being able to > > make a distinction in the pattern whether `byte` is of the pattern > > > > byte > > > > or > > > > (count byte) > > > > and if it's the latter construct a list of `count` `byte`s (via > > `(make-list > > count byte)` for example) and splice it in. This distinction needs to be > > made for each byte specification because I want to mix actual bytes and > > these "RLE- encoded" byte specifications. > > > > So I guess what I'm looking for is to have a `syntax-rules` inside a > > `syntax- rules` in a way. Can this be done? > > It cannot be done with pure 'syntax-rules' macros, but it can certainly > be done with procedural macros, sometimes called 'syntax-case' macros. > Procedural macros are quite general, allowing you to write arbitrary > Scheme code that runs at compile time to inspect the macro operands and > generate arbitrary code. > > I'll describe how to do this with macros further down, but let me begin > with the simple approach. > > As rain1 suggested, this can be accomplished most easily by writing a > normal procedure to convert your compact bytevector notation into a > bytevector, and then having your macro expand into code that calls that > procedure. Here's working code to do that: > > --8<---------------cut here---------------start------------->8--- > (use-modules (ice-9 match) > (srfi srfi-1) > (rnrs bytevectors)) > > (define (compact-bytevector segments) > (u8-list->bytevector > (append-map (match-lambda > ((count byte) (make-list count byte)) > (byte (list byte))) > segments))) > > (define-syntax test-cases > (syntax-rules () > ((_ title > (given (seg ...)) > ...) > (begin > (test-begin title) > (call-with-values (λ () (open-bytevector-output-port)) > (λ (out get-bv) > (pack given out) > (let ((received (get-bv)) > (expected (compact-bytevector '(seg ...)))) > (test-assert (bytevector=? received expected))))) > ... > (test-end title))))) > > > scheme@(guile-user)> (compact-bytevector '(#xDC (16 #x00))) > $2 = #vu8(220 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) > scheme@(guile-user)> ,expand (test-cases "Single precision floating point > numbers" ((make-vector 16 0) (#xDC (16 #x00)))) $3 = (begin > (test-begin > "Single precision floating point numbers") > (call-with-values > (lambda () (open-bytevector-output-port)) > (lambda (out get-bv) > (pack (make-vector 16 0) out) > (let ((received (get-bv)) > (expected (compact-bytevector '(220 (16 0))))) > (test-assert (bytevector=? received expected))))) > (test-end > "Single precision floating point numbers")) > scheme@(guile-user)> > --8<---------------cut here---------------end--------------->8--- > > Now, suppose it was important to do more of this work at macro expansion > time. For example, if efficiency was a concern, it might not be > acceptable to postpone the conversion of '(#xDC (16 #x00)) into > #vu8(220 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) until run time. > > It turns out that pure 'syntax-rules' macros are turing complete, but > they are limited in the ways that they can inspect the syntax objects > given to them as operands. In particular, they cannot inspect atomic > expressions, except to compare them with the finite set of literals in > the first operand to 'syntax-rules'. This is not sufficient to > interpret an arbitrary integer literal. It could only be done with > 'syntax-rules' macros if the 'count' field were represented using a > finite set of literals and/or list structure. E.g. it could be done if > the count were represented as a list of decimal digits like (1 4 2) for > 142. > > In cases like this, we would normally turn to procedural macros, > e.g. 'syntax-case' macros. Here's a straightforward approach, reusing > the 'compact-bytevector' procedure given above, but calling it at > compile time instead of at run time: > > --8<---------------cut here---------------start------------->8--- > (use-modules (ice-9 match) > (srfi srfi-1) > (rnrs bytevectors)) > > (define (compact-bytevector segments) > (u8-list->bytevector > (append-map (match-lambda > ((count byte) (make-list count byte)) > (byte (list byte))) > segments))) > > (define-syntax compact-bytevector-literal > (lambda (stx) > (syntax-case stx () > ((_ (seg ...)) > (compact-bytevector (syntax->datum #'(seg ...))))))) > > (define-syntax test-cases > (syntax-rules () > ((_ title > (given (seg ...)) > ...) > (begin > (test-begin title) > (call-with-values (λ () (open-bytevector-output-port)) > (λ (out get-bv) > (pack given out) > (let ((received (get-bv)) > (expected (compact-bytevector-literal (seg ...)))) > (test-assert (bytevector=? received expected))))) > ... > (test-end title))))) > --8<---------------cut here---------------end--------------->8--- > > Here, instead of having 'test-cases' expand into a procedure call to > 'compact-bytevector', it expands into a *macro* call to > 'compact-bytevector-literal'. The latter is a procedural macro, which > calls 'compact-bytevector' at compile time. > > This approach is sufficient in this case, but I sense in your question a > desire to be able to perform more general inspection on the macro > operands and generation of the resulting code. This particular example > is not ideally suited for this task, but the following example code > comes a bit closer: > > --8<---------------cut here---------------start------------->8--- > (define (segment-syntax->u8-list stx) > (syntax-case stx () > ((count byte) > (every number? (syntax->datum #'(count byte))) ;optional guard > (make-list (syntax->datum #'count) > (syntax->datum #'byte))) > (byte > (number? (syntax->datum #'byte)) ;optional guard > (list (syntax->datum #'byte))))) > > (define (compact-bytevector-syntax->bytevector stx) > (syntax-case stx () > ((seg ...) > (u8-list->bytevector > (append-map segment-syntax->u8-list > #'(seg ...)))))) > > (define-syntax compact-bytevector-literal > (lambda (stx) > (syntax-case stx () > ((_ (seg ...)) > (compact-bytevector-syntax->bytevector #'(seg ...)))))) > --8<---------------cut here---------------end--------------->8--- > > I've omitted the 'test-cases' macro here because it's unchanged from the > previous example. Here we have two normal procedures that use > 'syntax-case', which might be a bit confusing. These are procedures > that accept syntax objects as arguments, and return normal data > structures. > > In contrast to the previous example, which used 'syntax->datum' on the > entire compact-bytevector-literal, in this example we inspect and > destruct the syntax object itself using 'syntax-case'. This would be > needed in the more general case where identifiers (i.e. variable > references) might occur in the syntax objects. > > Hopefully this gives you some idea of what can be done, but I don't > think this is the best example to explore these possibilities, since in > this case the normal procedural approach in my first code excerpt above > is simplest and most likely sufficient. > > Regards, > Mark
