Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule
Yes, I see some of this at the bottom of the chapter 16 in the guide, and the recursion supported with (require http://docs.racket-lang.org/reference/require.html#%28form._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._require%29%29 (for-meta http://docs.racket-lang.org/reference/require.html#%28form._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._for-meta%29%29 n racket/base)). On Fri, Jan 16, 2015 at 11:21 AM, Thomas Lynch thomas.ly...@reasoningtechnology.com wrote: Thank you for that nice explanation. I'm reminded of the scope variables carried in Ruby. In Mathematica the renaming of module variables is explicit. There do not appear to be run phases there. Also thanks for the working example using syntax parse. Thus far I have working examples using define-syntax via datum and define-syntax with case (thanks to Matthais). The following questions might best be answered by pointing me to some other docs, haven't run across yet: Are there more than two phases? There is the syntax phase, i.e. compile time phase, and then a run phase. Lexical scope at the syntax phase would be static based textual layout of the program - i.e. lexical scope. Then the macros (syntax transformers) are gone (as the syntax has been transformed) after the syntax phase. At run time then, we have no access to the syntax object information? Academic question, but can syntax information, or other values be shared across phases? As an example, could one store the file name, line, and column number for an identifier, the make use of that later? How is it that we do work inside of a macro, for example by making use of require-for-syntax? Does that imply that the routines that we make use of in the macro for doing work had their own syntax phase, and now they are in run time phase during the macro's syntax phase? .. then if those worker routines make use of macros ... seems this recursion could be arbitrarily deep. Can the syntax analysis be explicitly invoked? Sort of like (apply ...) or (eval ..) ? Come to think of it, eval must do the syntax phase, then the run phase, so perhaps it is calls to that which causes the recursion. On Fri, Jan 16, 2015 at 11:01 AM, Alexander D. Knauth alexan...@knauth.org wrote: But I think it’s important that it doesn’t use gensym or something like that, it uses syntax-marks, which means you can break these lexical scoping rules if you want/need to by using either syntax-local-introduce or datum-syntax: #lang racket (require syntax/parse/define) (define-simple-macro (with-tables stem body ...) #:with table-author-id (syntax-local-introduce #'table-author) (let([table-publication (string-append stem _publication)] [table-author-id (string-append stem _author)] [table-bridge-publication-author (string-append stem _bridge_publication_author)] [table-unique-counters (string-append stem _unique_counters)] ) body ... )) (with-tables x table-author) ;”x_author On Jan 15, 2015, at 9:23 PM, Alexander McLin alex.mc...@gmail.com wrote: Warning I am still a Racket intermediate user but I've been studying syntactic extensions a lot the past several months. The problem here is macros in Racket have lexical scope just like procedures, they are hygienic macros. The identifiers you introduced in the with-tables macro only exist or refer to other bindings in the same lexical scope as where you originally wrote the macro. When you invoke the macro and pass in table-author, even though it is spelled the same as the identifier you wrote in the macro definition, they are not the same. When the macro expands, hygiene is implemented by renaming all identifiers in the macro to unique non-clashing symbols that don't conflict with others existing in the scope the macro is expanding in. The table-author identifier in the macro in the let form is renamed to something different like g6271 or something along those lines. Furthermore, you need to be careful about what you mean by evaluation. In the presence of macros, you have the concept of syntax phase(or compile-time or expand-time) evaluation versus run-time evaluation. When the macro is expanding, it does it thing, processing the original syntax into the new piece of syntax that replaces what was there previously such as (with-tables x table-author) which is then finally evaluated during run-time. (with-tables x table-author) will expand into something looking similar to the following, just to give you an idea of what macro expansion looks like: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author)) (g6445 (string-append x _unique_counters))) table-author) Note that the original table-author identifier has been replaced by a different identifier that still has the same binding you originally defined. The table-author identifier
Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule
Warning I am still a Racket intermediate user but I've been studying syntactic extensions a lot the past several months. The problem here is macros in Racket have lexical scope just like procedures, they are hygienic macros. The identifiers you introduced in the with-tables macro only exist or refer to other bindings in the same lexical scope as where you originally wrote the macro. When you invoke the macro and pass in table-author, even though it is spelled the same as the identifier you wrote in the macro definition, they are not the same. When the macro expands, hygiene is implemented by renaming all identifiers in the macro to unique non-clashing symbols that don't conflict with others existing in the scope the macro is expanding in. The table-author identifier in the macro in the let form is renamed to something different like g6271 or something along those lines. Furthermore, you need to be careful about what you mean by evaluation. In the presence of macros, you have the concept of syntax phase(or compile-time or expand-time) evaluation versus run-time evaluation. When the macro is expanding, it does it thing, processing the original syntax into the new piece of syntax that replaces what was there previously such as (with-tables x table-author) which is then finally evaluated during run-time. (with-tables x table-author) will expand into something looking similar to the following, just to give you an idea of what macro expansion looks like: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author)) (g6445 (string-append x _unique_counters))) table-author) Note that the original table-author identifier has been replaced by a different identifier that still has the same binding you originally defined. The table-author identifier you passed to the macro gets inserted in the body position and then the expanded code is evaluated at run-time and of course gives you a run-time error since table-author does not refer to anything and thus when it's evaluated, it is recognized as an undefined identifier. (with-tables x hello) works because what you get in return is: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author)) (g6445 (string-append x _unique_counters))) hello) hello is just a self-evaluating string giving you back hello from within the let form. On Thu, Jan 15, 2015 at 12:12 AM, Thomas Lynch thomas.ly...@reasoningtechnology.com wrote: I have a simple syntax rule: Welcome to Racket v5.2.1. racket@ (define-syntax-rule (with-tables stem body ...) (let( [table-publication (string-append stem _publication)] [table-author (string-append stem _author)] [table-bridge-publication-author (string-append stem _bridge_publication_author)] [table-unique-counters (string-append stem _unique_counters)] ) body ... )) Which works fine when I don't reference the environment defined by the let: racket@ racket@ (with-tables x hello) hello However when I pass it an identifier corresponding to one of the variables defined in the let: racket@ (with-tables x table-author) reference to undefined identifier: table-author stdin::1167: table-author The identifier passed in doesn't seem to be part of the local let context, but carried in a different context, or perhaps it was evaluated as an operand. I didn't expect either of those. Can someone point me at a description of the expected behavior, or give me a tip here on what is happening and why. ... in Wolfram language there is a 'Hold' operator for situations like this. Apparently inside the macro we have to do some evaluation to handle the work of the macro, is that why the operand is evaluated? Thanks in advance for explaining the evaluation/context model here. Thomas _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule
But I think it’s important that it doesn’t use gensym or something like that, it uses syntax-marks, which means you can break these lexical scoping rules if you want/need to by using either syntax-local-introduce or datum-syntax: #lang racket (require syntax/parse/define) (define-simple-macro (with-tables stem body ...) #:with table-author-id (syntax-local-introduce #'table-author) (let([table-publication (string-append stem _publication)] [table-author-id (string-append stem _author)] [table-bridge-publication-author (string-append stem _bridge_publication_author)] [table-unique-counters (string-append stem _unique_counters)] ) body ... )) (with-tables x table-author) ;”x_author On Jan 15, 2015, at 9:23 PM, Alexander McLin alex.mc...@gmail.com wrote: Warning I am still a Racket intermediate user but I've been studying syntactic extensions a lot the past several months. The problem here is macros in Racket have lexical scope just like procedures, they are hygienic macros. The identifiers you introduced in the with-tables macro only exist or refer to other bindings in the same lexical scope as where you originally wrote the macro. When you invoke the macro and pass in table-author, even though it is spelled the same as the identifier you wrote in the macro definition, they are not the same. When the macro expands, hygiene is implemented by renaming all identifiers in the macro to unique non-clashing symbols that don't conflict with others existing in the scope the macro is expanding in. The table-author identifier in the macro in the let form is renamed to something different like g6271 or something along those lines. Furthermore, you need to be careful about what you mean by evaluation. In the presence of macros, you have the concept of syntax phase(or compile-time or expand-time) evaluation versus run-time evaluation. When the macro is expanding, it does it thing, processing the original syntax into the new piece of syntax that replaces what was there previously such as (with-tables x table-author) which is then finally evaluated during run-time. (with-tables x table-author) will expand into something looking similar to the following, just to give you an idea of what macro expansion looks like: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author)) (g6445 (string-append x _unique_counters))) table-author) Note that the original table-author identifier has been replaced by a different identifier that still has the same binding you originally defined. The table-author identifier you passed to the macro gets inserted in the body position and then the expanded code is evaluated at run-time and of course gives you a run-time error since table-author does not refer to anything and thus when it's evaluated, it is recognized as an undefined identifier. (with-tables x hello) works because what you get in return is: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author)) (g6445 (string-append x _unique_counters))) hello) hello is just a self-evaluating string giving you back hello from within the let form. On Thu, Jan 15, 2015 at 12:12 AM, Thomas Lynch thomas.ly...@reasoningtechnology.com wrote: I have a simple syntax rule: Welcome to Racket v5.2.1. racket@ (define-syntax-rule (with-tables stem body ...) (let( [table-publication (string-append stem _publication)] [table-author (string-append stem _author)] [table-bridge-publication-author (string-append stem _bridge_publication_author)] [table-unique-counters (string-append stem _unique_counters)] ) body ... )) Which works fine when I don't reference the environment defined by the let: racket@ racket@ (with-tables x hello) hello However when I pass it an identifier corresponding to one of the variables defined in the let: racket@ (with-tables x table-author) reference to undefined identifier: table-author stdin::1167: table-author The identifier passed in doesn't seem to be part of the local let context, but carried in a different context, or perhaps it was evaluated as an operand. I didn't expect either of those. Can someone point me at a description of the expected behavior, or give me a tip here on what is happening and why. ... in Wolfram language there is a 'Hold' operator for situations like this. Apparently inside the macro we have to do some evaluation to handle the work of the macro, is that why the operand is evaluated? Thanks in advance for explaining the evaluation/context model here. Thomas _ Racket Developers list:
Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule
Thank you for that nice explanation. I'm reminded of the scope variables carried in Ruby. In Mathematica the renaming of module variables is explicit. There do not appear to be run phases there. Also thanks for the working example using syntax parse. Thus far I have working examples using define-syntax via datum and define-syntax with case (thanks to Matthais). The following questions might best be answered by pointing me to some other docs, haven't run across yet: Are there more than two phases? There is the syntax phase, i.e. compile time phase, and then a run phase. Lexical scope at the syntax phase would be static based textual layout of the program - i.e. lexical scope. Then the macros (syntax transformers) are gone (as the syntax has been transformed) after the syntax phase. At run time then, we have no access to the syntax object information? Academic question, but can syntax information, or other values be shared across phases? As an example, could one store the file name, line, and column number for an identifier, the make use of that later? How is it that we do work inside of a macro, for example by making use of require-for-syntax? Does that imply that the routines that we make use of in the macro for doing work had their own syntax phase, and now they are in run time phase during the macro's syntax phase? .. then if those worker routines make use of macros ... seems this recursion could be arbitrarily deep. Can the syntax analysis be explicitly invoked? Sort of like (apply ...) or (eval ..) ? Come to think of it, eval must do the syntax phase, then the run phase, so perhaps it is calls to that which causes the recursion. On Fri, Jan 16, 2015 at 11:01 AM, Alexander D. Knauth alexan...@knauth.org wrote: But I think it’s important that it doesn’t use gensym or something like that, it uses syntax-marks, which means you can break these lexical scoping rules if you want/need to by using either syntax-local-introduce or datum-syntax: #lang racket (require syntax/parse/define) (define-simple-macro (with-tables stem body ...) #:with table-author-id (syntax-local-introduce #'table-author) (let([table-publication (string-append stem _publication)] [table-author-id (string-append stem _author)] [table-bridge-publication-author (string-append stem _bridge_publication_author)] [table-unique-counters (string-append stem _unique_counters)] ) body ... )) (with-tables x table-author) ;”x_author On Jan 15, 2015, at 9:23 PM, Alexander McLin alex.mc...@gmail.com wrote: Warning I am still a Racket intermediate user but I've been studying syntactic extensions a lot the past several months. The problem here is macros in Racket have lexical scope just like procedures, they are hygienic macros. The identifiers you introduced in the with-tables macro only exist or refer to other bindings in the same lexical scope as where you originally wrote the macro. When you invoke the macro and pass in table-author, even though it is spelled the same as the identifier you wrote in the macro definition, they are not the same. When the macro expands, hygiene is implemented by renaming all identifiers in the macro to unique non-clashing symbols that don't conflict with others existing in the scope the macro is expanding in. The table-author identifier in the macro in the let form is renamed to something different like g6271 or something along those lines. Furthermore, you need to be careful about what you mean by evaluation. In the presence of macros, you have the concept of syntax phase(or compile-time or expand-time) evaluation versus run-time evaluation. When the macro is expanding, it does it thing, processing the original syntax into the new piece of syntax that replaces what was there previously such as (with-tables x table-author) which is then finally evaluated during run-time. (with-tables x table-author) will expand into something looking similar to the following, just to give you an idea of what macro expansion looks like: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author)) (g6445 (string-append x _unique_counters))) table-author) Note that the original table-author identifier has been replaced by a different identifier that still has the same binding you originally defined. The table-author identifier you passed to the macro gets inserted in the body position and then the expanded code is evaluated at run-time and of course gives you a run-time error since table-author does not refer to anything and thus when it's evaluated, it is recognized as an undefined identifier. (with-tables x hello) works because what you get in return is: (let ((g6191 (string-append x _publication)) (g6271 (string-append x _author)) (g6369 (string-append x _bridge_publication_author))
