Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule

[Thomas Lynch]

Yes, I see some of this at the bottom of the chapter 16 in the guide,  and
the recursion supported with
 (require

 (for-meta

 n racket/base)).


On Fri, Jan 16, 2015 at 11:21 AM, Thomas Lynch <
thomas.ly...@reasoningtechnology.com> wrote:

> Thank you for that nice explanation.  I'm reminded of the scope variables
> carried in Ruby.  In Mathematica the renaming of module variables is
> explicit.  There do not appear to be run phases there.  Also thanks for the
> working example using syntax parse.  Thus far I have working examples using
> define-syntax via datum and define-syntax with case (thanks to Matthais).
>
> The following questions might best be answered by pointing me to some
> other docs, haven't run across yet:
>
> Are there more than two phases?  There is the syntax phase, i.e. compile
> time phase, and then a run phase.  Lexical scope at the syntax phase would
> be static based textual layout of the program - i.e. lexical scope.  Then
> the macros (syntax transformers) are gone (as the syntax has been
> transformed) after the syntax phase.  At run time then, we have no access
> to the syntax object information?
>
> Academic question, but can syntax information, or other values be shared
> across phases?  As an example, could one store the file name, line, and
> column number for an identifier, the make use of that later?
>
> How is it that we do work inside of a macro, for example by making use of
> require-for-syntax?  Does that imply that the routines that we make use of
> in the macro for doing work had their own syntax phase, and now they are in
> run time phase during the macro's syntax phase?   .. then if those worker
> routines make use of macros ...   seems this recursion could be arbitrarily
> deep.
>
> Can the syntax analysis be explicitly invoked?   Sort of like (apply ...)
>  or (eval ..) ?  Come to think of it, eval must do the syntax phase, then
> the run phase, so perhaps it is calls to that which causes the recursion.
>
> On Fri, Jan 16, 2015 at 11:01 AM, Alexander D. Knauth <
> alexan...@knauth.org> wrote:
>
>> But I think it’s important that it doesn’t use gensym or something like
>> that, it uses syntax-marks, which means you can break these lexical scoping
>> rules if you want/need to by using either syntax-local-introduce or
>> datum->syntax:
>>
>> #lang racket
>> (require syntax/parse/define)
>> (define-simple-macro (with-tables stem body ...)
>>   #:with table-author-id (syntax-local-introduce #'table-author)
>>   (let([table-publication (string-append stem "_publication")]
>>[table-author-id (string-append stem "_author")]
>>[table-bridge-publication-author (string-append stem
>> "_bridge_publication_author")]
>>[table-unique-counters (string-append stem "_unique_counters")]
>>)
>> body ...
>> ))
>> (with-tables "x" table-author) ;”x_author"
>>
>>
>> On Jan 15, 2015, at 9:23 PM, Alexander McLin 
>> wrote:
>>
>> Warning I am still a Racket intermediate user but I've been studying
>> syntactic extensions a lot the past several months.
>>
>> The problem here is macros in Racket have lexical scope just like
>> procedures, they are hygienic macros. The identifiers you introduced in the
>> with-tables macro only exist or refer to other bindings in the same lexical
>> scope as where you originally wrote the macro.
>>
>> When you invoke the macro and pass in table-author, even though it is
>> spelled the same as the identifier you wrote in the macro definition, they
>> are not the same. When the macro expands, hygiene is implemented by
>> renaming all identifiers in the macro to unique non-clashing symbols that
>> don't conflict with others existing in the scope the macro is expanding in.
>>
>> The table-author identifier in the macro in the let form is renamed to
>> something different like g6271 or something along those lines.
>>
>> Furthermore, you need to be careful about what you mean by evaluation. In
>> the presence of macros, you have the concept of syntax phase(or
>> compile-time or expand-time) evaluation versus run-time evaluation. When
>> the macro is expanding, it does it thing, processing the original syntax
>> into the new piece of syntax that replaces what was there previously such
>> as (with-tables "x" table-author) which is then finally evaluated during
>> run-time.
>>
>> (with-tables "x" table-author) will expand into something looking similar
>> to the following, just to give you an idea of what macro expansion looks
>> like:
>>
>> (let ((g6191 (string-append "x" "_publication"))
>>(g6271 (string-append "x" "_author"))
>>(g6369 (string-append "x" "_bridge_publication_author"))
>>(g6445 (string-append "x" "_unique_counters")))
>>table-author)
>>
>> Note th

Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule

[Thomas Lynch]

Thank you for that nice explanation.  I'm reminded of the scope variables
carried in Ruby.  In Mathematica the renaming of module variables is
explicit.  There do not appear to be run phases there.  Also thanks for the
working example using syntax parse.  Thus far I have working examples using
define-syntax via datum and define-syntax with case (thanks to Matthais).

The following questions might best be answered by pointing me to some other
docs, haven't run across yet:

Are there more than two phases?  There is the syntax phase, i.e. compile
time phase, and then a run phase.  Lexical scope at the syntax phase would
be static based textual layout of the program - i.e. lexical scope.  Then
the macros (syntax transformers) are gone (as the syntax has been
transformed) after the syntax phase.  At run time then, we have no access
to the syntax object information?

Academic question, but can syntax information, or other values be shared
across phases?  As an example, could one store the file name, line, and
column number for an identifier, the make use of that later?

How is it that we do work inside of a macro, for example by making use of
require-for-syntax?  Does that imply that the routines that we make use of
in the macro for doing work had their own syntax phase, and now they are in
run time phase during the macro's syntax phase?   .. then if those worker
routines make use of macros ...   seems this recursion could be arbitrarily
deep.

Can the syntax analysis be explicitly invoked?   Sort of like (apply ...)
 or (eval ..) ?  Come to think of it, eval must do the syntax phase, then
the run phase, so perhaps it is calls to that which causes the recursion.

On Fri, Jan 16, 2015 at 11:01 AM, Alexander D. Knauth 
wrote:

> But I think it’s important that it doesn’t use gensym or something like
> that, it uses syntax-marks, which means you can break these lexical scoping
> rules if you want/need to by using either syntax-local-introduce or
> datum->syntax:
>
> #lang racket
> (require syntax/parse/define)
> (define-simple-macro (with-tables stem body ...)
>   #:with table-author-id (syntax-local-introduce #'table-author)
>   (let([table-publication (string-append stem "_publication")]
>[table-author-id (string-append stem "_author")]
>[table-bridge-publication-author (string-append stem
> "_bridge_publication_author")]
>[table-unique-counters (string-append stem "_unique_counters")]
>)
> body ...
> ))
> (with-tables "x" table-author) ;”x_author"
>
>
> On Jan 15, 2015, at 9:23 PM, Alexander McLin  wrote:
>
> Warning I am still a Racket intermediate user but I've been studying
> syntactic extensions a lot the past several months.
>
> The problem here is macros in Racket have lexical scope just like
> procedures, they are hygienic macros. The identifiers you introduced in the
> with-tables macro only exist or refer to other bindings in the same lexical
> scope as where you originally wrote the macro.
>
> When you invoke the macro and pass in table-author, even though it is
> spelled the same as the identifier you wrote in the macro definition, they
> are not the same. When the macro expands, hygiene is implemented by
> renaming all identifiers in the macro to unique non-clashing symbols that
> don't conflict with others existing in the scope the macro is expanding in.
>
> The table-author identifier in the macro in the let form is renamed to
> something different like g6271 or something along those lines.
>
> Furthermore, you need to be careful about what you mean by evaluation. In
> the presence of macros, you have the concept of syntax phase(or
> compile-time or expand-time) evaluation versus run-time evaluation. When
> the macro is expanding, it does it thing, processing the original syntax
> into the new piece of syntax that replaces what was there previously such
> as (with-tables "x" table-author) which is then finally evaluated during
> run-time.
>
> (with-tables "x" table-author) will expand into something looking similar
> to the following, just to give you an idea of what macro expansion looks
> like:
>
> (let ((g6191 (string-append "x" "_publication"))
>(g6271 (string-append "x" "_author"))
>(g6369 (string-append "x" "_bridge_publication_author"))
>(g6445 (string-append "x" "_unique_counters")))
>table-author)
>
> Note that the original table-author identifier has been replaced by a
> different identifier that still has the same binding you originally defined.
>
> The table-author identifier you passed to the macro gets inserted in the
> body position and then the expanded code is evaluated at run-time and of
> course gives you a run-time error since table-author does not refer to
> anything and thus when it's evaluated, it is recognized as an undefined
> identifier.
>
> (with-tables "x" "hello") works because what you get in return is:
>
> (let ((g6191 (string-append "x" "_publication"))
>(g6271 (string-append "x" "_author"))

Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule

[Alexander D. Knauth]

But I think it’s important that it doesn’t use gensym or something like that, 
it uses syntax-marks, which means you can break these lexical scoping rules if 
you want/need to by using either syntax-local-introduce or datum->syntax:

#lang racket
(require syntax/parse/define)
(define-simple-macro (with-tables stem body ...)
  #:with table-author-id (syntax-local-introduce #'table-author)
  (let([table-publication (string-append stem "_publication")]
   [table-author-id (string-append stem "_author")]
   [table-bridge-publication-author (string-append stem 
"_bridge_publication_author")]
   [table-unique-counters (string-append stem "_unique_counters")]
   )
body ...
))
(with-tables "x" table-author) ;”x_author"


On Jan 15, 2015, at 9:23 PM, Alexander McLin  wrote:

> Warning I am still a Racket intermediate user but I've been studying 
> syntactic extensions a lot the past several months.
> 
> The problem here is macros in Racket have lexical scope just like procedures, 
> they are hygienic macros. The identifiers you introduced in the with-tables 
> macro only exist or refer to other bindings in the same lexical scope as 
> where you originally wrote the macro.
> 
> When you invoke the macro and pass in table-author, even though it is spelled 
> the same as the identifier you wrote in the macro definition, they are not 
> the same. When the macro expands, hygiene is implemented by renaming all 
> identifiers in the macro to unique non-clashing symbols that don't conflict 
> with others existing in the scope the macro is expanding in.
> 
> The table-author identifier in the macro in the let form is renamed to 
> something different like g6271 or something along those lines.
> 
> Furthermore, you need to be careful about what you mean by evaluation. In the 
> presence of macros, you have the concept of syntax phase(or compile-time or 
> expand-time) evaluation versus run-time evaluation. When the macro is 
> expanding, it does it thing, processing the original syntax into the new 
> piece of syntax that replaces what was there previously such as (with-tables 
> "x" table-author) which is then finally evaluated during run-time.
> 
> (with-tables "x" table-author) will expand into something looking similar to 
> the following, just to give you an idea of what macro expansion looks like:
> 
> (let ((g6191 (string-append "x" "_publication"))
>(g6271 (string-append "x" "_author"))
>(g6369 (string-append "x" "_bridge_publication_author"))
>(g6445 (string-append "x" "_unique_counters")))
>table-author)
> 
> Note that the original table-author identifier has been replaced by a 
> different identifier that still has the same binding you originally defined.
> 
> The table-author identifier you passed to the macro gets inserted in the body 
> position and then the expanded code is evaluated at run-time and of course 
> gives you a run-time error since table-author does not refer to anything and 
> thus when it's evaluated, it is recognized as an undefined identifier.
> 
> (with-tables "x" "hello") works because what you get in return is:
> 
> (let ((g6191 (string-append "x" "_publication"))
>(g6271 (string-append "x" "_author"))
>(g6369 (string-append "x" "_bridge_publication_author"))
>(g6445 (string-append "x" "_unique_counters")))
>"hello")
> 
> "hello" is just a self-evaluating string giving you back "hello" from within 
> the let form.
> 
> On Thu, Jan 15, 2015 at 12:12 AM, Thomas Lynch 
>  wrote:
> I have a simple syntax rule:
> 
>   Welcome to Racket v5.2.1.
>   racket@> (define-syntax-rule (with-tables stem body ...)
> (let(
>   [table-publication (string-append stem "_publication")]
>   [table-author (string-append stem "_author")]
>   [table-bridge-publication-author (string-append stem 
> "_bridge_publication_author")]
>   [table-unique-counters (string-append stem "_unique_counters")]
>   )
>   body ...
>   ))
> 
> Which works fine when I don't reference the environment defined by the let:
> 
>   racket@>
>   racket@> (with-tables "x" "hello")
>   "hello"
> 
> 
> However when I pass it an identifier corresponding to one of the variables 
> defined in the let:
> 
>   racket@> (with-tables "x" table-author)
>   reference to undefined identifier: table-author
>   stdin::1167: table-author
> 
> The identifier passed in doesn't seem to be part of the local let context, 
> but carried in a different context, or perhaps it was evaluated as an 
> operand.  I didn't expect either of those.  Can someone point me at a 
> description of the expected behavior, or give me a tip here on what is 
> happening and why.
> 
> ... in Wolfram language there is a 'Hold' operator for situations like this.  
> Apparently inside the macro we have to do some evaluation to handle the work 
> of the macro,  is that why the operand is evaluated? 
> 
> Thanks in advance for explaining the evaluation/contex

Re: [racket-dev] question, issue(?) with the scope of identifiers passed into define-syntax-rule

[Alexander McLin]

Warning I am still a Racket intermediate user but I've been studying
syntactic extensions a lot the past several months.

The problem here is macros in Racket have lexical scope just like
procedures, they are hygienic macros. The identifiers you introduced in the
with-tables macro only exist or refer to other bindings in the same lexical
scope as where you originally wrote the macro.

When you invoke the macro and pass in table-author, even though it is
spelled the same as the identifier you wrote in the macro definition, they
are not the same. When the macro expands, hygiene is implemented by
renaming all identifiers in the macro to unique non-clashing symbols that
don't conflict with others existing in the scope the macro is expanding in.

The table-author identifier in the macro in the let form is renamed to
something different like g6271 or something along those lines.

Furthermore, you need to be careful about what you mean by evaluation. In
the presence of macros, you have the concept of syntax phase(or
compile-time or expand-time) evaluation versus run-time evaluation. When
the macro is expanding, it does it thing, processing the original syntax
into the new piece of syntax that replaces what was there previously such
as (with-tables "x" table-author) which is then finally evaluated during
run-time.

(with-tables "x" table-author) will expand into something looking similar
to the following, just to give you an idea of what macro expansion looks
like:

(let ((g6191 (string-append "x" "_publication"))
   (g6271 (string-append "x" "_author"))
   (g6369 (string-append "x" "_bridge_publication_author"))
   (g6445 (string-append "x" "_unique_counters")))
   table-author)

Note that the original table-author identifier has been replaced by a
different identifier that still has the same binding you originally defined.

The table-author identifier you passed to the macro gets inserted in the
body position and then the expanded code is evaluated at run-time and of
course gives you a run-time error since table-author does not refer to
anything and thus when it's evaluated, it is recognized as an undefined
identifier.

(with-tables "x" "hello") works because what you get in return is:

(let ((g6191 (string-append "x" "_publication"))
   (g6271 (string-append "x" "_author"))
   (g6369 (string-append "x" "_bridge_publication_author"))
   (g6445 (string-append "x" "_unique_counters")))
   "hello")

"hello" is just a self-evaluating string giving you back "hello" from
within the let form.

On Thu, Jan 15, 2015 at 12:12 AM, Thomas Lynch <
thomas.ly...@reasoningtechnology.com> wrote:

> I have a simple syntax rule:
>
>   Welcome to Racket v5.2.1.
>   racket@> (define-syntax-rule (with-tables stem body ...)
> (let(
>   [table-publication (string-append stem "_publication")]
>   [table-author (string-append stem "_author")]
>   [table-bridge-publication-author (string-append stem
> "_bridge_publication_author")]
>   [table-unique-counters (string-append stem "_unique_counters")]
>   )
>   body ...
>   ))
>
> Which works fine when I don't reference the environment defined by the let:
>
>
>   racket@>
>   racket@> (with-tables "x" "hello")
>
>   "hello"
>
>
> However when I pass it an identifier corresponding to one of the variables
> defined in the let:
>
>   racket@> (with-tables "x" table-author)
>   reference to undefined identifier: table-author
>   stdin::1167: table-author
>
> The identifier passed in doesn't seem to be part of the local let context,
> but carried in a different context, or perhaps it was evaluated as an
> operand.  I didn't expect either of those.  Can someone point me at a
> description of the expected behavior, or give me a tip here on what is
> happening and why.
>
> ... in Wolfram language there is a 'Hold' operator for situations like
> this.  Apparently inside the macro we have to do some evaluation to handle
> the work of the macro,  is that why the operand is evaluated?
>
> Thanks in advance for explaining the evaluation/context model here.
>
> Thomas
>
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
>
>
_
  Racket Developers list:
  http://lists.racket-lang.org/dev