Re: syntax-local-binding
Peter TB Brett pe...@peter-b.co.uk writes: It seems pretty clear to me that the only (debatable) downside to using Mark's implementation is that some definitions end up in the wrong module, while your implementation has several potentially *major* problems (including the necessity of providing universally unique gensyms) which Mark has managed to avoid. Let me just repeat that I consider universally unique gensyms a recipe for trouble. Having a compiler _depend_ on creating unreproducible output is going to be in the not really fun category for a _lot_ of things. I don't have much of a clue about the other differences. But this one _really_ comes at a dear price in its implications. -- David Kastrup
Re: syntax-local-binding
Hi Mark, On Tue 24 Jan 2012 03:11, Mark H Weaver m...@netris.org writes: you seem unabashedly content to lock us into using psyntax forever, This statement is an exaggeration. While I am content with psyntax now, all change is possible, with time. While it's important to think of the future (and the past), one must not forget about the present :) It concerns me when I see internal psyntax representations exported in our API. None of the interfaces that I proposed leak internal psyntax representations. syntax-local-binding provides binding information for an identifier. Racket provides similar procedures, and does not use psyntax. Therefore this information does not tie us to the psyntax implementation. syntax-locally-bound-identifiers can be implemented in any expander. It provides syntax objects. Syntax objects are not a psyntax implementation detail. syntax-module is a simple accessor. Racket provides the same accessor. Therefore it does not leak psyntax implementation details. If we can already foresee the need to deprecate an interface, wouldn't it be better not to add it in the first place? I don't see the need to deprecate them now, not more than any other identifier that we export. I just have one final request: please at least change the lexical environments in your `local-eval' implementation to use the future-proof `evaluator procedure' representation, as I have done in mine. For the reasons I mentioned in my mail yesterday at 12:52 UTC, I really don't see the point, as we have more effective means of dealing with future change than introducing an abstraction there. But if it will make you happy, sure. I'm quite tired of this topic ;-) Regards, Andy -- http://wingolog.org/
Re: syntax-local-binding
Andy Wingo wi...@pobox.com writes: On Tue 24 Jan 2012 11:30, Peter TB Brett pe...@peter-b.co.uk writes: It seems pretty clear to me that the only (debatable) downside to using Mark's implementation is that some definitions end up in the wrong module, while your implementation has several potentially *major* problems (including the necessity of providing universally unique gensyms) Let's be clear here: the universally-unique gensym issue is something that Guile *already* has, in version 2.0.0, 2.0.1, etc. I don't see why we need universally-unique gensyms unless your approach to `local-eval' is used. I've already explained why they are not needed for macros compiled in another session. It concerns me when I see internal psyntax representations exported in our API. None of the interfaces that I proposed leak internal psyntax representations. `syntax-local-binding' leaks the internal representations used for bindings. I gave an example where this would constrain our ability to change the binding representation for syntactic keywords, and your response was to point out that my particular example could be done in a different way without changing the representation. Your response demonstrated the weakness of my particular example, while underscoring my main point: that this constrains our internal representation choices. I just have one final request: please at least change the lexical environments in your `local-eval' implementation to use the future-proof `evaluator procedure' representation, as I have done in mine. For the reasons I mentioned in my mail yesterday at 12:52 UTC, I really don't see the point, as we have more effective means of dealing with future change than introducing an abstraction there. Your suggested more effective means is to introduce a new type lexical-environment-2 and continue supporting lexical-environment. No thanks. But if it will make you happy, sure. I'm quite tired of this topic ;-) Yes, it would make me happy, thank you. And believe me, I'm tired of this topic too. I thought I was mostly finished working on `local-eval' three weeks ago, when I produced my simple patch, which I _still_ think is superior to yours in the most important respect, namely in the commitments that it forces us to make. Regards, Mark
Unsafe psyntax label generation
Hi Andy, I'm worried about this change you recently committed: diff --git a/module/ice-9/psyntax.scm b/module/ice-9/psyntax.scm index 1bf3c32..fd33e98 100644 --- a/module/ice-9/psyntax.scm +++ b/module/ice-9/psyntax.scm @@ -636,7 +636,12 @@ ;; labels must be comparable with eq?, have read-write invariance, ;; and distinct from symbols. (define gen-label - (lambda () (symbol-string (gensym i + (let ((i 0)) +(lambda () + (let ((n i)) +;; FIXME: Use atomic ops. +(set! i (1+ n)) +(number-string n 36) (define gen-labels (lambda (ls) I guess you did this as an optimization, but as your comment above confesses, this optimization is not quite correct. Apart from the obvious lack of mutex on the global counter, there's another problem having to do with your implementation of `local-eval'. If we use your implementation of `local-eval', I think we may need universally-unique labels here. You decided that we didn't, because labels are always compared using `eq?' and labels from an earlier session will never be `eq?' to labels generated in a newer session, even if they have the same name. So far so good, but there's a case you didn't consider: `local-eval' combines syntax objects from two different sessions into a single syntax object (in the wrapper procedure), and thus there may be label name collisions. Now, if this combined syntax object is serialized as a compiled procedure, these labels with the same name will be optimized together into the same string object! I think this can actually happen, as follows: * Session A compiles procedure (foo), which uses (the-environment), to a .go file. * Session B loads the .go file and calls (foo) to create a lexical-environment. Then it uses `local-compile' to compile a local-expression that uses (the-environment) again. Maybe there's something I'm missing here, but if the change you made above was safe, I think the burden of proof is on you to explain why. Thanks, Mark
Re: syntax-local-binding
Hello, If we can already foresee the need to deprecate an interface, wouldn't it be better not to add it in the first place? I don't see the need to deprecate them now, not more than any other identifier that we export. I think this may be the key to this argument. There are two separate questions being debated as one question. Here they are: 1. Do we forsee a need to deprecate the syntax-local-binding functionality in the future, for instance to move away from psyntax? 2. Is this version of syntax-local-binding the best interface to this functionality? They are related questions, but they are distinct because someone could believe that it is good to expose this functionality but that the current syntax-local-bindings is not the best interface for us. I could be wrong, but I think that Andy answers maybe, but not for a long time to 1, and therefore thinks it's fine to include syntax-local-binding. Mark answers maybe, but definitely needs more thought before we make a commitment to 2, and therefore does not want to include syntax-local-binding. These are not contradictory positions. (Some of their other positions are contradictory, though :-) ). However, it does make me think that we should discuss the interface to syntax-local-binding more before releasing it (but don't take this too seriously, because I didn't follow the earlier threads much). Noah
mark uniqueness (Was: Re: syntax-local-binding)
On Tue 24 Jan 2012 14:25, Mark H Weaver m...@netris.org writes: I don't see why we need universally-unique gensyms I've already explained why they are not needed for macros compiled in another session. Ah, I forgot to reply to that. I found it: On Mon 16 Jan 2012 14:28, Mark H Weaver m...@netris.org writes: The reason it has not been a problem with macros is that, within a top-level macro (which are the only ones used across Guile sessions), the only syntax-objects that can be meaningfully _introduced_ into the expansion are top-level/module bindings. But these bindings have no associated labels or gensyms, because they're not in the wrap. See how this is a problem now where it wasn't before? Or am I missing something? Either you are missing something, or I am, or both of us -- that much is clear ;-) Psyntax associates marks with every identifier. Two identifiers are equal if they are symbolically equal, and they have the same marks. It would break hygiene if two identifiers that didn't come from the same place accidentally had the same marks. A fresh mark is placed on syntax returned from a macro expander, if the syntax was not present in the input. An easy way to do this would be simply: (define-syntax-rule (fresh-identifier) #'x) (define my-id (fresh-identifier)) All you need to do is to introduce that binding into a macro, and you might alias some other binding, because you have serialized the symbol and marks into a compiled file. This is admittedly far-fetched. But it can happen, and at the top-level. For example, our old friend: (define-syntax-rule (define-const x val) (begin (define t val) (define-syntax x (identifier-syntax t Here, `t' will have a fresh mark. Now, if in one compilation unit, I do: (define-const x 10) And in another, I do: (let ((t 20)) x) = ? You would expect the result to be 20. But I think it could be 20, if the marks on the two ts happened to collide. Am I missing something? :-) Andy -- http://wingolog.org/
Re: syntax-local-binding
Hello Mark :) Thanks again for your deep thoughts. This conversation is a bit stressful for the both of us, but we wouldn't be having it if we didn't both care about Guile. In the spirit of diffusing tension here, feel free to imagine all of my words as coming from Mr. Collins for the duration of this thread ;-) On Tue 24 Jan 2012 14:25, Mark H Weaver m...@netris.org writes: Andy Wingo wi...@pobox.com writes: None of the interfaces that I proposed leak internal psyntax representations. `syntax-local-binding' leaks the internal representations used for bindings. You mean, whether something is a lexical, or a macro, or a global, or whatever; OK. I think that leak is the wrong word here: leaks are inadvertent, whereas providing this information is what this function was designed to do; and furthermore it's not a detail of psyntax (cf Racket which does not use psyntax). Let me offer another example of its utility: writing a macro stepper. Again, for an example I'll have to link to the PLT folks' great work: http://docs.racket-lang.org/macro-debugger/index.html With syntax-local-binding, syntax-locally-bound-identifiers, and a couple hooks that get fired when a macro is expanded and reconstructed, you could implement a macro stepper for Guile. And, once we provide those hooks, you can implement this in a module. Pretty sweet, if you ask me! Perhaps, though, at this point we're just going to have to agree to disagree; surely we have plumbed the depths sufficiently. It's not satisfying, but I think we both made a great effort to communicate and convince -- and in some cases, to work on each other's code. We really need to move on here. I will re-post my patches taking into account your comments regarding the form of the environments. Regards, Andy -- http://wingolog.org/
Re: Unsafe psyntax label generation
On Tue 24 Jan 2012 15:01, Mark H Weaver m...@netris.org writes: `local-eval' combines syntax objects from two different sessions into a single syntax object (in the wrapper procedure), and thus there may be label name collisions. Now, if this combined syntax object is serialized as a compiled procedure, these labels with the same name will be optimized together into the same string object! A very good point! Cf. Aziz's psyntax/expander.ss from r6rs-libraries.dev: ;;; (two marks must be eq?-comparable, so we use a string ;;; of one char (this assumes that strings are mutable)). ;;; gen-mark generates a new unique mark (define (gen-mark) ;;; faster (string #\m)) ;;; every identifier in the program would have a label associated ;;; with it in its substitution. gen-label generates such labels. ;;; the labels have to have read/write eq? invariance to support ;;; separate compilation. (define gen-label (lambda (_) (gensym))) His gensyms are globally unique; he made the exact opposite choice that I did in the fd5985271fee3bcb6a290b6ad10525980a97ef8d; interesting. I wonder who is right here. Maybe there's something I'm missing here, but if the change you made above was safe, I think the burden of proof is on you to explain why. Indeed, and I think you found a good indication that it's not right. Andy -- http://wingolog.org/
Re: guile-library - make bug report
Hi David, On Fri 20 Jan 2012 19:17, David Pirotte da...@altosw.be writes: guile 2.0.3.150-88c0 guile-lib release-0.2.1-2-ge9fe22b ... make[1]: Entering directory `/usr/local/src/guile-lib/git-clone/src' ../dev-environ /usr/local/bin/guile-tools compile -o text/parse-lalr.go text/parse-lalr.scm ice-9/boot-9.scm:106:20: In procedure #procedure 9a66680 at ice-9/boot-9.scm:97:6 (thrown-k . args): ice-9/boot-9.scm:106:20: In procedure scm_read_expression: text/parse-lalr.scm:1418:45: unexpected ) make[1]: *** [text/parse-lalr.go] Error 1 make[1]: Leaving directory `/usr/local/src/guile-lib/git-clone/src' make: *** [all-recursive] Error 1 I can't reproduce this bug. I am at the same git version. Can anyone else reproduce this? Andy -- http://wingolog.org/
Re: mark uniqueness
Andy Wingo wi...@pobox.com writes: (define-syntax-rule (define-const x val) (begin (define t val) (define-syntax x (identifier-syntax t Here, `t' will have a fresh mark. Now, if in one compilation unit, I do: (define-const x 10) And in another, I do: (let ((t 20)) x) = ? You would expect the result to be 20. But I think it could be 20, if the marks on the two ts happened to collide. Ah yes, indeed you are right. Thanks for this explanation. I guess we need universally-unique gensyms for marks at least, regardless of which `local-eval' implementation we use. Thanks, Mark
Re: Unsafe psyntax label generation
Andy Wingo wi...@pobox.com writes: On Tue 24 Jan 2012 15:01, Mark H Weaver m...@netris.org writes: `local-eval' combines syntax objects from two different sessions into a single syntax object (in the wrapper procedure), and thus there may be label name collisions. Now, if this combined syntax object is serialized as a compiled procedure, these labels with the same name will be optimized together into the same string object! A very good point! Cf. Aziz's psyntax/expander.ss from r6rs-libraries.dev: ;;; (two marks must be eq?-comparable, so we use a string ;;; of one char (this assumes that strings are mutable)). ;;; gen-mark generates a new unique mark (define (gen-mark) ;;; faster (string #\m)) ;;; every identifier in the program would have a label associated ;;; with it in its substitution. gen-label generates such labels. ;;; the labels have to have read/write eq? invariance to support ;;; separate compilation. (define gen-label (lambda (_) (gensym))) His gensyms are globally unique; he made the exact opposite choice that I did in the fd5985271fee3bcb6a290b6ad10525980a97ef8d; interesting. I wonder who is right here. John Cowan's recent message on scheme-reports Re: fresh empty strings suggests that Ikarus does not merge equal string literals into a single object, so I guess in that case his `gen-mark' would work properly. Maybe there's something I'm missing here, but if the change you made above was safe, I think the burden of proof is on you to explain why. Indeed, and I think you found a good indication that it's not right. How would you like to fix this? Would you like me to make a new procedure that creates a universally-unique string? Most of `gensym's current code would be moved to that new procedure, and then `gensym' would use it. Thanks, Mark
Re: syntax-local-binding
Andy Wingo wi...@pobox.com writes: In the spirit of diffusing tension here, feel free to imagine all of my words as coming from Mr. Collins for the duration of this thread ;-) Hehe :) On Tue 24 Jan 2012 14:25, Mark H Weaver m...@netris.org writes: Andy Wingo wi...@pobox.com writes: None of the interfaces that I proposed leak internal psyntax representations. `syntax-local-binding' leaks the internal representations used for bindings. You mean, whether something is a lexical, or a macro, or a global, or whatever; OK. That's actually not what I meant, although I'm not convinced that we fully understand the implications of exposing even that much. One thing that is already clear is that `identifier-syntax' and `local-eval' were previously capable of emulating variables perfectly before, whereas in the presence of `syntax-local-binding' they no longer are. This is a perfect example of how added flexibility in one aspect can lead to _reduced_ flexibility in other aspects. I think we need more time to consider the implications of this. However, the more serious problem, and the one I was actually talking about, is the _second_ value returned by `syntax-local-binding', which exposes the representations of bindings stored in the cdrs of the psyntax `r' alist. Let me offer another example of its utility: writing a macro stepper. It's not the least bit surprising that exposing internal implementation details enables the creation of all kinds of nifty things as external programs that would ordinarily need to be internal. This is perfectly obvious. However, as you wisely said to me when I posted my first `local-eval' evaluator-only implementation, this stuff has a cost, and it has to justify itself. I assumed you meant the maintenance cost of supporting this advanced functionality indefinitely, and the constraints that it places on the freedom of future implementors. Was I right? In retrospect, based on your recent behavior, I wonder if that's what you meant or if you were talking about something completely different, because you seem to have completely abandoned your original restraint, and have now gone much farther than I would ever dare to go. Indeed, as you can see, I am very unhappy about how far you have gone with this. Perhaps, though, at this point we're just going to have to agree to disagree; This is a euphemism for sorry, but we're doing it my way. Multiple people here have expressed grave concerns about your approach, and not a single person has publicly expressed their support for adding all of these new interfaces you've designed. Even Ludovic has remained silent. And yet you apparently intend to rush all of this stuff into 2.0.4. After all, nothing could be worse than having `the-environment' in psyntax, even for single release. Is this your idea of a consensus process, which you so often advocate? Mark
