Re: Fixing the slib mess
Hello Mikael, A pleasure to see you around! On Mon 22 Oct 2012 01:11, Mikael Djurfeldt mik...@djurfeldt.com writes: When trying to use guile 2 for logic programming I discovered that the slib interface is again broken (and has been for quite some time). I am very sorry that I did not see this thread before hacking on this recently. Somehow over the past three or four months I just managed to drop everything and the inboxes filled without being filtered or drained in any way -- and to attack that I decided to just run through individual lists in order. A strange strategy, but it is good for honing the does something need to be done about this or can I drop it? instinct. Anyway I picked up something in the user list about Slib, looked into it, and then decided to fix it, without having seen this mail -- resulting in the recent patches to Slib CVS and Guile git. I'm sorry to have stepped on your toes here. In any case I didn't check it thoroughly, so surely there are issues yet to resolve. The implementation of the interface has two sides. One, the file ice-9/slib.scm, is owned by Guile. The other, slib/guile.init, is owned by slib. slib has such .init files for some common scheme implementations but I early on noticed that that the guile.init file is not really maintained. I decided that it would be more robust if slib.scm incorporated most of the interface so that it would be easy to update it as Guile changed. But of course slib also changed and at some point others felt that guile.init should contain most of the interface and the bulk of slib.scm was moved there. As we have seen, this didn't make things much better. Yes, in many ways I would like to have the interface in Guile. However it seems that time has shown that it really wants to live in slib -- probably because that's where people care most about slib. At least with Guile 2 we have managed to clean up many of the version dependent hacks, by just delegating to a fresh file for Guile 2. Anyway. Perhaps I did the wrong thing in fixing it this way? I would be very happy to commit anything you have. Please take a look at both Slib and Guile from their version control systems, and the recent patch about `include'. Aubrey seems quite responsive in dealing with patches, so if there is a change to make, I'm sure we can get it in. *But*, the proper implementation of syntax-toplevel? requires modification of psyntax.scm and adding it to the (system syntax) module. Do you have a new patch for this one? Regards, Andy -- http://wingolog.org/
Re: Fixing the slib mess
Hi Andy, No problem at all! In fact, apologies are entirely on my side: I thought I would get time to hack on this before and during Christmas, but this turned out not to be true. Great that you fixed it! If I have anything to add, I will of course bring that up. Now, I'm looking into porting Gerald Sussman's scmutils to Guile-2.0. I'm aware of an older port by Daniel Gildea but I don't think that uses GOOPS: I'm currently wondering if it could make sense to try to make an mit-scheme compatibility module providing the needed bindings. In that way a port could become easier to maintain and maybe such a module could also be useful for other mit-scheme software. Again, the amount of time I can spend on this is highly unpredictable... :( Best regards, Mikael On Mon, Jan 21, 2013 at 6:58 PM, Andy Wingo wi...@pobox.com wrote: Hello Mikael, A pleasure to see you around! On Mon 22 Oct 2012 01:11, Mikael Djurfeldt mik...@djurfeldt.com writes: When trying to use guile 2 for logic programming I discovered that the slib interface is again broken (and has been for quite some time). I am very sorry that I did not see this thread before hacking on this recently. Somehow over the past three or four months I just managed to drop everything and the inboxes filled without being filtered or drained in any way -- and to attack that I decided to just run through individual lists in order. A strange strategy, but it is good for honing the does something need to be done about this or can I drop it? instinct. Anyway I picked up something in the user list about Slib, looked into it, and then decided to fix it, without having seen this mail -- resulting in the recent patches to Slib CVS and Guile git. I'm sorry to have stepped on your toes here. In any case I didn't check it thoroughly, so surely there are issues yet to resolve. The implementation of the interface has two sides. One, the file ice-9/slib.scm, is owned by Guile. The other, slib/guile.init, is owned by slib. slib has such .init files for some common scheme implementations but I early on noticed that that the guile.init file is not really maintained. I decided that it would be more robust if slib.scm incorporated most of the interface so that it would be easy to update it as Guile changed. But of course slib also changed and at some point others felt that guile.init should contain most of the interface and the bulk of slib.scm was moved there. As we have seen, this didn't make things much better. Yes, in many ways I would like to have the interface in Guile. However it seems that time has shown that it really wants to live in slib -- probably because that's where people care most about slib. At least with Guile 2 we have managed to clean up many of the version dependent hacks, by just delegating to a fresh file for Guile 2. Anyway. Perhaps I did the wrong thing in fixing it this way? I would be very happy to commit anything you have. Please take a look at both Slib and Guile from their version control systems, and the recent patch about `include'. Aubrey seems quite responsive in dealing with patches, so if there is a change to make, I'm sure we can get it in. *But*, the proper implementation of syntax-toplevel? requires modification of psyntax.scm and adding it to the (system syntax) module. Do you have a new patch for this one? Regards, Andy -- http://wingolog.org/
Re: Fixing the slib mess
Howdy :) On Mon 21 Jan 2013 19:09, Mikael Djurfeldt mik...@djurfeldt.com writes: Now, I'm looking into porting Gerald Sussman's scmutils to Guile-2.0. I'm aware of an older port by Daniel Gildea but I don't think that uses GOOPS: Neat; GOOPS is definitely the way to go there ;-) I'm currently wondering if it could make sense to try to make an mit-scheme compatibility module providing the needed bindings. In that way a port could become easier to maintain and maybe such a module could also be useful for other mit-scheme software. Yes I think that can make a lot of sense. Another thing to think about is this guildhall thing that folks are building (especially Ian Price) -- seems that guile update guild install foo is starting to work. A guildhall MIT scheme compatibility package would be great; that would let us have a guildhall package for scmutils as well. Dunno, just thoughts for your Copious Free Time ;-) Cheers! Andy -- http://wingolog.org/
Re: Fixing the slib mess
Mikael Djurfeldt mik...@djurfeldt.com writes: On Tue, Oct 23, 2012 at 8:01 PM, Mark H Weaver m...@netris.org wrote: Anyway, here's another idea: after requiring a new slib package, iterate over the entire list of top-level bindings in the slib module and export everything. What do you think? I think it sounds like the best idea so far. I'll try to go with this. Sounds good. One more thing: ideally, any logic that peeks into Guile internals or is likely to change between Guile versions should be in slib.scm, and anything that's likely to change between slib versions should be in guile.init. Does that make sense? Three problems come to my mind: 1. guile.init is really mostly a kind of interface, meaning that changes in both Guile and slib can affect the same pieces of code. 2. guile.init is supposed to work with a series of Guile versions. If I now try to do a larger reorganization, I will likely break compatibility with some older Guile versions, especially if I start moving things back to ice-9/slib.scm. 3. I don't really have time currently to do a full reorganization. Okay, that's fine. I'd say go ahead and do whatever you have time for. We can always rethink the interface between slib.scm and guile.init for Guile 2.2, if someone volunteers to do that work. Thanks, Mark
Re: Fixing the slib mess
On Mon, Oct 22, 2012 at 11:51 PM, Mark H Weaver m...@netris.org wrote: It looks to me like your current implementation of 'syntax-toplevel?' is actually testing for a top-level _syntactic_ environment, but what you ought to be testing for here is slightly different. You are absolutely right. Thank you for spotting this. Unfortunately my scheming knowledge is a bit rusty. I'm not sure whether the wrap contains enough information to determine that. I don't think it does either. It might be easier to handle this with 'define-syntax-parameter' and 'syntax-parameterize'. The idea would be that within slib, 'define' would be a syntax parameter. Its default expansion would turn it into 'define-public', and also parameterize 'define' to mean 'base:define' within the body. If needed, you could also define 'let' and maybe some other things to parameterize 'define' within the body. Another option would be to make 'export' a syntax parameter, and parameterize it to a no-op within lexical contours such as 'define' and 'let'. What do you think? Correct me if I'm wrong, but doesn't this involve re-defining the syntax for all forms with bodies (in order to introduce the syntax-parameterize form)? I happen to be working on the reader lately. Would it help to implement SRFI-58 in our reader? While I think SRFI-58 support is great, I don't think slib needs it because it doesn't, to my knowledge, use read syntax.
Re: Fixing the slib mess
Hi Mikael, Mikael Djurfeldt mik...@djurfeldt.com writes: On Mon, Oct 22, 2012 at 11:51 PM, Mark H Weaver m...@netris.org wrote: It might be easier to handle this with 'define-syntax-parameter' and 'syntax-parameterize'. The idea would be that within slib, 'define' would be a syntax parameter. Its default expansion would turn it into 'define-public', and also parameterize 'define' to mean 'base:define' within the body. If needed, you could also define 'let' and maybe some other things to parameterize 'define' within the body. Another option would be to make 'export' a syntax parameter, and parameterize it to a no-op within lexical contours such as 'define' and 'let'. What do you think? Correct me if I'm wrong, but doesn't this involve re-defining the syntax for all forms with bodies (in order to introduce the syntax-parameterize form)? Yes, I guess it would, although perhaps you could come up with a smaller set of forms that would be sufficient for slib. I'm surprised that slib doesn't have a mechanism for making this job easier for Scheme systems that support modules. It seems like generating the list of exports should not be so difficult. Why doesn't slib use something like 'define-public' to facilitate this? Anyway, here's another idea: after requiring a new slib package, iterate over the entire list of top-level bindings in the slib module and export everything. What do you think? One more thing: ideally, any logic that peeks into Guile internals or is likely to change between Guile versions should be in slib.scm, and anything that's likely to change between slib versions should be in guile.init. Does that make sense? Thanks, Mark
Re: Fixing the slib mess
On Tue, Oct 23, 2012 at 8:01 PM, Mark H Weaver m...@netris.org wrote: Anyway, here's another idea: after requiring a new slib package, iterate over the entire list of top-level bindings in the slib module and export everything. What do you think? I think it sounds like the best idea so far. I'll try to go with this. One more thing: ideally, any logic that peeks into Guile internals or is likely to change between Guile versions should be in slib.scm, and anything that's likely to change between slib versions should be in guile.init. Does that make sense? Three problems come to my mind: 1. guile.init is really mostly a kind of interface, meaning that changes in both Guile and slib can affect the same pieces of code. 2. guile.init is supposed to work with a series of Guile versions. If I now try to do a larger reorganization, I will likely break compatibility with some older Guile versions, especially if I start moving things back to ice-9/slib.scm. 3. I don't really have time currently to do a full reorganization. Otherwise I concur with what you say. Problem 2 could be solved by asking Aubrey Jaffer (or who is currently maintaining slib) to include a new version of the file, guile2.init, in addition to guile.init... I'll think about your suggestions and try to come up with new patches. Best regards, Mikael D.
Re: Fixing the slib mess
Hi Mikael and welcome back! *But*, the proper implementation of syntax-toplevel? requires modification of psyntax.scm and adding it to the (system syntax) module. I didn't want to do this until I've had your comments, so the present patch has its own syntax-object accessors (which breaks abstraction and is therefore not a real solution). I should also say that I have not yet fixed the slib interface to the new Guile uniform arrays, so there's a lot of slib functionality which won't yet work. Comments? Can I add syntax-toplevel? to psyntax.scm and (system syntax)? Do you think it is reasonable to submit something along the line of guile.init.diff to slib guile.init? Best regards, Mikael Djurfeldt I can answer with some kind of suggestion here. in (system syntax) there is syntax-local-binding which you can use for example as (define-syntax f (lambda (x) (syntax-case x () ((_ x) (call-with-values (lambda () (syntax-local-binding #'x)) (lambda (x y) (pk x) (pk y))) #'#t Then, scheme@(guile-user) [1] (f +) ;;; (global) ;;; ((+ guile-user)) And, scheme@(guile-user) [1] (let ((s 1)) (f s)) ;;; (lexical) ;;; (s-490) (let ((s 1)) (define-syntax g (lambda (x) #'#f)) (f g)) ;;; (displaced-lexical) ;;; (#f) I'm not sure what exactly syntax-toplevel? does, but can you base it on syntax-local-binding? And if not is it possible to change syntax-local-binding so that you can use it? Regards Stefan
Re: Fixing the slib mess
On Mon, Oct 22, 2012 at 8:31 PM, Stefan Israelsson Tampe stefan.ita...@gmail.com wrote: Comments? Can I add syntax-toplevel? to psyntax.scm and (system syntax)? [...] I can answer with some kind of suggestion here. in (system syntax) there is syntax-local-binding which you can use for example as (define-syntax f (lambda (x) (syntax-case x () ((_ x) (call-with-values (lambda () (syntax-local-binding #'x)) (lambda (x y) (pk x) (pk y))) #'#t Then, scheme@(guile-user) [1] (f +) ;;; (global) ;;; ((+ guile-user)) And, scheme@(guile-user) [1] (let ((s 1)) (f s)) ;;; (lexical) ;;; (s-490) (let ((s 1)) (define-syntax g (lambda (x) #'#f)) (f g)) ;;; (displaced-lexical) ;;; (#f) I'm not sure what exactly syntax-toplevel? does, but can you base it on syntax-local-binding? And if not is it possible to change syntax-local-binding so that you can use it? Thanks, Stefan. (syntax-toplevel?) expands to #t if occurs in a context (position in the code if you prefer) where a (define x #f) would create/set! a global binding for x. It expands to #f otherwise. I had a look at syntax-local-binding, but decided that syntax-toplevel? was needed since the latter is not trying to determine the nature of an existing binding but rather the nature of the context. Of course oncould probe the context by first creating a new binding (with some random name) and then use syntax-local-binding to determine the nature of the context by looking at the new binding, but that seems somewhat invasive. :-)
Re: Fixing the slib mess
Yes in that case this stands on it's own! /Stefan On Mon, Oct 22, 2012 at 9:11 PM, Mikael Djurfeldt mik...@djurfeldt.comwrote: On Mon, Oct 22, 2012 at 8:31 PM, Stefan Israelsson Tampe stefan.ita...@gmail.com wrote: Comments? Can I add syntax-toplevel? to psyntax.scm and (system syntax)? [...] I can answer with some kind of suggestion here. in (system syntax) there is syntax-local-binding which you can use for example as (define-syntax f (lambda (x) (syntax-case x () ((_ x) (call-with-values (lambda () (syntax-local-binding #'x)) (lambda (x y) (pk x) (pk y))) #'#t Then, scheme@(guile-user) [1] (f +) ;;; (global) ;;; ((+ guile-user)) And, scheme@(guile-user) [1] (let ((s 1)) (f s)) ;;; (lexical) ;;; (s-490) (let ((s 1)) (define-syntax g (lambda (x) #'#f)) (f g)) ;;; (displaced-lexical) ;;; (#f) I'm not sure what exactly syntax-toplevel? does, but can you base it on syntax-local-binding? And if not is it possible to change syntax-local-binding so that you can use it? Thanks, Stefan. (syntax-toplevel?) expands to #t if occurs in a context (position in the code if you prefer) where a (define x #f) would create/set! a global binding for x. It expands to #f otherwise. I had a look at syntax-local-binding, but decided that syntax-toplevel? was needed since the latter is not trying to determine the nature of an existing binding but rather the nature of the context. Of course oncould probe the context by first creating a new binding (with some random name) and then use syntax-local-binding to determine the nature of the context by looking at the new binding, but that seems somewhat invasive. :-)
Re: Fixing the slib mess
On Mon, Oct 22, 2012 at 1:11 AM, Mikael Djurfeldt mik...@djurfeldt.com wrote: Comments? Can I add syntax-toplevel? to psyntax.scm and (system syntax)? Do you think it is reasonable to submit something along the line of guile.init.diff to slib guile.init? If I get an OK, then I would of course put some further work into this so that the full feature set of slib (including uniform arrays) gets operational. If people find my prodigious use of nested-ref ugly (but note that the original code already makes use of module system primitives), I could remove most of them as well.
Re: Fixing the slib mess
Hi Mikael! It's great to see you on guile-devel again, and it would be good to have a working slib on Guile 2. Thanks for working on this :) Mikael Djurfeldt mik...@djurfeldt.com writes: Comments? Can I add syntax-toplevel? to psyntax.scm and (system syntax)? FWIW, it sounds reasonable to add something like 'syntax-toplevel?'. However, if it's going to be added as a public API, we ought to be careful to get it right. Consider this top-level form: (let-syntax ((foo (syntax-rules () ((foo x) (quote x) (define bar (foo (a b Here, although the 'define' is expanded within a non-toplevel syntactic environment, after expansion it ends up within a top-level environment, and indeed 'bar' becomes a top-level binding. That's because the body of a 'let-syntax' or 'letrec-syntax' gets spliced into the outer context, sort of like a 'begin' but with some syntax bindings added. It looks to me like your current implementation of 'syntax-toplevel?' is actually testing for a top-level _syntactic_ environment, but what you ought to be testing for here is slightly different. I'm not sure what is the proper term, but for now I'll call it a top-level _execution_ environment. I'm also not quite sure how best to test for a top-level execution environment. I'm not sure whether the wrap contains enough information to determine that. It might be easier to handle this with 'define-syntax-parameter' and 'syntax-parameterize'. The idea would be that within slib, 'define' would be a syntax parameter. Its default expansion would turn it into 'define-public', and also parameterize 'define' to mean 'base:define' within the body. If needed, you could also define 'let' and maybe some other things to parameterize 'define' within the body. Another option would be to make 'export' a syntax parameter, and parameterize it to a no-op within lexical contours such as 'define' and 'let'. What do you think? I should also say that I have not yet fixed the slib interface to the new Guile uniform arrays, so there's a lot of slib functionality which won't yet work. I happen to be working on the reader lately. Would it help to implement SRFI-58 in our reader? Note that SRFI-58 array notation conflicts with Guile's own array notation, e.g. #1a(a b) is legal in both syntaxes and means different things. So we'd need to use a reader-directive such as #!srfi-58 to change the reader mode on that port. Would that help? Thanks for working on this! Regards, Mark
Fixing the slib mess
Dear Guile hackers, What nice work you are doing! For those who don't know me, I'm a Guile developer who has been doing other stuff for some time. When trying to use guile 2 for logic programming I discovered that the slib interface is again broken (and has been for quite some time). This easily happens because it is a very fragile interface. The way this is supposed to be used (and as documented in the manual), one does a (use-modules (ice-9 slib)) and can then do (require 'modular) etc. The module (ice-9 slib) forms a kind of sandbox so that all new definitions that are imported through require are loaded as local bindings in the (ice-9 slib) module and are exported through the public interface of (ice-9 slib). The implementation of the interface has two sides. One, the file ice-9/slib.scm, is owned by Guile. The other, slib/guile.init, is owned by slib. slib has such .init files for some common scheme implementations but I early on noticed that that the guile.init file is not really maintained. I decided that it would be more robust if slib.scm incorporated most of the interface so that it would be easy to update it as Guile changed. But of course slib also changed and at some point others felt that guile.init should contain most of the interface and the bulk of slib.scm was moved there. As we have seen, this didn't make things much better. I'll let you ponder on how to handle the fundamental problems with this interface, but, as a Guile user, I think it would be nice if the interface works as written in the manual. Attached to this email you'll find two patches. The patch to slib.scm copies a snippet of code from guile.init so that they agree with eachother and with the Guile reference manual on how to find slib in the filesystem. This patch for example makes SCHEME_LIBRARY_PATH work as described. I've tried to write the patch to guile.init so that it can play well with older Guile versions, but we should test this. In order to make it work with Guile 2, though, I had to introduce a new syntax binding syntax-toplevel?. Given a syntax object (as available within a syntax-case transformer), it decides if the object originates from top level context. It is used, as in the old memoizing macro transformer, to choose whether to define-public or just define. *But*, the proper implementation of syntax-toplevel? requires modification of psyntax.scm and adding it to the (system syntax) module. I didn't want to do this until I've had your comments, so the present patch has its own syntax-object accessors (which breaks abstraction and is therefore not a real solution). I should also say that I have not yet fixed the slib interface to the new Guile uniform arrays, so there's a lot of slib functionality which won't yet work. Comments? Can I add syntax-toplevel? to psyntax.scm and (system syntax)? Do you think it is reasonable to submit something along the line of guile.init.diff to slib guile.init? Best regards, Mikael Djurfeldt slib.scm.diff Description: Binary data guile.init.diff Description: Binary data
