Re: [racket-dev] proposal for moving to packages: repository
9 hours ago, Matthew Flatt wrote: At Wed, 22 May 2013 14:50:41 -0400, Eli Barzilay wrote: That's true, but the downside of changing the structure and having files and directories move post structure change will completely destroy the relevant edit history of the files, since it will not be carried over to the repos once it's split. It's possible that we're talking past each other due to me not getting this point. (Obligatory re-disclaimer: I consider the problem with forcing people to change their working environment much more severe.) Why is it not possible to carry over history? The history I want corresponds to `git log --follow' on each of the files that end up in a repository. I'm pretty sure that such a history of commits can be generated for any given set of files, even if no ready-made tool exists already (i.e., 'git' is plenty flexible that I can script it myself). Or maybe I'm missing some larger reason? The thing to remember is just how simple git is... There's no magical way to carry over a history artificially -- it's whatever is in the commits. To make this more concrete (and more verbose), in this context the point is that git filter-branch is a simple tool that basically replays the complete history, allowing you to plant various hooks to change the directory structure, commit messages or whatever. The new history is whatever new commits are in the revised repository, with no way to make up a history with anything else. Now, to make my first point about the potential loss of history that is inherent in the process -- say that you want to split out a drracket repo in a naive way: taking just that one directory. Since it's done naively, the resulting repository will not have the drscheme directory and its contents, which means that you lose all history of files that happened there. To try that (in a fresh clone, of course) -- first, look at the history of a random file in it: F=collects/drracket/private/app.rkt git log --format='%n%h %s' --name-only --follow -- $F Now do the revision: S=collects/drracket git filter-branch --prune-empty --subdirectory-filter $S -- --all And look at the same log line again, the history is gone: git log --format='%n%h %s' --name-only --follow -- $F If you look at the *new* file, you do see the history, but the revisions made in drscheme are gone: git log --format='%n%h %s' --name-only --follow -- private/app.rkt In any case, this danger is there no matter what, especially in our case since code has been moving around in the racket switch. I *hope* that most of it will be simple: like carrying along the drscheme directory with drracket, the scheme and mzlib with racket, etc. Later on, if these things move to compat packages, the irrelevant directories get removed from the repo without surgeries, so the history will still be there. This shows some of the tricks that might be involved in the current switch: if you'd want to have some compat package *now*, the right thing to do would be: * do a simple filter-branch to extract drscheme (and other such collections) in a new repository for compat * for drracket: do a filter-branch that keeps *both* directories in, then commit a removal of drscheme. (Optionally, use rebase to move the deletion backward...) Going back to the repo structure change that you want and the reason that I said that doing moves between the package directories post-restructure is destructive should be clear now: say that you move collects/A/x into foo/A/x as part of the restructure. Later you realize that A/x should go into the bar package instead so you just move it to bar/A/x. The history is now in, including the rename, but later on when bar is split into a separate repo, the history of the file is gone. Instead, it appears in the foo repository, ending up being deleted. One way to get around this is to avoid moving the file -- instead, do another filter-branch surgery. This will be a mess since each such change will mean rebuilding the repository with all the pain that this implies. Another way to get around it is to keep track of these moving commits, and when the time comes to split into package repos, you first do another surgery on the whole repo which moves foo/A/x to bar/A/x for all of the commits before the move (not after, since that could lead to other problems), and then do the split. This might work, but besides being very error-prone, it means doing the same kind of file-movement tracking that I'm talking about anyway. So take this all as saying that the movement of files between packages needs to be tracked anyway -- but with my suggestion the movement is delayed until it's known to be final before the repo split, which makes it more robust overall. But really, the much more tempting aspect for me is that this can be done now -- if you give me a list of packages and files, I can already do the movement script. Actually, in an
Re: [racket-dev] proposal for moving to packages: repository
9 hours ago, Carl Eastlund wrote: I was going to comment on the same thing. While a naive use of git filter-branch might not retain the history, it should be entirely possible to do something a little more intelligent and keep that history. Just to be clear, this is exactly what you can't get with filter-branch. Essentially each of the new repositories could keep the entire history of the original repository, followed by a massive move/rename, then moving forward with an individual package. This can work, but it is unrelated to filter-branch: it's basically starting each package repository from a clone of the monolithic repo, then move shuffle things around. This seems wrong to me in all kinds of ways -- but if someone wants to do this with *their* package (ie, not a package that I need to deal with), then it's certainly an option. (That's one of the big appeals of moving to packages for me: some code moves to packages which I can let myself Not Care About™. Knock youself out with tabs, spaces at ends of lines, braces in code, two spaces between bindings and values in `let's, and make sure that no file ends with a newline...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
On Thu, May 23, 2013 at 5:49 AM, Eli Barzilay e...@barzilay.org wrote: 9 hours ago, Carl Eastlund wrote: I was going to comment on the same thing. While a naive use of git filter-branch might not retain the history, it should be entirely possible to do something a little more intelligent and keep that history. Just to be clear, this is exactly what you can't get with filter-branch. Essentially each of the new repositories could keep the entire history of the original repository, followed by a massive move/rename, then moving forward with an individual package. This can work, but it is unrelated to filter-branch: it's basically starting each package repository from a clone of the monolithic repo, then move shuffle things around. This seems wrong to me in all kinds of ways -- but if someone wants to do this with *their* package (ie, not a package that I need to deal with), then it's certainly an option. It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
A few minutes ago, Carl Eastlund wrote: On Thu, May 23, 2013 at 5:49 AM, Eli Barzilay e...@barzilay.org wrote: 9 hours ago, Carl Eastlund wrote: I was going to comment on the same thing. While a naive use of git filter-branch might not retain the history, it should be entirely possible to do something a little more intelligent and keep that history. Just to be clear, this is exactly what you can't get with filter-branch. Essentially each of the new repositories could keep the entire history of the original repository, followed by a massive move/rename, then moving forward with an individual package. This can work, but it is unrelated to filter-branch: it's basically starting each package repository from a clone of the monolithic repo, then move shuffle things around. This seems wrong to me in all kinds of ways -- but if someone wants to do this with *their* package (ie, not a package that I need to deal with), then it's certainly an option. It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? The problem of filter-branch? It has no problems, it does exactly what it is supposed to do. If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. If the drracket files are irrelevant for the swindle package then they shouldn't be in the swindle repository -- and on the exact same token, the development history of drracket shouldn't be there either. (This is not new, BTW, I think that there was general concensus right from the start of the package talk that the monolithic repo is just a host for a bunch of separate projects.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
On Thu, May 23, 2013 at 6:57 AM, Eli Barzilay e...@barzilay.org wrote: A few minutes ago, Carl Eastlund wrote: On Thu, May 23, 2013 at 5:49 AM, Eli Barzilay e...@barzilay.org wrote: 9 hours ago, Carl Eastlund wrote: I was going to comment on the same thing. While a naive use of git filter-branch might not retain the history, it should be entirely possible to do something a little more intelligent and keep that history. Just to be clear, this is exactly what you can't get with filter-branch. Essentially each of the new repositories could keep the entire history of the original repository, followed by a massive move/rename, then moving forward with an individual package. This can work, but it is unrelated to filter-branch: it's basically starting each package repository from a clone of the monolithic repo, then move shuffle things around. This seems wrong to me in all kinds of ways -- but if someone wants to do this with *their* package (ie, not a package that I need to deal with), then it's certainly an option. It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? The problem of filter-branch? It has no problems, it does exactly what it is supposed to do. It has no problems? Where above you stated this is exactly what you can't get with filter-branch in reference to keeping our packages' relevant history. That sounds like a problem to me, in our current context. But filter-branch is not what I was talking about. I was talking about _not_ using filter-branch, and instead doing something that does keep history. If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. If the drracket files are irrelevant for the swindle package then they shouldn't be in the swindle repository -- and on the exact same token, the development history of drracket shouldn't be there either. (This is not new, BTW, I think that there was general concensus right from the start of the package talk that the monolithic repo is just a host for a bunch of separate projects.) Okay, then let's purge the history of irrelevant files, but keep the history of relevant files even if they weren't in the right directory. If the monolithic repo is just a host for a bunch of separate projects, shouldn't it be possible to tease out their more-or-less separate histories? --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
Just now, Carl Eastlund wrote: On Thu, May 23, 2013 at 6:57 AM, Eli Barzilay e...@barzilay.org wrote: A few minutes ago, Carl Eastlund wrote: It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? The problem of filter-branch? It has no problems, it does exactly what it is supposed to do. It has no problems? Where above you stated this is exactly what you can't get with filter-branch in reference to keeping our packages' relevant history. Relevant history is vague. The thing that you can't do with filter-branch is keep the complete history if you remove files from the history -- the files that are gone go with their history. But filter-branch is not what I was talking about. I was talking about _not_ using filter-branch, and instead doing something that does keep history. Like I said: what you're suggesting means keeping the full monolithic history of developement in the main repo, including all of the irrelevant files (which will be removed in the tip, but included in the repo). If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. If the drracket files are irrelevant for the swindle package then they shouldn't be in the swindle repository -- and on the exact same token, the development history of drracket shouldn't be there either. (This is not new, BTW, I think that there was general concensus right from the start of the package talk that the monolithic repo is just a host for a bunch of separate projects.) Okay, then let's purge the history of irrelevant files, but keep the history of relevant files even if they weren't in the right directory. If the monolithic repo is just a host for a bunch of separate projects, shouldn't it be possible to tease out their more-or-less separate histories? (*sigh*; please read the other email, where I went over this thoroughly.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
On Thu, May 23, 2013 at 7:09 AM, Eli Barzilay e...@barzilay.org wrote: Just now, Carl Eastlund wrote: On Thu, May 23, 2013 at 6:57 AM, Eli Barzilay e...@barzilay.org wrote: A few minutes ago, Carl Eastlund wrote: It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? The problem of filter-branch? It has no problems, it does exactly what it is supposed to do. It has no problems? Where above you stated this is exactly what you can't get with filter-branch in reference to keeping our packages' relevant history. Relevant history is vague. The thing that you can't do with filter-branch is keep the complete history if you remove files from the history -- the files that are gone go with their history. But filter-branch is not what I was talking about. I was talking about _not_ using filter-branch, and instead doing something that does keep history. Like I said: what you're suggesting means keeping the full monolithic history of developement in the main repo, including all of the irrelevant files (which will be removed in the tip, but included in the repo). If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. If the drracket files are irrelevant for the swindle package then they shouldn't be in the swindle repository -- and on the exact same token, the development history of drracket shouldn't be there either. (This is not new, BTW, I think that there was general concensus right from the start of the package talk that the monolithic repo is just a host for a bunch of separate projects.) Okay, then let's purge the history of irrelevant files, but keep the history of relevant files even if they weren't in the right directory. If the monolithic repo is just a host for a bunch of separate projects, shouldn't it be possible to tease out their more-or-less separate histories? (*sigh*; please read the other email, where I went over this thoroughly.) I just went over all your emails on this topic, and I can't find a single one where you addressed this specific proposal at all. I don't know which one of us is misunderstanding another on this point. --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
This has a scary feeling to it. Can you raise the level of discourse one level and perhaps figure out whether this is needed at all? I.e., find a different way to solve the problem? (What is the real problem?) On May 23, 2013, at 1:57 AM, Eric Dobson eric.n.dob...@gmail.com wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
There are several identifiers that get introduced via expansion that TR needs to give types to. Not all are exported. Sam probably knows the reasons why they are not. I had to use the same code when concocting an analysis of racket core forms. -Ian - Original Message - From: Matthias Felleisen matth...@ccs.neu.edu To: Eric Dobson eric.n.dob...@gmail.com Cc: dev dev@racket-lang.org Sent: Thu, 23 May 2013 08:54:36 -0400 (EDT) Subject: Re: [racket-dev] Constructing an identifier to an unexported binding This has a scary feeling to it. Can you raise the level of discourse one level and perhaps figure out whether this is needed at all? I.e., find a different way to solve the problem? (What is the real problem?) On May 23, 2013, at 1:57 AM, Eric Dobson eric.n.dob...@gmail.com wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On Thu, May 23, 2013 at 1:54 PM, Matthias Felleisen matth...@ccs.neu.edu wrote: Can you raise the level of discourse one level and perhaps figure out whether this is needed at all? I.e., find a different way to solve the problem? (What is the real problem?) This is important, and it's the second implementation of this feature. The previous one was substantially more painful, and involved local-expand and dumpster-diving. The reason we need a tool like this is that the expansion of some macros (examples include `for`, `delay`, `quasiquote`, among others) have runtime support functions that are not exported. Typed Racket needs to specify the type of these operations. You can see these specifications here: https://github.com/plt/racket/blob/master/collects/typed-racket/base-env/base-special-env.rkt . That means that it needs to construct those identifiers, although it never needs to execute code with those identifiers. The previous implementation was much slower here, remarkably. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
1. At some point, we had a macro that opened up modules and made all module-level identifiers available. Wouldn't a flavor of this macro work here and, if so, wouldn't it be cheaper? Or is this what you call dumpster-diving, traversing the expansion and extracting ids from there? 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? On May 23, 2013, at 9:22 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: On Thu, May 23, 2013 at 1:54 PM, Matthias Felleisen matth...@ccs.neu.edu wrote: Can you raise the level of discourse one level and perhaps figure out whether this is needed at all? I.e., find a different way to solve the problem? (What is the real problem?) This is important, and it's the second implementation of this feature. The previous one was substantially more painful, and involved local-expand and dumpster-diving. The reason we need a tool like this is that the expansion of some macros (examples include `for`, `delay`, `quasiquote`, among others) have runtime support functions that are not exported. Typed Racket needs to specify the type of these operations. You can see these specifications here: https://github.com/plt/racket/blob/master/collects/typed-racket/base-env/base-special-env.rkt . That means that it needs to construct those identifiers, although it never needs to execute code with those identifiers. The previous implementation was much slower here, remarkably. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On Thu, May 23, 2013 at 2:29 PM, Matthias Felleisen matth...@ccs.neu.edu wrote: 1. At some point, we had a macro that opened up modules and made all module-level identifiers available. Wouldn't a flavor of this macro work here and, if so, wouldn't it be cheaper? Or is this what you call dumpster-diving, traversing the expansion and extracting ids from there? You're probably thinking of `require/expose`. This isn't the same feature, because it gives you the right value, but not the right identifier. But it works similarly to the way the code Eric posted now works, using namespaces explicitly. 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? No. We can't specify types inside `racket/base` without making `racket/base` depend on Typed Racket. Sam On May 23, 2013, at 9:22 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: On Thu, May 23, 2013 at 1:54 PM, Matthias Felleisen matth...@ccs.neu.edu wrote: Can you raise the level of discourse one level and perhaps figure out whether this is needed at all? I.e., find a different way to solve the problem? (What is the real problem?) This is important, and it's the second implementation of this feature. The previous one was substantially more painful, and involved local-expand and dumpster-diving. The reason we need a tool like this is that the expansion of some macros (examples include `for`, `delay`, `quasiquote`, among others) have runtime support functions that are not exported. Typed Racket needs to specify the type of these operations. You can see these specifications here: https://github.com/plt/racket/blob/master/collects/typed-racket/base-env/base-special-env.rkt . That means that it needs to construct those identifiers, although it never needs to execute code with those identifiers. The previous implementation was much slower here, remarkably. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On May 23, 2013, at 9:34 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? No. We can't specify types inside `racket/base` without making `racket/base` depend on Typed Racket. 1. I was proposing a fundamental change to the language, with an eye toward Racket 2. 2. I was also proposing an experiment that temporarily creates such a dependency and we can then look for a refactoring that breaks the dependency again but in a way that supports the proper access to these base identifiers. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On Thu, May 23, 2013 at 9:39 AM, Matthias Felleisen matth...@ccs.neu.eduwrote: On May 23, 2013, at 9:34 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? No. We can't specify types inside `racket/base` without making `racket/base` depend on Typed Racket. 1. I was proposing a fundamental change to the language, with an eye toward Racket 2. 2. I was also proposing an experiment that temporarily creates such a dependency and we can then look for a refactoring that breaks the dependency again but in a way that supports the proper access to these base identifiers. It shouldn't be necessary to specify types inside racket/base; it's only necessary to make the identifiers available somehow. Then TR can do the type specification, but without using namespaces. Protecting the exported identifiers from misuse could be done by convention -- naming them unsafe-foo or exporting them from a submodule named private -- or by enforcement -- for instance, rather than providing them, instead exporting a phase 1 syntax object that contains them with appropriate syntax taints / dye packs so that they can be used for free-identifier=? but not put into expanded code. --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On Thu, May 23, 2013 at 2:39 PM, Matthias Felleisen matth...@ccs.neu.edu wrote: On May 23, 2013, at 9:34 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? No. We can't specify types inside `racket/base` without making `racket/base` depend on Typed Racket. 1. I was proposing a fundamental change to the language, with an eye toward Racket 2. 2. I was also proposing an experiment that temporarily creates such a dependency and we can then look for a refactoring that breaks the dependency again but in a way that supports the proper access to these base identifiers. Given that we're currently working on splitting the entire system to make this dependency impossible, I don't think this is a viable option currently. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
+1 On May 23, 2013, at 9:42 AM, Carl Eastlund c...@ccs.neu.edu wrote: On Thu, May 23, 2013 at 9:39 AM, Matthias Felleisen matth...@ccs.neu.edu wrote: On May 23, 2013, at 9:34 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? No. We can't specify types inside `racket/base` without making `racket/base` depend on Typed Racket. 1. I was proposing a fundamental change to the language, with an eye toward Racket 2. 2. I was also proposing an experiment that temporarily creates such a dependency and we can then look for a refactoring that breaks the dependency again but in a way that supports the proper access to these base identifiers. It shouldn't be necessary to specify types inside racket/base; it's only necessary to make the identifiers available somehow. Then TR can do the type specification, but without using namespaces. Protecting the exported identifiers from misuse could be done by convention -- naming them unsafe-foo or exporting them from a submodule named private -- or by enforcement -- for instance, rather than providing them, instead exporting a phase 1 syntax object that contains them with appropriate syntax taints / dye packs so that they can be used for free-identifier=? but not put into expanded code. --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On May 23, 2013, at 9:42 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: Given that we're currently working on splitting the entire system to make this dependency impossible, I don't think this is a viable option currently. Perhaps that's a mistake. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Parens/string quotes automatic behavior
Hello Racket devs, I'm working on tweaking how typing a double quote is handled in strings when DrRacket's auto parens mode is on, per recent post on the users list. If any of you use the mode and can offer feedback on the following, it'd be appreciated: In addition to handling Laurent's initial feature request (see message at bottom), I'm setting it up so that if the following string is in the DrRacket window: abcdefghi and you select the _def_ and press (double quote), then it places double-quotes around the def with additional quotes to ensure that the other two portions of the string are still validly delimited, i.e. abcdefghi One question is where to put the cursor following this operation? Should it be right inside the beginning of the lifted string, or in front of its double quotes, i.e.: abc|defghi or abc|defghi ? Another question is whether to space off the def string that is created from the surrounding ones, i.e. abc def ghi instead of abcdefghi? (As a side note: the Emacs paredit mode handles the situation of a double quotes typed inside a string by inserting an escaped \. It uses a separate key combination to handle splitting both strings and parenthesized expressions into two pieces.) Thanks, --- nadeem On May 22, 2013, at 7:07 AM, Laurent wrote: Hi, The new behavior of automatic parenthesis matching is really nice, but there is one problem with string quotes. For example, if the cursor is in the middle of a string and I type the string-quote symbol , it places a quote which cuts the current string and leaves the right part in a bad syntax. Most of the time, when I type a quote inside a string, it's because I want to split the string in two parts. To do that, I have to type string-quote, string-quote, delete (to remove the extra string-quote added by the paren-match behavior), and left to go back between the two strings, which is mildly annoying. Would it be possible (unless problematic) to have the default paren-match behavior for strings that splits the string instead of inserting a single string-quote, possibly unless the left symbol is a backslash? _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
This sounds like the right solution to me too. Robby On Thursday, May 23, 2013, Matthias Felleisen wrote: +1 On May 23, 2013, at 9:42 AM, Carl Eastlund c...@ccs.neu.edu javascript:; wrote: On Thu, May 23, 2013 at 9:39 AM, Matthias Felleisen matth...@ccs.neu.edu javascript:; wrote: On May 23, 2013, at 9:34 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edujavascript:; wrote: 2. Is it possible that we could solve the problem via a bootstrapping-only violation of our policy that you can add types to Racket w/o modifying existing modules? No. We can't specify types inside `racket/base` without making `racket/base` depend on Typed Racket. 1. I was proposing a fundamental change to the language, with an eye toward Racket 2. 2. I was also proposing an experiment that temporarily creates such a dependency and we can then look for a refactoring that breaks the dependency again but in a way that supports the proper access to these base identifiers. It shouldn't be necessary to specify types inside racket/base; it's only necessary to make the identifiers available somehow. Then TR can do the type specification, but without using namespaces. Protecting the exported identifiers from misuse could be done by convention -- naming them unsafe-foo or exporting them from a submodule named private -- or by enforcement -- for instance, rather than providing them, instead exporting a phase 1 syntax object that contains them with appropriate syntax taints / dye packs so that they can be used for free-identifier=? but not put into expanded code. --Carl _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [racket] Parens/string quotes automatic behavior
On Thursday, May 23, 2013, Nadeem Abdul Hamid wrote: Hello Racket devs, I'm working on tweaking how typing a double quote is handled in strings when DrRacket's auto parens mode is on, per recent post on the users list. If any of you use the mode and can offer feedback on the following, it'd be appreciated: In addition to handling Laurent's initial feature request (see message at bottom), I'm setting it up so that if the following string is in the DrRacket window: abcdefghi and you select the _def_ and press (double quote), then it places double-quotes around the def with additional quotes to ensure that the other two portions of the string are still validly delimited, i.e. abcdefghi One question is where to put the cursor following this operation? Should it be right inside the beginning of the lifted string, or in front of its double quotes, i.e.: abc|defghi or abc|defghi ? Oh: another possibility: keep the selection where it is (or, perhaps better, select the open quote, the def, and the close quote). Another question is whether to space off the def string that is created from the surrounding ones, i.e. abc def ghi instead of abcdefghi? (As a side note: the Emacs paredit mode handles the situation of a double quotes typed inside a string by inserting an escaped \. It uses a separate key combination to handle splitting both strings and parenthesized expressions into two pieces.) Thanks, --- nadeem On May 22, 2013, at 7:07 AM, Laurent wrote: Hi, The new behavior of automatic parenthesis matching is really nice, but there is one problem with string quotes. For example, if the cursor is in the middle of a string and I type the string-quote symbol , it places a quote which cuts the current string and leaves the right part in a bad syntax. Most of the time, when I type a quote inside a string, it's because I want to split the string in two parts. To do that, I have to type string-quote, string-quote, delete (to remove the extra string-quote added by the paren-match behavior), and left to go back between the two strings, which is mildly annoying. Would it be possible (unless problematic) to have the default paren-match behavior for strings that splits the string instead of inserting a single string-quote, possibly unless the left symbol is a backslash? _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [racket] Parens/string quotes automatic behavior
On May 23, 2013, at 8:13 AM, Robby Findler wrote: On Thursday, May 23, 2013, Nadeem Abdul Hamid wrote: Hello Racket devs, I'm working on tweaking how typing a double quote is handled in strings when DrRacket's auto parens mode is on, per recent post on the users list. If any of you use the mode and can offer feedback on the following, it'd be appreciated: In addition to handling Laurent's initial feature request (see message at bottom), I'm setting it up so that if the following string is in the DrRacket window: abcdefghi and you select the _def_ and press (double quote), then it places double-quotes around the def with additional quotes to ensure that the other two portions of the string are still validly delimited, i.e. abcdefghi One question is where to put the cursor following this operation? Should it be right inside the beginning of the lifted string, or in front of its double quotes, i.e.: abc|defghi or abc|defghi ? Oh: another possibility: keep the selection where it is (or, perhaps better, select the open quote, the def, and the close quote). +1 John _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [racket] Parens/string quotes automatic behavior
+1 indeed, that way you can follow easily with typing a paren, thus enclosing it again. Laurent Le 23 mai 2013 17:17, John Clements cleme...@brinckerhoff.org a écrit : On May 23, 2013, at 8:13 AM, Robby Findler wrote: On Thursday, May 23, 2013, Nadeem Abdul Hamid wrote: Hello Racket devs, I'm working on tweaking how typing a double quote is handled in strings when DrRacket's auto parens mode is on, per recent post on the users list. If any of you use the mode and can offer feedback on the following, it'd be appreciated: In addition to handling Laurent's initial feature request (see message at bottom), I'm setting it up so that if the following string is in the DrRacket window: abcdefghi and you select the _def_ and press (double quote), then it places double-quotes around the def with additional quotes to ensure that the other two portions of the string are still validly delimited, i.e. abcdefghi One question is where to put the cursor following this operation? Should it be right inside the beginning of the lifted string, or in front of its double quotes, i.e.: abc|defghi or abc|defghi ? Oh: another possibility: keep the selection where it is (or, perhaps better, select the open quote, the def, and the close quote). +1 John _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
Hi Eli: I'm trying to understand your point. Do I have this right? Background: The git history consists of a series checkpoints in time of the entire repository, not a collection of individual files. So, when I do git log x.rkt then what I get is essentially a filtered list (except where people didn't properly rebase, but lets ignore that) of those checkpoints: all the ones where x.rkt changed. Big Question: The issue is, then, when we split up the current repo into smaller repos, what are the series of checkpoints that we're going to make up for the individual repos? Right? Your Advice: And, IIUC, you're suggesting that the best way to deal with this question is to defer it until we are more sure of the actual split we want to make. So we don't mess with the history at all and instead just work at the level of some script that we can run to just use mv and company to move things around. When we know exactly what ends up going where, then we can figure out how to make up a new, useful history for the separate repositories. Is that the point? Robby On Thu, May 23, 2013 at 4:41 AM, Eli Barzilay e...@barzilay.org wrote: 9 hours ago, Matthew Flatt wrote: At Wed, 22 May 2013 14:50:41 -0400, Eli Barzilay wrote: That's true, but the downside of changing the structure and having files and directories move post structure change will completely destroy the relevant edit history of the files, since it will not be carried over to the repos once it's split. It's possible that we're talking past each other due to me not getting this point. (Obligatory re-disclaimer: I consider the problem with forcing people to change their working environment much more severe.) Why is it not possible to carry over history? The history I want corresponds to `git log --follow' on each of the files that end up in a repository. I'm pretty sure that such a history of commits can be generated for any given set of files, even if no ready-made tool exists already (i.e., 'git' is plenty flexible that I can script it myself). Or maybe I'm missing some larger reason? The thing to remember is just how simple git is... There's no magical way to carry over a history artificially -- it's whatever is in the commits. To make this more concrete (and more verbose), in this context the point is that git filter-branch is a simple tool that basically replays the complete history, allowing you to plant various hooks to change the directory structure, commit messages or whatever. The new history is whatever new commits are in the revised repository, with no way to make up a history with anything else. Now, to make my first point about the potential loss of history that is inherent in the process -- say that you want to split out a drracket repo in a naive way: taking just that one directory. Since it's done naively, the resulting repository will not have the drscheme directory and its contents, which means that you lose all history of files that happened there. To try that (in a fresh clone, of course) -- first, look at the history of a random file in it: F=collects/drracket/private/app.rkt git log --format='%n%h %s' --name-only --follow -- $F Now do the revision: S=collects/drracket git filter-branch --prune-empty --subdirectory-filter $S -- --all And look at the same log line again, the history is gone: git log --format='%n%h %s' --name-only --follow -- $F If you look at the *new* file, you do see the history, but the revisions made in drscheme are gone: git log --format='%n%h %s' --name-only --follow -- private/app.rkt In any case, this danger is there no matter what, especially in our case since code has been moving around in the racket switch. I *hope* that most of it will be simple: like carrying along the drscheme directory with drracket, the scheme and mzlib with racket, etc. Later on, if these things move to compat packages, the irrelevant directories get removed from the repo without surgeries, so the history will still be there. This shows some of the tricks that might be involved in the current switch: if you'd want to have some compat package *now*, the right thing to do would be: * do a simple filter-branch to extract drscheme (and other such collections) in a new repository for compat * for drracket: do a filter-branch that keeps *both* directories in, then commit a removal of drscheme. (Optionally, use rebase to move the deletion backward...) Going back to the repo structure change that you want and the reason that I said that doing moves between the package directories post-restructure is destructive should be clear now: say that you move collects/A/x into foo/A/x as part of the restructure. Later you realize that A/x should go into the bar package instead so you just move it to bar/A/x. The history is now in, including the rename, but later on when bar is split into a separate
Re: [racket-dev] Constructing an identifier to an unexported binding
On 05/23/2013 01:57 AM, Eric Dobson wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? There's another way around this issue, which is to avoid creating these identifiers at all. In other words, change the representation of the type environment to something that supports symbol+module pairs as keys in addition to identifiers. The easiest way to do that is to add in a hash table behind the current free-id-table, since the two tables would handle disjoint sets of identifiers. Ryan _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
On Thu, May 23, 2013 at 4:13 PM, Ryan Culpepper ry...@ccs.neu.edu wrote: On 05/23/2013 01:57 AM, Eric Dobson wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? There's another way around this issue, which is to avoid creating these identifiers at all. In other words, change the representation of the type environment to something that supports symbol+module pairs as keys in addition to identifiers. The easiest way to do that is to add in a hash table behind the current free-id-table, since the two tables would handle disjoint sets of identifiers. Ryan I would not have thought that'd work, but apparently identifier-binding will give one that information. Nice going, Ryan! --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
At Thu, 23 May 2013 07:09:17 -0400, Eli Barzilay wrote: Relevant history is vague. The history I want corresponds to `git log --follow' on each of the files that end up in a repository. The thing that you can't do with filter-branch is keep the complete history if you remove files from the history -- the files that are gone go with their history. That's true if you use `git filter-branch' in a particular way. I'll suggest an alternative way, which involves filtering the set of files in a commit-specific way. That is, the right set of files to keep for each commit are not the ones in the final place, but the ones whose history we need at each commit. To make sure I'm not confused, I've implemented this idea. My implementation is unlikely to be exactly right, yet, but I think it works as a proof of concept. The enclosed slice.rkt script takes a subdirectory and a destination directory. Run it in the top directory of a git repository, and it finds all the files in the given subdirectory, and then it closes over the history of each file via `git log --follow'. From that point, we could use the computed set of paths as the ones to keep during a `git filter-branch' on every commit, but that's not ideal. For example, a file in collection a that is destined for package a may have originated in b (think mzlib), where the same-named file sticks around in b after the copy. It's nicer and cleaner to have irrelevant files disappear after the relevant copy/move is made. So, I took one more step: slice.rkt constructs a range of commits during which the file should exist, based on when it was moved or copied. (Forks and merges are a minor obstacle, which the script works around by enlarging ranges to hit commits in the `--first-parent' traversal.) Conceptually, the result is a mapping from commit ids to paths, but that would be a big table to read on every `filter-branch' step, so it's reported as a table of commits with enter/leave transitions. The output of slice.rkt is files: state.rktd for the set of files to be kept in the initial commit, and actions.rktd to specify the transitions. The enclosed prune.rkt script works with `git filter-branch --index-filter'. It uses actions.rktd (read-only) and state.rktd (which it updates via transitions). The Racket git repo is large, so I've only tried the `git filter-branch' step so far on smaller repos, such as the iplt repository. In my clone of iplt, I `git mv'ed web/internal to ex/internal. Then, with the scripts in /tmp, racket /tmp/slice.rkt ex /tmp git filter-branch --index-filter racket /tmp/prune.rkt /tmp --prune-empty leaves the repo with only the files of ex, and `git log --follow' on various files looks right. I'll try on a clone of the Racket repo and report back. FWIW, before doing this for real, I'd want to add a `--msg-filter' that extends each commit message to add the original commit id, since we have references to the old ids in various places (and so it would be handy to have them in the new repos). slice.rkt Description: Binary data prune.rkt Description: Binary data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
Isn't that exactly what free-indentifier=? is checking for on identfiers with a module level binding? Or is there something else it does? On Thu, May 23, 2013 at 3:13 PM, Carl Eastlund c...@ccs.neu.edu wrote: On Thu, May 23, 2013 at 4:13 PM, Ryan Culpepper ry...@ccs.neu.edu wrote: On 05/23/2013 01:57 AM, Eric Dobson wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? There's another way around this issue, which is to avoid creating these identifiers at all. In other words, change the representation of the type environment to something that supports symbol+module pairs as keys in addition to identifiers. The easiest way to do that is to add in a hash table behind the current free-id-table, since the two tables would handle disjoint sets of identifiers. Ryan I would not have thought that'd work, but apparently identifier-binding will give one that information. Nice going, Ryan! --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
Essentially yes. It doesn't do anything else, but it needs an identifier to do it. Currently, TR starts with a module and a symbol, goes through an expensive process to forge an identifier from them, just to call free-identifier=? to compare based on the module and the symbol after all. Doing the comparison directly, without ever forging the identifier, would be quicker. On Thu, May 23, 2013 at 8:43 PM, Eric Dobson eric.n.dob...@gmail.comwrote: Isn't that exactly what free-indentifier=? is checking for on identfiers with a module level binding? Or is there something else it does? On Thu, May 23, 2013 at 3:13 PM, Carl Eastlund c...@ccs.neu.edu wrote: On Thu, May 23, 2013 at 4:13 PM, Ryan Culpepper ry...@ccs.neu.edu wrote: On 05/23/2013 01:57 AM, Eric Dobson wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? There's another way around this issue, which is to avoid creating these identifiers at all. In other words, change the representation of the type environment to something that supports symbol+module pairs as keys in addition to identifiers. The easiest way to do that is to add in a hash table behind the current free-id-table, since the two tables would handle disjoint sets of identifiers. Ryan I would not have thought that'd work, but apparently identifier-binding will give one that information. Nice going, Ryan! --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Constructing an identifier to an unexported binding
Right, but why cannot we forge an identifier easily? I'm happy getting an armed identifier. What are the reasons for preventing such a construction? On Thu, May 23, 2013 at 6:04 PM, Carl Eastlund c...@ccs.neu.edu wrote: Essentially yes. It doesn't do anything else, but it needs an identifier to do it. Currently, TR starts with a module and a symbol, goes through an expensive process to forge an identifier from them, just to call free-identifier=? to compare based on the module and the symbol after all. Doing the comparison directly, without ever forging the identifier, would be quicker. On Thu, May 23, 2013 at 8:43 PM, Eric Dobson eric.n.dob...@gmail.com wrote: Isn't that exactly what free-indentifier=? is checking for on identfiers with a module level binding? Or is there something else it does? On Thu, May 23, 2013 at 3:13 PM, Carl Eastlund c...@ccs.neu.edu wrote: On Thu, May 23, 2013 at 4:13 PM, Ryan Culpepper ry...@ccs.neu.edu wrote: On 05/23/2013 01:57 AM, Eric Dobson wrote: Some modules have macros which expand into identifiers that are not exported, as they want to protect those bindings. TR currently has the following code which allows it to generate an identifier which is free-identifier=? to what would appear in the output of the macros. define (make-template-identifier what where) (let ([name (module-path-index-resolve (module-path-index-join where #f))]) (parameterize ([current-namespace (make-empty-namespace)]) (namespace-attach-module (current-namespace) ''#%kernel) (parameterize ([current-module-declare-name name]) (eval `(,#'module any '#%kernel (#%provide ,what) (define-values (,what) #f (namespace-require `(for-template ,name)) (namespace-syntax-introduce (datum-syntax #f what) This turns out to be a slightly slow part of the initialization of TR. Does anyone know another way to get such an identifier? There's another way around this issue, which is to avoid creating these identifiers at all. In other words, change the representation of the type environment to something that supports symbol+module pairs as keys in addition to identifiers. The easiest way to do that is to add in a hash table behind the current free-id-table, since the two tables would handle disjoint sets of identifiers. Ryan I would not have thought that'd work, but apparently identifier-binding will give one that information. Nice going, Ryan! --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
