Re: [racket-dev] What is the policy on what is included in the core libraries?
On Tue, Feb 17, 2015 at 8:26 AM, Robby Findler ro...@eecs.northwestern.edu wrote: I don't think the libraries are sufficient as is, but I would resist adding aliases. Perhaps a better way to get people coming from Haskell would be to write an essay specifically aimed there? - Step 1: use variables. - Step 2: here are `for` loops! Instead of an essay, this could be a guide for Haskellers/MLers/ Whatever-ers, which would make it easy to get results into documentation searches, and that would make it easy to address issues that cannot be resolved with a new binding like the bad argument order in `take` or the different input function for the folds. (Another point for avoiding a `zip` binding is that it's much more likely to collide with user defined functions than it is in other languages.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! -- You received this message because you are subscribed to the Google Groups Racket Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+unsubscr...@googlegroups.com. To post to this group, send email to racket-...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/CALO-gusXr5Di4GBb3-xi3R5sL8G1CcAfNs3%2BBOB-7uM_AavGvA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-dev] Line editing in the default REPL
On Wed, Dec 17, 2014 at 12:35 PM, Tony Garnock-Jones to...@ccs.neu.edu wrote: If anyone reading this has an interesting or unusual terminal they like to use, please run $ raco pkg install ansi $ racket -l ansi/test-raw I didn't run it, but guessing common keys isn't too difficult, of course. I'm attaching a piece of code that I have that does that. It looks to me like my code is much more ambitious -- I wanted to be able to have almost all possible keys with modifiers, including function keys and plain characters, and that makes it messier. (I posted variations of this thing a few times in the past.) likely evolve into some form of a terminfo-like facility. So it's better to just write something that can deal with terminfo directly. Sadly, terminfo is a little anaemic with respect to the sequences it specifies. To get input capabilities even half as rich as, say, emacs, you have to go beyond what is given in the terminfo database in a couple of ways. Specifically, to parse shift/control/meta combinations you need to get into terminal-specific encodings that are not covered by terminfo. Sure, but those are usually not specific too. (You're probably talking about the DEC thing, right? There's also the rxvt/aterm thing which is different and more weird.) In any case, terminfo for reading keys is obviously not too difficult to manage. If that was all that you need, then it's very easy to write code that translates the terminfo database into some readable format that you can read in Racket. (My code is basically doing that parsing, so it's 90% of what you need for that.) But the thing is that terminfo is much more needed for the related things -- querying the terminal, using escape sequences to do the interactions (and doing that without restricting yourself to some small subset), and also sending sequences that setup the terminal (like rmkx/smkx which is what tripped me earlier, and IIRC, it's needed with at least st, possibly others too). it's the blindness of going in that direction and thinking that you *won't* fall into this trap. It remains to be seen whether there are any problems resulting from this approach at all. It looks like *you're* very aware of the issues, so why not take it? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! keys.rkt Description: Binary data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
[I'll reply to your email separately...] Sorry for not sending the results I had earlier re parsing input keys. Just to get this recorded here in case someone wants to do this in the future: I was first optimistic about the prospect of parsing keys in a way that works for all terminals, provided that it's limited to the few keys that are needed for line editing. I wrote some code to parse terminfo texts which I extracted on my linux installation, and find conflicts. Turns out that it's easy to do what I wanted only for the arrow keys, but then things got very messy. Specifically, there are terminals that use the opposite escape sequences for home/end or backspace/delete. (My best guess about the origin of some of this is that there were numeric pads that were laid out top-to-bottom as in phones, and the escape sequences were based on the kp keys.) It might be that the really bad conflicts were happen with irrelevant terminals, but there are definitely conflicts involving terminals that are relevant, like xrvt and gnome-terminal. My conclusion is that even for a very simple thing you need to consult $TERM to know what to do, and some simple attempt to do that would most likely evolve into some form of a terminfo-like facility. So it's better to just write something that can deal with terminfo directly. FWIW, I don't think that it's too hard -- specifically for reading keys, it's straightforward since you just get the escape sequence that you need to identify on input. I'm attaching the code that I wrote. (To run it you might need to adjust the location of the terminfo files at the top of the code; also, it relies on infocmp as a way to print them out.) If you run it you get a huge listing of all keys, with indications about conflicts. The reallly bad conflicts would be cases where some escape sequence is a prefix of another -- and those are rare (look for prefix) and happen only with some obscure F keys. (I initially just looked at these which is why I thought that it's doable.) Also, the `parse-decisions' thing in the code is my attempt to encode how a parser should work and have it show the rest of the conflicts. I planned to add more keys and keep minimal conflicts but then I got to the ones that don't have a good solution. As an example, this part of the output: end: \eOq (!! (aterm mrxvt rxvt ... xterm-xi): clash with home of (nsterm ... vt102)) (!! (aterm mrxvt rxvt ... xterm-xi): clash with key_f0 of (xterm-r5)) \e[4~ (!! (... cygwin ... linux putty ... xterm-vt220): clash with key_select of (... aterm gnome ... rxvt ... xterm-24 ...)) shows one of these problems, where end on rxvt (popular enough) correspond to home on vt102 (popular emulation target), and the last thing shows a conflict between cygwin/linux/putty (all popular) and aterm/gnome/rxvt (popular too). [Extended footnote re time investing in this nightmarish pit of depair: If you stare at this for a while, and consider using these results, you'll get a feeling of how bad it can be... Maybe vt102 is not used too much? Would there be any need for a select key, or maybe the same escape sequence is mapped to something else that is useful? The bottom line is that I don't really reject the noiseline-style attempt to get something that works for most usert -- it's the blindness of going in that direction and thinking that you *won't* fall into this trap. (Or, if you really won't since you'll refuse to accomodate bugreports, then you'll make someone else fall into this trap because of you.)] -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! std-keys.rkt Description: Binary data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Sat, Dec 13, 2014 at 7:00 PM, Leif Andersen l...@leifandersen.net wrote: Okay, here's another idea. I have parametrized the readline collection over the readline/edit/etc. library. You can do: (require editline) For the editline equivalent. It also falls back to no line editing (rather than throwing an exception), if the library is not there. The source is here: https://github.com/LeifAndersen/readline (For reference: it's in the editline branch...) I'm thinking we could distribute the editline collection with Racket (treating libedit the same way we currently treat other native packages such as libcairo). And then we can make xrepl default. Would this be doable? I looked at the code a little, but I couldn't find the place that does the fallback part. (You're defining a `libreadline-path' which AFAICS isn't used.) But it looks like you're using the same interface for both editline and readline -- right? If so, then I think that it's better to just keep the current setup, and do this: (define libreadline (ffi-lib libreadline '(5 6 4 ) #:fail (lambda () (ffi-lib libedit ... And since that addresses an API that is shared for the two, then it's not code that should be linked only with readline, which means that it's fine to it by default...? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
: the 27 is the generic prefix for these keys, 6 stands for the shift+control, 85 is the ascii of U, and ~ terminates the escape sequence. - Note BTW that the escape sequence parser is completely broken, since it assumes some given lengths in a very primitive way that would break in the presence of these generic escapes I mentioned above. It is important to know about the escape sequences so that even if you don't know how to translate some to keys you still know when the escape sequence ends so you can ignore the whole key and not leave leftovers behind. * Your tests are not great too -- they're similar to copy+paste tests which encapsulate a specific behavior, and the thing that really matters (that it actually works) is left untested. Again, I wrote all of this in a kind of hope that you'll do this, but practically speaking, this code is so far from working that if you care for your time it would be best to avoid it. In other words, you have almost nothing done there, compared to the amount of work that should be added. (But be careful of my cheap reverse psychology...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Line editing in the default REPL
On Wed, Dec 3, 2014 at 6:22 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Wed, Dec 3, 2014 at 6:10 PM, Eli Barzilay e...@barzilay.org wrote: If you're talking about implementing line editing yourself, then my personal reaction to that would be wonderful, but doing it properly is something that is difficult and easy to underestimate I've already done this (admittedly it only works on OS X, but most Linux terminals work exactly the same with a few different constants). You can see what I have so far here: https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt If this works, I'd be thoroughly impressed -- so I tried it. I ran through a bunch of configurations that I use: None of which was OS X, which was what Leif's email explicitly said it _only works on_. My point wasn't it doesn't work outside of OSX, I was talking about a few different constants being a bad underestimation: they're not few, they're not constant, and the interfaces for getting them is what you'd expect from something that started to grow in the 70s. I think we should wait till he has a version which he says works before telling people to avoid making contributions. You should re-read my email: if there's anything that I'm telling, it's the exact opposite. A point that I repeated more than once. The thing is that it would be easy to not say anything, and watch this approach of try a few escapes and see what works fail. And it will fail, because, again, they're not constant, and you can't just get some magic combination that works for 90% of the people -- even just xterm on just linux is something that can (and often is) configured in many different ways. So what I'm doing is the opposite: I point at how such a thing should be written for it to survive -- but I appreciate Leif's time enough to warn him that this is a much bigger thing than what he seems to think. (And to be clear, the reason for this is that I've seen probably around 3 to 5 serious attempts that got dumped; and I have encountered these issues multiple times, so I know that it's hard enough that even getting simple code to work, code that I intended only for my own use, even that was pretty hard, enough to go chase old VT references and obscure man pages.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: All the history for the code has been preserved, and for code that dates back before 2005, the history is extended back to the original CVS repository. See https://github.com/racket/games/ for an example of this. There's a failure in the import -- if you look at the commits of this repo (https://github.com/racket/games/commits/master), then starting from Robby's commit from Jul 17 (add a contract to make-card...) and going back, the original commit references are all bogus. If you build Racket from source from Git, that build now contains fewer packages. There is not yet an single-step way to get all of the split pkgs as git repositories; we plan to write a script for this soon. Any reason they are not git modules? (They've improved a lot, and they're even supported in github as links as long as they point to other GH repos. See for example: https://github.com/elibarzilay/test) To clone individual repositories, use the new `--clone` option for `raco pkg`, such as: raco pkg install --clone pkg-build git://github.com/racket/pkg-build or for packages that are grouped together in a single repository raco pkg install --clone remote-shell git://github.com/racket/remote-shell?path=remote-shell-lib git://github.com/racket/remote-shell?path=remote-shell git://github.com/racket/remote-shell?path=remote-shell-doc Note that the clones created by `raco pkg install` cannot be pushed to with the default origin remote. This is very obscure. Is there a compact description of what to do when you want to make a change in a file that is not on the main repo? (Hopefully there's obvious agreement on the need for this to be compact.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Splitting the Racket repository
On Sat, Nov 29, 2014 at 8:30 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay e...@barzilay.org wrote: On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: All the history for the code has been preserved, and for code that dates back before 2005, the history is extended back to the original CVS repository. See https://github.com/racket/games/ for an example of this. There's a failure in the import -- if you look at the commits of this repo (https://github.com/racket/games/commits/master), then starting from Robby's commit from Jul 17 (add a contract to make-card...) and going back, the original commit references are all bogus. For packages with old history, the shas are in reference to https://github.com/samth/old-plt-new/ (which should maybe have a better name). Yeah, very bad name but an even worse location... Since this is a major reorganization it would also be a reasonable to do a forced push of that thing to the main repository so the references make sense. (This is assuming that you cannot refer to the proper commits in the current repo for some reason.) If all else fails, you should at least include in the commits some pointer to a repo that has it. (Sounds picky, but having no reliable history is one thing that can make a company avoid using it.) Any reason they are not git modules? (They've improved a lot, and they're even supported in github as links as long as they point to other GH repos. See for example: https://github.com/elibarzilay/test) The goal is to have packages that are in the main distribution not have a particular special status, beyond being in the list of things that gets put in the distribution. I'm not talking about the distribution here -- just some meta repository that has everything in it in a convenient way for the people who have code in there. (And assuming that this meta repository will build with all of these packages that still not be related to a distribution.) (A different issue is organizing distributions: at some point in the past I pointed at the fact that git modules could be used for this, essentially being a collection of pointers to code that is included. But that bus has long gone, so I'm definitely not talking about that.) Also, a situation where you have to update two things when you do a commit is not ideal. I absolutely agree with that. Probably even more than you do, and certainly for a much longer time (that's the essense of my bundle-related rants years ago about spaghetti dependencies). I suppose that you're worrying about modules making it easy to do such cross-repo commits -- but it's no easier than just having a bunch of repos and doing a commit over them simultaneously. So it's as doable, and in both cases will be awkward enough to make it clear to the committer that something is wrong. To clone individual repositories, use the new `--clone` option for `raco pkg`, such as: [...] This is very obscure. Is there a compact description of what to do when you want to make a change in a file that is not on the main repo? You just need to edit and change the relevant repository -- nothing else is required. You don't have to use --clone or any other mechanism. [...] This doesn't answer my question -- so I think that I wasn't clear enough. Say that I have some change to a specific file. I want to know how to do it in the following two ways: 1. Get the equivalent of the whole distribution in the form of a bunch of repositories, including the repository that I want to change. I make the change in the file, re-build, test things out, and eventually push. This is the way that is more relevant for internal people, and I suppose that there would be some script that will do all of that. (The equivalent of a toplevel `make'). 2. The thing that is more relevant for others (and that's the important one): I have a normal installation, I find a bug -- what do I do now? I can't just report the bug, the file, and my patch since that's using a path in the distribution that is mostly unrelated to the source repo for the file. Ideally, there would be some way to turn some containing directory into a repository so I can commit my change, and push it to a fork to do a PR. But that's a little extreme -- so what I'm talking about is more like a way to say: get this repository and use it instead of the stuff in my distribution, so I can work and push from that. I'm guessing that this is the intention behind that command, but it's obscure since it's not obvious what to do. (And obviousness is important to encourage contributions.) As a concrete example: I find a bug in a file like share/pkgs/games/gcalc/gcalc.rkt and want to fork, edit, and push a fix. What do I do? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http
Re: [racket-dev] DrRacket PF1 Search Bug?
On Fri, Nov 21, 2014 at 4:26 PM, Robby Findler ro...@eecs.northwestern.edu wrote: Oh, absolutely. OK, I'll do that later today. The two candidates are the trampoline approach and the just move the documentation files over into the user space as if a package had been installed. Oh, *that*'s the other solution? That sounds pretty bad not only because it complicates file installation which is already very complex, but also because it's a significant weight. (For example, the current size I see is about 180M, multiply that for a department with a thousand users and you get unhappy admins.) I guess the latter is better because it means there will be more homogeneity in the sets of files and whatnot, such that these kinds of bugs will be less likely to be unnoticed by in house folk. I think you meant to say former (ie, the trampoline) -- yes, having a very different setup for end users is also pretty bad. IMO, it falls under the complication point above: installation layout is not only complex in itself, it's also a point that introduces many different options which are almost all never tested, and worse -- they can easily lead to very bad bugs. That is, I never noticed this problem because my documentation is already in the user-specific place because I build from source. The other option will make our users operate more like how my drracket operates. (I'm not following that.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DrRacket PF1 Search Bug?
On Fri, Nov 21, 2014 at 8:46 AM, Robby Findler ro...@eecs.northwestern.edu wrote: On Fri, Nov 21, 2014 at 12:34 AM, Eli Barzilay e...@barzilay.org wrote: Not that it matters, but did you try to see if it's the file permissions? Oh, they are! [...] And that was it!! If I run: $ xattr -d com.apple.quarantine /Applications/r/doc/search/index.html then osascript works again! [...] ... I don't know the ramifications of just calling xattr from inside DrRacket to fix things up. It may be better to just create the file in the user's directory instead. That (the first thing) sounds like a very bad idea, and one that will most likely fail in a case of a non-private machine. The second is also pretty bad since you'd be exluding a central setup in such a situation. (Not that I see any reason to have a lab of Macs...) And this is why I said that it's not really relevant -- AFAICT, the fact that things are setup in a way that prevents passing queries and fragments means that you just have to deal with it rather than try to play games with the filesystem. But of course there's another question of whether the trampoline will work -- the browser might itself detect the different owner and refuse to do the requested hop. Hopefully it doesn't. [And if it does, then I can only see ugly solutions like open a page with a known weird title, wait for it to come up and then send it a command to go to the real target... Or running a tiny web server at some known high port, and using an http://localhost:port/, and that web server will wait for a single request and spit out the forwarding line, dealing with multiple processes somehow (eg, some random number as a private identifying cookie).] -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DrRacket PF1 Search Bug?
On Fri, Nov 21, 2014 at 9:45 AM, Sam Tobin-Hochstadt sa...@cs.indiana.edu wrote: We have labs of macs here at IU. [*sigh* at that kind of spending...] And this is why I said that it's not really relevant -- AFAICT, the fact that things are setup in a way that prevents passing queries and fragments means that you just have to deal with it rather than try to play games with the filesystem. Could we just write a file with the query embedded in it, and open that? That is exactly what that trampoline is: a page with just a immediate redirection to the search file with the query. (Unless you mean write a version of the search page with a specific query hard-wired into it, which is a bad idea for a reasons...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DrRacket PF1 Search Bug?
On Fri, Nov 21, 2014 at 3:16 PM, Robby Findler ro...@eecs.northwestern.edu wrote: Oh, my apologies. I thought you meant something different. Yes, this works. Ah, in this case, the patch that I sent earlier should work fine. (I could do a pull request, but it should really be tested...) I don't have a good idea which is the better approach now. [Bear in mind that I don't really know what the issue with the file permissions is (looks more like some security attributes):] I think that playing with such attributes is a worse option -- it's something that I'm not sure would work in all settings (but see above), and if there is a way to make it work now, it might not be possible in the future. The trampoline approach is relatively robust and well behaved. There is a minor price of these temporary files, but they do get cleaned out (whenever `send-url/contents' is called), so it should settle on a small number that will not grow indefinitely[*], so this is not a problem. Also, on the it's already there side -- this trampoline approach is already there for the windows version, and it's needed for the `xdg-open' thing anyway, so abstracting it into a new (internal) function makes sense already, so in terms of additional code there's no cost for doing the mac version too. ([*] It might be a problem if, for example, your cat sits on the f1 key and the os doesn't take the focus away from drracket... It might be useful to change it from using a random temp file to a name based on a hash of the forwarded url, which will further minimize the number of files.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DrRacket PF1 Search Bug?
On Thu, Nov 20, 2014 at 5:22 PM, Robby Findler ro...@eecs.northwestern.edu wrote: Ah, thanks! I see that if it is trying to load the docs that are in /Applications then it runs the code below, which somehow magically drops the query argument by the time safari gets it. It's not because of the space, either; when I rename Racket v6.1.1.5 to just r, it also doesn't work. But if I change the path to one in my home directory (just like the one you write below), then I see it works. Puzzling. It's almost like there is special handling for /Applications instead of /Users. Something that might be relevant here: on Windows, the query (?...) or fragment (#...) bits of a URL are dropped when you run them. That's why I made it use a trampoline file: it creates a small HTML file which can open fine, and that file redirects to the full URL. At some point, Linux joined this with xdg-open not passing along these things also, and needed a similar hack (which IIRC, wasn't done). So it might be that OSX is doing something similar now. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DrRacket PF1 Search Bug?
On Thu, Nov 20, 2014 at 7:51 PM, Stephen De Gabrielle spdegabrie...@gmail.com wrote: in DrRacket: (send-url file:///Users/spdegabrielle/hello.html?q=aaaba) works (send-url file:///Applications/hello.html?q=xyz) works In that case I think that it's a different problem than what I described. (send-url file:///Applications/Racket%20v6.1.1.5/doc/search/index.html?q=xyz) fails This is weird. To debug it, the text thing I'd do is make that /Applications/hello.html show the location string -- perhaps the browser hides the query part. Another thing to look into is the permissions of the files -- maybe it descides to drop the query bit on a file that you don't personally own. in bash: Miriams-MacBook-Pro-2:~ spdegabrielle$ open file:///Users/spdegabrielle/hello.html?q= fails It shouldn't be `open' -- it's something with osascript. (I don't remember it now, but Robby had it in the other email.) I've just had a quick go at '(send-url/file ' but I cant remember how to escape the '?' Should be something like (send-url/file /Some/file.html #:query 123) but it's probably won't help to try it, since it uses the same functionality as the above. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DrRacket PF1 Search Bug?
On Thu, Nov 20, 2014 at 10:38 PM, Robby Findler ro...@eecs.northwestern.edu wrote: No, the browser isn't hiding the query part. Here are the content of two script files: $ cat a.scrpt open location file:///Applications/r/doc/search/index.html?q=xyz $ cat b.scrpt open location file:///Users/robby/Library/Racket/development/doc/search/index.html?q=xyz Running osascript a.scrpt doesn't work, but running osascript b.scrpt does work. And by work I mean that in both cases the corresponding webpage is visited in Safari, but in the first case, the q parameter disappears. Not that it matters, but did you try to see if it's the file permissions? Another thing: I googled osascript open url drops query and got a bunch of racket results, and dropping osascript make the results more sensible. So perhaps osascript is outdated? Some more searching makes it look like you can just run the open command directly, and that might make a difference. In any case, if it just doesn't work, then doing the trampoline thing should be easy -- it should do the same thing that `send-url/win' does: if the URL has a query or a fragment, then use `send-url/contents' with the same contents as the windows case. And since there's the linux case too, then it's worth an abstraction, something like the simple shuffle in the attached patch. (Completely untested, and wrapping xdg-open in the linux function since it looks like it's still as broken as I remember it to be.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! p Description: Binary data _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #28054: master branch updated
4d44878 Asumu Takikawa as...@racket-lang.org 2014-01-06 17:47 : | Move `chomp` function into its own util file : A pkgs/typed-racket-pkgs/typed-racket-lib/typed-racket/utils/string.rkt M .../typed-racket-lib/typed-racket/utils/tc-utils.rkt | 8 +--- Why not (string-trim s #:left? #f) ? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #28054: master branch updated
A few minutes ago, Sam Tobin-Hochstadt wrote: I think it's really `(string-tring s \n #:left? #f)`. Yes, but IIUC, the function that Asumu wrote is for tweaking output strings, which makes `string-trim' relevant, and my guess is that the extra trimming that it makes (other whitespaces, and any number of them) is probably fine. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Huh?
6 hours ago, Jens Axel Søgaard wrote: In DrRacket 6.0.0.1 open pkgs/draw-pkgs/draw-lib/racket/draw/private/record-dc.rkt and then click Run. I see: define-values: assignment disallowed; cannot re-define a constant constant: get-recorded-command1.1 Why? That's not new -- usually when you run the code you get a clean environment with the old instantiation gone, but some core files are shared across runs, so it's like trying to redefine things that are already defined. Many files in racket/* will do the same. To debug this kind of stuff it can be helpful to work on a copy of the file that you want to edit, and rename it back over the original (and recompile) when it's working. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Release for v6.0 has begun
Yesterday, Matthew Flatt wrote:
At Mon, 25 Nov 2013 09:56:45 -0500, Ryan Culpepper wrote:
On 11/25/2013 09:44 AM, Matthew Flatt wrote:
Here's the full comment:
The version string has one of the forms:
X.Y
X.Y.Z Z != 0
X.Y.Z.W W != 0
where each X, Y, Z, W is a non-negative exact integer, Y must not
exceed 99, and Z or W must not exceed 999. Y=90 means that this is
working towards {X+1}.0, and X.Y (Z=0, W=0) is an alpha version for
{X+1}.0; Z=900 means working towards X.{Y+1}, and X.Y.Z as an
alpha release.
Then intent is that when Z and W are 0, the string form of the version
number is just X.Y, not X.Y.Z.W.
How about this clarification?
... and X.Y (i.e., Z=0 and W=0, so Z and W are
omitted from the string form) ...
That's not the part that needs clarifying. I think that fact that the
string form drops final zeros is clear from lines 2-4.
The part that needs clarifying is how to choose the version number for
the alpha releases leading up to version {X+1}.0. (Really, how to choose
alpha version numbers in general, since I've had similar problems in the
past.) From this statement, X.Y (Z=0, W=0) is an alpha version for
{X+1}.0 (Y=90 already stated), I would expect that 5.91 would be a
fine alpha version number for 6.0. Is it? If not, what should the alpha
version number be?
I agree that 5.91 is the right alpha-version string, assuming that
it's intended as an alpha in the sense of our release rules (as
opposed to a release candidate, which has a non-zero W).
These two things were originally independent: the ability to specify
alpha-ness (the second .91) and release-ness (W=0), and that was used
by the old build script to make some decisions for what the installers
do. Assuming that this still matters, there is a problem with using
5.91 for the release process -- and instead it should be 5.91.0.1
to make it treated as a nightly build. For example, on Windows the
installer for a 5.91.0.1 wouldn't grab the suffix registration, but
5.91 would which makes it bad as something that you ask people to
try.
BTW, this is not the same meaning of alpha that is used in the
release checklist -- that one has the meaning of a release
candidate.
--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!
_
Racket Developers list:
http://lists.racket-lang.org/dev
Re: [racket-dev] Release for v6.0 has begun
Just now, Robby Findler wrote: So, IIUC, Ryan should have used 5.91.0.1 as the version number on the release branch? Yes. (Part of this is probably the result of the poor estimate that the Emacs code does for creating the checklist template -- mostly because I originally intended that to be just a recommendation, not a determination.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #27825: master branch updated
IIUC, this makes the limit thing -- and therefore sandboxes -- behave *very* differently. The original intention was that the time limit is on something similar to what you get with `time'. A very visible way to see the effect of this change: - ,r racket/sandbox - (define e (make-evaluator 'racket)) - (e '(define foo 1)) - (e '(thread (lambda () (sleep 5) (set! foo 2 #thread This used to happen immediately, with the thread continuing to run inside the sandbox. After your change, the last line hangs until the thread is done. Using a bigger sleeping time will make it throw an error when it previously didn't. Similarly, (make-module-evaluator #lang racket (thread (λ() (sleep 99 used to work and will throw an error now, and of course, any code that runs some kind of sandboxed server will probably break now. I think that instead of this, it'd be better to write a helper that runs a thunk and waits for it and for any generated threads to end, and suggest using this helper when you want to wait for all threads in a `with-limits'. (It might also be useful in the profiler, where a similar kind of wait-for-all is done.) On Friday, j...@racket-lang.org wrote: jay has updated `master' from e0026f5de4 to 79f8636e1e. http://git.racket-lang.org/plt/e0026f5de4..79f8636e1e =[ One Commit ]= Directory summary: 52.6% pkgs/racket-pkgs/racket-test/tests/racket/ 45.6% pkgs/sandbox-lib/racket/ ~~ 79f8636 Jay McCarthy j...@racket-lang.org 2013-11-22 14:25 : | Ensure that threads created within call-with-limits are accounted | during the time/space limits : A pkgs/racket-pkgs/racket-test/tests/racket/sandbox.rkt M pkgs/sandbox-lib/racket/sandbox.rkt | 81 ++-- M .../racket-test/tests/racket/sandbox.rktl | 48 M .../scribblings/reference/sandbox.scrbl | 4 + -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] unable to git fetch, or ssh -v to git.racket-lang.org ?
10 hours ago, John Clements wrote: This... doesn't look like something on my end? [...] Did you check if you were blacklisted? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Looking for volunteer(s)
Yesterday, Eli Barzilay wrote: I'm looking for someone to take over mailing list moderation and possibly other similar work like monitoring newsgroups etc. To clarify this a little: * The workload is very small -- Mailman has a web interface and an email one for doing moderation; but since I find these interactions cumbersome, I have a script that I run once every one to three days. And since the script is in racket, you can really hook it up to anything that is convenient. * It's really something that should be done by someone on one of the plt groups, since you'll need to know how to filter academic calls that are relevant or not, how to authorize pending requests to the edu lists, and you'll need to moderate some internal lists. It's also better if it's taken by someone who is actively reading the mailing lists, so you know when some pending post has already been sent since the sender subscribed and re-sent their email. * The other monitors are for keeping track of things in general and be present whenever needed. (This is why the newsgroup monitor is a good example: announcing releases there is still very useful, but doing so without tracking followups is not a good idea.) Still waiting, -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Looking for volunteer(s)
I'm looking for someone to take over mailing list moderation and possibly other similar work like monitoring newsgroups etc. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] DOS attack on planet?
I just looked into that, and it seems that there's something bad going on with some machine at BYU which started yesterday. (Ping: Jay.) The offending traffic comes from fltr5.byu.edu, at a very high rate. The new log file for the week had started at 2013-09-22 03:40 local time (about 12.5 hours ago) with 92000 queries for this period, and 85% of this traffic (about 78k, about a 100 hits per second) is coming from this BYU IP. Looking back, it seems that it's something recent that had started just yesterday, so whatever it is, it's new. Most of the traffic is basically a repeating loop of these 8 lines, shown below. (I will restart the server now, in an attempt to get whatever it is that causes this mess to crash.) 128.187.97.22 - - [22/Sep/2013:03:49:17 -0400] GET /servlets/pkg-info.ss HTTP/1.1 200 5650 - - 128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22dracula.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22cce%22%29 HTTP/1.1 404 79 - - 128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22dracula.plt%22maj=2min-lo=0min-hi=%23fpath=%28%22cce%22%29 HTTP/1.1 404 79 - - 128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22dracula.plt%22maj=5min-lo=0min-hi=%23fpath=%28%22cce%22%29 HTTP/1.1 404 41 - - 128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22drocaml.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22abromfie%22%29 HTTP/1.1 404 79 - - 128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22fasttest.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22cce%22%29 HTTP/1.1 404 79 - - 128.187.97.22 - - [22/Sep/2013:03:49:19 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22xmlrpc.plt%22maj=3min-lo=0min-hi=%23fpath=%28%22schematics%22%29 HTTP/1.1 404 79 - - 128.187.97.22 - - [22/Sep/2013:03:49:19 -0400] GET /servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22bystroTeX.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22amkhlv%22%29 HTTP/1.1 200 63508 - - 30 minutes ago, Robby Findler wrote: I don't think that the planet server itself doesn't keep enough information to say much about this, but the requests come via apache so there might be more information in a log file at that level that Eli might be able to tell us about. I do see lots of requests coming in for packages, tho. In addition to yours, dracula.plt fasttest.plt, drocaml.plt, and xmlrpc.plt seem to be being continuously asked for. Robby On Sun, Sep 22, 2013 at 12:40 PM, Andrei Mikhailov a.mk...@gmail.com wrote: Sorry if I am rising a false alarm. I noticed that there is a massive download of my package called bystroTeX : http://planet.racket-lang.org/display.ss?package= bystroTeX.pltowner=amkhlv All the downloads are of the (old) version 1.6 What is going on? Is it possible to figure out who downloads it? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Pinging BYU people!! (was: DOS attack on planet?)
Update: bringing it down for a few minutes didn't help, and the offending process continues its merciless traffic. I've added a temporary rule that effectively blacklists planet access from that IP address. (Apologies in case that's a shared machine.) All I see now, are failed attempts to get /servlets/pkg-info.ss (which are answered with a 403 to that IP). Can someone at BYU look into this? 20 minutes ago, Eli Barzilay wrote: I just looked into that, and it seems that there's something bad going on with some machine at BYU which started yesterday. (Ping: Jay.) The offending traffic comes from fltr5.byu.edu, at a very high rate. The new log file for the week had started at 2013-09-22 03:40 local time (about 12.5 hours ago) with 92000 queries for this period, and 85% of this traffic (about 78k, about a 100 hits per second) is coming from this BYU IP. Looking back, it seems that it's something recent that had started just yesterday, so whatever it is, it's new. Most of the traffic is basically a repeating loop of these 8 lines, shown below. (I will restart the server now, in an attempt to get whatever it is that causes this mess to crash.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)
(Note that instead of the apache rule I now switched to a firewall rule, so it won't even get 403 responses now.) 40 minutes ago, Eli Barzilay wrote: Update: bringing it down for a few minutes didn't help, and the offending process continues its merciless traffic. I've added a temporary rule that effectively blacklists planet access from that IP address. (Apologies in case that's a shared machine.) All I see now, are failed attempts to get /servlets/pkg-info.ss (which are answered with a 403 to that IP). Can someone at BYU look into this? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)
50 minutes ago, Jay McCarthy wrote: Next time, feel free to follow the directions on internal.racket-lang.org. I have no practical way to know whether it's actually one of your machines. (I did check that it's not an IP that is in our DNS.) Now that you've turn off its access, rather than just logging in and killing it, Nor do I know what it is that should be killed. (And I will certainly not going to ssh into your account and sniff around.) I can't test and see what the underlying problem was. Let me know when you have turn traffic back on. It's back on now. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)
A few minutes ago, Jay McCarthy wrote: In retrospect, I guess it's not so obvious that the package server contacts the old server regularly to build the compatibility version packages. Is this the package server?? The IP I have for that is 128.187.105.226, which is different from the IP that caused the traffic. This is why I couldn't guess what causes the traffic, and guessed some rogue experiment in indexing on some test machine. In any case, if it is the package server through some other machine, then it's best to change it so it comes from the actual server. It's back on now. Thanks... it looks like I'm still getting 403s though. Ah, sorry -- I forgot to remove the apache rule too. Should be working now. Also, since it's scanning the planet packages (at least looks like that), and those really don't change that often, then it'll be much better to do this scan much more infrequently -- like once every hour or so rather than once every two seconds... -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)
Just now, Jay McCarthy wrote: On Sun, Sep 22, 2013 at 6:53 PM, Eli Barzilay e...@barzilay.org wrote: In any case, if it is the package server through some other machine, then it's best to change it so it comes from the actual server. I don't know what's going on with that. It's in a VM, so maybe something is fishy when traffic leaves it versus when it comes to it? Ooh, that's pretty bad for a server. Having an IP address that doesn't resolve back to the IP name is nothing new these days, but having traffic from the server come via a different IP address is really not a good idea. Think about dealing with some kind of an external service, who would need to be aware of your traffic: having it come from a different IP address is something that would make it very hard. It would be a good idea to ask the people who manage that if it's possible to get the expected behavior. (FWIW, it might be some result of a firewall or something like that too. In NEU, our public machines are all in a DMZ network so they're not affected by such firewalling. (But it does mean dealing with a public machine -- for example, dealing with ssh dictionary attacks, not having some kind of expected weaknesses exposed like PHP and similar junkware, etc.)) It is supposed to do it weekly. I just turned it back on and did not get an error, so I'm not sure what the problem was. (The 403 errors totally filled the log, so I couldn't tell what the problem was earlier in the day.) So, I'm not sure what the problem was. I can tell you exactly when it happend -- the flood started with this entry: 128.187.97.22 - - [21/Sep/2013:22:10:10 -0400] GET /servlets/pkg-info.ss HTTP/1.1 200 5650 - - This was the first entry from that IP address for the whole week, so it was probably the weekly run which then went bad. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [racket] RacketCon full schedule
5 hours ago, Sam Tobin-Hochstadt wrote: I went to change this, but the current source of the web site is not in the meta/web directory. It is. In particular, it doesn't contain the Racket Videos announcement, which is where the RacketCon announcement should go. That part is in iplt, I added and updated the web page. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [racket] RacketCon full schedule
Just now, Sam Tobin-Hochstadt wrote: Thanks for updating! Is that the only part of the web page in iplt? No, see the contents of the web directory there. The reason that the blurbs are is that the rcon sources are there, and the reason for that is that they contain the pdfs etc which shouldn't be part of the code repository. More generally, it seems like we could move some or all of plt-services to a separate repository. Either that, or move it to iplt. It should remain publicly inaccessible since it contains sensitive information. (Sensitive in the security sense, like the web server configuration, etc.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [racket] RacketCon full schedule
Two hours ago, Sam Tobin-Hochstadt wrote: On Sep 18, 2013 10:33 PM, Eli Barzilay e...@barzilay.org wrote: Just now, Sam Tobin-Hochstadt wrote: Thanks for updating! Is that the only part of the web page in iplt? No, see the contents of the web directory there. The reason that the blurbs are is that the rcon sources are there, and the reason for that is that they contain the pdfs etc which shouldn't be part of the code repository. They have to be in some repository -- why I'd one worse than another? I didn't say that they shouldn't be. More generally, it seems like we could move some or all of plt-services to a separate repository. Either that, or move it to iplt. It should remain publicly inaccessible since it contains sensitive information. (Sensitive in the security sense, like the web server configuration, etc.) I don't understand -- plt-services is a directory on the 'meta' collection in the git repository. I was talking about the contents that is now in iplt. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Bug report not confirmed
On Friday, Eli Barzilay wrote: There was a permission problem with it, which is fixed now. Unfortunately, it looks like the contents of those messages are lost. Actually, I just remembered that the bug emails are routed through gmail, so there are copies of everything that was lost. (Expect a mini-flood of 6 messages...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Bug report not confirmed
Just now, Greg Hendershott wrote: On Thu, Sep 12, 2013 at 4:41 PM, Laurent laurent.ors...@gmail.com wrote: FYI, I have filed a bug report more than 3 hours ago but did not receive any confirmation. Is it possible the bug tracker is broken? There was a permission problem with it, which is fixed now. Unfortunately, it looks like the contents of those messages are lost. Plus me, recently: I had a bug where my email got munged from @gmail.com to @email.com: [...] No, this was an unrelated issue... It happened before the server change. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] call-with-limits memory bound isn't actually bounding memory usage
An hour ago, J. Ian Johnson wrote: Okay, stamourv made your response make sense. I added parameterize ([current-namespace (make-base-namespace)]) inside the thunk, [...] If you're going down that road (which makes sense, of course), then it would probably be much easier to just use the full sandbox. There's a long laundry list of things to deal with to get good isolation, and the sandbox is basically a convenience tool for that list. (IIRC, the gui stuff had a bunch of subtle points, like taking care of the eventspace etc.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
and in bold instead. * I'm ambivalent in the same way Neil is about stuffing a whole bunch of pages into a single big page. It's true that clicks cost, but scrolling cost too (both in terms of the needed user action and the crowded look of the page). Maybe a good middle ground is to keep the sub-pages as before, and have only prominent parts (= shorter bits) on the front page. Specifically, there are a lot of things in the community part that seem like they really don't fit a big front page thing (eg, the blog item, the emphasized #racket, the link to freenode.net, the browse the logs). It seems that the whole community part (and possibly others, like the documentation section) could use a list of quick links: RacketCon, Blog, Twitter; Mailing Lists, IRC, People; etc. * The freshest doesn't read like built from git head, IOW, it looks like a real release. * Either the code pointer should go to git.racket-lang.org, or just kill the git server. There is little point in investing efforts on maintaining a git server to save a few random people the tiny cost of a github account or to setup their own server -- especially when there's no sysadmin to manage it. * The textbooks are a nice touch, but I think that some of the other good course-level material (coursera, brown lectures, etc) could be included. Also, they do consume a lot of space, so I think that they're better at the end of the page (after a much-trimmed community part, assuming that the full page is still there) * The middle sentence in the book descriptions seem redundant in all cases. Also, the lack of periods look odd if it doesn't look like an ol list visually. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
30 minutes ago, Matthias Felleisen wrote: All in all, I really like the idea of renovating our home page. What I would like to see is that we get * a distinct home page * one that brings across that Racket is full spectrum technically -- you can gradually migrate from untyped to typed, from modules to components, ... educationally -- you can grow into Racket starting with Bootstrap, which I can't find socially -- you can join a variety of mailing lists, irc channels, twitter streams, etc. and that there are many kinds of Racket communities (edu, research, com, open source, secret cabals) and more This three-face spectrum thing sounds *really* good. Probably would work out well replacing the three grow sections. (And begs for some visual pun too.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
And a major bug (chrome, now on linux): resizing the browser doesn't reposition the text in the middle. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
Just now, Sam Tobin-Hochstadt wrote: On Tue, Aug 20, 2013 at 12:16 PM, Eli Barzilay e...@barzilay.org wrote: * The noise background image is still at the previous place which is now completely bogus. I don't understand what's wrong with the background image. The problem with the headings is known. It was originally intended to be behind the code example -- now it stays there, and in narrow mode it's behind some random text. (But see previous email: I really think that it doesn't fit the new layout and should just be removed.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
An hour ago, Sam Tobin-Hochstadt wrote:
On Tue, Aug 20, 2013 at 5:30 PM, Jose A. Ortega Ruiz j...@gnu.org wrote:
On Tue, Aug 20 2013, Neil Van Dyke wrote:
Yes, I scroll down the page and I see pictures and a simple
picture language, and it seems to be using them to introduce some
pretty simple and familiar concepts, so I assume it's for
children.
FWIW, i get the exact same feeling.
I think this is a sign that we need to revise the tutorial, not that
we need to not link to it. For example, Quick is the only tutorial
that doesn't assume the reader knows Lisp notation.
I can see where the problem is: assume that you're trying to convince
someone foreign to any lisp (or worse, someone who remembers some
parens from an undergrad course 20 years ago) to do the next project
in Racket.
Perhaps a good way to resolve this would be to write a
Semi-Quick: An Introduction to Racket Hacking
text, and have a standout blurb at the top of each of the two Quicks
describing the style of the text and referring to the other Quick if
they prefer a more {hacker-oriented,playful} kind of an intro. Maybe
even go with another one:
Quasi-Quick: An Introduction to Racket for the Secular Hacker
Obviously, the main issue is who will write all of these things -- but
I think that it's much easier than it seems, assuming permissions of
the respective authors:
Semi-Quick: take Prabhakar's intro (cs.uwaterloo.ca/~plragde/tyr/) and
make it terse enough to fit a single (longish) page. I think that
this is a fine choice for an aspiring hacker -- the only part that
is missing, the advanced section, is probably the part that should
at best be a very quick overview for a bunch of things that you
could do, mostly serving as a link hub.
Quasi-Quick: this seems like it fits well the new Learn Racket in Y
Minutes (learnxinyminutes.com/docs/racket/), where the text is more
overview-ish, so is more oriented towards people who already know a
few languages and are mainly interested at a syntactic/what's-new
kind of overview. The main thing that is needed is a tiny bit more
prose to make it more readable (but very little, since this audience
is more interested in the code bits).
A good writer can probably bring both to a good level in a few hours.
--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!
_
Racket Developers list:
http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
An hour ago, Sam Tobin-Hochstadt wrote: On Tue, Aug 20, 2013 at 10:50 AM, Eli Barzilay e...@barzilay.org wrote: I dislike the switch to something that looks less unique -- this feels more like yet-another-site of a language. I can't really put a finger on what makes that, but I think that the big factor is the dark header. (I didn't like it when Matthew initially suggested it, but I think it ended up being a good visual signature.) After trying this out, I agree, and I've switched to a black top bar. I've also added more space, as Laurent suggested, and fixed the cropping in the popup, as Stephen pointed out. I've revised the tag-line formatting, and darkened a few things, and taken some more of your suggestions that I've forgotten now. All of these look good to me. (Still a good amount of work to do, and I hope that it'll get there this time...) A quick idea that popped up today re the books: have a book bar as in the erlang page, but with popups that have the book descriptions. It can be much less intrusive while keeping the good general first-look impression, and it can accommodate many more books. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
A few minutes ago, Sam Tobin-Hochstadt wrote: On Tue, Aug 20, 2013 at 7:32 PM, Eli Barzilay e...@barzilay.org wrote: A quick idea that popped up today re the books: have a book bar as in the erlang page, but with popups that have the book descriptions. It can be much less intrusive while keeping the good general first-look impression, and it can accommodate many more books. What do you mean by pop-ups? Just a div that is shown next to a book with the description text. Roughly like the ones that appear next to screen shots in the old page: http://plt-scheme.org/screenshots/. (Actually that same code could be used, just make it a horizontal widget instead of vertical, add some space between the images, and make a much smaller big--small size difference. It does have a few bugs which I fixed later, as seen on my pages, eg, http://barzilay.org/random/stuff2.html) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Revising Racket's home page
A few minutes ago, Sam Tobin-Hochstadt wrote: On Tue, Aug 20, 2013 at 7:59 PM, Eli Barzilay e...@barzilay.org wrote: A few minutes ago, Sam Tobin-Hochstadt wrote: On Tue, Aug 20, 2013 at 7:32 PM, Eli Barzilay e...@barzilay.org wrote: A quick idea that popped up today re the books: have a book bar as in the erlang page, but with popups that have the book descriptions. It can be much less intrusive while keeping the good general first-look impression, and it can accommodate many more books. What do you mean by pop-ups? Just a div that is shown next to a book with the description text. Roughly like the ones that appear next to screen shots in the old page: http://plt-scheme.org/screenshots/. So, in the style of tooltips, rather than something to click on? You can have any html in there -- see for example the last blurb which has a link in it. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] sandbox and file-/directory-existence tests
A few minutes ago, Sam Tobin-Hochstadt wrote: On Mon, Aug 19, 2013 at 4:34 PM, Matthew Flatt mfl...@cs.utah.edu wrote: Is there a situation where allowing an arbitrary file- or directory-existence test would be bad? This all depends on how paranoid we want to be. There are certainly situations when this will be bad -- it lets you determine who else has an account on a computer, for example. But there are contexts where having GC be observable is a security hole as well, so we have to pick a spot on the continuum. Getting some hacker-useful information from an observable GC time is much harder than doing so from FS existence tests. Two quick examples: * On a unix machine, check if there's a /tmp/shadow file -- if there isn't then you have a machine that is a potential gold mine for hackers. * On a windows machine you can use some network drive or a drive of some random device for a kind of a local DOS attack. (There's probably a lot of similar things that are much more sophisticated; probe attacks in general are very common now.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #27090: master branch updated
About a month ago, Matthias Felleisen wrote: On Jul 7, 2013, at 12:08 AM, Eli Barzilay e...@barzilay.org wrote: It would be nice to have some way of doing it with contracts, but that's obviously impractical -- so I think that a good way to solve it and other such problems is to add a parameter for the name used in read errors, and making `raise-read-error' behave in a similar way to `raise-syntax-error' with the parameter used as the `foo:' part of the error message. Boy I like this. I can't remember how often I have code up something like '#:function' and thought this is a general problem worthy of a solution. My point was that this is a problem with readers -- where you often wrap some reading function (that can fail) by your own reader function, and any reading problems should therefore be attributed to your reader. That's why I think that a parameter would be a nice way to address this. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Require error
Here's a confusing require error that I ran into: - ,r tests/json ; stdin:2:3: cannot open module file ; module path: tests/json ; path: /home/eli/src/plt/pkgs/db-pkgs/db-test/tests/json.rkt ; system error: No such file or directory; errno=2 ; [,bt for context] The path: part has gone from being useless to being confusing... -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Fishy directory rename
Trying to sort out some apparent mess in the props, I ran into this possibly bad rename: pkgs/htdp-pkgs/htdp-lib/2htdp/utest -- pkgs/htdp-pkgs/htdp-test/2htdp/utest Before the move, the utest directory was not considered a test by the old distribution rules (since it wasn't in a tests directory), and after the move it is in a *-test package and therefore probably considered as tests that wouldn't be packaged in the release. I *think* that it's wrong, because I vaguely remember Matthias saying that he wants utest in the distribution, but I'm not sure. [The more disturbing point is that dealing with two of these paths and comparing it with the old paths made this thing ridiculously difficult to notice.] -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] git tag (was: package-system update)
About a month ago, Matthew Flatt wrote: At Thu, 18 Jul 2013 09:11:28 +0200, Togan Muftuoglu wrote: When I check the current HEAD tag of the master this is what I get. Is this what it should be if not can it be fixed so it reflects reality? ~/devel/git-sources/racket $ git describe --tags HEAD v5.0.1-12863-g3c0b799 I think this result reflects that we don't generally tag commits in the master branch. We tag releases, but those are normally commits (that were constructed via branches) that are not reachable from HEAD. It happens that v5.0.1 coincides with a commit reachable form HEAD (for reasons I don't know or remember), and so that've why you get a v5.0.1 release. Maybe it would be good to add a tag (more recent than v5.0.1) that doesn't include a version number, so it's not so misleading? Or is it worth adjusting our release process to introduce a tag that indicates when a release version branched from master? The current system is to have a `stable' branch always pointing at the most recently released version of the repository. Then on releases a `release' branch is made up based on `master'. When the release is done, `stable' is advanced to the point where `release' is, but this is done in an explicit way that makes it point to the previous `stable' as a parent -- so the two lines (`master' and `stable') are kept in parallel. This is why `git describe' looks weird -- as Matthew said, it just finds some tag in the past that happens to be the last one before this was started. This was done due to some problems with 3rd party packages who follow the repository -- I don't remember the details, but David Bremner was the one pointing at the problem so he might be able to explain it. (And if there's no problem, then it is probably better to not do that anymore.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] A story of a package split
40 minutes ago, Sam Tobin-Hochstadt wrote: On Tue, Aug 13, 2013 at 5:10 PM, Tony Garnock-Jones to...@ccs.neu.edu wrote: On 08/13/2013 05:05 PM, Nick Shelley wrote: I was mainly asking about the intermediate form because it seems like it could be useful, but I didn't understand how it would work. One major difference I've just spotted is that git submodules are tied to particular commit IDs, leading to detached-HEAD checkouts, where a looser approach can check out subrepos on real branches. Apparently this limitation has been lifted in the most recent git release. Yes -- there is a `--remote' flag to `submodule update' which tells git to grab whatever is current in the remote. (On a branch that eventually defaults to `master', but that can be configured.) BTW, there is also an `--init' flag which does the init if it wasn't done, so if you use both: git submodule update --init --remote you get all of the submodules in, using their current state. And you can use `git submodule status' to see if some submodule was updated away from the recorded sha1. One more issue is that this will still have all the sub-modules in a detached head state (which is arguably something that --remote should take care of), so instead of all of this, you can just go into every submodule and checkout its master explicitly: git submodule --init git submodule foreach git checkout master (But all of this should be mostly irrelevant if the repositries are really used as repositories -- rather than trying to force it into looking like you still have a single monolithic repository.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #27294: master branch updated
6 hours ago, mfl...@racket-lang.org wrote: 2ba615a Matthew Flatt mfl...@racket-lang.org 2013-08-10 19:12 : | directory-list: always sort results | | Consistently sorting shouldn't cost much relative to the | cost of `directory-list' (except for the path-bytes conversion?), | and it makes directory traversals (including archive packaging) | more deterministic across runs and platforms. | | (Eli suggested this a long time ago.) Yes, and as I said back then, with the new `path?' there's probably no reason to want unsorted results. [BTW, there's also only a minimal cost for existing code that sorts the results (probably lots of these), since `sort' does a check that the input is sorted -- however, there is more significant cost since such code converts the paths to strings/bytes and that's no longer needed...] What I had in mind, due to the cost, was to have an argument that specifies how things are sorted, so you could specify it as `values'. Then I figured that it could also be used to do other kinds of sorts, like put directories first, or even filter the results, which leads to an argument that looks like: (directory-list #:sort foo) = (foo (directory-list)) and that didn't seem right... (As a sidenote, it would be nice to have some more standard globbing support; I've written such things probably 2-3 times -- in the gui racket-implemented file dialog, and in meta things like the bundle script.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Package compatibility
Yesterday, Jay McCarthy wrote: Planet attempts to solve this problem technically by (a) having all collections be prefixed by author/package-name and (b) mandating a centralized server that enforces unique authors and unique package-names per author. Since Racket packages don't have a mandated central server we can't enforce uniqueness like that. (i.e. even if we mandated the prefix by package-name, there's no way to enforce unique package names across the universe without a central server.) Furthermore, bringing package-names in to the code would give us /internal linking/ which Racket packages are designed to avoid for other reasons. Since technical solutions to this problem are lacking, An easy way to make it a non problem would be to eliminate the extra indirection that is in package names: if the package that Carl is asking about is *named* data/red-black-tree then it clearly conflicts with another package with the same name. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] What command should DrDr run by default?
Yesterday, Robby Findler wrote: Raco test runs the file if there is not test sub module. Fwiw. That's what I thought first, with run being the same as what racket does. But it doesn't: it just runs the toplevel module as usual when the file is required (and does so unconditionally, of course), and then runs a `test' submodule if there is one. In the files in question, the tests are running in the `main' module, and raco test was therefore not running them. This seems like a fine behavior for raco test, but for drdr, it *should* run a `main' module if there is one, and if there is no `test' module. This avoids the need to abuse a `test' submodule as some semi-drdr-flag-thing, since files in some tests path shouldn't be required to have a `test' submodule. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Package compatibility
Just now, Jay McCarthy wrote: The problem with that is that there is no way to ensure that there is only one package named data/red-black-tree There's no need to ensure such a thing -- and IIUC, the current system doesn't do that neither modulo a bunch of blessed packages. and there can be two mutually incompatible universes of packages for Carl's rbts and mine, for instance. This is also something that could happen now in the same way. But I don't see anything wrong with having this ability: I can hack my own racket/list to do whatever I want, and therefore I should be able to make a package that has this hacked version of the file. I'd be mostly living in my own world whether this hack is done with a package system or directly. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] What command should DrDr run by default?
It doesn't make much sense to use a test submodule in files that are intended to be (only) tests. Earlier today, Robby Findler wrote: That's the plan. Thanks for fixing them. Robby On Sat, Jul 6, 2013 at 8:47 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote: I recently discovered that DrDr runs `raco test` by default on files, and has since February. However, the DrDr documentation says that it runs `racket -t`. I discovered this because the change caused many of the `net` tests to no longer actually run; see http://drdr.racket-lang.org/27042/pkgs/racket-pkgs/racket-test/tests/ net/url.rkt which appears to be successful, even though running `racket` on that file would have shown the error that I fixed in https://github.com/plt/racket/commit/a4e529a816 . These tests are a particularly bad case because they run all their tests in the `main` submodule, and thus are *not* executed by `raco test`. If the plan is for everything to run with `raco test`, I'll fix these tests. But what is the plan? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #27090: master branch updated
Yesterday, Sam Tobin-Hochstadt wrote: On Fri, Jul 5, 2013 at 6:50 PM, e...@racket-lang.org wrote: f90fe4c Eli Barzilay e...@racket-lang.org 2013-07-05 18:08 : | Get rid of the `#:function' keyword. | | These problems are always dealt with via an internal function instead of | making the name argument part of the visible API. : M pkgs/racket-pkgs/racket-doc/json/json.scrbl | 10 ++-- M racket/lib/collects/json/main.rkt | 33 +++--- I think we should keep the name argument part of the API. Imagine that I had a another function that used `read-json` or `write-json` internally, and I wanted the same good error messages that `jsexpr-bytes` has. Then I'd have to either perform string replacement on the exception, or create do something else equally ugly. I disagree with keeping this (but not with solving it) for two different reasons: 1. This is not a problem that is unique to this case, so a local solution is best avoided if there's a general one. 2. Generalizing this in some way, ie, adding `#:function' to functions that should fake their error messages is bad in exactly the way that justifies parameters. It would be nice to have some way of doing it with contracts, but that's obviously impractical -- so I think that a good way to solve it and other such problems is to add a parameter for the name used in read errors, and making `raise-read-error' behave in a similar way to `raise-syntax-error' with the parameter used as the `foo:' part of the error message. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] A grab-bag of issues
Yesterday, Matthew Flatt wrote: At Thu, 4 Jul 2013 12:52:10 -0400, Sam Tobin-Hochstadt wrote: 2. It's possible to get lots of undefined tags in the documentation builds. Should this be remedied with more build dependencies, or will it be fixed with the online documentation server, or something else? Mostly through an online documentation server and declaring certain documentation links to be indirect (which may sometimes involve adjusting the documentation content). If the documentation requires consulting a server, then why distribute them in the first place? There were a whole bunch of issues (most notably a lack of a full text search) that are much easier to deal with if the documentations are all on-line. 5. GitHub recently introduced Releases, which allow you to serve arbitrary downloads. This might be useful for the plt/libs repository. More info here: https://github.com/blog/1547-release-your-software . Even without this, we might want to switch to downloading native packages as zip files instead of requiring git to build from source. I imagined that we would having a catalog server for the native-library packages, but it's good to know about support for Releases from GitHub, and I'll look into that more. AFAICT, there is not much in this, and you get releases by just pushing tags -- so they're already there: https://github.com/plt/libs/releases (Also, serving individual files is fine from our git server.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #27071: master branch updated
On Tuesday, sa...@racket-lang.org wrote:
37c87ec Sam Tobin-Hochstadt sa...@racket-lang.org 2013-07-02 12:24
:
| Move json tests to racket-test pkg.
:
R {racket/lib/collects/json/tests =
pkgs/racket-pkgs/racket-test/tests/json}/json.rkt (100%)
Why did you do this move? Having tests bee in the same directory as
the code makes it much easier to maintain, and I mean same directory
in both the place where the file gets installed and being in the same
package.
(In case it's not obvious where I'm coming from: when I wanted to test
the recent commit, I resorted to grepping the tree for the file -- I
view such grepping as indicators of organizational failures that
should not be needed even after the repo is split, and commits like
this make things worse in this regard.)
--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!
_
Racket Developers list:
http://lists.racket-lang.org/dev
Re: [racket-dev] A grab-bag of issues
A few minutes ago, Matthew Flatt wrote: At Fri, 5 Jul 2013 18:55:32 -0400, Eli Barzilay wrote: Yesterday, Matthew Flatt wrote: At Thu, 4 Jul 2013 12:52:10 -0400, Sam Tobin-Hochstadt wrote: 2. It's possible to get lots of undefined tags in the documentation builds. Should this be remedied with more build dependencies, or will it be fixed with the online documentation server, or something else? Mostly through an online documentation server and declaring certain documentation links to be indirect (which may sometimes involve adjusting the documentation content). If the documentation requires consulting a server, then why distribute them in the first place? Reading documentation does require a server. Following a link from installed documentation to uninstalled documentation requires a server. OK (assuming a does not in the first sentence...). There were a whole bunch of issues (most notably a lack of a full text search) that are much easier to deal with if the documentations are all on-line. While I am interested in exploring installations where all documentation is online, there are also issues that seem harder to handle with exclusively online documentation (most notably, aspects of DrRacket's support for connecting code to documentation). (I think that these should be easy with just having the same files on the server, where drr can grab them and use them. That's except for jumping straight to some specific documentation, which would be done by the server, probably via a url that is formatted according to the source module etc.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #27093: master branch updated
10 minutes ago, Sam Tobin-Hochstadt wrote: On Fri, Jul 5, 2013 at 8:21 PM, e...@racket-lang.org wrote: + ;; The following two tests are not really correct: they rely on the URL + ;; decoding silently passing un-encoded text as is instead of barfing. (Eg, + ;; using these URLs in a browser and then copy-pasting it from the address + ;; should get you a properly encoded string instead.) This parenthetical is not really correct. In particular, you cannot rely on what's displaying in the browser address bar to be a correctly-encoded URL -- the address bar is part of the user interface, and not a reflection of the internal state. It doesn't talk about what is displayed. For some discussion on this point, see this thread: https://groups.google.com/d/topic/mozilla.dev.platform/V0XjJ-uQbcI/discussion In general, the whole URL parser should probably be changed/re-written to follow http://url.spec.whatwg.org/ but that's a longer-term project. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] checking package dependencies
Yesterday, Matthew Flatt wrote: [...] (Sidenote: it would be nice if `--fix-pkg-deps' or something similar could also report redundant dependencies.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] another Planet2 Express memo
A few minutes ago, Matthew Flatt wrote: To avoid problems, delete racket/lib/links.rktd and racket/lib/pkgs. [...] Quick note: a useful thing to know for such cases is git clean -dX -f which will make git remove all files that are ignored. Since the .gitignore files are all setup to list built stuff, it should leave you in a clean state with no pre-built stuff. It will not delete new files that are not ignored, so if there are new source files they will be kept. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] `raco link', links.rktd, pkgs, and config.rktd
Two hours ago, Matthew Flatt wrote: At Wed, 26 Jun 2013 01:04:13 -0400, Eli Barzilay wrote: Furthermore, config.rktd can provide a list of additional files/directories to search. This allows the main links.rktd and pkgs to act like /usr/lib things, while additional directories can act like /lib things. Can you explain this more? I don't see the connection. Sorry, I meant to write /usr/local/lib (instead of /usr/lib) and /usr/lib (instead of /lib). It that more clear? Yeah, that's what I hoped you meant... -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Package management
An hour and a half ago, Sam Tobin-Hochstadt wrote: On Thu, Jun 27, 2013 at 3:17 PM, Matthew Flatt mfl...@cs.utah.edu wrote: The package system has a notion of auto packages, which are packages that were automatically installed to satisfy dependencies. Also, `raco pkg remove' supports an `--auto' flag for automatically removing auto packages that have no non-auto packages depending on them. Note that this is also how the deb package manager (on Debian/Ubuntu/etc) works, so we have good precedent here. I think that all distros do that for a number of years now. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Things we could move out of the core
(BTW, there's a file/private/octree-quantize which should move to where file/gif is.) An hour ago, Robby Findler wrote: The sandbox, IMO, is a nice standalone library the does not need to be in the core. (Ditto for errortrace.) I like the definition of the core as minimum stuff to get pkgs running and we should be picky about what goes in -- even to the point of rewriting some code to move other stuff out. This doesn't make sense if things at such a small size turn to packages. It's true that a package is a unit of utility for end users, but it's also a unit of developement. IMO there are far better things to move out -- things that are bigger (in size and complexity) like (on a brief scan) contracts, gui, match, places, units, json, planet, xml, net, help.[*] Think of someone deploying a game on a phone: would they be happy with your rationale influencing their app's size? I think that errortrace has been one of the biggest offenders in terms of bad bugs of many varieties -- and moving it out won't help. (And if I were doing a game on a phone I'd worry about any of the above much more, btw.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] `raco link', links.rktd, pkgs, and config.rktd
8 hours ago, Matthew Flatt wrote: The config.rktd file in etc can now specify a location for the installation-wide links.rktd file and pkgs installed-package directory (with its pkgs.rktd). On a clean build I get an empty etc directory -- explained by moving the files that were in it -- but is it now a bug, or is it a placeholder for an optional config file? (Would there be anything else there?) Furthermore, config.rktd can provide a list of additional files/directories to search. This allows the main links.rktd and pkgs to act like /usr/lib things, while additional directories can act like /lib things. Can you explain this more? I don't see the connection. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Things we could move out of the core
Two.5 notes:
* IMO the sandbox and errortrace should stay in -- even if they're not
needed, they are both intimately tied to the core so there wouldn't
be any benefit from moving them to a package.
* I think that it makes sense to move compatibility into its own
package, and move the mzlib stuff into that same package.
* Maybe the same for data?
9 hours ago, Sam Tobin-Hochstadt wrote:
While moving some files around between packages, I realized that there
are a number of things that could be moved out of the core and into
packages. Here's a partial list of things that I think are not needed
at all by the rest of the core:
- racket/sandbox and the rest of the sandbox code
- data/{union-find, interval-map, splay-tree, skip-list, order, gvector, heap}
- srfi/4
- unstable/{options, contract}
- errortrace (once sandbox is removed)
- compatibility/*
- mzlib/{pconvert, class100, serialize, thread, transcr}
- a bunch of other parts of mzlib that are just reprovides
The following are libraries that could be removed with (I believe)
small changes to the rest of the core:
- rackunit
- srfi/13
- mzlib/unit200 (maybe, this might be hard)
- mzlib/kw
If this seems like a good idea to people, I'm happy to go ahead and
start doing the work.
--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!
_
Racket Developers list:
http://lists.racket-lang.org/dev
Re: [racket-dev] Mangaging `unstable` with packages
On Friday, Sam Tobin-Hochstadt wrote: # `unstable/list` - `remf`, `list-update` `list-set` `map/values`: move to `racket/list` - `group-by`: rename to `group`, add keyword argument `#:by` defaulting to `equal?`, move to `racket/list` All of these are used in the tree only by their authors, and almost all of them are used between zero and time. `remf' is the same as `filter-not'. `map/values' can be expressed in most cases more conveniently with `for/fold'. `group-by' is doing something weird -- see for example clojure's function by the same name that does something that looks more useful (IMO). Also, `group' sounds way too generic for something as specific as what it does. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Mangaging `unstable` with packages
A few minutes ago, Sam Tobin-Hochstadt wrote: On Mon, Jun 24, 2013 at 3:52 AM, Eli Barzilay e...@barzilay.org wrote: On Friday, Sam Tobin-Hochstadt wrote: # `unstable/list` - `remf`, `list-update` `list-set` `map/values`: move to `racket/list` - `group-by`: rename to `group`, add keyword argument `#:by` defaulting to `equal?`, move to `racket/list` All of these are used in the tree only by their authors, and almost all of them are used between zero and time. `remf' is the same as `filter-not'. This is false, as the example in the `remf` docs shows. s/same/almost the \1/ [We already went over this: I don't find the remove-first functionality too useful (I use (remove* (list x) l) *very* frequently), but with a function a `remf' that is not a `refm*' is even less useful and more surprising.] `map/values' can be expressed in most cases more conveniently with `for/fold'. I've used `map/values`, and I'm not its author. Also, it's a function that I've seen requested for years. (OK, sam added from carl; but you're still the only client in the tree.) `group-by' is doing something weird -- see for example clojure's function by the same name that does something that looks more useful (IMO). Also, `group' sounds way too generic for something as specific as what it does. `group-by`, and my proposed `group`, is what Haskell provides. I don't find the Clojure version nearly as compelling, since it combines a map and a grouping operation. Haskell does something more tame, and more predictable in the sense of splitting an ordered list. And yes, it looks like a good idea to have *that*. To be clear, the list-splitting functionality that I'd like to have: * Split a list to lists of length N (IIRC, clojure has some generalization where you specify an interval length + chunk length, so if you use 1,2 you get something like (x y z) = ((x y) (y z)).) * Split the list by some predicate over the elements, common use: take a list of strings and split it over \n items. Clojure has `partition-by' which is close, and the Haskell group-by can probably be used too (but possibly not conveniently enough to eliminate the need for what I want). (And as a sidenote, I agree with you that the clojure `group-by' is not too useful -- I think that this applies to many list functions that pretend that their input is a set.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] ready for the package switch?
Yesterday, Carl Eastlund wrote: On Tue, Jun 18, 2013 at 2:02 PM, Eli Barzilay e...@barzilay.org wrote: 20 minutes ago, Carl Eastlund wrote: [...] git rebase -s recursive -X rename-threshold=50% mflatt/pkg2 From a brief reading, I think that you're much better off with -X subtree=/some/path. There's also a subtree strategy, which you'd get with -s subtree, but it is guessing how to do the path shifting. I thought that too, but it didn't work when I tried it. Some of my files need to go in racket/lib, some in pkgs/racket-tests, and I think there might have been a third place. So unless everything is uniformly going to the same place, the subtree options don't help. In that case, I think that you should rebase each commit based on where it should go, and possibly split commits that touch different paths. At a higher level, things like this point to possible mistakes in package splitting which should be fixed. In some cases they might mean that a different split should happen, but in the case of the racket tests I think that it's far too broad and should be taken as a temporary package until all tests move out into their proper places. Maybe Matthew decided to make it a separate package because of this. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] ready for the package switch?
Yesterday, Matthew Flatt wrote: I don't think that will work right in the long run. There's currently no easy query to get all of the subcollections of a given collection. Instead of looking for subcollections of games, I think `games/main' should find all info.rkts that have a `game' definition (via `find-relevant-directories'). +8 for this, and also I think that it applies for other path-based conventions like tests and scribblings. (And that leads me back to the specification of these things in info files, of course.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #26989: master branch updated
Four hours ago, mfl...@racket-lang.org wrote: D collects/2htdp/image.rkt D collects/algol60/info.rkt D collects/browser/info.rkt D collects/compiler/embed-unit.rkt D collects/compiler/find-exe.rkt D collects/config/config.rkt D collects/config/config.scrbl [...] Note that there are a *lot* of files here that were not detected as renames, so as I described previously, that's a whole bunch of history that is likely to be lost. (It might also be due to different parameters that are used indirectly by the email script to produce these reports.) Do you still have a list of renames that were done? If you do, then I think that it's best to take the repo off-line, undo the commit, then redo it in two parts -- the renames and the changes, and finally force-push the result. Normally that would be a problem, but now the forced push shouldn't be too problematic. Without a list of renames, I can probably think of ways to automate much of the work, but it will still be a considerable effort to do it all. Also, not doing it now means that it will need to be done later if we care for keeping as much of the history as possible -- and the more changes are committed, the more difficult it will be to fix it later. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] [plt] Push #26989: master branch updated
30 minutes ago, Matthew Flatt wrote: At Wed, 19 Jun 2013 15:54:00 -0400, Eli Barzilay wrote: Four hours ago, mfl...@racket-lang.org wrote: D collects/2htdp/image.rkt D collects/algol60/info.rkt D collects/browser/info.rkt D collects/compiler/embed-unit.rkt D collects/compiler/find-exe.rkt D collects/config/config.rkt D collects/config/config.scrbl [...] Note that there are a *lot* of files here that were not detected as renames, When you use `git log --follow' on the destination file, do you get the expected history? It seems to work right for me the first five files above. The sixth and seventh really are gone. I haven't been able to find a file where `git log --follow' fails to do what I want, so far (not just right now, but in all of my experiments to date). These were just the first few -- I checked a few of the D files that have a corresponding A, and just sent it out with that. I'll try looking more into it now -- probably some script that finds the recent rename of all files and make sure that it produces an expected result. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] ready for the package switch?
20 minutes ago, Carl Eastlund wrote: [...] git rebase -s recursive -X rename-threshold=50% mflatt/pkg2 From a brief reading, I think that you're much better off with -X subtree=/some/path. There's also a subtree strategy, which you'd get with -s subtree, but it is guessing how to do the path shifting. rantI don't understand why version control systems don't take directories and renames more seriously, because this stuff is part of the development cycle and should be recorded like any other change./rant Meta: JFYI, not only does git not track renames, it doesn't do that *intentionally*. The exact arguments for/against are irrelevant, but the main point is that the decision was made very deliberately, and therefore you can't really accuse them of not taking it seriously enough even if you disagree with the decision. Concrete: I said this many times, and it's important to remind people of this especially now -- because git relies on detecting renames, it is *MUCH* better to commit just renames, without changes to file contents. Yes -- even if that means that there's a commit with an uncompilable tree, the potential minor hassle to future bisecters is smaller than the potential to bad rename detection damages. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
Yesterday, Matthew Flatt wrote: At Mon, 3 Jun 2013 10:36:51 -0400, Eli Barzilay wrote: (BTW, a possible source of confusion: I'm assuming that distribution must be done via archives and not via repository specs, since there should be some way to put the compiled files in there. I don't think that's the right assumption. We want it to have packages that combine source with built entities, but I don't think we want to require it. Requiring it would break the way GitHub repositories are meant to be used as packages. But they are required for all non-source things anyway, right? And given that the plt packages have binary distributions, then they should be someplace else anyway. If the sources *do* come from the repository, how does it get a specific version that corresponds to whatever was distributed in binary form? If I have v1.2.3 installed, and it came with web server as precompiled zo files, and I install the source -- how do I get that specific version of the source? The only thing I can think of is relying on tags but that's a bad assumption since it assumes that package owners keep and maintain the proper tags, and the release should come with adding tags in all of these repos -- and that contradicts letting people do whatever they want to do. This is going further off-current-topic, but when I talked to Ryan about possibilities for revising the release process, I thought that the thing that makes most sense is that at the branch creation time (which still happens as usual in the core package repo), package owners hand over a sha1 that corresponds to their recent version. That gets used by the build (possibly via a meta-repo that has these sha1s as submodule specifications) script which gets these versions to build the packages. The equivalent of me making a merge this commit request is now turning into asking Ryan to change my sha1 to a newer one, and Ryan can now use a diff between the two sha1s to review the changes that I made. The main point here is that it makes the package repositories be completely independent: package maintainers can keep a `release' branch and release tags if they want, or they can avoid the whole thing. And in fact I think that the much more common case would be to not having a release branch etc. But going back to the on-topic point: this whole plan assumes that the sha1 is what gets used to pull the content and build zipped package archives. You've mentioned at some point a binary catalog which seems unnecessary to me, but maybe there's something I don't get here.) Yes, binary packages are another way to address the underlying issue. I'm going to try to synthesize the progress and discussion so far in a new message, and hopefully I'll managed to explain binary packages this time around. It would be good to take the above into account, as another reason for doing this which is independent of actual binary files. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] PLaneT(2): Single vs multi-collection packages
Yesterday, Jay McCarthy wrote: and you should deal with the non-proof of concept method of specifying it in, for instance, the info file, which is now package info AND collect info. This shouldn't be a problem -- *most* of the information in the info files that we have now is really information that applies to packages. The only thing that I could find that doesn't is compile-omit-paths. In fact, as I said in the past, the creation of a new info file is the thing that is problematic. What if, for example, there is a `name' entry in both a package info file and in (one of) its collections? The first one would supposedly be used when talking about the package and the second one would be used in setup to indicate the compiled collection... Or what happens with the `version' entry in some of the existing collections? My guess is that the answer to both is that there shouldn't be entries that exist in both info files (eg, the current `version' fields in the collection info files should go away) -- and that shows that there shouldn't be ambiguity problems. Yesterday, Matthew Flatt wrote: Meanwhile, as we've started leaning on the package system and looked at how to fit our current collections into it, I think we're running into the same problem at another level: if web-server-lib and web-server-doc are both packages in a single web-server repository, and if they have to be separate directories, then we have yet another layer of directories that affect repository layout. Yes, that's related, and part of my motivation to avoid that level of splitting. I think that telling people that they should start their hacking work with git clone pltgit:pkg/web-server edit web-server/web-server-lib/web-server/web-server.rkt is making it obvious that there's some problem here... Maybe there's a more general answer to the directory-layout problem that would neatly cover single-collection packages. I originally thought (/hoped) that you would be able to place packages at any level. Going back to the info files -- this is what happens now, roughly -- you just specify an info file anywhere in the tree and it gets picked up as something very close to packages. (For example, the game info files, or drr tools in nested directories.) Three hours ago, Laurent wrote: Actually I use collection links instead of package links for my single-collection packages, which are Github clones. It kind of makes sense. (One more homogeneous possibility would be to add a `raco pkg install --link-single my-collect/package` command that is just a synonym for `raco link my-collect/package`. Yes, that's what I thought should happen -- that installed packages get linked as a single collection when they have a single collection. Actually I would much prefer to have `raco pkg install --link` work like `raco link` and have an additional `raco pkg install --link-multi` command instead.) +173, with --root being used for both. - When all you need is a file, just use a file. It should have all the necessary information (i.e., why not have a 'info' submodule? E.g. to deal with dependencies) (This would be nice, but won't work now, since there's no way to require toplevel files. But that's fine since it corresponds to the fact that you can't have a repository without a directory to hold it. So I view an `in-url' thing is the way for this super-cheap code distribution.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
On Thursday, Matthew Flatt wrote: You've sketched out the producer side, and I'm not sure of some about some of those details. The consumer side seems even more complex to me. It seems like the package system would have to keep track of which subpackages are installed for a package, provide an interface to the user (e.g., in the GUI) for subpackages, and be able to update a package with new subpackages --- all while tracking dependencies at the level of subpackages. I'm sure it can all be done, but I'm not sure how difficult or important it will be relative to everything else that still needs to be done, and I'm pretty sure it can't all be done right now. A very cheap way to do this is to use these sub-package specifications only for creating packages for distribution. With the obvious resulting package file names, this means that there is no change at all that is needed on the consumer side. And I think that while it looks like it reduces the whole thing considerably, it's still worth it for some important advantages that are still kept: * No need to move files around in a distribution-oriented layout in code repositories (which is important since people really maintain the freedom of puting files in a way that fits their needs logically). * No restriction on a few known -foo names -- keeping it open for new kinds of package kinds. (Related note: I spoke to Ryan about having db in the minimal core and the possible reason for this being due to sqlite -- it would be nice, and according to Ryan not too difficult, to split the db package into several db-foo backends, which hooks back into my point of some splittings that make sense in just one package.) * Minimize code hacks that compensate for the lack of such sub-packages. There will still be some hacks -- for example, it will be convenient in the beginning to just show available package names and the end user will need to figure out what they mean, and when later on you want to grouping back packages that really came from a single source for the UI, instead of some hack that is based on names, it can use this information instead. All of these are important IMO, but the last one is particularly good, since hacking something based on naming conventions means that these hacks will stay around for a very long time -- which will lead to yet more hacks. In the spirit of making things usable, I also worry about accumulation of command-line flags, a problem that plagued .plt file generation and installation (making it into black magic that not many people could handle). Instead of trying to create subpackages, we might stick with a naming convention. Even with a naming convention, it might make sense to extend the package-manager GUI so that it recognizes the convention and groups related packages together in a list view (but all the machinery remains at the package level). (This is an example for the kind of hacks where some of them can be avoided with the above, and with others I think that it will be easier to recover proper code after a transition period with such hacks.) Also, a package implementor is free to develop subpackages along the lines you suggest, but splitting out the subpackages into packages before handling them to a catalog. (And having scripts to do such splitting by developers would not be needed at all.) Or, we could just not try to slice packages as finely for now, so that the -lib and -docs packages of the experimental split are merged together --- leaving a future split possibly to subpackages. That will most likely result in a coarse layering that is close to build-deps up to the drracket layer, but still provide more separation for things that are outside the layer needed by DrRacket itself; for example, games, redex, htdp, plai, picturing-programs, and plot can still be separate packages, I think. FWIW, I'd view this (avoid fine slicing) as a possible way to avoid growing hacks, but I really like the idea of smaller packages and the benefits it comes with. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] PLaneT(2): Single vs multi-collection packages
Yesterday, Laurent wrote: On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote: To clarify, because of reasons that I won't go into on the list, the actual chances of me getting this implemented (and of such a change being accepted) are pretty much in the area of slim to none. That's a bummer. At first sight I'd have thought that it would have just been a matter of creating a directory with the same name as the package, and then decompressing the package into that directory. At the lower level, packages use links, and raco link's default mode of work is at the collection level. There is obviously a need for packages that correspond to collection roots, but I wanted to see the default being the same as the raco-link default (which fits most uses that don't come out of the current main tree). TBH, the chances of implementing an `in-url' are low too, but they're probably higher than going into the package system. That would be nice, but I'd much much prefer to see the single-collection package thing implemented. Well, the reason I think that plain collection packages are important is it will simplify dealing with packages, and lower the investment threshold that you need to publish a package. Along the same line, there should also be a way to put out just a single file (or very few) as a first step where I make some code public but with absolutely minimum effort on the publisher's side. See for example the Processing thread, where Sean said that people who do these kind of thing don't even know what source control is, let alone packages -- so the existence of such a crowd is a perfect reason for a single-file thing being another important point on the code distribution spectrum. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] member like assoc
+1, and maybe also add a racket2 item to get rid of all of the unnecessary functions? (I think that member/assoc/etc are better with `eq?' than using memq/assq/etc.) Two days ago, Matthias Felleisen wrote: +1, do it. On May 31, 2013, at 7:40 PM, Asumu Takikawa wrote: Hi all, Is it feasible to get `member` to have the same optional argument behavior as `assoc`? That is, to have an equality predicate as the third argument. I find myself writing the (imaginary) equivalent of things like: (member id some-list free-identifier=?) and it seems like it would be nicer with an extra `member` argument than with `memf`. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
50 minutes ago, Matthew Flatt wrote: At Mon, 3 Jun 2013 08:27:19 -0400, Eli Barzilay wrote: A very cheap way to do this is to use these sub-package specifications only for creating packages for distribution. With the obvious resulting package file names, this means that there is no change at all that is needed on the consumer side. I don't understand the suggestion. As a concrete example, can you sketch our how a user installs the web-server package without documentation, and how the package manager later knows to upgrade the web-server package to include documentation when requested by the user? On the building side, you'd run some command/script/whatever that goes over the distribution kinds and generates a package for each -- so if you have (define distributions '([lib *.rkt] [doc *.scrbl scribblings])) the result will be web-server-lib.zip, web-server-doc.zip, and web-server.zip where the last one contains dependencies on the other two. (So this last one is the only one that is special since it's completely made up.) Then all of these files will get distributed in the same way they would be with the directory/repo split. (BTW, a possible source of confusion: I'm assuming that distribution must be done via archives and not via repository specs, since there should be some way to put the compiled files in there. You've mentioned at some point a binary catalog which seems unnecessary to me, but maybe there's something I don't get here.) Updates would happen in a similar way, with new zip files. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] PLaneT(2): Single vs multi-collection packages
(I completely agree with you, so I'll take it off-line.) 30 minutes ago, Laurent wrote: On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote: Yesterday, Laurent wrote: On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote: To clarify, because of reasons that I won't go into on the list, the actual chances of me getting this implemented (and of such a change being accepted) are pretty much in the area of slim to none. That's a bummer. At first sight I'd have thought that it would have just been a matter of creating a directory with the same name as the package, and then decompressing the package into that directory. At the lower level, packages use links, and raco link's default mode of work is at the collection level. There is obviously a need for packages that correspond to collection roots, but I wanted to see the default being the same as the raco-link default (which fits most uses that don't come out of the current main tree). Not sure I was clear, or I don't exactly understand what you mean. And maybe I'm missing some important point here. But the more I think about it, the less I see why this double-directory thing should be the default behavior of the new package system (I can understand why it's useful for plt packages though). Even Carl's mischief package doesn't make use of collections like that. Jen's this-and-that does, though, but then every collection of his package must have a name that is different from any existing collection. (And actually I think that's a bad idea, and that he should split his package into several single-collection packages −unless they are tightly related− because I don't want to install all his collections if I want to only try one of them, unless there is a way to install only one collection of an existing package?) For example, my personal racket project tree is arranged somewhat like that: racket ├── project-B │ └── main.rkt ├── project-A │ └── main.rkt ├── not-yet(?)a-project-C ├── my-collects │ ├── slideshow │ ├── scribble │ ├── racket │ └── data └── misc ├── try-this.rkt └── try-that.rkt project-A and project-B are git clones. my-collects contains all my extensions to racket's collections, plus some more generally useful collections. For my personal use, I just do: raco link project-A project-B my-collects and all is fine. I tend to believe it's a quite common tree for racket users. In my-collects, I have names that would collide with racket's collections, so instead of having: (require orseau/racket/slideshow) as in the former package system, I would now have (require some-funny-name/slideshow) after renaming my-collects to some-funny-name to make sure it does not collide with anything else. ... and since I don't want to invent a funny name for each extension of a racket collection I do, I prefer to put them all in a single package and collection. Now if I were to follow the new package system, I'd need to change my tree to: racket ├── misc │ ├── try-that.rkt │ └── try-this.rkt ├── some-funny-name │ └── some-funny-name │ ├── data │ ├── racket │ ├── scribble │ └── slideshow ├── not-yet(?)a-project-D ├── project-A │ └── project-A │ └── main.rkt └── project-B └── project-B └── main.rkt which is a mess. And I would also need to change my git repos to match that structure, which looks awkward. /But/ I don't care if the tree of the packages I *install* look like that. So what I would like is to keep my tree like the first one, but the package system can still install like the second one. My proposal was that for single-collection packages, I could keep my tree like the first one, my git clones like project-A/main.rkt, and tell the package manager (probably in the info.rkt file) that it is a single-collection package so that when it installs it in the installed-package-directory-that-I-will-not-look-into it generates a tree like the second one. Well, the reason I think that plain collection packages are important is it will simplify dealing with packages, and lower the investment threshold that you need to publish a package. Along the same line, there should also be a way to put out just a single file (or very few) as a first step where I make some code public but with absolutely minimum effort on the publisher's side. See for example the Processing thread, where Sean said that people who do these kind of thing don't even know what source control is, let alone packages -- so the existence of such a crowd is a perfect reason for a single-file thing being another important point on the code distribution spectrum. I entirely agree with that, and I think it is of utmost importance for a programming
Re: [racket-dev] [plt] Push #26926: master branch updated
Ah, so this is what made the scribble/reader tests fail -- thanks for the fix. But there's still a possible problem: v5.3.4: - (raise-read-error foo 'src 1 2 3 4) ; src:1:2: foo [,bt for context] Before this commit: - (raise-read-error foo 'src 1 2 3 4) ; #procedure:...ntax/readerr.rkt:33:21foo [,bt for context] And now: - (raise-read-error foo 'src 1 2 3 4) ; src:1:1: foo [,bt for context] where the column number is different from the first run. 20 minutes ago, ro...@racket-lang.org wrote: robby has updated `master' from 54b45607a2 to 1f22800d51. http://git.racket-lang.org/plt/54b45607a2..1f22800d51 =[ One Commit ]= Directory summary: 9.8% collects/syntax/ 90.1% collects/tests/syntax/ ~~ 1f22800 Robby Findler ro...@racket-lang.org 2013-06-03 08:38 : | fix bug introduced in 6b2a4ff5 : M collects/syntax/readerr.rkt | 1 + A collects/tests/syntax/test-readerr.rkt =[ Overall Diff ]=== collects/syntax/readerr.rkt ~~~ --- OLD/collects/syntax/readerr.rkt +++ NEW/collects/syntax/readerr.rkt @@ -30,6 +30,7 @@ (format ~a~a (cond [(not (error-print-source-location)) ] [(srcloc-string (srcloc source-name line col pos span)) + = (lambda (s) (format ~a: s))] [else ]) collects/tests/syntax/test-readerr.rkt ~~ --- /dev/null +++ NEW/collects/tests/syntax/test-readerr.rkt @@ -0,0 +1,9 @@ +#lang racket/base +(require rackunit syntax/readerr) + +(check-exn + (λ (x) + (and (exn:fail:read:eof? x) +(regexp-match #rx^y[01: ]* x (exn-message x + (λ () (raise-read-eof-error +x y 1 1 1 1))) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] PLaneT(2): Single vs multi-collection packages
To clarify, because of reasons that I won't go into on the list, the actual chances of me getting this implemented (and of such a change being accepted) are pretty much in the area of slim to none. TBH, the chances of implementing an `in-url' are low too, but they're probably higher than going into the package system. 40 minutes ago, Laurent wrote: Ah, that's cool. Looking forward to it! And the in-url thing would be useful indeed for gists for example. Laurent On Thu, May 30, 2013 at 8:32 PM, Eli Barzilay e...@barzilay.org wrote: Yes, I really want to try and get to look into doing this. The thing is that multi-collection libraries are going to be a common case for plt packages (which will get pulled out from the main repository), but the single-collection ones are going to be the much more popular case elsewhere. (And I also still dream on doing some `url-in' to require a single file, since that's also an important mode of quick sharing. But that's another story.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] PLaneT(2): Single vs multi-collection packages
Yes, I really want to try and get to look into doing this. The thing is that multi-collection libraries are going to be a common case for plt packages (which will get pulled out from the main repository), but the single-collection ones are going to be the much more popular case elsewhere. (And I also still dream on doing some `url-in' to require a single file, since that's also an important mode of quick sharing. But that's another story.) 10 minutes ago, Jay McCarthy wrote: The Racket package system doesn't support packages that aren't collection roots. Eli has said that he wants to implement such a feature, but it is not available today. Jay On Thu, May 30, 2013 at 8:29 AM, Laurent laurent.ors...@gmail.com wrote: I'm willing to upgrade my packages for the new PLaneT system, but one thing that pushes me back is the fact that I need to create a directory for my package and a subdirectory for the collection(s). But most of my packages are (and will probably be) single-collection packages, and it hurts my logic to have to modify my directory tree for that purpose. I know there have been several discussions already about this, but perhaps there is a simple fix: Maybe there could be a `single-collection?' entry in the info.rkt file which, when true, would tell `raco pkg install' to create itself a directory for the package that has the same name as the collection it contains? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
On Friday, Matthew Flatt wrote: At Fri, 24 May 2013 12:44:35 -0400, Eli Barzilay wrote: * The script should also take care to deal with files that got removed in the past. Ditto. I don't believe that it's *not* doing this, so I did the double-check in the form of a test. You're right --- I misunderstood your example. (BTW, in case it wasn't obvious -- that was a typo, since the script is not doing it...) Still, I'm happy enough with the result in your example. The conversion does preserve `git log --follow' results for the files that survive, which was my intended spec. And maybe it's better to explain my interest as `git blame', since my main interest in the history of a file is often how/why a particular bit of code ended up as it is. Ah, yes -- in that case, I think that it's doing that (= maintaining the blame information) fine, but there are still things that you'll want to keep. (At least for some value of you...) (I'll get back to this later, since it's the main content.) * filter-branch one time using your script to reorganize the files according to packages * use filter-branch with a subdirectory filter 5 times to create each repository Total runtime: about 21 hours This latter use would end up with the final tree being exactly the same (since you're talking about doing the reorganization within git), but the history would be different since it's as if the files were there the whole time. I don't see how that works. Since my script leaves each file in its original location for old commits, I expect a subdirectory `filter-branch' to still drop history for the old locations. In any case, I'm happy to sort out that detail later. Ah yes, keeping the files in-place instead of shuffling them around is definitely much better. And yes, it means that it *will* take that large chunk of time for each extracted repository, but I think that it's definitely worth the effort. (Once there is a good way to do the whole trimming thing, I can easily script that onto a bunch of lab machines to do it all in parallel.) If we agree that `git mv' before splitting is practical, though, that's all I need for now. Yes -- with all of the above, and with the additional improvements that I'll suggest below. Actually, I'll just send that in a new email since it's long enough. From my perspective, the important thing is to have the ability to just edit and move files around to sort out packages, instead of having the indirection of a script that edits and moves files around. OK -- but I still think that it's worth it to save a second major change for people, and given that you've started with a suggestion for package splitting, maybe just go on with revising that for a short time and then just do the splitting without an intermediate period? For people who want to keep dealing with the whole tree, the layout is going to be the same so there won't be much difference anyway, and people who want to deal with just their corner will get more time to adjust and enjoy the benefits of dealing with just their corner quicker. BTW, it will potentially lead to more problems where my change to my own repo goes fine and I don't know that it got broke because of a change elsewhere since I didn't keep the other files in git form -- but this makes me think that the next release might be prone to such issues, so it's better to start earlier with the segregation rather than doing it later. (But OTOH, the builds and drdr will keep a high level of problem prevnetion, I hope.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
= In this case, the drdr2 file is just #lang racket, and git considers it an original copy of the second since it just adds a (exit 43) line (first example) or 5 (second). This happens elsewhere too, a lot with readers and info files: 145efa6 Jay McCarthy (1 year, 2 months ago) Adding #lang web-server/base C061 collects/frtime/lang/reader.rkt collects/web-server/base/lang/reader.rkt 32d2a9c Matthew Flatt (3 years, 1 month ago) fix scheme/load and racket/load C087 collects/slideshow/lang/reader.rkt collects/racket/load/lang/reader.rkt c7e723e Matthew Flatt (3 years, 1 month ago) somewhat rackety core docs C060 collects/frtime/opt/lang/reader.ss collects/racket/signature/lang/reader.ss C066 collects/frtime/reactive/lang/reader.ss collects/racket/unit/lang/reader.ss 09bed0d Kevin Tew (1 year, 3 months ago) Initial Distributed Places commit C100 collects/combinator-parser/info.rkt collects/racket/place/distributed/info.rkt e788903 Eli Barzilay (1 year, 9 months ago) Remove a bunch of no-longer-needed `compile-omit-paths', and move the few ones into the subcollections. C100 collects/embedded-gui/private/tests/info.rkt collects/tests/gracket/info.rkt C060 collects/embedded-gui/private/tests/info.rkt collects/tests/plai/info.rkt C056 collects/embedded-gui/private/tests/info.rkt collects/tests/planet/info.rkt ... There are also counter examples: 2d12431 Matthew Flatt (5 months ago) move and fixup docs for the help collection R052 collects/scribblings/tools/documentation-utils.scrbl collects/help/help.scrbl fd7d8a4 Ryan Culpepper (6 months ago) move lazy-require to racket/lazy-require R064 collects/unstable/lazy-require.rkt collects/racket/lazy-require.rkt b53e458 Matthew Flatt (9 months ago) add `racket/format' C068 collects/unstable/cat.rkt collects/racket/format.rkt = here the similarity is very low (the cutoff is 50%), but it is actually a correct identification of a move that should be kept. (These would also not be a problem had people done the move-in-a-separate-commit thing.) This can probably be improved a lot with a script that will count the lines of change for low percentages, and weigh in the line length of the file. (The log command line allows specifying threshold only in fractions, so it has to be a separate script). -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
-more has things that should all go into their respective packages, or into -typed variants of these packages. * The unstable-list-lib package has a pile of non-list stuff... * ...but I really think that the unstable stuff should go away: things that are used in a particular place should move back into being private utilities, and most other things just left behind. There are several things there that are the kind of extensions that could be made into their own packages anyway. * And I think that the above holds for the unstable-parameter-group which is used in two packages, but both are basically Neil's thing... * The unstable package has the docs for a bunch of unstable things left in it -- my guess is that it's too much of a messto deal with, which would support its dissemination... * It also has some bigger packages that really deserve their own thing like the latent-contracts thing: automata and temp-c. But for all of these, there's no real reason to have them included as unstable packages: for these things, I think that the only reason they were in unstable is as a kind of non-core packages, so now they can be just that. * The wxme-lib package has some things that seem like they might be packaged with other libraries (like some snips?), like the xml thing. * A note about some of the text in the toplevel readme file: I really don't like keeping a centralized libs repository -- it's true that it's harded to build binary code, and distribution can be trickier, but as long as this thing is there, it means that the plt packages are still more special. (Maybe there's a plan to eventually spread out these things too?) * There are some administrative files that should be copied across multiple repositories and/or revised for each. Specifically: .gitignore files, .gitattributes, and .mailmap. The first two should be copied for each new repository, the last one can be trimmed for the actual contributors of each package, based on its list of committers. * In the core racket package: - There is a bunch of doc/release-notes/* directories that should move out. - I wonder whether these things are really needed in the core package (otherwise I think that they should move out to separate packages or into existing ones): - acks - db (needed only when there are docs, or to generate them, right? would be also nice to make the sqlite part accessible as its own package) - errortrace - ffi/examples (move to an -examples package) and ffi/com* (into mzcom?); maybe also the objc and winapi things (move to gui or its own thing)? - json, xml (are they needed?? If so, is it possible to hack some limited versions to avoid dependencies on a fuller package?) - unstable (as above -- for these things, probably move them as utilities where they're used or make them into non-unstable) - Many net/* things (imap, sendmail, nntp, pop3, ...); IMO net/url too (and replaced by something much simpler that can deal with simple GETs, as in the get-libs thing) - openssl/test.pem (keeps tripping people -- maybe include the text verbatim in the docs and have people copy+paste it?) - rackunit - srfi/* (should just grab the missing functionality, eventually so maybe not now) - tests/* (looks like it's not done yet) - There's a weird .gitignore in racket/collects/pkg which looks like some erroneous leftover. Meanwhile, part of this experiment is defining -lib packages that do not provide documentation, which means they can have fewer build-time dependencies than they would need for documentation. The -docs packages provide the corresponding documentation. (I'm not sure about the naming convention; I just had to pick something for now.) Naturally, the X package pulls in both the X-lib and X-doc packages. To clarify, after staring at this for some hours, I really prefer it if there's a way to keep it all together and do everything within the package system. (At least havine 3 repos for N packages sound like a very unfun thing to deal with.) But I do want the ability to have these partial-contents packages built for distribution. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
Yesterday, Sam Tobin-Hochstadt wrote: One question -- a bunch of unstable seems to be in the typed-racket-lib package. Why is that? Since these things are used in TR only, I think that they should move back. Ie, you get a trimmed TR repo with unstable/stuff which you move to a better place on your own... (Or do so now and save some splitting work.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
Yesterday, Robby Findler wrote: This looks great to me! I don't have a good sense of what level of granularity is the right one, but I naturally would have gone even finer grained with drracket: the macro-debugger, pkg/gui, and maybe even the gui-debugger I would have separated out. (Probably you were focused on lower-level things first, tho.) Also for the macro-debugger: I think it is useful by itself, without drracket (I would like to use it for files that in drracket's implementation, for example). So I think there should be two packages there: macro-debugger-standalone with the give me a file and show the macro debugger's result on it interface and macro-debugger-plugin pkg that fits into drracket. Besides the usable-by-itself point, I think that there's a strong point for code owners... In this case the owners are very different (and my guess is that any contributions to the code you do/might make are a kind of a suggestion), so it's better to split them; and also there is no sharing of code + maintenance (which is why this point doesn't apply to core stuff). -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
An hour ago, Neil Toronto wrote: I expect `plot' to depend on `math' in the near-ish future. Right now, `math' already depends on `plot' to build its docs. IOW, I expect the sources to be interdependent, but as binaries, `plot' will depend on `math' but `math' won't depend on `plot'. Can the package system handle that, or will I have to merge the two or factor out a common parent? I'm guessing that you won't -- I don't know enough details, but one thing that I can guarantee is that this kind of situation is nothing compared to some of the existing dependency spaghetti... -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
20 minutes ago, Matthew Flatt wrote: Yes. Package dependencies can be cyclic (unlike module dependencies), and packages can have cyclic build dependencies without cyclic run dependencies. (*sigh*) So the idea of acyclic package graph as module containers is dead? But to explain my sigh -- I'm not surprised given my other comment (sent prematurely); I just hope to see it still as a desirable goal. Maybe some quick package-level graph to show cycles and try to reduce them at easy points? (I think that still having lots of cycles will make things more difficult with packages, so that will be one form of discouragement, but there will also be problems that are harder to deal with.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] experiment reorganizing the repo into packages
getting some super flexible thing (and fall into the dist-spec pit). Another problem is specifying dependencies, which can be in a natural way: (define deps '([lib (readline lib) (scribble lib) (macro-debugger text lib)] [he hebrew-dictionary] ...)) There's more missing technicalities, but I think (= hope) that something along these lines can work. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
Yesterday, Robby Findler wrote: Hi Eli: I'm trying to understand your point. Do I have this right? Background: The git history consists of a series checkpoints in time of the entire repository, not a collection of individual files. Yes, although the difference between entire repository and individual files is mostly theoretical. The main point is that the log history is made from changes to *content* -- you can't have some made up history planter artificially for a file. (And it is the same for most CMSs if not all; the main difference here is that git doesn't keep meta information about copying and renaming.) So, when I do git log x.rkt then what I get is essentially a filtered list (except where people didn't properly rebase, but lets ignore that) of those checkpoints: all the ones where x.rkt changed. Exactly. (I don't get the rebase comment though -- even without rebasing what you get is this filtered history.) Big Question: The issue is, then, when we split up the current repo into smaller repos, what are the series of checkpoints that we're going to make up for the individual repos? Right? Yes, but it can get a bit subtle. Like I said, the `filter-branch' tool is basically replaying the entire history, giving you points to inject hooks that can modify the tree or the commits, etc. Note that in all uses that were mentioned, there was a --prune-empty flag, which means that commits that didn't have any change are dropped. I'm mentioning this because some people might have an illusion that it's better to *not* do that and keep these commits. Here's an example why this is not useful: say that you have this edit sequence: foo@somewhere creates A/x, with log message created x bar@somewhere edits it, with log message edited x baz@somewhere moved it into B/x, with log renamed A to B If at this point you use any git tools, they can see the real history. For exmaple, you can use `blame' to see which lines were written by which user. Also, assuming that these are all the changes, a git log will show the three commits as they appear above. Now, if you you use filter-branch to modify the repository and keep only the B directory, but you *don't* use the --prune-empty flag: the fact that you want to keep these other commits won't help -- the full history would have the three commits with the same three messages, but doing a log for just the file would show only the commits for the file, so the first two commits won't be shown. Similarly, blame can't show anything useful -- you'll only see baz@somewhere as the author of the entire file. And the reason this makes sense is that the full commit history has the first two commits, but they had no change -- so there's nothing that ties them to the file in the trimmed repository, let alone something that relates them to specific lines in the file... (Two notes: (a) This is just a demonstration -- obviously, this is a trimming that is done in a bad way since it dropped A even though it's part of the history of B. (B) Actually, it looks like the --subdirectory-filter drops empty commits anyway, but the above explains why it makes sense to do that.) Your Advice: And, IIUC, you're suggesting that the best way to deal with this question is to defer it until we are more sure of the actual split we want to make. So we don't mess with the history at all The point is that every such messing-with-history should be done very carefuly and checked thoroughly, since the chance to mess things up is very real. In the above, it's obvious that I should have not droped A in the filter -- but if it's some random single file which you had in the framework collection, out of tons of other files in the drracket package, then it's unlikely that I will catch it -- which is why I prefer using tools for these things and resolve all such issues with the people who know about the code. and instead just work at the level of some script that we can run to just use mv and company to move things around. When we know exactly what ends up going where, then we can figure out how to make up a new, useful history for the separate repositories. Is that the point? The thing is that having two such filters (one to restructure the big repository and one to split it) is both increasing chances for making mistakes, and making the job of the second restructure much harder to do. To the point where doing it manually is infeasible, which is why I said that it will guarantee losing history. (And I'll reply to Matthew's suggested tool next.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
8 hours ago, Matthew Flatt wrote: At Thu, 23 May 2013 07:09:17 -0400, Eli Barzilay wrote: Relevant history is vague. The history I want corresponds to `git log --follow' on each of the files that end up in a repository. (In this context this is clear; the problem in Carl's post is that it seemed like he was suggesting keeping the whole repository and doing the split by removing material from clones -- which is and even fuller history, but one that has large parts that are irrelevant.) That's true if you use `git filter-branch' in a particular way. I'll suggest an alternative way, which involves filtering the set of files in a commit-specific way. That is, the right set of files to keep for each commit are not the ones in the final place, but the ones whose history we need at each commit. If that can be done reliabely, then of course it makes it possible to do the split reliabley after the first restructure. It does come with a set of issues though... [... scripts description ...] Here are a bunch of things that I thought about as I went over this. In no particular order, probably not exhaustive, and possibly repetitive: * Minor: better to use `find-executable-path' since it's common to find systems (like mine) with an antique git in /usr/bin and a modern one elsewhere. (In my case, both scripts failed since /usr/bin has an antique version.) * There is an important point of fragility here: you're relying on git to be able to find all of the relevant file movements (renames and copies), which might not always be correct. On one hand, you don't want to miss these operations, and on the other you don't want to have a low-enough threshold to identify bogus copies and renames. * Because of this, I think that it's really best to inspect the results manually. The danger of bogus copies, for example, is real, especially with small and very boilerplate-ish files like info.rkt files. If there's a mistaken identification of such a copy you can end up with a bogus directory kept in the trimmed repo. In addition, consider this information that the script detects via git for a specific commit: A/f1.ss renamed to B/f1.rkt A/f2.ss renamed to B/f2.rkt ... A/f47.ss renamed to B/f47.rkt A/f48.ss renamed to B/f48.rkt A/f49.ss deleted A/f50.ss deleted B/f49.rkt created B/f49.rkt created For a human reviewer, it's pretty clear that this is just a misidentification of two more moves (likely to happen with the kind of restructures that we did in the past, where a single commit both moves a file, and changes its contents). This is why on one hand I *really* like to use such scripts (to make sure that I don't miss such things), but OTOH I want to review the analysis results to see potential problems and either fix them manually or figure out a way to improve the analysis and run it again. * Also, I'd worry about file movements on top of paths that existed under a different final path at some point, and exactly situations like you described, where a file was left behind, but that file is completely new and should be considered separate (as in the case of a file move and a stub created in its place). * The script should also take care to deal with files that got removed in the past. For example, the drscheme collection has some file which gets removed, and later (completely unrelated) most of the contents migrated to drracket. If the result of the analysis is that most of the material moved this way, and because of that you decide to keep the old drscheme collection -- you'd also want to keep that file that disappeared before the move, since it's part of the relevant history. So I'd modify this script to run on the *complete* repository -- the whole tree and all commits -- and generate information about movements. Possibly do what your script is does for the whole tree, then add a second step that runs and looks for such files that are unaccounted for in the results, and decide what to do with them. I think that this also means that it makes sense to create a global database of all file movements in a single scan, instead of running it for each package. * Technical: I thought that it might make sense to use a racket server (with netcat for the actual command), or have it compile a /bin/sh script to do the actual work instead of using `racket/kernel' for speed. However, when I tried it on the plt tree, it started with spitting out new commits rapidly, but eventually slowed down to more than a second between commits, so probably even the kernel trick is not helping much... * Actually, given the huge amount of time it's running (see next bullet), it's probably best to make it do the movements from all paths at the same time. In this specific context, this means that it scans the package-restructured repo (from the first step) into a package-restructured repo
Re: [racket-dev] proposal for moving to packages: binary vs source
[Note subject change...] Two days ago, Eric Dobson wrote: For binary vs source, I think you are providing a good argument for the usefulness of a no source distribution. Some people want to use tools written in Racket, and the fact that the tools are written in Racket is immaterial to them. They should be able to have just the binary versions. There have been a bunch of concerns expressed about the question of distributing sources or not -- but I think that generally speaking, there shouldn't be any problems at all. Here's a list of things that contribute to not having such concerns: 1. The eventual goal would be to have very easy selection of packages that you want to install. Either with (a) a bunch of installers, (b) possibly doing this by just a different URL that will have the installers listed in it as arguments, or (c) with a post-install dialog that will ask you for additional packages to install. (In the (c) case, it could also detect packages that you had decided to install previously, and re-use the same list.) The bottom line is that if *you* want to get the sources, then it should be extremely easy to just have them installed (c), or create installers that include the sources (a;b) which you'll use. The main point here is that using packages will make such variations very easy to implement, and make it easy for you to add sources or provide popular options based on demand. 2. With the geiser/drracket concern about reduced functionality because there are no sources: the information about the source of bindings is still there. (Ie, things work fine if you remove a random source file from a current installation -- the only difference is that the actual source file is not there.) Now, I'm assuming that there is some way with the package system to know for any given file which package it came from. With this information, I think that it would be easy to do something like this: * In drr, if you try to jump to a definition for a function whose source is not included, you get a popup telling you that you don't have the source, and list an on-line URL where the source can be found (which is inferrable from the package information) as well as a one-button-click option to install the source and then open the file. * Geiser could do exactly the same, and also use something like `url-handler-mode' to visit the source file directly from the on-line source in addition to offering to install the sources. 3. I think that there should be an option for package owners to decide how their package gets installed, so for example, if realm must be distributed with its sources, it can just specify that and avoid the stripping that other packages would go through. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
Four hours ago, Matthew Flatt wrote: At Fri, 24 May 2013 03:26:45 -0400, Eli Barzilay wrote: If that can be done reliabely, then of course it makes it possible to do the split reliabley after the first restructure. Great! Let's do that, because I remain convinced that it's going to be a lot easier. I'm really surprised. Given that you consider this a *lot* easier, and that I consider it (reorganization + split) a lot messier, I think that I'm still not getting something. * Also, I'd worry about file movements on top of paths that existed under a different final path at some point I believe the file-lifetime computation in slice.rkt takes care of that. That's what it looks like, but I'd double-check to make sure that it happens. * The script should also take care to deal with files that got removed in the past. Ditto. I don't believe that it's *not* doing this, so I did the double-check in the form of a test. When I run it, I see these bad things (which I expected to happen, so wrote it as a test): * The c file got completely lost (this is the pre-reorganization file deletion scenario) * The b file got lost too (post-reorg deletion) * The history of e during the A days got lost, since it was not recognized as a rename in the A-B move due to being edited too. = The first two are things that a script can deal with doing some kind of scan like I mentioned (go over the full history of the full tree). = The third one is something that requires human judgment *but* if the A/e historic file is considered as deleted, and if deleted files from the original directories are included with doing the above, then it should still be there in the rewritten repo. Test file attached; probably need to do very little other than adjusting the paths to the two racket scripts. b Description: Binary data * Actually, given the huge amount of time it's running (see next bullet), it's probably best to make it do the movements from all paths at the same time. There's no need to move anything while extracting a repository slice; the movements happen before. What I'm saying is that if filter-branch using your script takes 20 hours, and you want to use it to split the repo to 5 packages, and if a simple filter-branch with a subdirectory filter takes a few minutes, then instead of: * filter-branch using your script 5 times to create each repository Total runtime: more than 4 days you do this: * filter-branch one time using your script to reorganize the files according to packages * use filter-branch with a subdirectory filter 5 times to create each repository Total runtime: about 21 hours This latter use would end up with the final tree being exactly the same (since you're talking about doing the reorganization within git), but the history would be different since it's as if the files were there the whole time. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
9 hours ago, Matthew Flatt wrote: At Wed, 22 May 2013 14:50:41 -0400, Eli Barzilay wrote: That's true, but the downside of changing the structure and having files and directories move post structure change will completely destroy the relevant edit history of the files, since it will not be carried over to the repos once it's split. It's possible that we're talking past each other due to me not getting this point. (Obligatory re-disclaimer: I consider the problem with forcing people to change their working environment much more severe.) Why is it not possible to carry over history? The history I want corresponds to `git log --follow' on each of the files that end up in a repository. I'm pretty sure that such a history of commits can be generated for any given set of files, even if no ready-made tool exists already (i.e., 'git' is plenty flexible that I can script it myself). Or maybe I'm missing some larger reason? The thing to remember is just how simple git is... There's no magical way to carry over a history artificially -- it's whatever is in the commits. To make this more concrete (and more verbose), in this context the point is that git filter-branch is a simple tool that basically replays the complete history, allowing you to plant various hooks to change the directory structure, commit messages or whatever. The new history is whatever new commits are in the revised repository, with no way to make up a history with anything else. Now, to make my first point about the potential loss of history that is inherent in the process -- say that you want to split out a drracket repo in a naive way: taking just that one directory. Since it's done naively, the resulting repository will not have the drscheme directory and its contents, which means that you lose all history of files that happened there. To try that (in a fresh clone, of course) -- first, look at the history of a random file in it: F=collects/drracket/private/app.rkt git log --format='%n%h %s' --name-only --follow -- $F Now do the revision: S=collects/drracket git filter-branch --prune-empty --subdirectory-filter $S -- --all And look at the same log line again, the history is gone: git log --format='%n%h %s' --name-only --follow -- $F If you look at the *new* file, you do see the history, but the revisions made in drscheme are gone: git log --format='%n%h %s' --name-only --follow -- private/app.rkt In any case, this danger is there no matter what, especially in our case since code has been moving around in the racket switch. I *hope* that most of it will be simple: like carrying along the drscheme directory with drracket, the scheme and mzlib with racket, etc. Later on, if these things move to compat packages, the irrelevant directories get removed from the repo without surgeries, so the history will still be there. This shows some of the tricks that might be involved in the current switch: if you'd want to have some compat package *now*, the right thing to do would be: * do a simple filter-branch to extract drscheme (and other such collections) in a new repository for compat * for drracket: do a filter-branch that keeps *both* directories in, then commit a removal of drscheme. (Optionally, use rebase to move the deletion backward...) Going back to the repo structure change that you want and the reason that I said that doing moves between the package directories post-restructure is destructive should be clear now: say that you move collects/A/x into foo/A/x as part of the restructure. Later you realize that A/x should go into the bar package instead so you just move it to bar/A/x. The history is now in, including the rename, but later on when bar is split into a separate repo, the history of the file is gone. Instead, it appears in the foo repository, ending up being deleted. One way to get around this is to avoid moving the file -- instead, do another filter-branch surgery. This will be a mess since each such change will mean rebuilding the repository with all the pain that this implies. Another way to get around it is to keep track of these moving commits, and when the time comes to split into package repos, you first do another surgery on the whole repo which moves foo/A/x to bar/A/x for all of the commits before the move (not after, since that could lead to other problems), and then do the split. This might work, but besides being very error-prone, it means doing the same kind of file-movement tracking that I'm talking about anyway. So take this all as saying that the movement of files between packages needs to be tracked anyway -- but with my suggestion the movement is delayed until it's known to be final before the repo split, which makes it more robust overall. But really, the much more tempting aspect for me is that this can be done now -- if you give me a list of packages and files, I can already do the movement script. Actually
Re: [racket-dev] proposal for moving to packages: repository
9 hours ago, Carl Eastlund wrote: I was going to comment on the same thing. While a naive use of git filter-branch might not retain the history, it should be entirely possible to do something a little more intelligent and keep that history. Just to be clear, this is exactly what you can't get with filter-branch. Essentially each of the new repositories could keep the entire history of the original repository, followed by a massive move/rename, then moving forward with an individual package. This can work, but it is unrelated to filter-branch: it's basically starting each package repository from a clone of the monolithic repo, then move shuffle things around. This seems wrong to me in all kinds of ways -- but if someone wants to do this with *their* package (ie, not a package that I need to deal with), then it's certainly an option. (That's one of the big appeals of moving to packages for me: some code moves to packages which I can let myself Not Care About™. Knock youself out with tabs, spaces at ends of lines, braces in code, two spaces between bindings and values in `let's, and make sure that no file ends with a newline...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
A few minutes ago, Carl Eastlund wrote: On Thu, May 23, 2013 at 5:49 AM, Eli Barzilay e...@barzilay.org wrote: 9 hours ago, Carl Eastlund wrote: I was going to comment on the same thing. While a naive use of git filter-branch might not retain the history, it should be entirely possible to do something a little more intelligent and keep that history. Just to be clear, this is exactly what you can't get with filter-branch. Essentially each of the new repositories could keep the entire history of the original repository, followed by a massive move/rename, then moving forward with an individual package. This can work, but it is unrelated to filter-branch: it's basically starting each package repository from a clone of the monolithic repo, then move shuffle things around. This seems wrong to me in all kinds of ways -- but if someone wants to do this with *their* package (ie, not a package that I need to deal with), then it's certainly an option. It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? The problem of filter-branch? It has no problems, it does exactly what it is supposed to do. If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. If the drracket files are irrelevant for the swindle package then they shouldn't be in the swindle repository -- and on the exact same token, the development history of drracket shouldn't be there either. (This is not new, BTW, I think that there was general concensus right from the start of the package talk that the monolithic repo is just a host for a bunch of separate projects.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] proposal for moving to packages: repository
Just now, Carl Eastlund wrote: On Thu, May 23, 2013 at 6:57 AM, Eli Barzilay e...@barzilay.org wrote: A few minutes ago, Carl Eastlund wrote: It doesn't seem wrong to me. It's an accurate representation of the history of the project, which is exactly what git is for retaining. Where does the problem come from? The problem of filter-branch? It has no problems, it does exactly what it is supposed to do. It has no problems? Where above you stated this is exactly what you can't get with filter-branch in reference to keeping our packages' relevant history. Relevant history is vague. The thing that you can't do with filter-branch is keep the complete history if you remove files from the history -- the files that are gone go with their history. But filter-branch is not what I was talking about. I was talking about _not_ using filter-branch, and instead doing something that does keep history. Like I said: what you're suggesting means keeping the full monolithic history of developement in the main repo, including all of the irrelevant files (which will be removed in the tip, but included in the repo). If git filter-branch doesn't maintain the history we need, it's not the right tool for the job. If the drracket files are irrelevant for the swindle package then they shouldn't be in the swindle repository -- and on the exact same token, the development history of drracket shouldn't be there either. (This is not new, BTW, I think that there was general concensus right from the start of the package talk that the monolithic repo is just a host for a bunch of separate projects.) Okay, then let's purge the history of irrelevant files, but keep the history of relevant files even if they weren't in the right directory. If the monolithic repo is just a host for a bunch of separate projects, shouldn't it be possible to tease out their more-or-less separate histories? (*sigh*; please read the other email, where I went over this thoroughly.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
