Re: [racket-dev] What is the policy on what is included in the core libraries?

2015-02-17 Thread Eli Barzilay
On Tue, Feb 17, 2015 at 8:26 AM, Robby Findler
ro...@eecs.northwestern.edu wrote:
 I don't think the libraries are sufficient as is, but I would resist
 adding aliases.

 Perhaps a better way to get people coming from Haskell would be to
 write an essay specifically aimed there?

   - Step 1: use variables.
   - Step 2: here are `for` loops!

Instead of an essay, this could be a guide for Haskellers/MLers/
Whatever-ers, which would make it easy to get results into documentation
searches, and that would make it easy to address issues that cannot be
resolved with a new binding like the bad argument order in `take` or the
different input function for the folds.

(Another point for avoiding a `zip` binding is that it's much more
likely to collide with user defined functions than it is in other
languages.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

-- 
You received this message because you are subscribed to the Google Groups 
Racket Developers group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-dev+unsubscr...@googlegroups.com.
To post to this group, send email to racket-...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-dev/CALO-gusXr5Di4GBb3-xi3R5sL8G1CcAfNs3%2BBOB-7uM_AavGvA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: [racket-dev] Line editing in the default REPL

2014-12-23 Thread Eli Barzilay
On Wed, Dec 17, 2014 at 12:35 PM, Tony Garnock-Jones to...@ccs.neu.edu wrote:

 If anyone reading this has an interesting or unusual terminal they
 like to use, please run

 $ raco pkg install ansi
 $ racket -l ansi/test-raw

I didn't run it, but guessing common keys isn't too difficult, of
course.  I'm attaching a piece of code that I have that does that.  It
looks to me like my code is much more ambitious -- I wanted to be able
to have almost all possible keys with modifiers, including function keys
and plain characters, and that makes it messier.  (I posted variations
of this thing a few times in the past.)


 likely evolve into some form of a terminfo-like facility.  So it's
 better to just write something that can deal with terminfo directly.

 Sadly, terminfo is a little anaemic with respect to the sequences it
 specifies. To get input capabilities even half as rich as, say, emacs,
 you have to go beyond what is given in the terminfo database in a couple
 of ways. Specifically, to parse shift/control/meta combinations you need
 to get into terminal-specific encodings that are not covered by terminfo.

Sure, but those are usually not specific too.  (You're probably talking
about the DEC thing, right?  There's also the rxvt/aterm thing which is
different and more weird.)

In any case, terminfo for reading keys is obviously not too difficult to
manage.  If that was all that you need, then it's very easy to write
code that translates the terminfo database into some readable format
that you can read in Racket.  (My code is basically doing that parsing,
so it's 90% of what you need for that.)  But the thing is that terminfo
is much more needed for the related things -- querying the terminal,
using escape sequences to do the interactions (and doing that without
restricting yourself to some small subset), and also sending sequences
that setup the terminal (like rmkx/smkx which is what tripped me
earlier, and IIRC, it's needed with at least st, possibly others too).


 it's the blindness of going in that direction and thinking that you
 *won't* fall into this trap.

 It remains to be seen whether there are any problems resulting from
 this approach at all.

It looks like *you're* very aware of the issues, so why not take it?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!


keys.rkt
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Line editing in the default REPL

2014-12-13 Thread Eli Barzilay
[I'll reply to your email separately...]

Sorry for not sending the results I had earlier re parsing input keys.
Just to get this recorded here in case someone wants to do this in the
future:

I was first optimistic about the prospect of parsing keys in a way that
works for all terminals, provided that it's limited to the few keys that
are needed for line editing.  I wrote some code to parse terminfo texts
which I extracted on my linux installation, and find conflicts.  Turns
out that it's easy to do what I wanted only for the arrow keys, but then
things got very messy.  Specifically, there are terminals that use the
opposite escape sequences for home/end or backspace/delete.  (My best
guess about the origin of some of this is that there were numeric pads
that were laid out top-to-bottom as in phones, and the escape sequences
were based on the kp keys.)  It might be that the really bad conflicts
were happen with irrelevant terminals, but there are definitely
conflicts involving terminals that are relevant, like xrvt and
gnome-terminal.

My conclusion is that even for a very simple thing you need to consult
$TERM to know what to do, and some simple attempt to do that would most
likely evolve into some form of a terminfo-like facility.  So it's
better to just write something that can deal with terminfo directly.
FWIW, I don't think that it's too hard -- specifically for reading keys,
it's straightforward since you just get the escape sequence that you
need to identify on input.

I'm attaching the code that I wrote.  (To run it you might need to
adjust the location of the terminfo files at the top of the code; also,
it relies on infocmp as a way to print them out.)  If you run it you
get a huge listing of all keys, with indications about conflicts.  The
reallly bad conflicts would be cases where some escape sequence is a
prefix of another -- and those are rare (look for prefix) and happen
only with some obscure F keys.  (I initially just looked at these which
is why I thought that it's doable.)

Also, the `parse-decisions' thing in the code is my attempt to encode
how a parser should work and have it show the rest of the conflicts.  I
planned to add more keys and keep minimal conflicts but then I got to
the ones that don't have a good solution.  As an example, this part of
the output:

end:
  \eOq (!! (aterm mrxvt rxvt ... xterm-xi):
   clash with home of (nsterm ... vt102))
   (!! (aterm mrxvt rxvt ... xterm-xi):
   clash with key_f0 of (xterm-r5))
  \e[4~ (!! (... cygwin ... linux putty ... xterm-vt220):
clash with key_select of (... aterm gnome ... rxvt ...
xterm-24 ...))

shows one of these problems, where end on rxvt (popular enough)
correspond to home on vt102 (popular emulation target), and the last
thing shows a conflict between cygwin/linux/putty (all popular) and
aterm/gnome/rxvt (popular too).

[Extended footnote re time investing in this nightmarish pit of depair:
If you stare at this for a while, and consider using these results,
you'll get a feeling of how bad it can be...  Maybe vt102 is not used
too much?  Would there be any need for a select key, or maybe the same
escape sequence is mapped to something else that is useful?

The bottom line is that I don't really reject the noiseline-style
attempt to get something that works for most usert -- it's the
blindness of going in that direction and thinking that you *won't* fall
into this trap.  (Or, if you really won't since you'll refuse to
accomodate bugreports, then you'll make someone else fall into this trap
because of you.)]

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!


std-keys.rkt
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Line editing in the default REPL

2014-12-13 Thread Eli Barzilay
On Sat, Dec 13, 2014 at 7:00 PM, Leif Andersen l...@leifandersen.net wrote:
 Okay, here's another idea.

 I have parametrized the readline collection over the readline/edit/etc.
 library. You can do:

 (require editline)

 For the editline equivalent.

 It also falls back to no line editing (rather than throwing an
 exception), if the library is not there.

 The source is here: https://github.com/LeifAndersen/readline

(For reference: it's in the editline branch...)

 I'm thinking we could distribute the editline collection with Racket
 (treating libedit the same way we currently treat other native
 packages such as libcairo). And then we can make xrepl default.

 Would this be doable?

I looked at the code a little, but I couldn't find the place that does
the fallback part.  (You're defining a `libreadline-path' which AFAICS
isn't used.)  But it looks like you're using the same interface for both
editline and readline -- right?  If so, then I think that it's better to
just keep the current setup, and do this:

(define libreadline (ffi-lib libreadline '(5 6 4 )
 #:fail (lambda () (ffi-lib libedit ...

And since that addresses an API that is shared for the two, then it's
not code that should be linked only with readline, which means that it's
fine to it by default...?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Line editing in the default REPL

2014-12-03 Thread Eli Barzilay
: the 27 is the generic prefix
for these keys, 6 stands for the shift+control, 85 is the ascii
of U, and ~ terminates the escape sequence.
  - Note BTW that the escape sequence parser is completely broken,
since it assumes some given lengths in a very primitive way that
would break in the presence of these generic escapes I mentioned
above.  It is important to know about the escape sequences so that
even if you don't know how to translate some to keys you still know
when the escape sequence ends so you can ignore the whole key and
not leave leftovers behind.

* Your tests are not great too -- they're similar to copy+paste tests
  which encapsulate a specific behavior, and the thing that really
  matters (that it actually works) is left untested.

Again, I wrote all of this in a kind of hope that you'll do this, but
practically speaking, this code is so far from working that if you care
for your time it would be best to avoid it.  In other words, you have
almost nothing done there, compared to the amount of work that should be
added.  (But be careful of my cheap reverse psychology...)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Line editing in the default REPL

2014-12-03 Thread Eli Barzilay
On Wed, Dec 3, 2014 at 6:22 PM, Sam Tobin-Hochstadt
sa...@cs.indiana.edu wrote:
 On Wed, Dec 3, 2014 at 6:10 PM, Eli Barzilay e...@barzilay.org wrote:

 If you're talking about implementing line editing yourself, then my
 personal reaction to that would be wonderful, but doing it properly
 is something that is difficult and easy to underestimate

 I've already done this (admittedly it only works on OS X, but most
 Linux terminals work exactly the same with a few different
 constants). You can see what I have so far here:
 https://github.com/LeifAndersen/racket-line-editor/blob/master/main.rkt

 If this works, I'd be thoroughly impressed -- so I tried it.  I ran
 through a bunch of configurations that I use:

 None of which was OS X, which was what Leif's email explicitly said it
 _only works on_.

My point wasn't it doesn't work outside of OSX, I was talking about a
few different constants being a bad underestimation: they're not few,
they're not constant, and the interfaces for getting them is what you'd
expect from something that started to grow in the 70s.


 I think we should wait till he has a version which he says works
 before telling people to avoid making contributions.

You should re-read my email: if there's anything that I'm telling, it's
the exact opposite.  A point that I repeated more than once.

The thing is that it would be easy to not say anything, and watch this
approach of try a few escapes and see what works fail.  And it will
fail, because, again, they're not constant, and you can't just get some
magic combination that works for 90% of the people -- even just xterm on
just linux is something that can (and often is) configured in many
different ways.  So what I'm doing is the opposite: I point at how such
a thing should be written for it to survive -- but I appreciate Leif's
time enough to warn him that this is a much bigger thing than what he
seems to think.

(And to be clear, the reason for this is that I've seen probably around
3 to 5 serious attempts that got dumped; and I have encountered these
issues multiple times, so I know that it's hard enough that even getting
simple code to work, code that I intended only for my own use, even that
was pretty hard, enough to go chase old VT references and obscure man
pages.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Splitting the Racket repository

2014-11-29 Thread Eli Barzilay
On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt
sa...@cs.indiana.edu wrote:

 All the history for the code has been preserved, and for code that
 dates back before 2005, the history is extended back to the original
 CVS repository. See https://github.com/racket/games/ for an example of
 this.

There's a failure in the import -- if you look at the commits of this
repo (https://github.com/racket/games/commits/master), then starting
from Robby's commit from Jul 17 (add a contract to make-card...) and
going back, the original commit references are all bogus.


 If you build Racket from source from Git, that build now contains
 fewer packages. There is not yet an single-step way to get all of the
 split pkgs as git repositories; we plan to write a script for this
 soon.

Any reason they are not git modules?  (They've improved a lot, and
they're even supported in github as links as long as they point to other
GH repos.  See for example: https://github.com/elibarzilay/test)


 To clone individual repositories, use the new `--clone` option for
 `raco pkg`, such as:

 raco pkg install --clone pkg-build git://github.com/racket/pkg-build

 or for packages that are grouped together in a single repository

 raco pkg install --clone remote-shell
 git://github.com/racket/remote-shell?path=remote-shell-lib
 git://github.com/racket/remote-shell?path=remote-shell
 git://github.com/racket/remote-shell?path=remote-shell-doc

 Note that the clones created by `raco pkg install` cannot be pushed to
 with the default origin remote.

This is very obscure.  Is there a compact description of what to do when
you want to make a change in a file that is not on the main repo?

(Hopefully there's obvious agreement on the need for this to be
compact.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Splitting the Racket repository

2014-11-29 Thread Eli Barzilay
On Sat, Nov 29, 2014 at 8:30 PM, Sam Tobin-Hochstadt
sa...@cs.indiana.edu wrote:
 On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay e...@barzilay.org wrote:
 On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt
 sa...@cs.indiana.edu wrote:

 All the history for the code has been preserved, and for code that
 dates back before 2005, the history is extended back to the original
 CVS repository. See https://github.com/racket/games/ for an example of
 this.

 There's a failure in the import -- if you look at the commits of this
 repo (https://github.com/racket/games/commits/master), then starting
 from Robby's commit from Jul 17 (add a contract to make-card...) and
 going back, the original commit references are all bogus.

 For packages with old history, the shas are in reference to
 https://github.com/samth/old-plt-new/ (which should maybe have a
 better name).

Yeah, very bad name but an even worse location...  Since this is a major
reorganization it would also be a reasonable to do a forced push of that
thing to the main repository so the references make sense.  (This is
assuming that you cannot refer to the proper commits in the current repo
for some reason.)  If all else fails, you should at least include in the
commits some pointer to a repo that has it.

(Sounds picky, but having no reliable history is one thing that can make
a company avoid using it.)


 Any reason they are not git modules?  (They've improved a lot, and
 they're even supported in github as links as long as they point to
 other GH repos.  See for example:
 https://github.com/elibarzilay/test)

 The goal is to have packages that are in the main distribution not
 have a particular special status, beyond being in the list of things
 that gets put in the distribution.

I'm not talking about the distribution here -- just some meta repository
that has everything in it in a convenient way for the people who have
code in there.  (And assuming that this meta repository will build with
all of these packages that still not be related to a distribution.)

(A different issue is organizing distributions: at some point in the
past I pointed at the fact that git modules could be used for this,
essentially being a collection of pointers to code that is included.
But that bus has long gone, so I'm definitely not talking about that.)


 Also, a situation where you have to update two things when you do a
 commit is not ideal.

I absolutely agree with that.  Probably even more than you do, and
certainly for a much longer time (that's the essense of my
bundle-related rants years ago about spaghetti dependencies).

I suppose that you're worrying about modules making it easy to do such
cross-repo commits -- but it's no easier than just having a bunch of
repos and doing a commit over them simultaneously.  So it's as doable,
and in both cases will be awkward enough to make it clear to the
committer that something is wrong.


 To clone individual repositories, use the new `--clone` option for
 `raco pkg`, such as:
 [...]

 This is very obscure.  Is there a compact description of what to do
 when you want to make a change in a file that is not on the main
 repo?

 You just need to edit and change the relevant repository -- nothing
 else is required. You don't have to use --clone or any other
 mechanism.  [...]

This doesn't answer my question -- so I think that I wasn't clear
enough.  Say that I have some change to a specific file.  I want to know
how to do it in the following two ways:

1. Get the equivalent of the whole distribution in the form of a bunch
   of repositories, including the repository that I want to change.  I
   make the change in the file, re-build, test things out, and
   eventually push.

   This is the way that is more relevant for internal people, and I
   suppose that there would be some script that will do all of that.
   (The equivalent of a toplevel `make').

2. The thing that is more relevant for others (and that's the important
   one): I have a normal installation, I find a bug -- what do I do now?
   I can't just report the bug, the file, and my patch since that's
   using a path in the distribution that is mostly unrelated to the
   source repo for the file.

   Ideally, there would be some way to turn some containing directory
   into a repository so I can commit my change, and push it to a fork to
   do a PR.  But that's a little extreme -- so what I'm talking about is
   more like a way to say: get this repository and use it instead of the
   stuff in my distribution, so I can work and push from that.

   I'm guessing that this is the intention behind that command, but it's
   obscure since it's not obvious what to do.  (And obviousness is
   important to encourage contributions.)

   As a concrete example: I find a bug in a file like
   share/pkgs/games/gcalc/gcalc.rkt and want to fork, edit, and push a
   fix.  What do I do?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http

Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-22 Thread Eli Barzilay
On Fri, Nov 21, 2014 at 4:26 PM, Robby Findler
ro...@eecs.northwestern.edu wrote:

 Oh, absolutely.

OK, I'll do that later today.


 The two candidates are the trampoline approach and the just move the
 documentation files over into the user space as if a package had been
 installed.

Oh, *that*'s the other solution?  That sounds pretty bad not only
because it complicates file installation which is already very complex,
but also because it's a significant weight.  (For example, the current
size I see is about 180M, multiply that for a department with a thousand
users and you get unhappy admins.)

 I guess the latter is better because it means there will be more
 homogeneity in the sets of files and whatnot, such that these kinds of
 bugs will be less likely to be unnoticed by in house folk.

I think you meant to say former (ie, the trampoline) -- yes, having a
very different setup for end users is also pretty bad.  IMO, it falls
under the complication point above: installation layout is not only
complex in itself, it's also a point that introduces many different
options which are almost all never tested, and worse -- they can easily
lead to very bad bugs.


 That is, I never noticed this problem because my documentation is
 already in the user-specific place because I build from source. The
 other option will make our users operate more like how my drracket
 operates.

(I'm not following that.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-21 Thread Eli Barzilay
On Fri, Nov 21, 2014 at 8:46 AM, Robby Findler
ro...@eecs.northwestern.edu wrote:
 On Fri, Nov 21, 2014 at 12:34 AM, Eli Barzilay e...@barzilay.org wrote:
 Not that it matters, but did you try to see if it's the file
 permissions?

 Oh, they are!

 [...]

 And that was it!!

 If I run:

   $ xattr -d com.apple.quarantine /Applications/r/doc/search/index.html

 then osascript works again!
 [...]

 ... I don't know the ramifications of just calling xattr from inside
 DrRacket to fix things up. It may be better to just create the file in
 the user's directory instead.

That (the first thing) sounds like a very bad idea, and one that will
most likely fail in a case of a non-private machine.  The second is also
pretty bad since you'd be exluding a central setup in such a situation.
(Not that I see any reason to have a lab of Macs...)

And this is why I said that it's not really relevant -- AFAICT, the fact
that things are setup in a way that prevents passing queries and
fragments means that you just have to deal with it rather than try to
play games with the filesystem.

But of course there's another question of whether the trampoline will
work -- the browser might itself detect the different owner and refuse
to do the requested hop.  Hopefully it doesn't.


[And if it does, then I can only see ugly solutions like open a page
with a known weird title, wait for it to come up and then send it a
command to go to the real target...  Or running a tiny web server at
some known high port, and using an http://localhost:port/, and that web
server will wait for a single request and spit out the forwarding line,
dealing with multiple processes somehow (eg, some random number as a
private identifying cookie).]

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-21 Thread Eli Barzilay
On Fri, Nov 21, 2014 at 9:45 AM, Sam Tobin-Hochstadt
sa...@cs.indiana.edu wrote:

 We have labs of macs here at IU.

[*sigh* at that kind of spending...]


 And this is why I said that it's not really relevant -- AFAICT, the fact
 that things are setup in a way that prevents passing queries and
 fragments means that you just have to deal with it rather than try to
 play games with the filesystem.

 Could we just write a file with the query embedded in it, and open that?

That is exactly what that trampoline is: a page with just a immediate
redirection to the search file with the query.

(Unless you mean write a version of the search page with a specific
query hard-wired into it, which is a bad idea for a reasons...)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-21 Thread Eli Barzilay
On Fri, Nov 21, 2014 at 3:16 PM, Robby Findler
ro...@eecs.northwestern.edu wrote:

 Oh, my apologies. I thought you meant something different.

 Yes, this works.

Ah, in this case, the patch that I sent earlier should work fine.
(I could do a pull request, but it should really be tested...)

 I don't have a good idea which is the better approach now.

[Bear in mind that I don't really know what the issue with the file
permissions is (looks more like some security attributes):]

I think that playing with such attributes is a worse option -- it's
something that I'm not sure would work in all settings (but see above),
and if there is a way to make it work now, it might not be possible in
the future.  The trampoline approach is relatively robust and well
behaved.  There is a minor price of these temporary files, but they do
get cleaned out (whenever `send-url/contents' is called), so it should
settle on a small number that will not grow indefinitely[*], so this is
not a problem.

Also, on the it's already there side -- this trampoline approach is
already there for the windows version, and it's needed for the
`xdg-open' thing anyway, so abstracting it into a new (internal)
function makes sense already, so in terms of additional code there's no
cost for doing the mac version too.


([*] It might be a problem if, for example, your cat sits on the f1 key
and the os doesn't take the focus away from drracket...  It might be
useful to change it from using a random temp file to a name based on a
hash of the forwarded url, which will further minimize the number of
files.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-20 Thread Eli Barzilay
On Thu, Nov 20, 2014 at 5:22 PM, Robby Findler
ro...@eecs.northwestern.edu wrote:
 Ah, thanks! I see that if it is trying to load the docs that are in
 /Applications then it runs the code below, which somehow magically
 drops the query argument by the time safari gets it. It's not because
 of the space, either; when I rename Racket v6.1.1.5 to just r, it
 also doesn't work. But if I change the path to one in my home
 directory (just like the one you write below), then I see it works.

 Puzzling. It's almost like there is special handling for /Applications
 instead of /Users.

Something that might be relevant here: on Windows, the query (?...) or
fragment (#...)  bits of a URL are dropped when you run them.  That's
why I made it use a trampoline file: it creates a small HTML file
which can open fine, and that file redirects to the full URL.

At some point, Linux joined this with xdg-open not passing along these
things also, and needed a similar hack (which IIRC, wasn't done).  So it
might be that OSX is doing something similar now.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-20 Thread Eli Barzilay
On Thu, Nov 20, 2014 at 7:51 PM, Stephen De Gabrielle
spdegabrie...@gmail.com wrote:
 in DrRacket:

  (send-url file:///Users/spdegabrielle/hello.html?q=aaaba)

 works

 (send-url file:///Applications/hello.html?q=xyz)

 works

In that case I think that it's a different problem than what I
described.


 (send-url
 file:///Applications/Racket%20v6.1.1.5/doc/search/index.html?q=xyz)

 fails

This is weird.  To debug it, the text thing I'd do is make that
/Applications/hello.html show the location string -- perhaps the browser
hides the query part.  Another thing to look into is the permissions of
the files -- maybe it descides to drop the query bit on a file that you
don't personally own.


 in bash:

 Miriams-MacBook-Pro-2:~ spdegabrielle$ open
 file:///Users/spdegabrielle/hello.html?q=

 fails

It shouldn't be `open' -- it's something with osascript.  (I don't
remember it now, but Robby had it in the other email.)


 I've just had a quick go at '(send-url/file ' but I cant remember how to
 escape the '?'

Should be something like

(send-url/file /Some/file.html #:query 123)

but it's probably won't help to try it, since it uses the same
functionality as the above.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DrRacket PF1 Search Bug?

2014-11-20 Thread Eli Barzilay
On Thu, Nov 20, 2014 at 10:38 PM, Robby Findler
ro...@eecs.northwestern.edu wrote:
 No, the browser isn't hiding the query part.

 Here are the content of two script files:

 $  cat a.scrpt
 open location file:///Applications/r/doc/search/index.html?q=xyz
 $  cat b.scrpt
 open location 
 file:///Users/robby/Library/Racket/development/doc/search/index.html?q=xyz

 Running osascript a.scrpt doesn't work, but running osascript
 b.scrpt does work. And by work I mean that in both cases the
 corresponding webpage is visited in Safari, but in the first case, the
 q parameter disappears.

Not that it matters, but did you try to see if it's the file
permissions?

Another thing: I googled osascript open url drops query and got a
bunch of racket results, and dropping osascript make the results more
sensible.  So perhaps osascript is outdated?  Some more searching
makes it look like you can just run the open command directly, and that
might make a difference.


In any case, if it just doesn't work, then doing the trampoline thing
should be easy -- it should do the same thing that `send-url/win' does:
if the URL has a query or a fragment, then use `send-url/contents' with
the same contents as the windows case.

And since there's the linux case too, then it's worth an abstraction,
something like the simple shuffle in the attached patch.  (Completely
untested, and wrapping xdg-open in the linux function since it looks
like it's still as broken as I remember it to be.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!


p
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #28054: master branch updated

2014-01-13 Thread Eli Barzilay
 4d44878 Asumu Takikawa as...@racket-lang.org 2014-01-06 17:47
 :
 | Move `chomp` function into its own util file
 :
   A pkgs/typed-racket-pkgs/typed-racket-lib/typed-racket/utils/string.rkt
   M .../typed-racket-lib/typed-racket/utils/tc-utils.rkt | 8 +---

Why not (string-trim s #:left? #f) ?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #28054: master branch updated

2014-01-13 Thread Eli Barzilay
A few minutes ago, Sam Tobin-Hochstadt wrote:
 I think it's really `(string-tring s \n #:left? #f)`.

Yes, but IIUC, the function that Asumu wrote is for tweaking output
strings, which makes `string-trim' relevant, and my guess is that the
extra trimming that it makes (other whitespaces, and any number of
them) is probably fine.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Huh?

2013-12-02 Thread Eli Barzilay
6 hours ago, Jens Axel Søgaard wrote:
 In DrRacket 6.0.0.1 open
 
pkgs/draw-pkgs/draw-lib/racket/draw/private/record-dc.rkt
 
 and then click Run.
 
 I see:
 
   define-values: assignment disallowed;
cannot re-define a constant
 constant: get-recorded-command1.1
 
 Why?

That's not new -- usually when you run the code you get a clean
environment with the old instantiation gone, but some core files are
shared across runs, so it's like trying to redefine things that are
already defined.  Many files in racket/* will do the same.  To debug
this kind of stuff it can be helpful to work on a copy of the file
that you want to edit, and rename it back over the original (and
recompile) when it's working.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Release for v6.0 has begun

2013-11-26 Thread Eli Barzilay
Yesterday, Matthew Flatt wrote:
 At Mon, 25 Nov 2013 09:56:45 -0500, Ryan Culpepper wrote:
  On 11/25/2013 09:44 AM, Matthew Flatt wrote:
   Here's the full comment:
  
   The version string has one of the forms:
  X.Y
  X.Y.Z Z != 0
  X.Y.Z.W   W != 0
   where each X, Y, Z, W is a non-negative exact integer, Y must not
   exceed 99, and Z or W must not exceed 999.  Y=90 means that this is
   working towards {X+1}.0, and X.Y (Z=0, W=0) is an alpha version for
   {X+1}.0; Z=900 means working towards X.{Y+1}, and X.Y.Z as an
   alpha release.
  
   Then intent is that when Z and W are 0, the string form of the version
   number is just X.Y, not X.Y.Z.W.
  
   How about this clarification?
  
 ... and X.Y (i.e., Z=0 and W=0, so Z and W are
   omitted from the string form) ...
  
  That's not the part that needs clarifying. I think that fact that the 
  string form drops final zeros is clear from lines 2-4.
  
  The part that needs clarifying is how to choose the version number for 
  the alpha releases leading up to version {X+1}.0. (Really, how to choose 
  alpha version numbers in general, since I've had similar problems in the 
  past.) From this statement, X.Y (Z=0, W=0) is an alpha version for 
  {X+1}.0 (Y=90 already stated), I would expect that 5.91 would be a 
  fine alpha version number for 6.0. Is it? If not, what should the alpha 
  version number be?
 
 I agree that 5.91 is the right alpha-version string, assuming that
 it's intended as an alpha in the sense of our release rules (as
 opposed to a release candidate, which has a non-zero W).

These two things were originally independent: the ability to specify
alpha-ness (the second .91) and release-ness (W=0), and that was used
by the old build script to make some decisions for what the installers
do.  Assuming that this still matters, there is a problem with using
5.91 for the release process -- and instead it should be 5.91.0.1
to make it treated as a nightly build.  For example, on Windows the
installer for a 5.91.0.1 wouldn't grab the suffix registration, but
5.91 would which makes it bad as something that you ask people to
try.

BTW, this is not the same meaning of alpha that is used in the
release checklist -- that one has the meaning of a release
candidate.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Release for v6.0 has begun

2013-11-26 Thread Eli Barzilay
Just now, Robby Findler wrote:
 So, IIUC, Ryan should have used 
 
   5.91.0.1
 
 as the version number on the release branch?

Yes.


(Part of this is probably the result of the poor estimate that the
Emacs code does for creating the checklist template -- mostly because
I originally intended that to be just a recommendation, not a
determination.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #27825: master branch updated

2013-11-25 Thread Eli Barzilay
IIUC, this makes the limit thing -- and therefore sandboxes -- behave
*very* differently.  The original intention was that the time limit is
on something similar to what you get with `time'.

A very visible way to see the effect of this change:

- ,r racket/sandbox
- (define e (make-evaluator 'racket))
- (e '(define foo 1))
- (e '(thread (lambda () (sleep 5) (set! foo 2
#thread

This used to happen immediately, with the thread continuing to run
inside the sandbox.  After your change, the last line hangs until the
thread is done.  Using a bigger sleeping time will make it throw an
error when it previously didn't.  Similarly,

(make-module-evaluator #lang racket (thread (λ() (sleep 99

used to work and will throw an error now, and of course, any code that
runs some kind of sandboxed server will probably break now.

I think that instead of this, it'd be better to write a helper that
runs a thunk and waits for it and for any generated threads to end,
and suggest using this helper when you want to wait for all threads in
a `with-limits'.  (It might also be useful in the profiler, where a
similar kind of wait-for-all is done.)


On Friday, j...@racket-lang.org wrote:
 jay has updated `master' from e0026f5de4 to 79f8636e1e.
   http://git.racket-lang.org/plt/e0026f5de4..79f8636e1e
 
 =[ One Commit ]=
 Directory summary:
   52.6% pkgs/racket-pkgs/racket-test/tests/racket/
   45.6% pkgs/sandbox-lib/racket/
 
 ~~
 
 79f8636 Jay McCarthy j...@racket-lang.org 2013-11-22 14:25
 :
 | Ensure that threads created within call-with-limits are accounted
 | during the time/space limits
 :
   A pkgs/racket-pkgs/racket-test/tests/racket/sandbox.rkt
   M pkgs/sandbox-lib/racket/sandbox.rkt   | 81 
 ++--
   M .../racket-test/tests/racket/sandbox.rktl | 48 
   M .../scribblings/reference/sandbox.scrbl   |  4 +

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] unable to git fetch, or ssh -v to git.racket-lang.org ?

2013-11-22 Thread Eli Barzilay
10 hours ago, John Clements wrote:
 This... doesn't look like something on my end?
 [...]

Did you check if you were blacklisted?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Looking for volunteer(s)

2013-10-04 Thread Eli Barzilay
Yesterday, Eli Barzilay wrote:
 I'm looking for someone to take over mailing list moderation and
 possibly other similar work like monitoring newsgroups etc.

To clarify this a little:

* The workload is very small -- Mailman has a web interface and an
  email one for doing moderation; but since I find these interactions
  cumbersome, I have a script that I run once every one to three days.
  And since the script is in racket, you can really hook it up to
  anything that is convenient.

* It's really something that should be done by someone on one of the
  plt groups, since you'll need to know how to filter academic calls
  that are relevant or not, how to authorize pending requests to the
  edu lists, and you'll need to moderate some internal lists.  It's
  also better if it's taken by someone who is actively reading the
  mailing lists, so you know when some pending post has already been
  sent since the sender subscribed and re-sent their email.

* The other monitors are for keeping track of things in general and
  be present whenever needed.  (This is why the newsgroup monitor is
  a good example: announcing releases there is still very useful, but
  doing so without tracking followups is not a good idea.)

Still waiting,
-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


[racket-dev] Looking for volunteer(s)

2013-10-03 Thread Eli Barzilay
I'm looking for someone to take over mailing list moderation and
possibly other similar work like monitoring newsgroups etc.
-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] DOS attack on planet?

2013-09-22 Thread Eli Barzilay
I just looked into that, and it seems that there's something bad going
on with some machine at BYU which started yesterday.  (Ping: Jay.)

The offending traffic comes from fltr5.byu.edu, at a very high rate.
The new log file for the week had started at 2013-09-22 03:40 local
time (about 12.5 hours ago) with 92000 queries for this period, and
85% of this traffic (about 78k, about a 100 hits per second) is coming
from this BYU IP.  Looking back, it seems that it's something recent
that had started just yesterday, so whatever it is, it's new.  Most of
the traffic is basically a repeating loop of these 8 lines, shown below.

(I will restart the server now, in an attempt to get whatever it is
that causes this mess to crash.)

128.187.97.22 - - [22/Sep/2013:03:49:17 -0400] GET /servlets/pkg-info.ss 
HTTP/1.1 200 5650 - -
128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22dracula.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22cce%22%29
 HTTP/1.1 404 79 - -
128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22dracula.plt%22maj=2min-lo=0min-hi=%23fpath=%28%22cce%22%29
 HTTP/1.1 404 79 - -
128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22dracula.plt%22maj=5min-lo=0min-hi=%23fpath=%28%22cce%22%29
 HTTP/1.1 404 41 - -
128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22drocaml.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22abromfie%22%29
 HTTP/1.1 404 79 - -
128.187.97.22 - - [22/Sep/2013:03:49:18 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22fasttest.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22cce%22%29
 HTTP/1.1 404 79 - -
128.187.97.22 - - [22/Sep/2013:03:49:19 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22xmlrpc.plt%22maj=3min-lo=0min-hi=%23fpath=%28%22schematics%22%29
 HTTP/1.1 404 79 - -
128.187.97.22 - - [22/Sep/2013:03:49:19 -0400] GET 
/servlets/planet-servlet.ss?lang=%225.90.0.9%22name=%22bystroTeX.plt%22maj=1min-lo=0min-hi=%23fpath=%28%22amkhlv%22%29
 HTTP/1.1 200 63508 - -




30 minutes ago, Robby Findler wrote:
 I don't think that the planet server itself doesn't keep enough
 information to say much about this, but the requests come via apache
 so there might be more information in a log file at that level that
 Eli might be able to tell us about.
 
 I do see lots of requests coming in for packages, tho. In addition
 to yours, dracula.plt fasttest.plt, drocaml.plt, and xmlrpc.plt seem
 to be being continuously asked for.
 
 Robby
 
 On Sun, Sep 22, 2013 at 12:40 PM, Andrei Mikhailov a.mk...@gmail.com wrote:
 
 Sorry if I am rising a false alarm. I noticed that there is a massive
 download of my package
 called bystroTeX : http://planet.racket-lang.org/display.ss?package=
 bystroTeX.pltowner=amkhlv
 All the downloads are of the (old) version 1.6

 What is going on?
 Is it possible to figure out who downloads it?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


[racket-dev] Pinging BYU people!! (was: DOS attack on planet?)

2013-09-22 Thread Eli Barzilay
Update: bringing it down for a few minutes didn't help, and the
offending process continues its merciless traffic.  I've added a
temporary rule that effectively blacklists planet access from that IP
address.  (Apologies in case that's a shared machine.)  All I see now,
are failed attempts to get /servlets/pkg-info.ss (which are answered
with a 403 to that IP).

Can someone at BYU look into this?



20 minutes ago, Eli Barzilay wrote:
 I just looked into that, and it seems that there's something bad going
 on with some machine at BYU which started yesterday.  (Ping: Jay.)
 
 The offending traffic comes from fltr5.byu.edu, at a very high rate.
 The new log file for the week had started at 2013-09-22 03:40 local
 time (about 12.5 hours ago) with 92000 queries for this period, and
 85% of this traffic (about 78k, about a 100 hits per second) is coming
 from this BYU IP.  Looking back, it seems that it's something recent
 that had started just yesterday, so whatever it is, it's new.  Most of
 the traffic is basically a repeating loop of these 8 lines, shown below.
 
 (I will restart the server now, in an attempt to get whatever it is
 that causes this mess to crash.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)

2013-09-22 Thread Eli Barzilay
(Note that instead of the apache rule I now switched to a firewall
rule, so it won't even get 403 responses now.)


40 minutes ago, Eli Barzilay wrote:
 Update: bringing it down for a few minutes didn't help, and the
 offending process continues its merciless traffic.  I've added a
 temporary rule that effectively blacklists planet access from that IP
 address.  (Apologies in case that's a shared machine.)  All I see now,
 are failed attempts to get /servlets/pkg-info.ss (which are answered
 with a 403 to that IP).
 
 Can someone at BYU look into this?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)

2013-09-22 Thread Eli Barzilay
50 minutes ago, Jay McCarthy wrote:
 Next time, feel free to follow the directions on
 internal.racket-lang.org.

I have no practical way to know whether it's actually one of your
machines.  (I did check that it's not an IP that is in our DNS.)


 Now that you've turn off its access, rather than just logging in and
 killing it,

Nor do I know what it is that should be killed.  (And I will
certainly not going to ssh into your account and sniff around.)


 I can't test and see what the underlying problem was. Let me know
 when you have turn traffic back on.

It's back on now.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)

2013-09-22 Thread Eli Barzilay
A few minutes ago, Jay McCarthy wrote:
 
 In retrospect, I guess it's not so obvious that the package server
 contacts the old server regularly to build the compatibility version
 packages.

Is this the package server??  The IP I have for that is
128.187.105.226, which is different from the IP that caused the
traffic.  This is why I couldn't guess what causes the traffic, and
guessed some rogue experiment in indexing on some test machine.

In any case, if it is the package server through some other machine,
then it's best to change it so it comes from the actual server.


  It's back on now.
 
 Thanks... it looks like I'm still getting 403s though.

Ah, sorry -- I forgot to remove the apache rule too.  Should be
working now.

Also, since it's scanning the planet packages (at least looks like
that), and those really don't change that often, then it'll be much
better to do this scan much more infrequently -- like once every hour
or so rather than once every two seconds...

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Pinging BYU people!! (was: DOS attack on planet?)

2013-09-22 Thread Eli Barzilay
Just now, Jay McCarthy wrote:
 On Sun, Sep 22, 2013 at 6:53 PM, Eli Barzilay e...@barzilay.org wrote:
 
  In any case, if it is the package server through some other
  machine, then it's best to change it so it comes from the actual
  server.
 
 I don't know what's going on with that. It's in a VM, so maybe
 something is fishy when traffic leaves it versus when it comes to
 it?

Ooh, that's pretty bad for a server.  Having an IP address that
doesn't resolve back to the IP name is nothing new these days, but
having traffic from the server come via a different IP address is
really not a good idea.  Think about dealing with some kind of an
external service, who would need to be aware of your traffic: having
it come from a different IP address is something that would make it
very hard.

It would be a good idea to ask the people who manage that if it's
possible to get the expected behavior.

(FWIW, it might be some result of a firewall or something like that
too.  In NEU, our public machines are all in a DMZ network so they're
not affected by such firewalling.  (But it does mean dealing with a
public machine -- for example, dealing with ssh dictionary attacks,
not having some kind of expected weaknesses exposed like PHP and
similar junkware, etc.))


 It is supposed to do it weekly. I just turned it back on and did not
 get an error, so I'm not sure what the problem was. (The 403 errors
 totally filled the log, so I couldn't tell what the problem was
 earlier in the day.) So, I'm not sure what the problem was.

I can tell you exactly when it happend -- the flood started with this
entry:

128.187.97.22 - - [21/Sep/2013:22:10:10 -0400] GET /servlets/pkg-info.ss 
HTTP/1.1 200 5650 - -

This was the first entry from that IP address for the whole week, so
it was probably the weekly run which then went bad.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [racket] RacketCon full schedule

2013-09-18 Thread Eli Barzilay
5 hours ago, Sam Tobin-Hochstadt wrote:
 I went to change this, but the current source of the web site is not
 in the meta/web directory.

It is.


 In particular, it doesn't contain the Racket Videos announcement,
 which is where the RacketCon announcement should go.

That part is in iplt, I added and updated the web page.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [racket] RacketCon full schedule

2013-09-18 Thread Eli Barzilay
Just now, Sam Tobin-Hochstadt wrote:
 Thanks for updating! Is that the only part of the web page in iplt?

No, see the contents of the web directory there.  The reason that
the blurbs are is that the rcon sources are there, and the reason for
that is that they contain the pdfs etc which shouldn't be part of the
code repository.


 More generally, it seems like we could move some or all of
 plt-services to a separate repository.

Either that, or move it to iplt.  It should remain publicly
inaccessible since it contains sensitive information.  (Sensitive in
the security sense, like the web server configuration, etc.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [racket] RacketCon full schedule

2013-09-18 Thread Eli Barzilay
Two hours ago, Sam Tobin-Hochstadt wrote:
 On Sep 18, 2013 10:33 PM, Eli Barzilay e...@barzilay.org wrote:
 
  Just now, Sam Tobin-Hochstadt wrote:
   Thanks for updating! Is that the only part of the web page in
   iplt?
 
  No, see the contents of the web directory there.  The reason
  that the blurbs are is that the rcon sources are there, and the
  reason for that is that they contain the pdfs etc which shouldn't
  be part of the code repository.
 
 They have to be in some repository -- why I'd one worse than another?

I didn't say that they shouldn't be.


   More generally, it seems like we could move some or all of
   plt-services to a separate repository.
 
  Either that, or move it to iplt.  It should remain publicly
  inaccessible since it contains sensitive information.  (Sensitive
  in the security sense, like the web server configuration, etc.)
 
 I don't understand -- plt-services is a directory on the 'meta'
 collection in the git repository.

I was talking about the contents that is now in iplt.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Bug report not confirmed

2013-09-16 Thread Eli Barzilay
On Friday, Eli Barzilay wrote:
 
 There was a permission problem with it, which is fixed now.
 Unfortunately, it looks like the contents of those messages are
 lost.

Actually, I just remembered that the bug emails are routed through
gmail, so there are copies of everything that was lost.  (Expect a
mini-flood of 6 messages...)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Bug report not confirmed

2013-09-13 Thread Eli Barzilay
Just now, Greg Hendershott wrote:
 On Thu, Sep 12, 2013 at 4:41 PM, Laurent laurent.ors...@gmail.com wrote:
  FYI, I have filed a bug report more than 3 hours ago but did not receive any
  confirmation.
 
 Is it possible the bug tracker is broken?

There was a permission problem with it, which is fixed now.
Unfortunately, it looks like the contents of those messages are lost.


 Plus me, recently: I had a bug where my email got munged from
 @gmail.com to @email.com:
 [...]

No, this was an unrelated issue...  It happened before the server
change.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] call-with-limits memory bound isn't actually bounding memory usage

2013-09-11 Thread Eli Barzilay
An hour ago, J. Ian Johnson wrote:
 Okay, stamourv made your response make sense. I added parameterize
 ([current-namespace (make-base-namespace)]) inside the thunk, [...]

If you're going down that road (which makes sense, of course), then it
would probably be much easier to just use the full sandbox.  There's a
long laundry list of things to deal with to get good isolation, and
the sandbox is basically a convenience tool for that list.  (IIRC, the
gui stuff had a bunch of subtle points, like taking care of the
eventspace etc.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
 and in bold
  instead.

* I'm ambivalent in the same way Neil is about stuffing a whole bunch
  of pages into a single big page.  It's true that clicks cost, but
  scrolling cost too (both in terms of the needed user action and the
  crowded look of the page).  Maybe a good middle ground is to keep
  the sub-pages as before, and have only prominent parts (= shorter
  bits) on the front page.

  Specifically, there are a lot of things in the community part that
  seem like they really don't fit a big front page thing (eg, the blog
  item, the emphasized #racket, the link to freenode.net, the
  browse the logs).  It seems that the whole community part (and
  possibly others, like the documentation section) could use a list of
  quick links: RacketCon, Blog, Twitter; Mailing Lists, IRC, People;
  etc.

* The freshest doesn't read like built from git head, IOW, it
  looks like a real release.

* Either the code pointer should go to git.racket-lang.org, or just
  kill the git server.  There is little point in investing efforts on
  maintaining a git server to save a few random people the tiny cost
  of a github account or to setup their own server -- especially when
  there's no sysadmin to manage it.

* The textbooks are a nice touch, but I think that some of the other
  good course-level material (coursera, brown lectures, etc) could be
  included.  Also, they do consume a lot of space, so I think that
  they're better at the end of the page (after a much-trimmed
  community part, assuming that the full page is still there)

* The middle sentence in the book descriptions seem redundant in all
  cases.  Also, the lack of periods look odd if it doesn't look like
  an ol list visually.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
30 minutes ago, Matthias Felleisen wrote:
 
 All in all, I really like the idea of renovating our home page. What
 I would like to see is that we get
 
  * a distinct home page 
  * one that brings across that Racket is full spectrum 
   technically -- you can gradually migrate from untyped to typed, from 
 modules to components, ...
   educationally -- you can grow into Racket starting with Bootstrap, 
 which I can't find 
   socially -- you can join a variety of mailing lists, irc channels, 
 twitter streams, etc. 
   and that there are many kinds of Racket communities 
 (edu, research, com, open source, secret cabals) 
   and more

This three-face spectrum thing sounds *really* good.  Probably would
work out well replacing the three grow sections.  (And begs for some
visual pun too.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
And a major bug (chrome, now on linux): resizing the browser doesn't
reposition the text in the middle.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
Just now, Sam Tobin-Hochstadt wrote:
 On Tue, Aug 20, 2013 at 12:16 PM, Eli Barzilay e...@barzilay.org wrote:
 
  * The noise background image is still at the previous place
which is now completely bogus.
 
 I don't understand what's wrong with the background image.  The
 problem with the headings is known.

It was originally intended to be behind the code example -- now it
stays there, and in narrow mode it's behind some random text.  (But
see previous email: I really think that it doesn't fit the new layout
and should just be removed.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
An hour ago, Sam Tobin-Hochstadt wrote:
 On Tue, Aug 20, 2013 at 5:30 PM, Jose A. Ortega Ruiz j...@gnu.org wrote:
  On Tue, Aug 20 2013, Neil Van Dyke wrote:
 
  Yes, I scroll down the page and I see pictures and a simple
  picture language, and it seems to be using them to introduce some
  pretty simple and familiar concepts, so I assume it's for
  children.
 
  FWIW, i get the exact same feeling.
 
 I think this is a sign that we need to revise the tutorial, not that
 we need to not link to it. For example, Quick is the only tutorial
 that doesn't assume the reader knows Lisp notation.

I can see where the problem is: assume that you're trying to convince
someone foreign to any lisp (or worse, someone who remembers some
parens from an undergrad course 20 years ago) to do the next project
in Racket.

Perhaps a good way to resolve this would be to write a

Semi-Quick: An Introduction to Racket Hacking

text, and have a standout blurb at the top of each of the two Quicks
describing the style of the text and referring to the other Quick if
they prefer a more {hacker-oriented,playful} kind of an intro.  Maybe
even go with another one:

Quasi-Quick: An Introduction to Racket for the Secular Hacker

Obviously, the main issue is who will write all of these things -- but
I think that it's much easier than it seems, assuming permissions of
the respective authors:

Semi-Quick: take Prabhakar's intro (cs.uwaterloo.ca/~plragde/tyr/) and
  make it terse enough to fit a single (longish) page.  I think that
  this is a fine choice for an aspiring hacker -- the only part that
  is missing, the advanced section, is probably the part that should
  at best be a very quick overview for a bunch of things that you
  could do, mostly serving as a link hub.

Quasi-Quick: this seems like it fits well the new Learn Racket in Y
  Minutes (learnxinyminutes.com/docs/racket/), where the text is more
  overview-ish, so is more oriented towards people who already know a
  few languages and are mainly interested at a syntactic/what's-new
  kind of overview.  The main thing that is needed is a tiny bit more
  prose to make it more readable (but very little, since this audience
  is more interested in the code bits).

A good writer can probably bring both to a good level in a few hours.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
An hour ago, Sam Tobin-Hochstadt wrote:
 On Tue, Aug 20, 2013 at 10:50 AM, Eli Barzilay e...@barzilay.org wrote:
  I dislike the switch to something that looks less unique -- this
  feels more like yet-another-site of a language.  I can't really
  put a finger on what makes that, but I think that the big factor
  is the dark header.  (I didn't like it when Matthew initially
  suggested it, but I think it ended up being a good visual
  signature.)
 
 After trying this out, I agree, and I've switched to a black top
 bar.  I've also added more space, as Laurent suggested, and fixed
 the cropping in the popup, as Stephen pointed out.  I've revised the
 tag-line formatting, and darkened a few things, and taken some more
 of your suggestions that I've forgotten now.

All of these look good to me.  (Still a good amount of work to do, and
I hope that it'll get there this time...)

A quick idea that popped up today re the books: have a book bar as in
the erlang page, but with popups that have the book descriptions.  It
can be much less intrusive while keeping the good general first-look
impression, and it can accommodate many more books.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
A few minutes ago, Sam Tobin-Hochstadt wrote:
 On Tue, Aug 20, 2013 at 7:32 PM, Eli Barzilay e...@barzilay.org wrote:
  A quick idea that popped up today re the books: have a book bar as
  in the erlang page, but with popups that have the book
  descriptions.  It can be much less intrusive while keeping the
  good general first-look impression, and it can accommodate many
  more books.
 
 What do you mean by pop-ups?

Just a div that is shown next to a book with the description text.
Roughly like the ones that appear next to screen shots in the old
page: http://plt-scheme.org/screenshots/.

(Actually that same code could be used, just make it a horizontal
widget instead of vertical, add some space between the images,
and make a much smaller big--small size difference.  It does have a
few bugs which I fixed later, as seen on my pages, eg,
http://barzilay.org/random/stuff2.html)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Revising Racket's home page

2013-08-20 Thread Eli Barzilay
A few minutes ago, Sam Tobin-Hochstadt wrote:
 On Tue, Aug 20, 2013 at 7:59 PM, Eli Barzilay e...@barzilay.org wrote:
  A few minutes ago, Sam Tobin-Hochstadt wrote:
  On Tue, Aug 20, 2013 at 7:32 PM, Eli Barzilay e...@barzilay.org wrote:
   A quick idea that popped up today re the books: have a book bar as
   in the erlang page, but with popups that have the book
   descriptions.  It can be much less intrusive while keeping the
   good general first-look impression, and it can accommodate many
   more books.
 
  What do you mean by pop-ups?
 
  Just a div that is shown next to a book with the description text.
  Roughly like the ones that appear next to screen shots in the old
  page: http://plt-scheme.org/screenshots/.
 
 So, in the style of tooltips, rather than something to click on?

You can have any html in there -- see for example the last blurb which
has a link in it.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] sandbox and file-/directory-existence tests

2013-08-19 Thread Eli Barzilay
A few minutes ago, Sam Tobin-Hochstadt wrote:
 On Mon, Aug 19, 2013 at 4:34 PM, Matthew Flatt mfl...@cs.utah.edu wrote:
 
  Is there a situation where allowing an arbitrary file- or
  directory-existence test would be bad?
 
 This all depends on how paranoid we want to be.  There are certainly
 situations when this will be bad -- it lets you determine who else
 has an account on a computer, for example.  But there are contexts
 where having GC be observable is a security hole as well, so we have
 to pick a spot on the continuum.

Getting some hacker-useful information from an observable GC time is
much harder than doing so from FS existence tests.  Two quick
examples:

  * On a unix machine, check if there's a /tmp/shadow file -- if there
isn't then you have a machine that is a potential gold mine for
hackers.

  * On a windows machine you can use some network drive or a drive of
some random device for a kind of a local DOS attack.

(There's probably a lot of similar things that are much more
sophisticated; probe attacks in general are very common now.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #27090: master branch updated

2013-08-14 Thread Eli Barzilay
About a month ago, Matthias Felleisen wrote:
 
 On Jul 7, 2013, at 12:08 AM, Eli Barzilay e...@barzilay.org wrote:
 
  It would be nice to have some way of doing it with contracts, but
  that's obviously impractical -- so I think that a good way to
  solve it and other such problems is to add a parameter for the
  name used in read errors, and making `raise-read-error' behave in
  a similar way to `raise-syntax-error' with the parameter used as
  the `foo:' part of the error message.
 
 Boy I like this. I can't remember how often I have code up something
 like '#:function' and thought this is a general problem worthy of a
 solution.

My point was that this is a problem with readers -- where you often
wrap some reading function (that can fail) by your own reader
function, and any reading problems should therefore be attributed to
your reader.  That's why I think that a parameter would be a nice way
to address this.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


[racket-dev] Require error

2013-08-14 Thread Eli Barzilay
Here's a confusing require error that I ran into:

  - ,r tests/json
  ; stdin:2:3: cannot open module file
  ;   module path: tests/json
  ;   path: /home/eli/src/plt/pkgs/db-pkgs/db-test/tests/json.rkt
  ;   system error: No such file or directory; errno=2
  ; [,bt for context]

The path: part has gone from being useless to being confusing...

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


[racket-dev] Fishy directory rename

2013-08-14 Thread Eli Barzilay
Trying to sort out some apparent mess in the props, I ran into this
possibly bad rename:

pkgs/htdp-pkgs/htdp-lib/2htdp/utest
--
pkgs/htdp-pkgs/htdp-test/2htdp/utest

Before the move, the utest directory was not considered a test by
the old distribution rules (since it wasn't in a tests directory),
and after the move it is in a *-test package and therefore probably
considered as tests that wouldn't be packaged in the release.

I *think* that it's wrong, because I vaguely remember Matthias saying
that he wants utest in the distribution, but I'm not sure.

[The more disturbing point is that dealing with two of these paths and
comparing it with the old paths made this thing ridiculously difficult
to notice.]

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


[racket-dev] git tag (was: package-system update)

2013-08-13 Thread Eli Barzilay
About a month ago, Matthew Flatt wrote:
 At Thu, 18 Jul 2013 09:11:28 +0200, Togan Muftuoglu wrote:
  When I check the current HEAD tag of the master this is what I get. Is
  this what it should be if not can it be fixed so it reflects reality?
  
 ~/devel/git-sources/racket $ git describe --tags HEAD
 v5.0.1-12863-g3c0b799
 
 I think this result reflects that we don't generally tag commits in the
 master branch. We tag releases, but those are normally commits (that
 were constructed via branches) that are not reachable from HEAD. It
 happens that v5.0.1 coincides with a commit reachable form HEAD (for
 reasons I don't know or remember), and so that've why you get a
 v5.0.1 release.
 
 Maybe it would be good to add a tag (more recent than v5.0.1) that
 doesn't include a version number, so it's not so misleading? Or is it
 worth adjusting our release process to introduce a tag that indicates
 when a release version branched from master?

The current system is to have a `stable' branch always pointing at the
most recently released version of the repository.  Then on releases a
`release' branch is made up based on `master'.  When the release is
done, `stable' is advanced to the point where `release' is, but this
is done in an explicit way that makes it point to the previous
`stable' as a parent -- so the two lines (`master' and `stable') are
kept in parallel.  This is why `git describe' looks weird -- as
Matthew said, it just finds some tag in the past that happens to be
the last one before this was started.

This was done due to some problems with 3rd party packages who follow
the repository -- I don't remember the details, but David Bremner was
the one pointing at the problem so he might be able to explain it.
(And if there's no problem, then it is probably better to not do that
anymore.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] A story of a package split

2013-08-13 Thread Eli Barzilay
40 minutes ago, Sam Tobin-Hochstadt wrote:
 On Tue, Aug 13, 2013 at 5:10 PM, Tony Garnock-Jones to...@ccs.neu.edu wrote:
  On 08/13/2013 05:05 PM, Nick Shelley wrote:
 
  I was mainly asking about the intermediate form because it seems
  like it could be useful, but I didn't understand how it would
  work.
 
  One major difference I've just spotted is that git submodules are
  tied to particular commit IDs, leading to detached-HEAD checkouts,
  where a looser approach can check out subrepos on real branches.
 
 Apparently this limitation has been lifted in the most recent git
 release.

Yes -- there is a `--remote' flag to `submodule update' which tells
git to grab whatever is current in the remote.  (On a branch that
eventually defaults to `master', but that can be configured.)  BTW,
there is also an `--init' flag which does the init if it wasn't done,
so if you use both:

git submodule update --init --remote

you get all of the submodules in, using their current state.  And you
can use `git submodule status' to see if some submodule was updated
away from the recorded sha1.  One more issue is that this will still
have all the sub-modules in a detached head state (which is arguably
something that --remote should take care of), so instead of all of
this, you can just go into every submodule and checkout its master
explicitly:

git submodule --init
git submodule foreach git checkout master


(But all of this should be mostly irrelevant if the repositries are
really used as repositories -- rather than trying to force it into
looking like you still have a single monolithic repository.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #27294: master branch updated

2013-08-11 Thread Eli Barzilay
6 hours ago, mfl...@racket-lang.org wrote:
 
 2ba615a Matthew Flatt mfl...@racket-lang.org 2013-08-10 19:12
 :
 | directory-list: always sort results
 |
 | Consistently sorting shouldn't cost much relative to the
 | cost of `directory-list' (except for the path-bytes conversion?),
 | and it makes directory traversals (including archive packaging)
 | more deterministic across runs and platforms.
 |
 | (Eli suggested this a long time ago.)

Yes, and as I said back then, with the new `path?' there's probably
no reason to want unsorted results.  [BTW, there's also only a minimal
cost for existing code that sorts the results (probably lots of
these), since `sort' does a check that the input is sorted -- however,
there is more significant cost since such code converts the paths to
strings/bytes and that's no longer needed...]

What I had in mind, due to the cost, was to have an argument that
specifies how things are sorted, so you could specify it as `values'.
Then I figured that it could also be used to do other kinds of sorts,
like put directories first, or even filter the results, which leads to
an argument that looks like:

(directory-list #:sort foo) = (foo (directory-list))

and that didn't seem right...

(As a sidenote, it would be nice to have some more standard globbing
support; I've written such things probably 2-3 times -- in the gui
racket-implemented file dialog, and in meta things like the bundle
script.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Package compatibility

2013-07-08 Thread Eli Barzilay
Yesterday, Jay McCarthy wrote:
 Planet attempts to solve this problem technically by (a) having all
 collections be prefixed by author/package-name and (b) mandating
 a centralized server that enforces unique authors and unique
 package-names per author. Since Racket packages don't have a
 mandated central server we can't enforce uniqueness like that. (i.e.
 even if we mandated the prefix by package-name, there's no way to
 enforce unique package names across the universe without a central
 server.) Furthermore, bringing package-names in to the code would
 give us /internal linking/ which Racket packages are designed to
 avoid for other reasons.
 
 Since technical solutions to this problem are lacking,

An easy way to make it a non problem would be to eliminate the extra
indirection that is in package names: if the package that Carl is
asking about is *named* data/red-black-tree then it clearly
conflicts with another package with the same name.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] What command should DrDr run by default?

2013-07-08 Thread Eli Barzilay
Yesterday, Robby Findler wrote:
 Raco test runs the file if there is not test sub module. Fwiw. 

That's what I thought first, with run being the same as what racket
does.  But it doesn't: it just runs the toplevel module as usual when
the file is required (and does so unconditionally, of course), and
then runs a `test' submodule if there is one.  In the files in
question, the tests are running in the `main' module, and raco test
was therefore not running them.

This seems like a fine behavior for raco test, but for drdr, it
*should* run a `main' module if there is one, and if there is no
`test' module.  This avoids the need to abuse a `test' submodule as
some semi-drdr-flag-thing, since files in some tests path shouldn't
be required to have a `test' submodule.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Package compatibility

2013-07-08 Thread Eli Barzilay
Just now, Jay McCarthy wrote:
 The problem with that is that there is no way to ensure that there
 is only one package named data/red-black-tree

There's no need to ensure such a thing -- and IIUC, the current system
doesn't do that neither modulo a bunch of blessed packages.


 and there can be two mutually incompatible universes of packages for
 Carl's rbts and mine, for instance.

This is also something that could happen now in the same way.  But I
don't see anything wrong with having this ability: I can hack my own
racket/list to do whatever I want, and therefore I should be able to
make a package that has this hacked version of the file.  I'd be
mostly living in my own world whether this hack is done with a package
system or directly.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] What command should DrDr run by default?

2013-07-06 Thread Eli Barzilay
It doesn't make much sense to use a test submodule in files that are
intended to be (only) tests.


Earlier today, Robby Findler wrote:
 That's the plan. Thanks for fixing them.
 
 Robby
 
 
 On Sat, Jul 6, 2013 at 8:47 AM, Sam Tobin-Hochstadt sa...@ccs.neu.edu wrote:
 
 I recently discovered that DrDr runs `raco test` by default on files,
 and has since February. However, the DrDr documentation says that it
 runs `racket -t`.  I discovered this because the change caused many of
 the `net` tests to no longer actually run; see
 
     http://drdr.racket-lang.org/27042/pkgs/racket-pkgs/racket-test/tests/
 net/url.rkt
 
 which appears to be successful, even though running `racket` on that
 file would have shown the error that I fixed in
 https://github.com/plt/racket/commit/a4e529a816 . These tests are a
 particularly bad case because they run all their tests in the `main`
 submodule, and thus are *not* executed by `raco test`. If the plan is
 for everything to run with `raco test`, I'll fix these tests.  But
 what is the plan?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #27090: master branch updated

2013-07-06 Thread Eli Barzilay
Yesterday, Sam Tobin-Hochstadt wrote:
 On Fri, Jul 5, 2013 at 6:50 PM,  e...@racket-lang.org wrote:
 
 
  f90fe4c Eli Barzilay e...@racket-lang.org 2013-07-05 18:08
  :
  | Get rid of the `#:function' keyword.
  |
  | These problems are always dealt with via an internal function instead of
  | making the name argument part of the visible API.
  :
M pkgs/racket-pkgs/racket-doc/json/json.scrbl | 10 ++--
M racket/lib/collects/json/main.rkt   | 33 
  +++---
 
 I think we should keep the name argument part of the API.  Imagine
 that I had a another function that used `read-json` or `write-json`
 internally, and I wanted the same good error messages that
 `jsexpr-bytes` has. Then I'd have to either perform string
 replacement on the exception, or create do something else equally
 ugly.

I disagree with keeping this (but not with solving it) for two
different reasons:

1. This is not a problem that is unique to this case, so a local
   solution is best avoided if there's a general one.

2. Generalizing this in some way, ie, adding `#:function' to functions
   that should fake their error messages is bad in exactly the way
   that justifies parameters.

It would be nice to have some way of doing it with contracts, but
that's obviously impractical -- so I think that a good way to solve it
and other such problems is to add a parameter for the name used in
read errors, and making `raise-read-error' behave in a similar way to
`raise-syntax-error' with the parameter used as the `foo:' part of the
error message.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] A grab-bag of issues

2013-07-05 Thread Eli Barzilay
Yesterday, Matthew Flatt wrote:
 At Thu, 4 Jul 2013 12:52:10 -0400, Sam Tobin-Hochstadt wrote:
 
  2. It's possible to get lots of undefined tags in the documentation
  builds.  Should this be remedied with more build dependencies, or
  will it be fixed with the online documentation server, or
  something else?
 
 Mostly through an online documentation server and declaring certain
 documentation links to be indirect (which may sometimes involve
 adjusting the documentation content).

If the documentation requires consulting a server, then why distribute
them in the first place?  There were a whole bunch of issues (most
notably a lack of a full text search) that are much easier to deal
with if the documentations are all on-line.


  5. GitHub recently introduced Releases, which allow you to serve
  arbitrary downloads. This might be useful for the plt/libs
  repository.  More info here:
  https://github.com/blog/1547-release-your-software . Even without
  this, we might want to switch to downloading native packages as
  zip files instead of requiring git to build from source.
 
 I imagined that we would having a catalog server for the
 native-library packages, but it's good to know about support for
 Releases from GitHub, and I'll look into that more.

AFAICT, there is not much in this, and you get releases by just
pushing tags -- so they're already there:

https://github.com/plt/libs/releases

(Also, serving individual files is fine from our git server.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #27071: master branch updated

2013-07-05 Thread Eli Barzilay
On Tuesday, sa...@racket-lang.org wrote:
 
 37c87ec Sam Tobin-Hochstadt sa...@racket-lang.org 2013-07-02 12:24
 :
 | Move json tests to racket-test pkg.
 :
   R {racket/lib/collects/json/tests = 
 pkgs/racket-pkgs/racket-test/tests/json}/json.rkt (100%)

Why did you do this move?  Having tests bee in the same directory as
the code makes it much easier to maintain, and I mean same directory
in both the place where the file gets installed and being in the same
package.

(In case it's not obvious where I'm coming from: when I wanted to test
the recent commit, I resorted to grepping the tree for the file -- I
view such grepping as indicators of organizational failures that
should not be needed even after the repo is split, and commits like
this make things worse in this regard.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] A grab-bag of issues

2013-07-05 Thread Eli Barzilay
A few minutes ago, Matthew Flatt wrote:
 At Fri, 5 Jul 2013 18:55:32 -0400, Eli Barzilay wrote:
  Yesterday, Matthew Flatt wrote:
   At Thu, 4 Jul 2013 12:52:10 -0400, Sam Tobin-Hochstadt wrote:
   
2. It's possible to get lots of undefined tags in the documentation
builds.  Should this be remedied with more build dependencies, or
will it be fixed with the online documentation server, or
something else?
   
   Mostly through an online documentation server and declaring certain
   documentation links to be indirect (which may sometimes involve
   adjusting the documentation content).
  
  If the documentation requires consulting a server, then why distribute
  them in the first place?
 
 Reading documentation does require a server. Following a link from
 installed documentation to uninstalled documentation requires a server.

OK (assuming a does not in the first sentence...).


  There were a whole bunch of issues (most notably a lack of a full
  text search) that are much easier to deal with if the
  documentations are all on-line.
 
 While I am interested in exploring installations where all
 documentation is online, there are also issues that seem harder to
 handle with exclusively online documentation (most notably, aspects
 of DrRacket's support for connecting code to documentation).

(I think that these should be easy with just having the same files on
the server, where drr can grab them and use them.  That's except for
jumping straight to some specific documentation, which would be done
by the server, probably via a url that is formatted according to the
source module etc.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #27093: master branch updated

2013-07-05 Thread Eli Barzilay
10 minutes ago, Sam Tobin-Hochstadt wrote:
 On Fri, Jul 5, 2013 at 8:21 PM,  e...@racket-lang.org wrote:
  +  ;; The following two tests are not really correct: they rely on the URL
  +  ;; decoding silently passing un-encoded text as is instead of barfing.  
  (Eg,
  +  ;; using these URLs in a browser and then copy-pasting it from the 
  address
  +  ;; should get you a properly encoded string instead.)
 
 This parenthetical is not really correct.  In particular, you cannot
 rely on what's displaying in the browser address bar to be a
 correctly-encoded URL -- the address bar is part of the user
 interface, and not a reflection of the internal state.

It doesn't talk about what is displayed.


 For some discussion on this point, see this thread:
 https://groups.google.com/d/topic/mozilla.dev.platform/V0XjJ-uQbcI/discussion
 
 In general, the whole URL parser should probably be changed/re-written
 to follow http://url.spec.whatwg.org/ but that's a longer-term
 project.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] checking package dependencies

2013-06-30 Thread Eli Barzilay
Yesterday, Matthew Flatt wrote:
 [...]

(Sidenote: it would be nice if `--fix-pkg-deps' or something similar
could also report redundant dependencies.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] another Planet2 Express memo

2013-06-28 Thread Eli Barzilay
A few minutes ago, Matthew Flatt wrote:
 
 To avoid problems, delete racket/lib/links.rktd and
 racket/lib/pkgs. [...]

Quick note: a useful thing to know for such cases is

git clean -dX -f

which will make git remove all files that are ignored.  Since the
.gitignore files are all setup to list built stuff, it should leave
you in a clean state with no pre-built stuff.  It will not delete
new files that are not ignored, so if there are new source files they
will be kept.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] `raco link', links.rktd, pkgs, and config.rktd

2013-06-27 Thread Eli Barzilay
Two hours ago, Matthew Flatt wrote:
 At Wed, 26 Jun 2013 01:04:13 -0400, Eli Barzilay wrote:
   Furthermore, config.rktd can provide a list of additional
   files/directories to search. This allows the main links.rktd
   and pkgs to act like /usr/lib things, while additional
   directories can act like /lib things.
  
  Can you explain this more?  I don't see the connection.
 
 Sorry, I meant to write /usr/local/lib (instead of /usr/lib) and
 /usr/lib (instead of /lib). It that more clear?

Yeah, that's what I hoped you meant...

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Package management

2013-06-27 Thread Eli Barzilay
An hour and a half ago, Sam Tobin-Hochstadt wrote:
 On Thu, Jun 27, 2013 at 3:17 PM, Matthew Flatt mfl...@cs.utah.edu wrote:
  The package system has a notion of auto packages, which are
  packages that were automatically installed to satisfy
  dependencies. Also, `raco pkg remove' supports an `--auto' flag
  for automatically removing auto packages that have no non-auto
  packages depending on them.
 
 Note that this is also how the deb package manager (on
 Debian/Ubuntu/etc) works, so we have good precedent here.

I think that all distros do that for a number of years now.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Things we could move out of the core

2013-06-26 Thread Eli Barzilay
(BTW, there's a file/private/octree-quantize which should move to
where file/gif is.)

An hour ago, Robby Findler wrote:
 The sandbox, IMO, is a nice standalone library the does not need to
 be in the core. (Ditto for errortrace.)
 
 I like the definition of the core as minimum stuff to get pkgs
 running and we should be picky about what goes in -- even to the
 point of rewriting some code to move other stuff out. 

This doesn't make sense if things at such a small size turn to
packages.  It's true that a package is a unit of utility for end
users, but it's also a unit of developement.  IMO there are far better
things to move out -- things that are bigger (in size and complexity)
like (on a brief scan) contracts, gui, match, places, units, json,
planet, xml, net, help.[*]


 Think of someone deploying a game on a phone: would they be happy
 with your rationale influencing their app's size?

I think that errortrace has been one of the biggest offenders in terms
of bad bugs of many varieties -- and moving it out won't help.  (And
if I were doing a game on a phone I'd worry about any of the above
much more, btw.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] `raco link', links.rktd, pkgs, and config.rktd

2013-06-25 Thread Eli Barzilay
8 hours ago, Matthew Flatt wrote:
 The config.rktd file in etc can now specify a location for the
 installation-wide links.rktd file and pkgs installed-package
 directory (with its pkgs.rktd).

On a clean build I get an empty etc directory -- explained by moving
the files that were in it -- but is it now a bug, or is it a
placeholder for an optional config file?  (Would there be anything
else there?)


 Furthermore, config.rktd can provide a list of additional
 files/directories to search. This allows the main links.rktd and
 pkgs to act like /usr/lib things, while additional directories
 can act like /lib things.

Can you explain this more?  I don't see the connection.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Things we could move out of the core

2013-06-25 Thread Eli Barzilay
Two.5 notes:

* IMO the sandbox and errortrace should stay in -- even if they're not
  needed, they are both intimately tied to the core so there wouldn't
  be any benefit from moving them to a package.

* I think that it makes sense to move compatibility into its own
  package, and move the mzlib stuff into that same package.

* Maybe the same for data?


9 hours ago, Sam Tobin-Hochstadt wrote:
 While moving some files around between packages, I realized that there
 are a number of things that could be moved out of the core and into
 packages.  Here's a partial list of things that I think are not needed
 at all by the rest of the core:
 
 - racket/sandbox and the rest of the sandbox code
 - data/{union-find, interval-map, splay-tree, skip-list, order, gvector, heap}
 - srfi/4
 - unstable/{options, contract}
 - errortrace (once sandbox is removed)
 - compatibility/*
 - mzlib/{pconvert, class100, serialize, thread, transcr}
 - a bunch of other parts of mzlib that are just reprovides
 
 The following are libraries that could be removed with (I believe)
 small changes to the rest of the core:
 
 - rackunit
 - srfi/13
 - mzlib/unit200 (maybe, this might be hard)
 - mzlib/kw
 
 If this seems like a good idea to people, I'm happy to go ahead and
 start doing the work.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Mangaging `unstable` with packages

2013-06-24 Thread Eli Barzilay
On Friday, Sam Tobin-Hochstadt wrote:
 
 # `unstable/list`
 
 - `remf`, `list-update` `list-set` `map/values`: move to
   `racket/list`
 - `group-by`: rename to `group`, add keyword argument `#:by`
 defaulting to `equal?`, move to `racket/list`

All of these are used in the tree only by their authors, and almost
all of them are used between zero and time.

`remf' is the same as `filter-not'.

`map/values' can be expressed in most cases more conveniently with
`for/fold'.

`group-by' is doing something weird -- see for example clojure's
function by the same name that does something that looks more useful
(IMO).  Also, `group' sounds way too generic for something as specific
as what it does.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] Mangaging `unstable` with packages

2013-06-24 Thread Eli Barzilay
A few minutes ago, Sam Tobin-Hochstadt wrote:
 On Mon, Jun 24, 2013 at 3:52 AM, Eli Barzilay e...@barzilay.org wrote:
  On Friday, Sam Tobin-Hochstadt wrote:
 
  # `unstable/list`
 
  - `remf`, `list-update` `list-set` `map/values`: move to
`racket/list`
  - `group-by`: rename to `group`, add keyword argument `#:by`
  defaulting to `equal?`, move to `racket/list`
 
  All of these are used in the tree only by their authors, and almost
  all of them are used between zero and time.
 
  `remf' is the same as `filter-not'.
 
 This is false, as the example in the `remf` docs shows.

s/same/almost the \1/

[We already went over this: I don't find the remove-first
functionality too useful (I use (remove* (list x) l) *very*
frequently), but with a function a `remf' that is not a `refm*' is
even less useful and more surprising.]

  `map/values' can be expressed in most cases more conveniently with
  `for/fold'.
 
 I've used `map/values`, and I'm not its author. Also, it's a
 function that I've seen requested for years.

(OK, sam added from carl; but you're still the only client in the
tree.)


  `group-by' is doing something weird -- see for example clojure's
  function by the same name that does something that looks more
  useful (IMO).  Also, `group' sounds way too generic for something
  as specific as what it does.
 
 `group-by`, and my proposed `group`, is what Haskell provides.  I
 don't find the Clojure version nearly as compelling, since it
 combines a map and a grouping operation.

Haskell does something more tame, and more predictable in the sense of
splitting an ordered list.  And yes, it looks like a good idea to have
*that*.

To be clear, the list-splitting functionality that I'd like to have:

* Split a list to lists of length N (IIRC, clojure has some
  generalization where you specify an interval length + chunk length,
  so if you use 1,2 you get something like (x y z) = ((x y) (y z)).)

* Split the list by some predicate over the elements, common use: take
  a list of strings and split it over \n items.  Clojure has
  `partition-by' which is close, and the Haskell group-by can probably
  be used too (but possibly not conveniently enough to eliminate the
  need for what I want).

(And as a sidenote, I agree with you that the clojure `group-by' is
not too useful -- I think that this applies to many list functions
that pretend that their input is a set.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #26989: master branch updated

2013-06-20 Thread Eli Barzilay
Yesterday, Eli Barzilay wrote:
 
 [...] some script that finds the recent rename of all files and make
 sure that it produces an expected result.

Suspicious undetected renames (excluding info files, git files, etc):

  pkgs/drracket/drracket/acks.rkt
  pkgs/drracket/drracket/drracket.extreg
  pkgs/drracket/drracket/drracket.startmenu
  pkgs/drracket/help/bug-report.scrbl
  pkgs/drracket/setup/plt-installer.scrbl
  pkgs/drracket/version/tool.scrbl
  pkgs/gui-pkgs/gui-lib/mred/installer.rkt
  pkgs/gui-pkgs/gui-lib/racket/gui/installer.rkt
  pkgs/htdp/scribblings/htdp-langs/htdp-ptr.scrbl
  pkgs/racket-pkgs/racket-doc/file/scribblings/gif.scrbl
  pkgs/racket-pkgs/racket-doc/scribblings/raco/config.scrbl
  pkgs/racket-pkgs/racket-index/scribblings/main/acks.scrbl
  racket/lib/collects/compiler/private/configdir.rkt
  racket/lib/collects/raco/main.extreg
  racket/lib/collects/setup/private/setup-relative.rkt
  racket/lib/collects/setup/winstrip.rkt
  racket/src/racket/collects-path.rkt
  racket/src/racket/src/cstartup.inc


Going from the other direction, these are the files that existed before
the rename commit that do not appear anywhere in the lists of rename
chains:

  collects/config/.gitignore
  collects/config/config.rkt
  collects/config/config.scrbl
  collects/config/info.rkt
  collects/config/main.rkt
  collects/file/scribblings/gif.scrbl
  collects/games/paint-by-numbers/solution-sets/directory
  collects/images/info.rkt
  collects/meta/pkg-index/planet-compat/.gitignore
  collects/mred/installer.rkt
  collects/mzscheme/info.rkt
  collects/racket/gui/installer.rkt
  collects/racket/place/distributed/info.rkt
  collects/rackunit/info.rkt
  collects/scheme/signature/lang/reader.rkt
  collects/scribblings/main/acks.scrbl
  collects/setup/configtab.rkt
  collects/slideshow/info.rkt
  collects/teachpack/data.csv
  collects/tests/datalog/paren-examples/ancestor.txt
  collects/tests/datalog/paren-examples/bidipath.txt
  collects/tests/datalog/paren-examples/empty.txt
  collects/tests/datalog/paren-examples/laps.txt
  collects/tests/datalog/paren-examples/long.txt
  collects/tests/datalog/paren-examples/path.txt
  collects/tests/datalog/paren-examples/pq.txt
  collects/tests/datalog/paren-examples/revpath.txt
  collects/tests/datalog/paren-examples/says.txt
  collects/tests/datalog/paren-examples/sym.txt
  collects/tests/datalog/paren-examples/true.txt
  collects/tests/eopl/chapter3/proc-lang/proc-rep/lang.rkt
  collects/tests/eopl/chapter4/call-by-reference/pairval1.rkt
  collects/tests/eopl/chapter4/call-by-reference/pairval2.rkt
  collects/tests/eopl/chapter4/call-by-reference/pairvals.rkt
  collects/tests/eopl/chapter4/implicit-refs/store.rkt
  collects/tests/eopl/chapter4/mutable-pairs/pairval2.rkt
  collects/tests/eopl/chapter4/mutable-pairs/pairvals.rkt
  collects/tests/eopl/chapter5/letrec-lang/environments.rkt
  collects/tests/eopl/chapter5/letrec-lang/lang.rkt
  collects/tests/eopl/chapter5/thread-lang/store.rkt
  collects/tests/eopl/chapter6/cps-side-effects-lang/store.rkt
  collects/tests/eopl/chapter7/inferred/data-structures.rkt
  collects/tests/eopl/chapter8/full-system/data-structures.rkt
  collects/tests/eopl/chapter8/full-system/environments.rkt
  collects/tests/eopl/chapter8/simplemodules/checker.rkt
  collects/tests/eopl/chapter8/simplemodules/data-structures.rkt
  collects/tests/eopl/chapter8/simplemodules/environments.rkt
  collects/tests/eopl/chapter9/classes/store.rkt
  collects/tests/eopl/chapter9/typed-oo/store.rkt
  collects/tests/pkg/test-pkgs/pkg-a-third/pkg-a/main.rkt
  collects/tests/pkg/test-pkgs/pkg-test1-manifest-error/pkg-test1/main.rkt
  collects/tests/pkg/test-pkgs/pkg-test1-v2/pkg-test1/conflict.rkt
  collects/tests/pkg/test-pkgs/pkg-test1-v2/pkg-test1/main.rkt
  collects/tests/pkg/test-pkgs/pkg-test1/pkg-test1/conflict.rkt
  collects/tests/pkg/test-pkgs/pkg-test1/pkg-test1/main.rkt
  collects/tests/pkg/test-pkgs/pkg-test1/pkg-test1/update.rkt
  collects/tests/pkg/test-pkgs/pkg-test3-v2/info.rkt
  collects/tests/racket/benchmarks/shootout/typed/nbody-vec-non-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nbody-vec-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nestedloop-non-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nestedloop-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nothing-non-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nothing-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nsieve-non-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nsieve-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nsievebits-non-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/nsievebits-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/partialsums-non-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/partialsums-optimizing.rkt
  collects/tests/racket/benchmarks/shootout/typed/pidigits-non

Re: [racket-dev] [plt] Push #26989: master branch updated

2013-06-20 Thread Eli Barzilay
An hour and a half ago, Matthew Flatt wrote:
 
 [...]

(Sorry, I meant it to just be informative, not to argue a point...  I
should have written a conclusion that it's fine to not redo the
reorganization commit.)


pkgs/gui-pkgs/gui-lib/mred/installer.rkt
pkgs/gui-pkgs/gui-lib/racket/gui/installer.rkt
 
 This strikes me as actual history loss (due to many changes in a
 small file).

There could be a similar loss of lots of useful stuff: I'm guessing
that much of the stuff in pkgs/plt-services/meta/build will go away
soon, and if it's deleted before the repository split then it would be
lost -- and there are lots of small edits due of subtle OS related
things that would be very useful to keep around.

  Going from the other direction, these are the files that existed
  before the rename commit that do not appear anywhere in the lists
  of rename chains:
 
 Some are truly gone, and some are covered above. I don't understand
 why most of them are in the list, though, since `git follow --log'
 on each new location works for me (for the handful that I tried).

What I did was run a log with --follow over each file in the
repository, collect the results with these name histories in one file,
and the paths in this least are ones that don't appear anywhere in
this resulting file.  (I can put the file out somewhere if it'll help;
it took a long time to produce.)


  I didn't do more of this because I realized that this is not too
  useful now, with most (or all) of the bad detections happening in
  the past.
 
 Well, exactly.

(I think that these bad detections should be addressed, but not in
this commit.)


 So, for the big reorganization of the tree, we seem to be losing
 history (that separate move and change commits would clearly
 preserve) on only a couple of 40-line files. I am more than
 satisfied with that result, compared to what we get from day-to-day
 use of git --- and particularly compared to the pain of trying to
 stage moves before edits in the reorganization.

My worry about this commit is mostly gone.  There are some problematic
places but it's obviously better than past commits.  So no need to
re-do it (this commit).  I still disagree on the pains of moving
before splitting -- but not strongly and definitely not worth it to
discuss it again.

--

But one thing that I think is important is to still do the manual
thing: extract all the information before the split in some human
read-writeable way, manually go over it and identify proper renames
for all the obvious cases, then do the split based on this edited
information.  For example, the whole history of collects/meta/build
would be preserved in the distro-build even when the files are not
actually there.

And BTW, I can obviously do much of this work when the time for a
split comes.  It's going to be very tedious, but my guess is that for
anyone else it will be even more tedious and error prone.  (And BTW#2,
it's still something that I'm doing mostly because I got burned in the
past when I suggested and/or did things that lost history
information.)

But again: this is all for the future split, not for now, and I'm
definitely not saying that the reorganization should not be done.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] ready for the package switch?

2013-06-19 Thread Eli Barzilay
Yesterday, Carl Eastlund wrote:
 On Tue, Jun 18, 2013 at 2:02 PM, Eli Barzilay e...@barzilay.org wrote:
 
 20 minutes ago, Carl Eastlund wrote:
  [...]
 
    git rebase -s recursive -X rename-threshold=50% mflatt/pkg2
 
 From a brief reading, I think that you're much better off with
 -X subtree=/some/path.  There's also a subtree strategy, which
 you'd get with -s subtree, but it is guessing how to do the
 path shifting.
 
 I thought that too, but it didn't work when I tried it.  Some of my
 files need to go in racket/lib, some in pkgs/racket-tests, and I
 think there might have been a third place.  So unless everything is
 uniformly going to the same place, the subtree options don't help.

In that case, I think that you should rebase each commit based on
where it should go, and possibly split commits that touch different
paths.

At a higher level, things like this point to possible mistakes in
package splitting which should be fixed.  In some cases they might
mean that a different split should happen, but in the case of the
racket tests I think that it's far too broad and should be taken as a
temporary package until all tests move out into their proper places.
Maybe Matthew decided to make it a separate package because of
this.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] ready for the package switch?

2013-06-19 Thread Eli Barzilay
Yesterday, Matthew Flatt wrote:
 
 I don't think that will work right in the long run. There's
 currently no easy query to get all of the subcollections of a given
 collection.
 
 Instead of looking for subcollections of games, I think
 `games/main' should find all info.rkts that have a `game'
 definition (via `find-relevant-directories').

+8 for this, and also I think that it applies for other path-based
conventions like tests and scribblings.  (And that leads me back to
the specification of these things in info files, of course.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #26989: master branch updated

2013-06-19 Thread Eli Barzilay
Four hours ago, mfl...@racket-lang.org wrote:
   D collects/2htdp/image.rkt
   D collects/algol60/info.rkt
   D collects/browser/info.rkt
   D collects/compiler/embed-unit.rkt
   D collects/compiler/find-exe.rkt
   D collects/config/config.rkt
   D collects/config/config.scrbl
   [...]

Note that there are a *lot* of files here that were not detected as
renames, so as I described previously, that's a whole bunch of history
that is likely to be lost.  (It might also be due to different
parameters that are used indirectly by the email script to produce
these reports.)

Do you still have a list of renames that were done?  If you do, then I
think that it's best to take the repo off-line, undo the commit, then
redo it in two parts -- the renames and the changes, and finally
force-push the result.  Normally that would be a problem, but now the
forced push shouldn't be too problematic.

Without a list of renames, I can probably think of ways to automate
much of the work, but it will still be a considerable effort to do it
all.  Also, not doing it now means that it will need to be done later
if we care for keeping as much of the history as possible -- and the
more changes are committed, the more difficult it will be to fix it
later.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] [plt] Push #26989: master branch updated

2013-06-19 Thread Eli Barzilay
30 minutes ago, Matthew Flatt wrote:
 At Wed, 19 Jun 2013 15:54:00 -0400, Eli Barzilay wrote:
  Four hours ago, mfl...@racket-lang.org wrote:
 D collects/2htdp/image.rkt
 D collects/algol60/info.rkt
 D collects/browser/info.rkt
 D collects/compiler/embed-unit.rkt
 D collects/compiler/find-exe.rkt
 D collects/config/config.rkt
 D collects/config/config.scrbl
 [...]
  
  Note that there are a *lot* of files here that were not detected as
  renames,
 
 When you use `git log --follow' on the destination file, do you get
 the expected history?
 
 It seems to work right for me the first five files above. The sixth
 and seventh really are gone.
 
 I haven't been able to find a file where `git log --follow' fails to
 do what I want, so far (not just right now, but in all of my
 experiments to date).

These were just the first few -- I checked a few of the D files that
have a corresponding A, and just sent it out with that.  I'll try
looking more into it now -- probably some script that finds the recent
rename of all files and make sure that it produces an expected result.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] ready for the package switch?

2013-06-18 Thread Eli Barzilay
20 minutes ago, Carl Eastlund wrote:
 [...]
 
   git rebase -s recursive -X rename-threshold=50% mflatt/pkg2

From a brief reading, I think that you're much better off with
-X subtree=/some/path.  There's also a subtree strategy, which you'd
get with -s subtree, but it is guessing how to do the path shifting.


 rantI don't understand why version control systems don't take
 directories and renames more seriously, because this stuff is part
 of the development cycle and should be recorded like any other
 change./rant

Meta: JFYI, not only does git not track renames, it doesn't do that
*intentionally*.  The exact arguments for/against are irrelevant, but
the main point is that the decision was made very deliberately, and
therefore you can't really accuse them of not taking it seriously
enough even if you disagree with the decision.

Concrete: I said this many times, and it's important to remind people
of this especially now -- because git relies on detecting renames, it
is *MUCH* better to commit just renames, without changes to file
contents.  Yes -- even if that means that there's a commit with an
uncompilable tree, the potential minor hassle to future bisecters is
smaller than the potential to bad rename detection damages.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-06-04 Thread Eli Barzilay
Yesterday, Matthew Flatt wrote:
 At Mon, 3 Jun 2013 10:36:51 -0400, Eli Barzilay wrote:
  (BTW, a possible source of confusion: I'm assuming that
  distribution must be done via archives and not via repository
  specs, since there should be some way to put the compiled files in
  there.
 
 I don't think that's the right assumption. We want it to have
 packages that combine source with built entities, but I don't think
 we want to require it. Requiring it would break the way GitHub
 repositories are meant to be used as packages.

But they are required for all non-source things anyway, right?  And
given that the plt packages have binary distributions, then they
should be someplace else anyway.

If the sources *do* come from the repository, how does it get a
specific version that corresponds to whatever was distributed in
binary form?  If I have v1.2.3 installed, and it came with web server
as precompiled zo files, and I install the source -- how do I get that
specific version of the source?  The only thing I can think of is
relying on tags but that's a bad assumption since it assumes that
package owners keep and maintain the proper tags, and the release
should come with adding tags in all of these repos -- and that
contradicts letting people do whatever they want to do.

This is going further off-current-topic, but when I talked to Ryan
about possibilities for revising the release process, I thought that
the thing that makes most sense is that at the branch creation time
(which still happens as usual in the core package repo), package
owners hand over a sha1 that corresponds to their recent version.
That gets used by the build (possibly via a meta-repo that has these
sha1s as submodule specifications) script which gets these versions to
build the packages.  The equivalent of me making a merge this commit
request is now turning into asking Ryan to change my sha1 to a newer
one, and Ryan can now use a diff between the two sha1s to review the
changes that I made.  The main point here is that it makes the package
repositories be completely independent: package maintainers can keep a
`release' branch and release tags if they want, or they can avoid the
whole thing.  And in fact I think that the much more common case would
be to not having a release branch etc.

But going back to the on-topic point: this whole plan assumes that the
sha1 is what gets used to pull the content and build zipped package
archives.


  You've mentioned at some point a binary catalog which seems
  unnecessary to me, but maybe there's something I don't get here.)
 
 Yes, binary packages are another way to address the underlying
 issue.
 
 I'm going to try to synthesize the progress and discussion so far in
 a new message, and hopefully I'll managed to explain binary packages
 this time around.

It would be good to take the above into account, as another reason for
doing this which is independent of actual binary files.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-04 Thread Eli Barzilay
Yesterday, Jay McCarthy wrote:
 and you should deal with the non-proof of concept method of
 specifying it in, for instance, the info file, which is now package
 info AND collect info.

This shouldn't be a problem -- *most* of the information in the info
files that we have now is really information that applies to packages.
The only thing that I could find that doesn't is compile-omit-paths.
In fact, as I said in the past, the creation of a new info file is the
thing that is problematic.

What if, for example, there is a `name' entry in both a package info
file and in (one of) its collections?  The first one would supposedly
be used when talking about the package and the second one would be
used in setup to indicate the compiled collection...  Or what happens
with the `version' entry in some of the existing collections?  My
guess is that the answer to both is that there shouldn't be entries
that exist in both info files (eg, the current `version' fields in the
collection info files should go away) -- and that shows that there
shouldn't be ambiguity problems.


Yesterday, Matthew Flatt wrote:
 
 Meanwhile, as we've started leaning on the package system and looked
 at how to fit our current collections into it, I think we're running
 into the same problem at another level: if web-server-lib and
 web-server-doc are both packages in a single web-server
 repository, and if they have to be separate directories, then we
 have yet another layer of directories that affect repository layout.

Yes, that's related, and part of my motivation to avoid that level of
splitting.  I think that telling people that they should start their
hacking work with

  git clone pltgit:pkg/web-server
  edit web-server/web-server-lib/web-server/web-server.rkt

is making it obvious that there's some problem here...


 Maybe there's a more general answer to the directory-layout
 problem that would neatly cover single-collection packages.

I originally thought (/hoped) that you would be able to place packages
at any level.  Going back to the info files -- this is what happens
now, roughly -- you just specify an info file anywhere in the tree and
it gets picked up as something very close to packages.  (For example,
the game info files, or drr tools in nested directories.)


Three hours ago, Laurent wrote:
 
 Actually I use collection links instead of package links for my
 single-collection packages, which are Github clones. It kind of
 makes sense.  (One more homogeneous possibility would be to add a
 `raco pkg install --link-single my-collect/package` command that
 is just a synonym for `raco link my-collect/package`.

Yes, that's what I thought should happen -- that installed packages
get linked as a single collection when they have a single collection.


 Actually I would much prefer to have `raco pkg install --link` work
 like `raco link` and have an additional `raco pkg install
 --link-multi` command instead.)

+173, with --root being used for both.


 - When all you need is a file, just use a file. It should have all
 the necessary information (i.e., why not have a 'info' submodule?
 E.g. to deal with dependencies)

(This would be nice, but won't work now, since there's no way to
require toplevel files.  But that's fine since it corresponds to the
fact that you can't have a repository without a directory to hold it.
So I view an `in-url' thing is the way for this super-cheap code
distribution.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-06-03 Thread Eli Barzilay
On Thursday, Matthew Flatt wrote:
 
 You've sketched out the producer side, and I'm not sure of some
 about some of those details.  The consumer side seems even more
 complex to me.  It seems like the package system would have to keep
 track of which subpackages are installed for a package, provide an
 interface to the user (e.g., in the GUI) for subpackages, and be
 able to update a package with new subpackages --- all while tracking
 dependencies at the level of subpackages. I'm sure it can all be
 done, but I'm not sure how difficult or important it will be
 relative to everything else that still needs to be done, and I'm
 pretty sure it can't all be done right now.

A very cheap way to do this is to use these sub-package specifications
only for creating packages for distribution.  With the obvious
resulting package file names, this means that there is no change at
all that is needed on the consumer side.  And I think that while it
looks like it reduces the whole thing considerably, it's still worth
it for some important advantages that are still kept:

* No need to move files around in a distribution-oriented layout in
  code repositories (which is important since people really maintain
  the freedom of puting files in a way that fits their needs
  logically).

* No restriction on a few known -foo names -- keeping it open for
  new kinds of package kinds.  (Related note: I spoke to Ryan about
  having db in the minimal core and the possible reason for this being
  due to sqlite -- it would be nice, and according to Ryan not too
  difficult, to split the db package into several db-foo backends,
  which hooks back into my point of some splittings that make sense in
  just one package.)

* Minimize code hacks that compensate for the lack of such
  sub-packages.  There will still be some hacks -- for example, it
  will be convenient in the beginning to just show available package
  names and the end user will need to figure out what they mean, and
  when later on you want to grouping back packages that really came
  from a single source for the UI, instead of some hack that is based
  on names, it can use this information instead.

All of these are important IMO, but the last one is particularly good,
since hacking something based on naming conventions means that these
hacks will stay around for a very long time -- which will lead to yet
more hacks.  In the spirit of making things usable, I also worry about
accumulation of command-line flags, a problem that plagued .plt file
generation and installation (making it into black magic that not many
people could handle).


 Instead of trying to create subpackages, we might stick with a
 naming convention. Even with a naming convention, it might make
 sense to extend the package-manager GUI so that it recognizes the
 convention and groups related packages together in a list view (but
 all the machinery remains at the package level).

(This is an example for the kind of hacks where some of them can be
avoided with the above, and with others I think that it will be easier
to recover proper code after a transition period with such hacks.)


 Also, a package implementor is free to develop subpackages along
 the lines you suggest, but splitting out the subpackages into
 packages before handling them to a catalog.

(And having scripts to do such splitting by developers would not be
needed at all.)


 Or, we could just not try to slice packages as finely for now, so
 that the -lib and -docs packages of the experimental split are
 merged together --- leaving a future split possibly to subpackages.
 That will most likely result in a coarse layering that is close to
 build-deps up to the drracket layer, but still provide more
 separation for things that are outside the layer needed by DrRacket
 itself; for example, games, redex, htdp, plai,
 picturing-programs, and plot can still be separate packages, I
 think.

FWIW, I'd view this (avoid fine slicing) as a possible way to avoid
growing hacks, but I really like the idea of smaller packages and the
benefits it comes with.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Eli Barzilay
Yesterday, Laurent wrote:
 On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote:
 
 To clarify, because of reasons that I won't go into on the list,
 the actual chances of me getting this implemented (and of such a
 change being accepted) are pretty much in the area of slim to
 none.  
 
 That's a bummer. At first sight I'd have thought that it would have
 just been a matter of creating a directory with the same name as the
 package, and then decompressing the package into that directory.

At the lower level, packages use links, and raco link's default mode
of work is at the collection level.  There is obviously a need for
packages that correspond to collection roots, but I wanted to see the
default being the same as the raco-link default (which fits most uses
that don't come out of the current main tree).


 TBH, the chances of implementing an `in-url' are low too, but
 they're probably higher than going into the package system.
 
 That would be nice, but I'd much much prefer to see the
 single-collection package thing implemented.

Well, the reason I think that plain collection packages are important
is it will simplify dealing with packages, and lower the investment
threshold that you need to publish a package.  Along the same line,
there should also be a way to put out just a single file (or very few)
as a first step where I make some code public but with absolutely
minimum effort on the publisher's side.  See for example the
Processing thread, where Sean said that people who do these kind of
thing don't even know what source control is, let alone packages --
so the existence of such a crowd is a perfect reason for a single-file
thing being another important point on the code distribution spectrum.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] member like assoc

2013-06-03 Thread Eli Barzilay
+1, and maybe also add a racket2 item to get rid of all of the
unnecessary functions?  (I think that member/assoc/etc are better with
`eq?' than using memq/assq/etc.)


Two days ago, Matthias Felleisen wrote:
 
 +1, do it. 
 
 On May 31, 2013, at 7:40 PM, Asumu Takikawa wrote:
 
  Hi all,
  
  Is it feasible to get `member` to have the same optional argument
  behavior as `assoc`? That is, to have an equality predicate as the third
  argument.
  
  I find myself writing the (imaginary) equivalent of things like:
   (member id some-list free-identifier=?)
  
  and it seems like it would be nicer with an extra `member` argument than
  with `memf`.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-06-03 Thread Eli Barzilay
50 minutes ago, Matthew Flatt wrote:
 At Mon, 3 Jun 2013 08:27:19 -0400, Eli Barzilay wrote:
  
  A very cheap way to do this is to use these sub-package
  specifications only for creating packages for distribution.  With
  the obvious resulting package file names, this means that there is
  no change at all that is needed on the consumer side.
 
 I don't understand the suggestion. As a concrete example, can you
 sketch our how a user installs the web-server package without
 documentation, and how the package manager later knows to upgrade
 the web-server package to include documentation when requested by
 the user?

On the building side, you'd run some command/script/whatever that goes
over the distribution kinds and generates a package for each -- so if
you have

  (define distributions
'([lib *.rkt]
  [doc *.scrbl scribblings]))

the result will be web-server-lib.zip, web-server-doc.zip, and
web-server.zip where the last one contains dependencies on the other
two.  (So this last one is the only one that is special since it's
completely made up.)  Then all of these files will get distributed in
the same way they would be with the directory/repo split.

(BTW, a possible source of confusion: I'm assuming that distribution
must be done via archives and not via repository specs, since there
should be some way to put the compiled files in there.  You've
mentioned at some point a binary catalog which seems unnecessary to
me, but maybe there's something I don't get here.)

Updates would happen in a similar way, with new zip files.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-03 Thread Eli Barzilay
(I completely agree with you, so I'll take it off-line.)


30 minutes ago, Laurent wrote:
 On Mon, Jun 3, 2013 at 2:44 PM, Eli Barzilay e...@barzilay.org wrote:
 
 Yesterday, Laurent wrote:
  On Sun, Jun 2, 2013 at 1:47 PM, Eli Barzilay e...@barzilay.org wrote:
 
      To clarify, because of reasons that I won't go into on the list,
      the actual chances of me getting this implemented (and of such a
      change being accepted) are pretty much in the area of slim to
      none.  
 
  That's a bummer. At first sight I'd have thought that it would have
  just been a matter of creating a directory with the same name as the
  package, and then decompressing the package into that directory.

 At the lower level, packages use links, and raco link's default mode
 of work is at the collection level.  There is obviously a need for
 packages that correspond to collection roots, but I wanted to see the
 default being the same as the raco-link default (which fits most uses
 that don't come out of the current main tree).
 
 Not sure I was clear, or I don't exactly understand what you mean.
 And maybe I'm missing some important point here.
 
 But the more I think about it, the less I see why this
 double-directory thing should be the default behavior of the new package
 system (I can understand why it's useful for plt packages though).
 Even Carl's mischief package doesn't make use of collections like
 that.
 Jen's this-and-that does, though, but then every collection of his
 package must have a name that is different from any existing collection.
 (And actually I think that's a bad idea, and that he should split his
 package into several single-collection packages −unless they are
 tightly related− because I don't want to install all his collections if I want
 to only try one of them, unless there is a way to install only one
 collection of an existing package?)
 
 For example, my personal racket project tree is arranged somewhat like
 that:
 racket
 ├── project-B
 │   └── main.rkt
 ├── project-A
 │   └── main.rkt
 ├── not-yet(?)a-project-C
 ├── my-collects
 │   ├── slideshow
 │   ├── scribble
 │   ├── racket
 │   └── data
 └── misc
     ├── try-this.rkt
     └── try-that.rkt
 
 project-A and project-B are git clones.
 my-collects contains all my extensions to racket's collections,
 plus some more generally useful collections.
 
 For my personal use, I just do:
   raco link project-A project-B my-collects
 and all is fine.
 I tend to believe it's a quite common tree for racket users.
 
 In my-collects, I have names that would collide with racket's collections,
 so instead of having:
 (require orseau/racket/slideshow)
 as in the former package system, I would now have
 (require some-funny-name/slideshow)
 after renaming my-collects to some-funny-name to make sure it does not
 collide with anything else.
 
 ... and since I don't want to invent a funny name for each extension of a
 racket collection I do, I prefer to put them all in a single package and
 collection.
 
 Now if I were to follow the new package system, I'd need to change my
 tree to:
 racket
 ├── misc
 │   ├── try-that.rkt
 │   └── try-this.rkt
 ├── some-funny-name
 │   └── some-funny-name
 │   ├── data
 │   ├── racket
 │   ├── scribble
 │   └── slideshow
 ├── not-yet(?)a-project-D
 ├── project-A
 │   └── project-A
 │   └── main.rkt
 └── project-B
     └── project-B
     └── main.rkt
 
 which is a mess. And I would also need to change my git repos to
 match that structure, which looks awkward.
 /But/ I don't care if the tree of the packages I *install* look like that.
 
 So what I would like is to keep my tree like the first one, but the
 package system can still install like the second one.
 
 My proposal was that for single-collection packages, I could keep
 my tree like the first one, my git clones like project-A/main.rkt, and
 tell the package manager (probably in the info.rkt file) that it is a
 single-collection package so that when it installs it in the
 installed-package-directory-that-I-will-not-look-into it generates a tree
 like the second one.
 
 Well, the reason I think that plain collection packages are important
 is it will simplify dealing with packages, and lower the investment
 threshold that you need to publish a package.  Along the same line,
 there should also be a way to put out just a single file (or very few)
 as a first step where I make some code public but with absolutely
 minimum effort on the publisher's side.  See for example the
 Processing thread, where Sean said that people who do these kind of
 thing don't even know what source control is, let alone packages --
 so the existence of such a crowd is a perfect reason for a single-file
 thing being another important point on the code distribution spectrum.
 
 I entirely agree with that, and I think it is of utmost importance for a
 programming

Re: [racket-dev] [plt] Push #26926: master branch updated

2013-06-03 Thread Eli Barzilay
Ah, so this is what made the scribble/reader tests fail -- thanks for
the fix.  But there's still a possible problem:

v5.3.4:
- (raise-read-error foo 'src 1 2 3 4)
; src:1:2: foo [,bt for context]

Before this commit:
- (raise-read-error foo 'src 1 2 3 4)
; #procedure:...ntax/readerr.rkt:33:21foo [,bt for context]

And now:
- (raise-read-error foo 'src 1 2 3 4)
; src:1:1: foo [,bt for context]

where the column number is different from the first run.



20 minutes ago, ro...@racket-lang.org wrote:
 robby has updated `master' from 54b45607a2 to 1f22800d51.
   http://git.racket-lang.org/plt/54b45607a2..1f22800d51
 
 =[ One Commit ]=
 Directory summary:
9.8% collects/syntax/
   90.1% collects/tests/syntax/
 
 ~~
 
 1f22800 Robby Findler ro...@racket-lang.org 2013-06-03 08:38
 :
 | fix bug introduced in 6b2a4ff5
 :
   M collects/syntax/readerr.rkt | 1 +
   A collects/tests/syntax/test-readerr.rkt
 
 =[ Overall Diff ]===
 
 collects/syntax/readerr.rkt
 ~~~
 --- OLD/collects/syntax/readerr.rkt
 +++ NEW/collects/syntax/readerr.rkt
 @@ -30,6 +30,7 @@
(format ~a~a
(cond [(not (error-print-source-location)) ]
  [(srcloc-string (srcloc source-name line col pos span))
 + =
   (lambda (s)
 (format ~a:  s))]
  [else ])
 
 collects/tests/syntax/test-readerr.rkt
 ~~
 --- /dev/null
 +++ NEW/collects/tests/syntax/test-readerr.rkt
 @@ -0,0 +1,9 @@
 +#lang racket/base
 +(require rackunit syntax/readerr)
 +
 +(check-exn
 + (λ (x) 
 +   (and (exn:fail:read:eof? x)
 +(regexp-match #rx^y[01: ]* x (exn-message x
 + (λ () (raise-read-eof-error
 +x y 1 1 1 1)))

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-06-02 Thread Eli Barzilay
To clarify, because of reasons that I won't go into on the list, the
actual chances of me getting this implemented (and of such a change
being accepted) are pretty much in the area of slim to none.  TBH,
the chances of implementing an `in-url' are low too, but they're
probably higher than going into the package system.


40 minutes ago, Laurent wrote:
 Ah, that's cool. Looking forward to it!
 And the in-url thing would be useful indeed for gists for example.
 
 Laurent
 
 On Thu, May 30, 2013 at 8:32 PM, Eli Barzilay e...@barzilay.org wrote:
 
 Yes, I really want to try and get to look into doing this.  The
 thing is that multi-collection libraries are going to be a
 common case for plt packages (which will get pulled out from the
 main repository), but the single-collection ones are going to be
 the much more popular case elsewhere.

 (And I also still dream on doing some `url-in' to require a
 single file, since that's also an important mode of quick
 sharing.  But that's another story.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] PLaneT(2): Single vs multi-collection packages

2013-05-30 Thread Eli Barzilay
Yes, I really want to try and get to look into doing this.  The thing
is that multi-collection libraries are going to be a common case for
plt packages (which will get pulled out from the main repository), but
the single-collection ones are going to be the much more popular case
elsewhere.

(And I also still dream on doing some `url-in' to require a single
file, since that's also an important mode of quick sharing.  But
that's another story.)


10 minutes ago, Jay McCarthy wrote:
 The Racket package system doesn't support packages that aren't
 collection roots. Eli has said that he wants to implement such a
 feature, but it is not available today.
 
 Jay
 
 On Thu, May 30, 2013 at 8:29 AM, Laurent laurent.ors...@gmail.com wrote:
  I'm willing to upgrade my packages for the new PLaneT system, but one thing
  that pushes me back is the fact that I need to create a directory for my
  package and a subdirectory for the collection(s).
 
  But most of my packages are (and will probably be) single-collection
  packages, and it hurts my logic to have to modify my directory tree for that
  purpose.
 
  I know there have been several discussions already about this, but perhaps
  there is a simple fix:
  Maybe there could be a `single-collection?' entry in the info.rkt file
  which, when true, would tell `raco pkg install' to create itself a directory
  for the package that has the same name as the collection it contains?

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] proposal for moving to packages: repository

2013-05-29 Thread Eli Barzilay
On Friday, Matthew Flatt wrote:
 At Fri, 24 May 2013 12:44:35 -0400, Eli Barzilay wrote:
* The script should also take care to deal with files that got
  removed in the past.
   
   Ditto.
  
  I don't believe that it's *not* doing this, so I did the
  double-check in the form of a test.
 
 You're right --- I misunderstood your example.

(BTW, in case it wasn't obvious -- that was a typo, since the script
is not doing it...)


 Still, I'm happy enough with the result in your example. The
 conversion does preserve `git log --follow' results for the files
 that survive, which was my intended spec. And maybe it's better to
 explain my interest as `git blame', since my main interest in the
 history of a file is often how/why a particular bit of code ended up
 as it is.

Ah, yes -- in that case, I think that it's doing that (= maintaining
the blame information) fine, but there are still things that you'll
want to keep.  (At least for some value of you...)

(I'll get back to this later, since it's the main content.)


* filter-branch one time using your script to reorganize the files
  according to packages
* use filter-branch with a subdirectory filter 5 times to create
  each repository
Total runtime: about 21 hours
  
  This latter use would end up with the final tree being exactly the
  same (since you're talking about doing the reorganization within
  git), but the history would be different since it's as if the
  files were there the whole time.
 
 I don't see how that works. Since my script leaves each file in its
 original location for old commits, I expect a subdirectory
 `filter-branch' to still drop history for the old locations. In any
 case, I'm happy to sort out that detail later.

Ah yes, keeping the files in-place instead of shuffling them around is
definitely much better.  And yes, it means that it *will* take that
large chunk of time for each extracted repository, but I think that
it's definitely worth the effort.  (Once there is a good way to do the
whole trimming thing, I can easily script that onto a bunch of lab
machines to do it all in parallel.)


 If we agree that `git mv' before splitting is practical, though,
 that's all I need for now.

Yes -- with all of the above, and with the additional improvements
that I'll suggest below.  Actually, I'll just send that in a new email
since it's long enough.


 From my perspective, the important thing is to have the ability to
 just edit and move files around to sort out packages, instead of
 having the indirection of a script that edits and moves files
 around.

OK -- but I still think that it's worth it to save a second major
change for people, and given that you've started with a suggestion for
package splitting, maybe just go on with revising that for a short
time and then just do the splitting without an intermediate period?
For people who want to keep dealing with the whole tree, the layout
is going to be the same so there won't be much difference anyway, and
people who want to deal with just their corner will get more time to
adjust and enjoy the benefits of dealing with just their corner
quicker.

BTW, it will potentially lead to more problems where my change to my
own repo goes fine and I don't know that it got broke because of a
change elsewhere since I didn't keep the other files in git form --
but this makes me think that the next release might be prone to such
issues, so it's better to start earlier with the segregation rather
than doing it later.  (But OTOH, the builds and drdr will keep a high
level of problem prevnetion, I hope.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] proposal for moving to packages: repository

2013-05-29 Thread Eli Barzilay

= In this case, the drdr2 file is just #lang racket, and git considers
   it an original copy of the second since it just adds a (exit 43)
   line (first example) or 5 (second).  This happens elsewhere too, a
   lot with readers and info files:

145efa6 Jay McCarthy (1 year, 2 months ago) Adding #lang web-server/base
  C061  collects/frtime/lang/reader.rkt collects/web-server/base/lang/reader.rkt
32d2a9c Matthew Flatt (3 years, 1 month ago) fix scheme/load and racket/load
  C087  collects/slideshow/lang/reader.rkt  
collects/racket/load/lang/reader.rkt
c7e723e Matthew Flatt (3 years, 1 month ago) somewhat rackety core docs
  C060  collects/frtime/opt/lang/reader.ss  
collects/racket/signature/lang/reader.ss
  C066  collects/frtime/reactive/lang/reader.ss 
collects/racket/unit/lang/reader.ss

09bed0d Kevin Tew (1 year, 3 months ago) Initial Distributed Places commit
  C100  collects/combinator-parser/info.rkt 
collects/racket/place/distributed/info.rkt
e788903 Eli Barzilay (1 year, 9 months ago) Remove a bunch of no-longer-needed 
`compile-omit-paths', and move the few ones into the subcollections.
  C100  collects/embedded-gui/private/tests/info.rkt
collects/tests/gracket/info.rkt
  C060  collects/embedded-gui/private/tests/info.rkt
collects/tests/plai/info.rkt
  C056  collects/embedded-gui/private/tests/info.rkt
collects/tests/planet/info.rkt
  ...

There are also counter examples:

2d12431 Matthew Flatt (5 months ago) move and fixup docs for the help 
collection
  R052  collects/scribblings/tools/documentation-utils.scrbl
collects/help/help.scrbl

fd7d8a4 Ryan Culpepper (6 months ago) move lazy-require to racket/lazy-require
  R064  collects/unstable/lazy-require.rkt  collects/racket/lazy-require.rkt

b53e458 Matthew Flatt (9 months ago) add `racket/format'
  C068  collects/unstable/cat.rkt   collects/racket/format.rkt

= here the similarity is very low (the cutoff is 50%), but it is
   actually a correct identification of a move that should be kept.
   (These would also not be a problem had people done the
   move-in-a-separate-commit thing.)

This can probably be improved a lot with a script that will count the
lines of change for low percentages, and weigh in the line length of the
file.  (The log command line allows specifying threshold only in
fractions, so it has to be a separate script).


-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-05-29 Thread Eli Barzilay
-more has things that should all go into their
  respective packages, or into -typed variants of these packages.

* The unstable-list-lib package has a pile of non-list stuff...

* ...but I really think that the unstable stuff should go away: things
  that are used in a particular place should move back into being
  private utilities, and most other things just left behind.  There
  are several things there that are the kind of extensions that could
  be made into their own packages anyway.

* And I think that the above holds for the unstable-parameter-group
  which is used in two packages, but both are basically Neil's
  thing...

* The unstable package has the docs for a bunch of unstable things
  left in it -- my guess is that it's too much of a messto deal with,
  which would support its dissemination...

* It also has some bigger packages that really deserve their own thing
  like the latent-contracts thing: automata and temp-c.  But for all
  of these, there's no real reason to have them included as unstable
  packages: for these things, I think that the only reason they were
  in unstable is as a kind of non-core packages, so now they can be
  just that.

* The wxme-lib package has some things that seem like they might be
  packaged with other libraries (like some snips?), like the xml
  thing.

* A note about some of the text in the toplevel readme file: I really
  don't like keeping a centralized libs repository -- it's true that
  it's harded to build binary code, and distribution can be trickier,
  but as long as this thing is there, it means that the plt packages
  are still more special.  (Maybe there's a plan to eventually spread
  out these things too?)

* There are some administrative files that should be copied across
  multiple repositories and/or revised for each.  Specifically:
  .gitignore files, .gitattributes, and .mailmap.  The first two
  should be copied for each new repository, the last one can be
  trimmed for the actual contributors of each package, based on its
  list of committers.

* In the core racket package:

  - There is a bunch of doc/release-notes/* directories that should
move out.

  - I wonder whether these things are really needed in the core
package (otherwise I think that they should move out to separate
packages or into existing ones):
- acks
- db (needed only when there are docs, or to generate them, right?
  would be also nice to make the sqlite part accessible as its own
  package)
- errortrace
- ffi/examples (move to an -examples package) and ffi/com* (into
  mzcom?); maybe also the objc and winapi things (move to gui or
  its own thing)?
- json, xml (are they needed??  If so, is it possible to hack some
  limited versions to avoid dependencies on a fuller package?)
- unstable (as above -- for these things, probably move them as
  utilities where they're used or make them into non-unstable)
- Many net/* things (imap, sendmail, nntp, pop3, ...); IMO net/url
  too (and replaced by something much simpler that can deal with
  simple GETs, as in the get-libs thing)
- openssl/test.pem (keeps tripping people -- maybe include the
  text verbatim in the docs and have people copy+paste it?)
- rackunit
- srfi/* (should just grab the missing functionality, eventually
  so maybe not now)
- tests/* (looks like it's not done yet)

  - There's a weird .gitignore in racket/collects/pkg which looks like
some erroneous leftover.


 Meanwhile, part of this experiment is defining -lib packages that
 do not provide documentation, which means they can have fewer
 build-time dependencies than they would need for documentation. The
 -docs packages provide the corresponding documentation. (I'm not
 sure about the naming convention; I just had to pick something for
 now.)  Naturally, the X package pulls in both the X-lib and
 X-doc packages.

To clarify, after staring at this for some hours, I really prefer it
if there's a way to keep it all together and do everything within the
package system.  (At least havine 3 repos for N packages sound like a
very unfun thing to deal with.)  But I do want the ability to have
these partial-contents packages built for distribution.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-05-29 Thread Eli Barzilay
Yesterday, Sam Tobin-Hochstadt wrote:
 
 One question -- a bunch of unstable seems to be in the
 typed-racket-lib package.  Why is that?

Since these things are used in TR only, I think that they should move
back.  Ie, you get a trimmed TR repo with unstable/stuff which you
move to a better place on your own...  (Or do so now and save some
splitting work.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-05-29 Thread Eli Barzilay
Yesterday, Robby Findler wrote:
 This looks great to me!
 
 I don't have a good sense of what level of granularity is the right
 one, but I naturally would have gone even finer grained with
 drracket: the macro-debugger, pkg/gui, and maybe even the
 gui-debugger I would have separated out. (Probably you were focused
 on lower-level things first, tho.) 
 
 Also for the macro-debugger: I think it is useful by itself, without
 drracket (I would like to use it for files that in drracket's
 implementation, for example). So I think there should be two
 packages there: macro-debugger-standalone with the give me a file
 and show the macro debugger's result on it interface and
 macro-debugger-plugin pkg that fits into drracket.

Besides the usable-by-itself point, I think that there's a strong
point for code owners...  In this case the owners are very different
(and my guess is that any contributions to the code you do/might make
are a kind of a suggestion), so it's better to split them; and also
there is no sharing of code + maintenance (which is why this point
doesn't apply to core stuff).

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-05-29 Thread Eli Barzilay
An hour ago, Neil Toronto wrote:
 
 I expect `plot' to depend on `math' in the near-ish future. Right
 now, `math' already depends on `plot' to build its docs. IOW, I
 expect the sources to be interdependent, but as binaries, `plot'
 will depend on `math' but `math' won't depend on `plot'. Can the
 package system handle that, or will I have to merge the two or
 factor out a common parent?

I'm guessing that you won't -- I don't know enough details, but one
thing that I can guarantee is that this kind of situation is nothing
compared to some of the existing dependency spaghetti...

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-05-29 Thread Eli Barzilay
20 minutes ago, Matthew Flatt wrote:
 
 Yes. Package dependencies can be cyclic (unlike module
 dependencies), and packages can have cyclic build dependencies
 without cyclic run dependencies.

(*sigh*)  So the idea of acyclic package graph as module containers is
dead?

But to explain my sigh -- I'm not surprised given my other comment
(sent prematurely); I just hope to see it still as a desirable goal.
Maybe some quick package-level graph to show cycles and try to reduce
them at easy points?

(I think that still having lots of cycles will make things more
difficult with packages, so that will be one form of discouragement,
but there will also be problems that are harder to deal with.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] experiment reorganizing the repo into packages

2013-05-29 Thread Eli Barzilay
 getting some super flexible
thing (and fall into the dist-spec pit).  Another problem is
specifying dependencies, which can be in a natural way:

  (define deps
'([lib (readline lib) (scribble lib) (macro-debugger text lib)]
  [he hebrew-dictionary]
  ...))

There's more missing technicalities, but I think (= hope) that
something along these lines can work.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] proposal for moving to packages: repository

2013-05-24 Thread Eli Barzilay
Yesterday, Robby Findler wrote:
 Hi Eli: I'm trying to understand your point. Do I have this right?
 
 Background: The git history consists of a series checkpoints in time
 of the entire repository, not a collection of individual files.

Yes, although the difference between entire repository and
individual files is mostly theoretical.  The main point is that the
log history is made from changes to *content* -- you can't have some
made up history planter artificially for a file.  (And it is the same
for most CMSs if not all; the main difference here is that git doesn't
keep meta information about copying and renaming.)


 So, when I do git log x.rkt then what I get is essentially a
 filtered list (except where people didn't properly rebase, but lets
 ignore that) of those checkpoints: all the ones where x.rkt
 changed.

Exactly.  (I don't get the rebase comment though -- even without
rebasing what you get is this filtered history.)


 Big Question: The issue is, then, when we split up the current repo
 into smaller repos, what are the series of checkpoints that we're
 going to make up for the individual repos? Right? 

Yes, but it can get a bit subtle.  Like I said, the `filter-branch'
tool is basically replaying the entire history, giving you points to
inject hooks that can modify the tree or the commits, etc.  Note that
in all uses that were mentioned, there was a --prune-empty flag,
which means that commits that didn't have any change are dropped.  I'm
mentioning this because some people might have an illusion that it's
better to *not* do that and keep these commits.  Here's an example why
this is not useful: say that you have this edit sequence:

  foo@somewhere creates A/x, with log message created x
  bar@somewhere edits it, with log message edited x
  baz@somewhere moved it into B/x, with log renamed A to B

If at this point you use any git tools, they can see the real history.
For exmaple, you can use `blame' to see which lines were written by
which user.  Also, assuming that these are all the changes, a git log
will show the three commits as they appear above.

Now, if you you use filter-branch to modify the repository and keep
only the B directory, but you *don't* use the --prune-empty flag:
the fact that you want to keep these other commits won't help -- the
full history would have the three commits with the same three
messages, but doing a log for just the file would show only the
commits for the file, so the first two commits won't be shown.
Similarly, blame can't show anything useful -- you'll only see
baz@somewhere as the author of the entire file.  And the reason this
makes sense is that the full commit history has the first two commits,
but they had no change -- so there's nothing that ties them to the
file in the trimmed repository, let alone something that relates them
to specific lines in the file...

(Two notes: (a) This is just a demonstration -- obviously, this is a
trimming that is done in a bad way since it dropped A even though it's
part of the history of B.  (B) Actually, it looks like the
--subdirectory-filter drops empty commits anyway, but the above
explains why it makes sense to do that.)


 Your Advice: And, IIUC, you're suggesting that the best way to deal
 with this question is to defer it until we are more sure of the
 actual split we want to make. So we don't mess with the history at
 all

The point is that every such messing-with-history should be done very
carefuly and checked thoroughly, since the chance to mess things up is
very real.  In the above, it's obvious that I should have not droped A
in the filter -- but if it's some random single file which you had in
the framework collection, out of tons of other files in the drracket
package, then it's unlikely that I will catch it -- which is why I
prefer using tools for these things and resolve all such issues with
the people who know about the code.


 and instead just work at the level of some script that we can run to
 just use mv and company to move things around.  When we know
 exactly what ends up going where, then we can figure out how to make
 up a new, useful history for the separate repositories.
 
 Is that the point?

The thing is that having two such filters (one to restructure the big
repository and one to split it) is both increasing chances for making
mistakes, and making the job of the second restructure much harder to
do.  To the point where doing it manually is infeasible, which is why
I said that it will guarantee losing history.

(And I'll reply to Matthew's suggested tool next.)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] proposal for moving to packages: repository

2013-05-24 Thread Eli Barzilay
8 hours ago, Matthew Flatt wrote:
 At Thu, 23 May 2013 07:09:17 -0400, Eli Barzilay wrote:
  Relevant history is vague.
 
 The history I want corresponds to `git log --follow' on each of the
 files that end up in a repository.

(In this context this is clear; the problem in Carl's post is that it
seemed like he was suggesting keeping the whole repository and doing
the split by removing material from clones -- which is and even fuller
history, but one that has large parts that are irrelevant.)


 That's true if you use `git filter-branch' in a particular way. I'll
 suggest an alternative way, which involves filtering the set of
 files in a commit-specific way. That is, the right set of files to
 keep for each commit are not the ones in the final place, but the
 ones whose history we need at each commit.

If that can be done reliabely, then of course it makes it possible to
do the split reliabley after the first restructure.  It does come with
a set of issues though...

 [... scripts description ...]

Here are a bunch of things that I thought about as I went over this.
In no particular order, probably not exhaustive, and possibly
repetitive:

* Minor: better to use `find-executable-path' since it's common to
  find systems (like mine) with an antique git in /usr/bin and a
  modern one elsewhere.  (In my case, both scripts failed since
  /usr/bin has an antique version.)

* There is an important point of fragility here: you're relying on git
  to be able to find all of the relevant file movements (renames and
  copies), which might not always be correct.  On one hand, you don't
  want to miss these operations, and on the other you don't want to
  have a low-enough threshold to identify bogus copies and renames.

* Because of this, I think that it's really best to inspect the
  results manually.  The danger of bogus copies, for example, is real,
  especially with small and very boilerplate-ish files like info.rkt
  files.  If there's a mistaken identification of such a copy you can
  end up with a bogus directory kept in the trimmed repo.  In
  addition, consider this information that the script detects via git
  for a specific commit:

A/f1.ss renamed to B/f1.rkt
A/f2.ss renamed to B/f2.rkt
...
A/f47.ss renamed to B/f47.rkt
A/f48.ss renamed to B/f48.rkt
A/f49.ss deleted
A/f50.ss deleted
B/f49.rkt created
B/f49.rkt created

  For a human reviewer, it's pretty clear that this is just a
  misidentification of two more moves (likely to happen with the kind
  of restructures that we did in the past, where a single commit both
  moves a file, and changes its contents).  This is why on one hand I
  *really* like to use such scripts (to make sure that I don't miss
  such things), but OTOH I want to review the analysis results to see
  potential problems and either fix them manually or figure out a way
  to improve the analysis and run it again.

* Also, I'd worry about file movements on top of paths that existed
  under a different final path at some point, and exactly situations
  like you described, where a file was left behind, but that file is
  completely new and should be considered separate (as in the case of
  a file move and a stub created in its place).

* The script should also take care to deal with files that got removed
  in the past.  For example, the drscheme collection has some file
  which gets removed, and later (completely unrelated) most of the
  contents migrated to drracket.  If the result of the analysis is
  that most of the material moved this way, and because of that you
  decide to keep the old drscheme collection -- you'd also want to
  keep that file that disappeared before the move, since it's part of
  the relevant history.

  So I'd modify this script to run on the *complete* repository -- the
  whole tree and all commits -- and generate information about
  movements.  Possibly do what your script is does for the whole tree,
  then add a second step that runs and looks for such files that are
  unaccounted for in the results, and decide what to do with them.

  I think that this also means that it makes sense to create a global
  database of all file movements in a single scan, instead of running
  it for each package.

* Technical: I thought that it might make sense to use a racket server
  (with netcat for the actual command), or have it compile a /bin/sh
  script to do the actual work instead of using `racket/kernel' for
  speed.  However, when I tried it on the plt tree, it started with
  spitting out new commits rapidly, but eventually slowed down to more
  than a second between commits, so probably even the kernel trick is
  not helping much...

* Actually, given the huge amount of time it's running (see next
  bullet), it's probably best to make it do the movements from all
  paths at the same time.  In this specific context, this means that
  it scans the package-restructured repo (from the first step) into a
  package-restructured repo

Re: [racket-dev] proposal for moving to packages: binary vs source

2013-05-24 Thread Eli Barzilay
[Note subject change...]

Two days ago, Eric Dobson wrote:
 For binary vs source, I think you are providing a good argument for
 the usefulness of a no source distribution. Some people want to use
 tools written in Racket, and the fact that the tools are written in
 Racket is immaterial to them. They should be able to have just the
 binary versions.

There have been a bunch of concerns expressed about the question of
distributing sources or not -- but I think that generally speaking,
there shouldn't be any problems at all.  Here's a list of things that
contribute to not having such concerns:

1. The eventual goal would be to have very easy selection of packages
   that you want to install.  Either with (a) a bunch of installers,
   (b) possibly doing this by just a different URL that will have the
   installers listed in it as arguments, or (c) with a post-install
   dialog that will ask you for additional packages to install.  (In
   the (c) case, it could also detect packages that you had decided to
   install previously, and re-use the same list.)

   The bottom line is that if *you* want to get the sources, then it
   should be extremely easy to just have them installed (c), or create
   installers that include the sources (a;b) which you'll use.  The
   main point here is that using packages will make such variations
   very easy to implement, and make it easy for you to add sources or
   provide popular options based on demand.

2. With the geiser/drracket concern about reduced functionality
   because there are no sources: the information about the source of
   bindings is still there.  (Ie, things work fine if you remove a
   random source file from a current installation -- the only
   difference is that the actual source file is not there.)

   Now, I'm assuming that there is some way with the package system to
   know for any given file which package it came from.  With this
   information, I think that it would be easy to do something like
   this:

 * In drr, if you try to jump to a definition for a function whose
   source is not included, you get a popup telling you that you
   don't have the source, and list an on-line URL where the source
   can be found (which is inferrable from the package information)
   as well as a one-button-click option to install the source and
   then open the file.

 * Geiser could do exactly the same, and also use something like
   `url-handler-mode' to visit the source file directly from the
   on-line source in addition to offering to install the sources.

3. I think that there should be an option for package owners to decide
   how their package gets installed, so for example, if realm must be
   distributed with its sources, it can just specify that and avoid
   the stripping that other packages would go through.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] proposal for moving to packages: repository

2013-05-24 Thread Eli Barzilay
Four hours ago, Matthew Flatt wrote:
 At Fri, 24 May 2013 03:26:45 -0400, Eli Barzilay wrote:
  If that can be done reliabely, then of course it makes it possible to
  do the split reliabley after the first restructure.
 
 Great! Let's do that, because I remain convinced that it's going to
 be a lot easier.

I'm really surprised.  Given that you consider this a *lot* easier,
and that I consider it (reorganization + split) a lot messier, I think
that I'm still not getting something.


  * Also, I'd worry about file movements on top of paths that
existed under a different final path at some point
 
 I believe the file-lifetime computation in slice.rkt takes care of
 that.

That's what it looks like, but I'd double-check to make sure that it
happens.


  * The script should also take care to deal with files that got
removed in the past.
 
 Ditto.

I don't believe that it's *not* doing this, so I did the double-check
in the form of a test.  When I run it, I see these bad things (which I
expected to happen, so wrote it as a test):

* The c file got completely lost (this is the pre-reorganization
  file deletion scenario)

* The b file got lost too (post-reorg deletion)

* The history of e during the A days got lost, since it was not
  recognized as a rename in the A-B move due to being edited too.

= The first two are things that a script can deal with doing some
   kind of scan like I mentioned (go over the full history of the full
   tree).

= The third one is something that requires human judgment *but* if
   the A/e historic file is considered as deleted, and if deleted
   files from the original directories are included with doing the
   above, then it should still be there in the rewritten repo.

Test file attached; probably need to do very little other than
adjusting the paths to the two racket scripts.



b
Description: Binary data


  * Actually, given the huge amount of time it's running (see next
bullet), it's probably best to make it do the movements from all
paths at the same time.
 
 There's no need to move anything while extracting a repository
 slice; the movements happen before.

What I'm saying is that if filter-branch using your script takes 20
hours, and you want to use it to split the repo to 5 packages, and if
a simple filter-branch with a subdirectory filter takes a few minutes,
then instead of:

  * filter-branch using your script 5 times to create each repository
  Total runtime: more than 4 days

you do this:

  * filter-branch one time using your script to reorganize the files
according to packages
  * use filter-branch with a subdirectory filter 5 times to create
each repository
  Total runtime: about 21 hours

This latter use would end up with the final tree being exactly the
same (since you're talking about doing the reorganization within git),
but the history would be different since it's as if the files were
there the whole time.

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!
_
  Racket Developers list:
  http://lists.racket-lang.org/dev


Re: [racket-dev] proposal for moving to packages: repository

2013-05-23 Thread Eli Barzilay
9 hours ago, Matthew Flatt wrote:
 At Wed, 22 May 2013 14:50:41 -0400, Eli Barzilay wrote:
  That's true, but the downside of changing the structure and having
  files and directories move post structure change will completely
  destroy the relevant edit history of the files, since it will not
  be carried over to the repos once it's split.
 
 It's possible that we're talking past each other due to me not getting
 this point.

(Obligatory re-disclaimer: I consider the problem with forcing people
to change their working environment much more severe.)


 Why is it not possible to carry over history?
 
 The history I want corresponds to `git log --follow' on each of the
 files that end up in a repository. I'm pretty sure that such a
 history of commits can be generated for any given set of files, even
 if no ready-made tool exists already (i.e., 'git' is plenty flexible
 that I can script it myself).
 
 Or maybe I'm missing some larger reason?

The thing to remember is just how simple git is...  There's no magical
way to carry over a history artificially -- it's whatever is in the
commits.

To make this more concrete (and more verbose), in this context the
point is that git filter-branch is a simple tool that basically
replays the complete history, allowing you to plant various hooks to
change the directory structure, commit messages or whatever.  The new
history is whatever new commits are in the revised repository, with no
way to make up a history with anything else.

Now, to make my first point about the potential loss of history that
is inherent in the process -- say that you want to split out a
drracket repo in a naive way: taking just that one directory.  Since
it's done naively, the resulting repository will not have the
drscheme directory and its contents, which means that you lose all
history of files that happened there.  To try that (in a fresh clone,
of course) -- first, look at the history of a random file in it:

  F=collects/drracket/private/app.rkt
  git log --format='%n%h %s' --name-only --follow -- $F

Now do the revision:

  S=collects/drracket
  git filter-branch --prune-empty --subdirectory-filter $S -- --all

And look at the same log line again, the history is gone:

  git log --format='%n%h %s' --name-only --follow -- $F

If you look at the *new* file, you do see the history, but the
revisions made in drscheme are gone:

  git log --format='%n%h %s' --name-only --follow -- private/app.rkt

In any case, this danger is there no matter what, especially in our
case since code has been moving around in the racket switch.  I
*hope* that most of it will be simple: like carrying along the
drscheme directory with drracket, the scheme and mzlib with
racket, etc.  Later on, if these things move to compat packages,
the irrelevant directories get removed from the repo without
surgeries, so the history will still be there.  This shows some of the
tricks that might be involved in the current switch: if you'd want to
have some compat package *now*, the right thing to do would be:

  * do a simple filter-branch to extract drscheme (and other such
collections) in a new repository for compat

  * for drracket: do a filter-branch that keeps *both* directories
in, then commit a removal of drscheme.  (Optionally, use rebase
to move the deletion backward...)

Going back to the repo structure change that you want and the reason
that I said that doing moves between the package directories
post-restructure is destructive should be clear now: say that you move
collects/A/x into foo/A/x as part of the restructure.  Later you
realize that A/x should go into the bar package instead so you just
move it to bar/A/x.  The history is now in, including the rename, but
later on when bar is split into a separate repo, the history of the
file is gone.  Instead, it appears in the foo repository, ending up
being deleted.

One way to get around this is to avoid moving the file -- instead, do
another filter-branch surgery.  This will be a mess since each such
change will mean rebuilding the repository with all the pain that this
implies.  Another way to get around it is to keep track of these
moving commits, and when the time comes to split into package repos,
you first do another surgery on the whole repo which moves foo/A/x to
bar/A/x for all of the commits before the move (not after, since that
could lead to other problems), and then do the split.

This might work, but besides being very error-prone, it means doing
the same kind of file-movement tracking that I'm talking about anyway.
So take this all as saying that the movement of files between packages
needs to be tracked anyway -- but with my suggestion the movement is
delayed until it's known to be final before the repo split, which
makes it more robust overall.



But really, the much more tempting aspect for me is that this can be
done now -- if you give me a list of packages and files, I can already
do the movement script.

Actually

Re: [racket-dev] proposal for moving to packages: repository

2013-05-23 Thread Eli Barzilay
9 hours ago, Carl Eastlund wrote:
 I was going to comment on the same thing.  While a naive use of git
 filter-branch might not retain the history, it should be entirely
 possible to do something a little more intelligent and keep that
 history.

Just to be clear, this is exactly what you can't get with
filter-branch.


 Essentially each of the new repositories could keep the entire
 history of the original repository, followed by a massive
 move/rename, then moving forward with an individual package.

This can work, but it is unrelated to filter-branch: it's basically
starting each package repository from a clone of the monolithic repo,
then move  shuffle things around.

This seems wrong to me in all kinds of ways -- but if someone wants to
do this with *their* package (ie, not a package that I need to deal
with), then it's certainly an option.

(That's one of the big appeals of moving to packages for me: some code
moves to packages which I can let myself Not Care About™.  Knock
youself out with tabs, spaces at ends of lines, braces in code, two
spaces between bindings and values in `let's, and make sure that no
file ends with a newline...)

-- 
  ((lambda (x) (x x)) (lambda (x) (x x)))  Eli Barzilay:
http://barzilay.org/   Maze is Life!

_
  Racket Developers list:
  http://lists.racket-lang.org/dev


  1   2   3   4   5   6   7   8   9   >