[racket-dev] toward a new Racket macro expander

[Matthew Flatt]

I've been working on a new macro expander for Racket, and I'm starting
to think that it will work. The new expander is not completely
compatible with the current expander --- and that will be an issue if
we eventually go forward with the change --- but most existing code
still works.

Here's a report on my current experiment:

 http://www.cs.utah.edu/~mflatt/scope-sets/


The goals for a new expander are

 1. to replace a complicated representation of syntax objects with a
simpler one (and, as a result, avoid some performance and
submodule-re-expansion problems that have been too difficult to fix
with the current expander);

 2. to find a simpler model of binding than the current one, so that
it's easier to explain and reason about scope and macros; and

 3. to implement the new expander in Racket instead of C.

I have possibly succeeded on 1, possibly succeeded to some degree on 2,
and temporarily given up on 3.

(I know it sounds crazy to continue with a C implementation of macro
expansion. Goals 1 and 2 were impossible for me without being able to
try ideas at scale, though, and it was too daunting to tackle a full
reimplementation of the expander and a restructuring of the VM at the
same time. Continuing in the comfortable-for-only-me C environment was
the way to make progress for now, and I look forward to tackling 3 as a
next step.)


I've pushed my current implementation to the "scope" branches of

 https://github.com/mflatt/racket
 https://github.com/mflatt/compiler [for `raco decompile` updates]

I don't recommend bothering with the implementation, for now, unless
you're especially interested in trying some examples. The current
version builds well enough to run DrRacket, but there some build
errors. I have a lot of work to do, still.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-dev+unsubscr...@googlegroups.com.
To post to this group, send email to racket-...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-dev/20150226172633.2725A6501AA%40mail-svr1.cs.utah.edu.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-dev] make & --clone installed pkgs

[Matthew Flatt]

At Tue, 17 Feb 2015 19:59:38 -0500, Sam Tobin-Hochstadt wrote:
> On Tue, Feb 17, 2015 at 6:41 PM, Matthew Flatt  wrote:
> > At Tue, 17 Feb 2015 14:12:54 -0500, Sam Tobin-Hochstadt wrote:
> > Does another system have a Racket-like in-place option (that works
> > better)?
> 
> I haven't used it, but GHC has an in-place build option where you can
> install packages; see [1].

I don't have a lot of experience with GHC, but I have talked to some
GHC and Cabal developers about how they work with the package system.
Those discussions did not lead me to believe that they have an
especially smooth system for working with packages and updates in-place
--- and, in particular, that it's not the way they normally work. I'd
be happy to hear more from someone who routinely works that way with
GHC, though.

> I think the closer analogy is to what other software does when you run
> both `make` and `make install` analagous to Racket's unix-style
> installation, since that's the recommended way of building (eg
> Python), just as in-place is the recommended way of building Racket
> from source. I don't think any of those systems update packages when
> running `make install`.

I'm not sure what you're getting at here. We seem to agree that the
usual `make` plus `make install` is like Racket's `make unix-style`,
neither of which updates packages (other than the ones the makefile
knows about).


> > At Tue, 17 Feb 2015 17:40:36 -0500, Matthias Felleisen wrote:
> >> Speaking as the user I am, I really like it that make updates
> >> my extra-pkgs.
> >
> > Package scope provides one a way to get these different behaviors. The
> > current `make` updates only packages that are in installation scope,
> > and it also sets installation scope to be the default, so that's why
> > `make` tends to update everything that is installed. Maybe Sam should
> > install additional packages in user scope, and then `make` won't try to
> > update them.
> 
> I expect that the packages that update for Matthias on `make` are
> packages in "main-distribution"

Ah, no. I've helped Matthias when problems break his installation, I've
noticed that he installs packages not in "main-distribution" (e.g.,
"marketplace"), and I believe he really does want those updated.

I had that context in mind but didn't think to spell it out as I should
have.


> As an aside, the reason I don't install in user scope is that I switch
> between Racket implementations regularly, which would lead to
> out-of-date zo errors for all my user packages (instead, I get
> multiple copies of the packages).

You can give each installation a different name (using `raco pkg config
--set name ...`) to avoid the collision. That would be an extra step in
setting up each new installation, though.


I don't have a strong opinion on whether `make` should update packages
outside of "main-distribution", but the feedback I'm getting is

 * Sam doesn't think they should be updated --- but he also doesn't
   want packages in "main-distribution" updated, so he's going to use
   `make as-is`.

 * Everyone else who has spoken up seems to prefer an updating `make`,
   so far.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-dev+unsubscr...@googlegroups.com.
To post to this group, send email to racket-...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-dev/20150218125203.667906501B8%40mail-svr1.cs.utah.edu.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-dev] make & --clone installed pkgs

[Matthew Flatt]

At Tue, 17 Feb 2015 14:12:54 -0500, Sam Tobin-Hochstadt wrote:
> Regardless of that, though, I think we should switch to updating only
> "main-distribution" (and perhaps "main-distribution-tests"). I doubt
> people expect `make` in the Racket source tree to update their
> software somewhere else on their machine -- I certainly would be very
> unpleasantly surprised if that happened to me when rebuilding some
> other language I had installed.

I have trouble seeing the comparison to other software. I don't know of
anything else with an in-place build option, where you can install or
link additional packages in place, and you can also pull core updates,
and somehow it all works. The examples I have seen are all more like
`make unix-style`, where there's a clear separation of the source and
the installed form. In that case, the issues we're looking at here
don't come up.

Does another system have a Racket-like in-place option (that works
better)?


At Tue, 17 Feb 2015 17:40:36 -0500, Matthias Felleisen wrote:
> Speaking as the user I am, I really like it that make updates 
> my extra-pkgs. 

Package scope provides one a way to get these different behaviors. The
current `make` updates only packages that are in installation scope,
and it also sets installation scope to be the default, so that's why
`make` tends to update everything that is installed. Maybe Sam should
install additional packages in user scope, and then `make` won't try to
update them.

Another possibility is that the packages installed by `make` should go
into a scope that's even deeper than installation scope. That's how the
pre-split organization worked, and I sometimes think that we should go
back to that. But it's more complex, and it would work less well for
Matthias and others who benefit from the current `make` behavior.

Overall, my sense is that the current defaults work right for people
with the least expertise, but we can refine our tools to provide
options to people with more expertise.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-dev+unsubscr...@googlegroups.com.
To post to this group, send email to racket-...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-dev/20150217234131.86F396501B8%40mail-svr1.cs.utah.edu.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-dev] make & --clone installed pkgs

[Matthew Flatt]

Continuing in reverse order:

- My sense is that the switch to `make` so that it updates packages,
which was a result of

 http://lists.racket-lang.org/users/archive/2015-January/065345.html

has been a good change for most people most of the time.

The `as-is` target is currently available for building without updating
packages. I think it would make sense to introduce `make
as-is-the-default` to effectively change the default `make` target to
be `make as-is`. (And there would be `make with-update` to get the
current behavior of `make` after `make as-is-the-default`, plus `make
with-update-the-default` for completeness.)


- It makes sense to me to add a `--pull` option to `raco pkg update`
for specifying the behavior of the `git pull` step for linked clones. I
imagine that the default would be `--pull ff-only` for the current
behavior, while `--pull try` would ignore fast-forward failures. I
think I'd also like `--pull rebase`.

I'm skeptical that making `--pull try` the default would be a good
idea. People have trouble seeing warnings.


Would those changes work?


At Tue, 17 Feb 2015 10:38:43 -0500, Sam Tobin-Hochstadt wrote:
> I think there are two seperable issues here:
> 
> 1. Can we make `raco pkg update -a` better/more robust in this case?
> 
> 2. Should `make` run `raco pkg update -a`?
> 
> In reverse order:
> 
> - I think `make`, by default, shouldn't update anything, and that we
> should have a different Makefile target which updates things when
> people want that explicitly. The current behavior is especially
> problematic because it updates things that aren't in
> "main-distribution", meaning that it's making potentially arbitrary
> breaking changes to software on your computer (not just to core
> Racket).
> 
> This could be more inconvenient for someone working widely on core
> packages, but if they wanted the current behavior it would be just
> `make update` (or some other name) instead of `make`. As someone who
> does work on a lot of core packages, I'd prefer greater explicitness.
> 
> - I think `raco pkg update p` where `p` is a cloned package should
> only do anything if (a) the currently-checked-out branch is the one in
> the pkg source and (b) the `git merge --ff-only` command would
> succeed. Otherwise, I think it should just print a message and leave
> the repository as it is. I think that's what I wanted all the times
> that this operation has failed in my experience.
> 
> Sam
> 
> On Tue, Feb 17, 2015 at 9:54 AM, Robby Findler
>  wrote:
> > Sam and I have run into a situation where `make` fails because we've
> > set up clone pkgs and made local modifications in a way that makes the
> > git commands fail [*].
> >
> > My guess is that the right thing to do is for me to know about these
> > pkgs and do something special when running make. I'm thinking that I
> > could let make finish the initial steps until it gets to the step
> > where it updates the pkgs and then do the update step myself and then
> > run `make as-is`. But the problem with this is that I don't see what
> > command I can run that will update all of the pkgs except the
> > problematic ones. Like I could imagine a `raco pkg update
> > --all-except-clones` or something, but that feels a bit strange as
> > there could be other development modes that would also run into
> > similar problems. Maybe `raco pkg update
> > --all-things-from-this-catalog
> > ` or something
> > along those lines is the way to go? In general, it seems right for me
> > to run some commands whose complications are roughly proportional to
> > the number of pkgs that I have installed as clones (and where I'm
> > actively developing them) but not to run some commands that require me
> > to do something special for each pkg that is installed.
> >
> > Any ideas? Or am I just missing the right command?
> >
> > Thanks,
> > Robby
> >
> > [*] In my case, in case this suggests a better/different approach to a
> > resolution: the `raco pkg update` step eventually gets to this git
> > command:
> >
> >   git merge --ff-only 
> >
> > where the  is the checksum from the pkg server, I believe. In my
> > case, this is a different branch than is currently checked in my
> > clone'd pkg and so the git merge command fails (and that command
> > failing seems like the right behavior for the setup I'd like to be
> > able to use).
> >
> > --
> > You received this message because you are subscribed to the Google Groups 
> "Racket Developers" group.
> > To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-dev+unsubscr...@googlegroups.com.
> > To post to this group, send email to racket-...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/racket-dev/CAL3TdONOjL3_y8U_A1ZUoz_1Z%3DE3HjV
> V8by9e%2B2dS-W2mc51pg%40mail.gmail.com.
> > For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Racket Dev

Re: [racket-dev] file change notifications

[Matthew Flatt]

It's good to know that canceling a filesystem-change event can be
expensive on Windows, in which case they probably shouldn't be used on
Windows when resolving module paths. I'll investigate more and remove
the use.

Meanwhile, there's not a good way to disable the current use in v6.1,
but the hack

  (unsafe-vector-set! (system-type 'fs-change) 2 #f)

should work to disable it.


At Mon, 2 Feb 2015 12:14:12 -0800, Dan Liebgold wrote:
> Hi -
> 
> I'm doing a little profiling of Racket 6.1 on windows. It seems like
> cancelling a filesystem-change-evt can take many seconds, and one is
> cancelled for the addon-dir directory (which defaults to my roaming
> profile) during every run. I think it interacts badly with my virus checker.
> 
> Is there a way to disable this? I'm not using the addon-dir, so any
> notifications shouldn't be important.
> 
> Thanks,
> -- 
> Dan Liebgold[dan.liebg...@gmail.com]
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [plt] Push #29677: master branch updated

[Matthew Flatt]

I noticed when I made that change that the ".dep" format is still not
documented anywhere. I'll document it soon.

Here's a first cut at the grammar of the value that is stored in a
".dep" file via `write`:

  = (list 
(cons  )
 ...)
  = (cons 'indirect )
   | 
  = (cons 'ext ) ; non-module dependency
  |  ; module dependency
  = (cons 'collects ) ; a collection-relative path
 |   ; other path (not portable)
   

At Fri, 30 Jan 2015 22:29:11 -0800, Eric Dobson wrote:
> This change seemed to change the format of .dep files, likely as intended
> to add the indirect dependencies. Is there any documentation of what the
> format is supposed to be? Currently I've just been trying to read cm.rkt
> and understand how it treats them.
> 
> On Thu, Jan 8, 2015 at 9:31 AM,  wrote:
> 
> > mflatt has updated `master' from c56c9250f1 to 95e85ec5bd.
> >   http://git.racket-lang.org/plt/c56c9250f1..95e85ec5bd
> >
> > =[ 2 Commits ]==
> > Directory summary:
> >   45.1% pkgs/racket-doc/scribblings/raco/
> >    4.7% pkgs/racket-doc/scribblings/reference/
> >   47.5% racket/collects/compiler/
> >
> > ~~
> >
> > fe9a04d Matthew Flatt  2015-01-08 09:11
> > :
> > | doc tweaks for `raco {setup,make}`
> > :
> >   M pkgs/racket-doc/scribblings/raco/make.scrbl  |  4 ++--
> >   M pkgs/racket-doc/scribblings/raco/setup.scrbl | 22
> > --
> >
> > ~~
> >
> > 95e85ec Matthew Flatt  2015-01-08 09:57
> > :
> > | add support for indirect CM dependencies; use in `lazy-require`
> > |
> > | If module M in package P imports module N from package Q,
> > | and if N has a `lazy-require` for a module in R that is
> > | triggered during the compilation of M, then P doesn't really
> > | depend on R; P depends on Q, and Q depends on R, and P
> > | shoudn't necessarily know anything about Q. At the same time,
> > | a change to the file in R means that M must be recompiled.
> > | So, continue to track the compilation dependency, but mark
> > | it as "indirect" so that the package-dependency checker can
> > | ignore the dependency.
> > :
> >   M pkgs/racket-doc/scribblings/raco/make.scrbl   | 33 -
> >   M racket/collects/compiler/cm-accomplice.rkt| 14 +++---
> >   M racket/collects/compiler/cm.rkt   | 49
> > ++--
> >   M racket/collects/racket/lazy-require.rkt   |  2 +-
> >   M racket/collects/setup/private/pkg-deps.rkt|  1 +
> >   M .../racket-doc/scribblings/reference/syntax.scrbl |  8 ++--
> >
> > =[ Overall Diff ]===
> >
> > pkgs/racket-doc/scribblings/raco/make.scrbl
> > ~~~
> > --- OLD/pkgs/racket-doc/scribblings/raco/make.scrbl
> > +++ NEW/pkgs/racket-doc/scribblings/raco/make.scrbl
> > @@ -123,7 +123,7 @@ would create only @filepath{compiled/b_rkt.zo} and
> >
> >  @; --
> >
> > -@section{Dependency Files}
> > +@section[#:tag "Dependency Files"]{Dependency Files}
> >
> >  In addition to a bytecode file, @exec{raco make} creates a file
> >  @filepath{compiled/@nonterm{name}_@nonterm{ext}.dep} that records
> > @@ -538,7 +538,7 @@ messages are instances of a
> > @racket[parallel-compile-event] prefab structure:
> >
> >  @racketblock[
> >(struct parallel-compile-event (worker event) #:prefab)
> > -].
> > +]
> >
> >  The worker field is the index of the worker that the created the event.
> > The event
> >  field is a @racket[compile-event] as document in
> > @@ -550,25 +550,36 @@ field is a @racket[compile-event] as document in
> >
> >  @defmodule[compiler/cm-accomplice]
> >
> > -@defproc[(register-external-file [file (and path? complete-path?)])
> > void?]{
> > +@defproc[(register-external-file [file (and path? complete-path?)]
> > + [#:indirect? indirect? any/c #f])
> > + void?]{
> >
> > -Logs a message (see @racket[log-message]) at level @racket['info] to
> > -a logger named @racket['cm-accomplice]. The
> > -message data is a @racketidfont{file-dependency} prefab structure type
> > -with two fields; the first field's value is @racket[file] and the second
> > -field's value is @r

Re: [racket-dev] feature request: thread-safe memoize-evt

[Matthew Flatt]

Hi Jan,

Interesting problem!

I think I see what you mean: There's no way to combine the completion
of an event plus saving its value as an atomic operation, except by
putting the synchronization in its own thread. But if you put the
synchronization in its own thread, then there's no way to prevent that
thread's synchronization when a consumer synchronization (i.e., one
that is waiting for the thread's result) picks a different event than
the one represented by the thread.

It's easy to make a complete+save combination atomic if it's built into
the scheduler. So, I can easily imagine adding a simpler primitive,
`once-evt`. The event OE created by `(once-evt E)` could save the first
result produced by E, but not attempt to have only a single wait on E.
That is, if OE1 is synchronized in multiple threads, then it would be
like synchronizing E in multiple threads, but only the first result
from E will be saved.

Unfortunately, `once-evt` isn't enough to implement `memoize-evt`. The
troublesome case is then thread T1 is synchronizing on OE1, T1 gets
suspended, and T2 starts waiting on OE1. In that case, you'd like T2 to
take over the wait, even if it means restarting E. You can detect that
T1 is suspended and have T2 start waiting on E, but there's no way to
cancel the wait of E in T1.

Building `memoize-evt` into the core doesn't the avoid the need to, at
some level, cancel T1's wait on E. I'll keep thinking about it, but it
looks like that would require deep changes to the scheduler.

Would the simpler `once-evt` work in your situation, or do you need the
guarantee that only one wait of E happens at a time?

Matthew

At Wed, 28 Jan 2015 13:49:51 +0100, Jan Dvořák wrote:
> Hi,
> 
> I would like to ask for another extension of the Racket's event handling
> system. A `memoize-evt` constructor with following semantics:
> 
> Given a base event E_b, memoize-evt will produce a memoizing event E_m.
> Synchronizing on E_m from any number of threads will block until a
> single value is produced by E_b. This value is then stored inside the
> E_m. From that moment on, E_m will always be immediately ready for
> synchronization and produce the stored value in all waiting threads.
> 
> The single-threaded implementation is a trivial guard-evt + replace-evt
> + (wrap-evt always-evt) combo, but for thread-safety a temporary thread
> would be needed to wait for the base event and the solution would have
> different semantics then rest of the event system.
> 
> A lower-level approach would also be possible; create something along
> the lines of a dynamic-wind-evt that would, with some cleverness, allow
> for generic thread-safe events via locking. Or create a locked-wrap-evt
> constructor that will not be ready until it's handler finishes.
> 
> Hoping that I am not being too bothersome,
> from Prague with thanks,
> Jan Dvorak


_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] build memory use (was: internal error during gc)

[Matthew Flatt]

At Thu, 29 Jan 2015 15:34:37 -0300, Gustavo Massaccesi wrote:
> If there are some easy technical details and advice, you can write a
> nice blog post about this.

Good idea. I have some advice on finalization (don't do it), and I
could write about how Racket tries to help when you can't follow that
advice.

> At line 109:
> +  (define max-val (apply max (map car measurements)))
> +  (define max-time (caddr (last measurements)))
> If you make multiple png at the same time, all of them get the same
> Peak and Duration.

Thanks! I had fixed that in http://github.com/racket/plt-build-plot,
but I forgot to update the service that uploads "plot.rkt" to the web
site. It will be fixed after the next run.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Full transparency (was: dev Digest, Vol 72, Issue 31)

[Matthew Flatt]

At Wed, 28 Jan 2015 16:21:51 -0700, Byron Davies wrote:
> Your code, commented:
> 
> (define orig-i (current-inspector))  ; saves the original inspector
> (define sub-i (make-inspector orig-i))  ;make a new inspector whose parent
> is the original inspector
> 
> (current-inspector sub-i)  ;makes the new inspector the current inspector
> (struct a (x))  ; creates a structure using the new inspector as the
> default inspector
> (define v (a 1))  ; creates an instance of the new structure
> (current-inspector orig-i) ;reverts the inspector to the original (the
> parent of the new inspector)
> 
> I see how this works, but I'm a little confused about why it works.  I see
> that the new inspector is a child of the old one, and I read in the
> reference chapter that access is determined not by the inspector in force
> at creation time, but by the parent of that inspector, i.e., the old
> inspector. I can't find any description of the "power" of an inspector,
> except that the parent is more powerful.
>
> Are there degrees of power? Or if you have access to the parent do you have
> all the power you can have? 

There are degrees only in that you can have a hierarchy of inspectors.
Inspector I is more powerful than inspector J if I is an ancestor of J.

I'll try to improve the docs, such as replacing "more powerful than"
with "an ancestor of".

> I see that the inspector gives you access to
> the data in a structure instance, but does it also give you access to
> meta-data, so that I know that the name of the first field in struct a is x?

You get access to all the metadata.

It turns out that fields currently have only positions, not names, but
that choice was not a good one. We plan to add support for field names
in the near future, in which case the information will be accessible
through an inspector.

> I also don't understand how the root inspector works.  I have found that
> setting (current-inspector root-inspector) delivers endless left parens for
> the (a 1) example, presumably because the display function recursively
> tries to inspect the components of the struct, all the way down.

That's a problem in the pretty printer. The pretty printer's
implementation includes

 (cond
   
   [(struct? v) ]
   
   [(unquoted? v) ]
   )

where `unquoted` is an internal structure. By setting the inspector to
the root inspector, a value that satisfies `unquoted?` also satisfies
`struct?`, and so printing doesn't reach the intended case. I'll push a
repair.


> Finally, does this also work for classes?

Yes. Reflective access to information via `object-info` and
`class-info` is controlled by inspectors.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Fwd: Sandbox evaluation problem - files with comment boxes

[Matthew Flatt]

Yes, it's a bug in v6.1.1. I've just pushed a repair for the next
build.

Do you need a workaround for v6.1.1? Your variant that sets
`sandbox-namespace-specs` is what I would have tried; unfortunately,
that runs into a second bug in v6.1.1 that I recently fixed. If you
need a workaround other than using the next snapshot, I can look harder
for one.

Thanks for the report!

At Mon, 26 Jan 2015 21:27:31 -0500, Nadeem Abdul Hamid wrote:
> Ah, I am still struggling with this sandbox problem! (previous message to
> 'users' below) I'm starting to feel like maybe there's a bug in 6.1.1, or
> else something changed in recent versions that I'm missing. The following
> screenshots show the entire contents of the BSL file that I'm trying to
> load using make-module-evaluator, with gracket v6.1.1 failing the second
> time you try to load the file (or another one) that contains a comment box,
> while racket v6.1.1 and gracket 6.0 don't have this behavior:
> 
> http://cs.berry.edu/~nhamid/temp/foo.png
> http://cs.berry.edu/~nhamid/temp/foo60.png
> 
> ???
> 
> --- nadeem
> 
> 
> 
> -- Forwarded message --
> From: Nadeem Abdul Hamid 
> Date: Sun, Jan 25, 2015 at 10:35 PM
> Subject: Sandbox evaluation problem - files with comment boxes
> To: users 
> 
> 
> 
> I'm trying to create a simple sandbox evaluator (to load in programs in
> *SL). I have the following code:
> 
> #lang racket
> (require racket/sandbox)
> (define E
>   (parameterize ([sandbox-path-permissions
>   '([write "/var/folders"]
> [exists "/"]
> [read "/"])]
>  )
> (make-module-evaluator (string->path "test-file.rkt"
> 
> 
> This works fine as long as test-file.rkt does *not* contain a comment box.
> If the file contains a comment box, then the following error occurs:
> 
> /Applications/Racket
> v6.1.1/share/pkgs/snip-lib/racket/snip/private/load-one.rkt:21:2:
> dynamic-require: unknown module
>   module name: # v6.1.1/share/pkgs/gui-lib/framework/main.rkt">
> 
> 
> 
> I thought maybe parameterizing sandbox-namespace-specs with 'framework
> might do something:
> 
> (require racket/sandbox racket/gui)
> (define E
>   (parameterize ([sandbox-path-permissions
>   '([write "/var/folders"]
> [exists "/"]
> [read "/"])]
>  [sandbox-namespace-specs
>   (list make-gui-namespace 'framework)]
>  )
> (make-module-evaluator (string->path "lab01-insulin.rkt"
> 
> But it results in:
>namespace-attach-module: a different instance of the same module is
> already in the destination namespace
>   module name: "/Applications/Racket
> v6.1.1/collects/racket/stxparam-exptime.rkt"
> 
> 
> Any suggestions?
> 
> Thanks!
> 
> --- nadeem
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] (reposted from users) Noisy compiler at PLTSTDERR=info

[Matthew Flatt]

At Wed, 21 Jan 2015 11:22:38 -0500, Tony Garnock-Jones wrote:
> Over the past few months, more and more subsystems have started logging
> at info level as part of regular compilation.
> 
> I prefer having PLTSTDERR=info in order to catch log-info that happens
> at runtime, and find the compile-time log output quite distracting.
> 
> My current workaround is to set
> 
> PLTSTDERR="info warning@cm warning@compiler/cm warning@module-prefetch
> warning@setup/parallel-build warning@cm-accomplice
> warning@online-check-syntax"
> 
> but this is quite verbose, and as subsystem log messages come and go
> I'll need to keep updating it.
> 
> Could one of the following ideas be worth exploring?
> 
>  1. Have a superlogger for these (and other?) compile-time-ish loggers,
> so that I could write "info warning@compilation", excluding the
> noise in one fell swoop
> 
>  2. Have a phase number associated with logging, so I could say
> "info@0 warning@1", or similar

I have a suggestion below, but it's based on a guess of what you really
want, and it might be a bad guess.


Here's my train of thought:

On 1 above: I've been uncertain of the best way to organize logging
from the start, but the idea of grouping topics hierarchically (such as
a "compilation" topic group) doesn't sound promising. The original
implementation had a notion of hierarchy that I thought might play out
this way, but it didn't. Overall, it seems to be difficult to fit
topics into a hierarchy a priori, in much the same way that it has been
difficult or awkward to group exceptions hierarchically (and few Racket
libraries attempt to extend the exception hierarchy). I guess there's
often a mismatch between the producer's original idea of organization
the and consumer's eventual idea.

On 2 above: I'm not sure why compilation or phase 1 is special. As more
and more libraries use logging, it seems like they will generate noise
(from your perspective) at run time, too.


More generally:

The most successful step in the logging system's evolution was the move
toward "@" specifications that include specific s
of interest. In a growing sea of information, knowing which pieces you
want and specifying those pieces makes sense. Trying to exclude a
category of information is much harder, because it's hard to know all
the information that will be available to exclude.

At the same time, it makes sense to have "error" messages visible all
the time for all topics, because users should see errors that they
didn't anticipate. Similarly, it makes sense for a user to request
extra unanticipated information at the "warning" level.

In this sense, the "info" label is ambiguous. The "info" level could be
like "warning": information that a human consumer might be interested
in without anticipating it, but (unlike "warning") not a suggestion of
a problem. Or "info" could be more like "debug": details that are only
useful when you're looking closely at a specific subsystem. A potential
distinction between "info" and "debug" is that the former makes sense
to the consumer of the subsystem as opposed to the producer.

My only idea, then, is that we're missing a level somewhere between
"debug" and "warning". My guess is that you're getting too much
information from "info", because you wanted a human-volume level of
status information; sometimes "info" is used for that. Other uses of
"info" are more like "debug" --- not really human-volume --- and that's
the part you don't want to see. It happens that many of the current
"debug"-like uses of "info" are in compilation subsystems.

If I'm guessing right, then we could introduce a new level and sort out
existing uses:

 * fatal - last-ditch communication channel before termination
 * error - errors
 * warning - info for humans that alerts a potential mistake
 * info - info for humans on status and progress
 * detail - info for subsystem consumers, not necessarily human-volume
 * debug - info for subsystem implementers, not necessarily human-volume

The idea is that most of the compilation messages could move to
"detail". (Or maybe it makes sense to add the level between "info" and
"warning"; it becomes a transition and compatibility question as much
as anything.)

In retrospect, I realize that I've often struggled with choosing "info"
vs. "debug" levels precisely because I haven't been clear on which is
meant as human-volume --- or even that it makes sense to consider a
human-volume vs. machine-volume distinction. Other times, I've
struggled with "info" vs. "debug" because of a internal vs. external
distinction.


Does any of that connect to your actual situation and goals?

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Full transparency

[Matthew Flatt]

I don't think you want to do anything with the compiler or macros.
Instead, it's a matter of having a sufficiently powerful inspector
(which is the concept of "inspectability" turned into a language
construct).

If you have just

 (struct a (x))
 (a 1)

then the result will print as `#`. But if you use

 (define orig-i (current-inspector))
 (define sub-i (make-inspector orig-i))

 (current-inspector sub-i)
 (struct a (x))
 (define v (a 1))
 (current-inspector orig-i)

 v

Then, the result will print as `(a 1)`. That's because the structure
declaration is creates under an inspector `sub-i` (which is the current
inspector at the time) that is subordinate to the inspector `orig-i`
that is in place when the structure instance is printed.

The current inspector is determined dynamically, which means that if
you're loading some code, you can set the inspector while loading the
code. For example, if "a.rkt" is

 #lang racket/base
 (provide v)

 (struct a (x))
 (define v (a 1))

then

 (define v
   (parameterize ([current-inspector (make-inspector)])
 (dynamic-require "a.rkt" 'v)))
 v

will print the `a` instance transparently.


To protect libraries, there's no safe way to access a root inspector
that controls all structure types when you start Racket. Nothing is
safe from unsafe code, though, and here's an unsafe way to access the
root inspector:

 #lang racket/base
 (require ffi/unsafe)

 (define-cstruct _Scheme_Inspector
   ([stag _short]
[keyex _short]
[depth _int]
[superior _racket]))

 (define root-inspector
   (Scheme_Inspector-superior
((get-ffi-obj 'scheme_get_initial_inspector
  #f
  (_fun -> (_gcable _Scheme_Inspector-pointer))

Using `root-inspector`, you can inspect any structure instance.

At Wed, 21 Jan 2015 23:46:10 -0700, Byron Davies wrote:
> Nice parry!  What may be straightforward to you may not be so obvious to
> me.  But I'll take a look.
> 
> I'm deep into a project using Racket for weakest precondition analysis.
> Every time I'm debugging it seems like I have to write another
> special-purpose accessor, or export some existing accessor up through
> multiple levels in order to get at the data I need at the top-level.  I
> remember how easy it was with the Lisp Machine to navigate through data no
> matter what it was.
> 
> The Lisp Machine offered total transparency, with no real way to protect
> data, to the benefit of the developer.  Racket offers total opacity, to the
> benefit of code security.  I'm hoping there's a middle ground, where
> transparency can be turned on and off.
> 
> Byron
> 
> On Wed, Jan 21, 2015 at 12:20 PM, Matthias Felleisen 
> wrote:
> 
> >
> > Sounds like a straightforward change to the existing macros. Why don't you
> > create a fork and experiment?
> >
> >
> > On Jan 21, 2015, at 1:15 PM, Byron Davies 
> > wrote:
> >
> > > Or, more conservatively, every struct and object in a given package,
> > file, or set of files.
> > >
> > > On Wed, Jan 21, 2015 at 11:03 AM, Byron Davies 
> > wrote:
> > > Would it be easy to create a compiler flag that would make every struct
> > and object transparent?  This would then make it easy to create a Lisp
> > Machine-style Inspector that would be able to roam through every data
> > structure during debugging.
> > >
> > > Byron
> > >
> > >
> > > _
> > >  Racket Developers list:
> > >  http://lists.racket-lang.org/dev
> >
> >
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Compiling a Debuggable Windows Build

[Matthew Flatt]

We don't have build products handy on a server, but it shouldn't be
difficult to build yourself using Visual Studio 10. I recommend setting
up the command line using "vcvarsall.bat x86_amd64", and then build
from the Git repo using "nmake win32-in-place".

That process will give you debugging information in ".pdb" files, at
least for Racket. It will still build with optimization (so detailed
debugging wouldn't be fun), and there won't be debugging information
for the pre-built graphics and GUI libraries. It might be enough
information to help pinpoint the problem, though.

At Tue, 20 Jan 2015 10:25:04 +0100, Christian Eitner wrote:
> Hello everyone,
> 
> I'm trying to analyze bug 14927, which simply states that I cannot
> save any kind of definitions file from DrRacket on Win7 64. The
> message is "Racket GUI application has stopped working", i.e., the
> windows crash dialog.
> 
> I'm pretty sure the problem is related to my machine only, since on a
> colleague's PC everything works fine.
> 
> So I need a Windows build which I can debug. Is there something on the
> build servers which I could use?
> 
> I could build locally, having both a Visual Studio (10, 12, 13) and a
> Cygwin 64 environment, or even cross-compile on Ubuntu if that was
> easier/possible.
> 
> BTW, how are the provided Windows builds being generated?
> 
> Any hints would be welcome.
> 
> Thanks for your consideration,
> 
> Christian
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [plt] Push #29680: master branch updated

[Matthew Flatt]

I'll adjust the docs to clarify that the permission change followed by
delete is a non-atomic sequence, with no attempt to revert a permission
change if the delete fails.

Ending up with just the permission change is one possible outcome, and
I hope the clarification will also make other outcomes more apparent.
For example, it could happen that X's permission is changed, then X is
concurrently replaced by a new file again without write permission, and
the still-in-progress delete fails due to a permission failure after
all.

At Wed, 14 Jan 2015 09:31:37 -0600, Robby Findler wrote:
> Is it perhaps worth being more explicit about this possibility in the
> docs? I'm thinking of a sentence that says "when  is
> set, delete-file may have only the effect of changing the permissions
> on the file" or similar.
> 
> Robby
> 
> On Wed, Jan 14, 2015 at 8:29 AM, Matthew Flatt  wrote:
> > At Wed, 14 Jan 2015 09:07:08 -0500, Neil Toronto wrote:
> >> On 01/13/2015 02:00 PM, mfl...@racket-lang.org wrote:
> >> > 9f3c82c Matthew Flatt  2015-01-13 08:47
> >> > :
> >> > | Windows: change `delete-{file,directory}` to attempt permission 
> correction
> >> > |
> >> > | If a file or directory delete fails, try adjusting the file or 
> >> > directory
> >> > | permissions to allow writes, then try deleting again. This process 
> should
> >> > | provide a more Unix-like experience and make programs behave more
> >> > | consistently.
> >> > |
> >> > | A new `current-force-delete-permissions` parameter provides access to
> >> > | the raw native behavior.
> >>
> >> If I'm understand the new behavior correctly, it's possible for a
> >> failing `delete-file` to raise an exception, having changed the file to
> >> be writable. Do I have that right, and is that OK?
> >
> > That is correct. I'm not too happy about it, but I think it's a good
> > trade-off relative to the old behavior (as a default).
> >
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [plt] Push #29680: master branch updated

[Matthew Flatt]

At Wed, 14 Jan 2015 09:07:08 -0500, Neil Toronto wrote:
> On 01/13/2015 02:00 PM, mfl...@racket-lang.org wrote:
> > 9f3c82c Matthew Flatt  2015-01-13 08:47
> > :
> > | Windows: change `delete-{file,directory}` to attempt permission correction
> > |
> > | If a file or directory delete fails, try adjusting the file or directory
> > | permissions to allow writes, then try deleting again. This process should
> > | provide a more Unix-like experience and make programs behave more
> > | consistently.
> > |
> > | A new `current-force-delete-permissions` parameter provides access to
> > | the raw native behavior.
> 
> If I'm understand the new behavior correctly, it's possible for a 
> failing `delete-file` to raise an exception, having changed the file to 
> be writable. Do I have that right, and is that OK?

That is correct. I'm not too happy about it, but I think it's a good
trade-off relative to the old behavior (as a default).

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] segfault with #%variable-reference

[Matthew Flatt]

It wasn't that I forgot to implement pieces, after all. The problem was
a bug in the byte compiler's handling of `#%variable-reference` when
inlining. I've pushed a repair.

At Mon, 12 Jan 2015 19:24:56 -0700, Matthew Flatt wrote:
> It's supposed to be safe; the behavior in this example is definitely a
> bug.
> 
> The `#%variable-reference` form used to work only on top-level and
> module variables. It seems that I forgot to fill in some pieces when I
> made `#%variable-reference` work on local bindings (several years ago,
> mainly for use with `variable-reference-constant?` in the expansion of
> certain keyword-function forms).
> 
> Thanks for the report!
> 
> At Mon, 12 Jan 2015 19:31:52 -0500, Jon Zeppieri wrote:
> > I'm not sure if #%variable-reference is supposed to be unsafe or not
> > (it's not mentioned in the documentation), but it looks like an
> > attempt to get the location of an identifier that is neither top-level
> > nor module-level results in a hard crash:
> > 
> > ===
> > #lang racket/base
> > 
> > (define (go)
> >   (define foo 3)
> >   (#%variable-reference foo))
> > 
> > (go)
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] segfault with #%variable-reference

[Matthew Flatt]

It's supposed to be safe; the behavior in this example is definitely a
bug.

The `#%variable-reference` form used to work only on top-level and
module variables. It seems that I forgot to fill in some pieces when I
made `#%variable-reference` work on local bindings (several years ago,
mainly for use with `variable-reference-constant?` in the expansion of
certain keyword-function forms).

Thanks for the report!

At Mon, 12 Jan 2015 19:31:52 -0500, Jon Zeppieri wrote:
> I'm not sure if #%variable-reference is supposed to be unsafe or not
> (it's not mentioned in the documentation), but it looks like an
> attempt to get the location of an identifier that is neither top-level
> nor module-level results in a hard crash:
> 
> ===
> #lang racket/base
> 
> (define (go)
>   (define foo 3)
>   (#%variable-reference foo))
> 
> (go)
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] git problem during make

[Matthew Flatt]

I'm not sure what you have in mind with some of the steps, such as step
4 (using `raco pkg` or `raco link`?).

The trace suggests that "gui-lib" is linked as a clone at the `raco
pkg` level. Using `raco link -r` to adjust links installed by the
package manager has a difficult-to-predict effect, but it would not
include unlinking the clone at the `raco pkg` level, so that may be the
source of the problem.

At Sun, 11 Jan 2015 20:00:45 +0100, Jens Axel Søgaard wrote:
> The following problem occurs while running
>  make CPUS=4
> in the plt repository.
> 
>  error: 'merge' is not possible because you have unmerged files.
> 
> And that's confusing, because git status doesn't report anything.
> 
> mba-5:plt soegaard$ git status
> On branch master
> Your branch is up-to-date with 'origin/master'.
> Untracked files:
>   (use "git add ..." to include in what will be committed)
> 
> The full transcript is below.
> 
> Here is what I did to get in this state.
> 
> 1. Updated and built the racket repository
> 2. Create a new directory ~/clones/ and cloned github.org/racket/gui
> 3. Added the cloned gui libs to with raco
> 4. Changed things
> 5. Built racket with modified gui in clones.
> 5. Removed the clones  with raco link -r 
> 6. Updated the repository with git pull
> 7. Tried make.
> 
> So there are unmerged files in ~/clones/... but they are no longer linked to.
> 
> > ~/racket-from-plt-git/plt/racket/bin/raco link -l | grep clone
> >
> 
> /Jens Axel
> 
> 
> 
> mba-5:plt soegaard$ make CPUS=4
> 
> if [ "4" = "" ] ; \
> 
>  then make plain-in-place PKGS="main-distribution
> main-distribution-test" ; \
> 
>  else make cpus-in-place CPUS="4" PKGS="main-distribution
> main-distribution-test" ; fi
> 
> make -j 4 plain-in-place JOB_OPTIONS="-j 4" PKGS="main-distribution
> main-distribution-test"
> 
> make base
> 
> mkdir -p build/config
> 
> echo '#hash((links-search-files . ()))' > build/config/config.rktd
> 
> mkdir -p racket/src/build
> 
> make racket/src/build/Makefile
> 
> make[4]: `racket/src/build/Makefile' is up to date.
> 
> cd racket/src/build; make reconfigure
> 
> make Makefile
> 
> make[5]: `Makefile' is up to date.
> 
> cd racket/src/build; make SELF_RACKET_FLAGS="-G `cd ../../../build/config; 
> pwd`"
> 
> make 3m
> 
> cd racket; make 3m
> 
> make cgc
> 
> make common
> 
> make g-c
> 
> cd sgc; make ../libmzgc.a
> 
> make[10]: `../libmzgc.a' is up to date.
> 
> make foreign-stuff
> 
> cd ../foreign; make all
> 
> make foreign.o
> 
> make[11]: `foreign.o' is up to date.
> 
> make libffi_ON
> 
> make libffi/libffi.la
> 
> make[12]: `libffi/libffi.la' is up to date.
> 
> make dynlib
> 
> cd dynsrc; make dynlib
> 
> make ../mzdyn.o
> 
> make[10]: `../mzdyn.o' is up to date.
> 
> make ../starter
> 
> make[10]: `../starter' is up to date.
> 
> make mzlibrary
> 
> cd src; make all
> 
> make mzobjects
> 
> make[10]: Nothing to be done for `mzobjects'.
> 
> make libracket.a
> 
> make[9]: `libracket.a' is up to date.
> 
> make racketcgc
> 
> make[8]: `racketcgc' is up to date.
> 
> cd gc2; make all
> 
> make check-sdep
> 
> ../racketcgc -G /Users/soegaard/racket-from-plt-git/plt/build/config
> -cqu ../../../racket/gc2/check-sdep.rkt
> 
> make xsrc/precomp.h
> 
> make[8]: `xsrc/precomp.h' is up to date.
> 
> make xobjects
> 
> make[8]: Nothing to be done for `xobjects'.
> 
> make ../racket3m
> 
> make[8]: `../racket3m' is up to date.
> 
> cd dynsrc; make dynlib3m
> 
> make ../mzdyn3m.o
> 
> make[8]: `../mzdyn3m.o' is up to date.
> 
> cd gc2; make ../racket3m
> 
> make[7]: `../racket3m' is up to date.
> 
> make gracket-3m
> 
> cd gracket; make 3m
> 
> make resources
> 
> :
> 
> cd gc2; make 3m
> 
> make ../gracket3m
> 
> make ../GRacket3m.app/Contents/MacOS/GRacket3m
> 
> make[10]: `../GRacket3m.app/Contents/MacOS/GRacket3m' is up to date.
> 
> cd racket/src/build; make install SELF_RACKET_FLAGS="-G `cd
> ../../../build/config; pwd`" PLT_SETUP_OPTIONS="-j 4 "
> 
> make install-3m
> 
> make install-3m-common
> 
> make install-common-first
> 
> mkdir -p "/Users/soegaard/racket-from-plt-git/plt/racket/bin"
> "/Users/soegaard/racket-from-plt-git/plt/racket/collects"
> "/Users/soegaard/racket-from-plt-git/plt/racket/doc"
> "/Users/soegaard/racket-from-plt-git/plt/racket/lib"
> "/Users/soegaard/racket-from-plt-git/plt/racket/include"
> "/Users/soegaard/racket-from-plt-git/plt/racket/lib"
> "/Users/soegaard/racket-from-plt-git/plt/racket/share"
> "/Users/soegaard/racket-from-plt-git/plt/racket/etc"
> "/Users/soegaard/racket-from-plt-git/plt/racket/share/applications"
> "/Users/soegaard/racket-from-plt-git/plt/racket/man"
> 
> cd racket; make install-3m
> 
> make headers
> 
> ./racketcgc -cqu ../../racket/mkincludes.rkt
> "/Users/soegaard/racket-from-plt-git/plt/racket/include"
> "../../racket" .
> 
> Making /Users/soegaard/racket-from-plt-git/plt/racket/include
> 
> Done.
> 
> make osx-install
> 
> make unix-install
> 
> cd ..; rm -f "/Users/soegaard/racket-from-plt-git/plt/racke

Re: [racket-dev] having zo files from two versions

[Matthew Flatt]

I see. If you compile with v5.2.1 first, then it puts files immediately
in "compiled", and those will be found by v6.1.

If you compile v6.1 first, then the files go in a subdirectory of
"compiled", and they won't interfere with v5.2.1, while v6.1 will
continue to find the files in the subdirectory. That's what I tried, so
I didn't think of the problem with the other order.

Were you able to solve this by compiling with v6.1 first? Or some other
approach?

At Tue, 6 Jan 2015 15:04:40 -0800, Dan Liebgold wrote:
> Ok, that seems to be exactly what I'm looking for -- however I tried it and
> it eventually seems to wind up using just the "compiled" path during raco
> setup.
> 
> Example output is here (I've echoed PLTCOMPILEDROOTS at the top, and note
> this is under windows): http://pasterack.org/pastes/45913
> 
> Is that enough context to see the issue?
> 
> On Tue, Jan 6, 2015 at 11:58 AM, Matthew Flatt  wrote:
> 
> > At Tue, 06 Jan 2015 14:14:22 -0500, Neil Van Dyke wrote:
> > > Dan Liebgold wrote on 01/06/2015 02:00 PM:
> > > > What is a straightforward way to designate the "compiled" directory to
> > > > look for zo files in that can be based on the Racket version?  I'd
> > > > like to have Racket 5.2.1 and 6.1 running in parallel to aid in
> > > > upgrading our version.
> > > > Thanks!
> > >
> > > I'd like for this to be the default behavior for Racket.
> > >
> > > One possible way: Insert a directory level between the `compiled`
> > > directory and its contents, with the directory named with the Racket
> > > version number with which the code is compiled.  Only that Racket
> > > version will run that compiled code.
> >
> > Although it's not the default, you can get that behavior by setting the
> > `PLTCOMPILEDROOTS` environment variable to
> >
> >  compiled/@(version):
> >
> > The trailing ":" allows compiled files to be found in "compiled" in the
> > installation, while newly generated ".zo" files are put in a
> > version-specific subdirectory.
> >
> > See also
> >
> >  http://lists.racket-lang.org/dev/archive/2012-September/010386.html
> >
> >
> 
> 
> -- 
> Dan Liebgold[dan.liebg...@gmail.com]
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] segfault during make

[Matthew Flatt]

Do you have the latest "images-doc" and/or "gui-lib" packages?

Recently, there was a problem with the "images" documentation where it
tried to use `racket/gui` at document-build time. At the same time,
there was also a problem in `racket/gui` that could cause a crash
(mainly on Mac OS X) after `racket/gui` was used during
documentation-build time. Both of those problems have been fixed.

At Tue, 06 Jan 2015 22:14:53 +, Spencer Florence wrote:
> Hey all,
> 
> When I try to run `make` on the current head the build segfaults while
> making the documentation. The last few lines of the output are:
> 
> raco setup: 2 rendering:
> /pfds/pfds/scribblings/functional-data-structures.scrbl
> raco setup: 1 rendering:
> /future-visualizer/future-visualizer/scribblings/future-visualizer.scrbl
> raco setup: 1 rendering: /games/scribblings/games.scrbl
> raco setup: 1 rendering:
> /racket-doc/scribblings/getting-started/getting-started.scrbl
> raco setup: 1 rendering: /games/gl-board-game/gl-board-game.scrbl
> raco setup: 1 rendering: /htdp-doc/graphics/scribblings/graphics.scrbl
> raco setup: 1 rendering: /gui-doc/scribblings/gui/gui.scrbl
> raco setup: 0 rendering: /racket-doc/help/help.scrbl
> raco setup: 0 rendering: /htdp-doc/htdp/htdp.scrbl
> raco setup: 0 rendering:
> /htdp-doc/scribblings/htdp-langs/htdp-langs.scrbl
> raco setup: 3 rendering: /html-doc/html/html.scrbl
> raco setup: 3 rendering: /images-doc/images/scribblings/images.scrbl
> raco setup: 1 rendering: /racket-doc/scribblings/inside/inside.scrbl
> raco setup: 1 rendering: /racket-doc/json/json.scrbl
> raco setup: 1 rendering: /lazy/lazy.scrbl
> raco setup: 1 rendering:
> /macro-debugger/macro-debugger/macro-debugger.scrbl
> raco setup: 0 rendering: /make/make.scrbl
> raco setup: 0 rendering: /math-doc/math/scribblings/math.scrbl
> raco setup: 1 rendering: /racket-doc/scribblings/more/more.scrbl
> raco setup: 1 rendering: /gui-doc/mrlib/scribblings/mrlib.scrbl
> raco setup: 1 rendering: /mysterx/scribblings/mysterx.scrbl
> raco setup: 1 rendering: /mzcom/mzcom.scrbl
> raco setup: 1 rendering:
> /compatibility-doc/mzlib/scribblings/mzlib.scrbl
> raco setup: 1 rendering: /mzscheme-doc/mzscheme/mzscheme.scrbl
> raco setup: 1 rendering: /net-doc/net/scribblings/net.scrbl
> raco setup: 1 rendering: /racket-doc/openssl/openssl.scrbl
> raco setup: 1 rendering:
> /optimization-coach/optimization-coach/scribblings/optimization-coach.scrb
> l
> raco setup: 1 rendering:
> /parser-tools-doc/parser-tools/parser-tools.scrbl
> raco setup: 1 rendering: /pict-doc/pict/scribblings/pict.scrbl
> raco setup: 3 rendering:
> /pict-snip-doc/scribblings/pict-snip/pict-snip.scrbl
> make[1]: *** [plain-in-place] Segmentation fault: 11
> 
> I'm on OSX 10.10.1.
> 
> Anyone know whats going on/how to debug this?
> 
> --spencer
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] lib changes between versions

[Matthew Flatt]

If your library's name is also `json` you could put in a place that is
included in PLTCOLLECTS when you run v5.2.1, but use a PLTCOLLECTS that
doesn't include it when your run v6.1 (in much the same way that you
use different PLTCOMPILEDROOTS settings for the different versions).

If your library has a different name, you could similarly point
PLTCOLLECTS at it for v.5.2.1, and use a different implementation ---
accessed either via PLTCOLLECTS or by installing as a package --- that
wraps the main-distribution one for v6.1.

At Wed, 7 Jan 2015 17:56:16 -0800, Dan Liebgold wrote:
> Actually this issue is still perplexing me. In 5.2.1 I have my own json lib
> which provides jsexpr->string. In 6.1 it's part of the distribution's
> collects directory.
> 
> Is there a command line for racket that'll cause it to find mine under
> 5.2.1 and the standard lib in 6.1 (skipping mine)?
> 
> Dan
> 
> On Wed, Jan 7, 2015 at 4:51 PM, Dan Liebgold  wrote:
> 
> > Ugh. Never mind... the old json lib is mine. Carry on :)
> >
> > On Wed, Jan 7, 2015 at 4:45 PM, Dan Liebgold 
> > wrote:
> >
> >>
> >> I'm maintaining the same racket code between Racket version 5.2.1 and
> >> 6.1. One thing that changed between those version was the json to string
> >> (and vice versa) lib functions.
> >>
> >> Is there a straightforward way to define those functions so they'll work
> >> with both lib versions?
> >>
> >> Thanks,
> >> --
> >> Dan Liebgold[dan.liebg...@gmail.com]
> >>
> >
> >
> >
> > --
> > Dan Liebgold[dan.liebg...@gmail.com]
> >
> 
> 
> 
> -- 
> Dan Liebgold[dan.liebg...@gmail.com]
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] LTS Racket?

[Matthew Flatt]

Some of us have discussed this in recent months. So far, our conclusion
has been "not yet".

At Mon, 05 Jan 2015 10:56:01 +, Stephen De Gabrielle wrote:
> (Sorry if this is the wrong list)
> 
> I just saw the LTS Haskell announcement, and it made me wonder if there is
> a racket equivalent?
> 
> I've had occasion to use packages from older versions of racket recently
> and the ability to run quite old code unchanged is remarkable, despite many
> significant changes to racket itself in the past few years.
> 
> The rate of change in Racket *is* terrifyingly but because of the
> remarkable backwards compatibility I'm not terribly concerned.
> 
> Kind regards and HNY,
> Stephen
> 
> Announcing LTS Haskell 1.0 | FP Complete
> https://www.fpcomplete.com/blog/2015/01/announcing-lts-haskell-1-0
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Build error

[Matthew Flatt]

I wasn't able to replicate the problem, but I think I see what could go
wrong. I've pushed an attempt at a repair.

At Wed, 7 Jan 2015 18:31:35 +0100, Jens Axel Søgaard wrote:
> Hi All,
> 
> I got a contract when I tried to build racket. Any ideas?
> 
> The main error is below.
> Entire log is here:
> https://gist.github.com/soegaard/aa8ea014e043a899a5a2
> 
> /Jens Axel
> 
> 
> package-source->name+type: contract violation
>   expected: (or/c #f (or/c (quote name) (quote file) (quote dir)
> (quote git) (quote github) (quote clone) (quote file-url) (quote
> dir-url) (quote link) (quote static-link)))
>   given: 'catalog
>   in: the 2nd argument of
>   (->*
>(string?
> (or/c
>  #f
>  (or/c 'name 'file 'dir 'git  'github 'clone  'file-url
> 'dir-url  'link'static-link)))
>
> 
>   contract from: /pkg/name.rkt
> 
>   blaming: /pkg/private/install.rkt
> 
>(assuming the contract is correct)
> 
>   at: /pkg/name.rkt:11.3
> 
>   context...:
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/racket/contract/private/
> blame.rkt:143:0:
> raise-blame-error16
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/racket/contract/private/
> arrow-val-first.rkt:283:3
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/pkg/private/install.rkt:
> 925:2:
> update-loop
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/racket/list.rkt:429:2:
> append-map
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/pkg/private/install.rkt:
> 1097:0:
> pkg-update195
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/racket/file.rkt:368:8
> 
>
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/racket/file.rkt:357:0:
> call-with-file-lock40
> 
>/Users/soegaard/racket-from-plt-git/plt/racket/collects/pkg/main.rkt:307:16
> 
>(submod 
> /Users/soegaard/racket-from-plt-git/plt/racket/collects/pkg/main.rkt
> main): [running body]
> 
>/Users/soegaard/racket-from-plt-git/plt/racket/collects/pkg/raco.rkt:
> [traversing imports]
> 
>/Users/soegaard/racket-from-plt-git/plt/racket/collects/raco/raco.rkt:
> [running body]
> 
>/Users/soegaard/racket-from-plt-git/plt/racket/collects/raco/main.rkt:
> [running body]
> 
> make[2]: *** [plain-in-place] Error 1
> 
> make[1]: *** [cpus-in-place] Error 2
> 
> make: *** [in-place] Error 2
> 
> 
> -- 
> --
> Jens Axel Søgaard
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] having zo files from two versions

[Matthew Flatt]

At Tue, 06 Jan 2015 14:14:22 -0500, Neil Van Dyke wrote:
> Dan Liebgold wrote on 01/06/2015 02:00 PM:
> > What is a straightforward way to designate the "compiled" directory to 
> > look for zo files in that can be based on the Racket version?  I'd 
> > like to have Racket 5.2.1 and 6.1 running in parallel to aid in 
> > upgrading our version.
> > Thanks!
> 
> I'd like for this to be the default behavior for Racket.
> 
> One possible way: Insert a directory level between the `compiled` 
> directory and its contents, with the directory named with the Racket 
> version number with which the code is compiled.  Only that Racket 
> version will run that compiled code.

Although it's not the default, you can get that behavior by setting the
`PLTCOMPILEDROOTS` environment variable to

 compiled/@(version):

The trailing ":" allows compiled files to be found in "compiled" in the
installation, while newly generated ".zo" files are put in a
version-specific subdirectory.

See also

 http://lists.racket-lang.org/dev/archive/2012-September/010386.html

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] having zo files from two versions

[Matthew Flatt]

Does it work to set PLTCOMPILEDROOTS so that it's used by version 6.1,
while version 5.2.1 would still read and write files at the default
location?

The documentation now tracks changes since v6.0, while changes relative
to previous versions are recorded separately (and less usefully) in the
release notes.

At Tue, 6 Jan 2015 11:46:11 -0800, Dan Liebgold wrote:
> That method isn't available in 5.2.1, unfortunately. Oh, and wishlist item:
> the reference docs specify the version the function was introduced in.
> 
> On Tue, Jan 6, 2015 at 11:39 AM, Matthew Flatt  wrote:
> 
> > You probably want to set `current-compiled-file-roots`, which can be
> > initialized by the `PLTCOMPILEDROOTS` environment variable, instead.
> >
> > At Tue, 6 Jan 2015 11:27:35 -0800, Dan Liebgold wrote:
> > > Hmmm... so this should be as easy to implement as:
> > >
> > >   (use-compiled-file-paths (list (build-path "compiled" (version
> > >
> > > Right?  Trying it out now.
> > >
> > > Dan
> > >
> > > On Tue, Jan 6, 2015 at 11:14 AM, Neil Van Dyke 
> > wrote:
> > >
> > > > Dan Liebgold wrote on 01/06/2015 02:00 PM:
> > > >
> > > >> What is a straightforward way to designate the "compiled" directory to
> > > >> look for zo files in that can be based on the Racket version?  I'd
> > like to
> > > >> have Racket 5.2.1 and 6.1 running in parallel to aid in upgrading our
> > > >> version.
> > > >> Thanks!
> > > >>
> > > >
> > > > I'd like for this to be the default behavior for Racket.
> > > >
> > > > One possible way: Insert a directory level between the `compiled`
> > > > directory and its contents, with the directory named with the Racket
> > > > version number with which the code is compiled.  Only that Racket
> > version
> > > > will run that compiled code.
> > > >
> > > > (When I've been using Racket version A to run a code tree, and then use
> > > > Racket version B with the same code tree, I don't mind waiting for a
> > > > recompile of the code tree, and I'd really prefer it not disturb the
> > code
> > > > compiled with Racket version A.  I also don't mind cluttering disk
> > space
> > > > with compiled code for Racket versions that I no longer use; I can
> > always
> > > > make a script to clean those up.)
> > > >
> > > > Neil V.
> > > >
> > > >
> > >
> > >
> > > --
> > > Dan Liebgold[dan.liebg...@gmail.com]
> > > _
> > >   Racket Developers list:
> > >   http://lists.racket-lang.org/dev
> >
> 
> 
> 
> -- 
> Dan Liebgold[dan.liebg...@gmail.com]
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] having zo files from two versions

[Matthew Flatt]

You probably want to set `current-compiled-file-roots`, which can be
initialized by the `PLTCOMPILEDROOTS` environment variable, instead.

At Tue, 6 Jan 2015 11:27:35 -0800, Dan Liebgold wrote:
> Hmmm... so this should be as easy to implement as:
> 
>   (use-compiled-file-paths (list (build-path "compiled" (version
> 
> Right?  Trying it out now.
> 
> Dan
> 
> On Tue, Jan 6, 2015 at 11:14 AM, Neil Van Dyke  wrote:
> 
> > Dan Liebgold wrote on 01/06/2015 02:00 PM:
> >
> >> What is a straightforward way to designate the "compiled" directory to
> >> look for zo files in that can be based on the Racket version?  I'd like to
> >> have Racket 5.2.1 and 6.1 running in parallel to aid in upgrading our
> >> version.
> >> Thanks!
> >>
> >
> > I'd like for this to be the default behavior for Racket.
> >
> > One possible way: Insert a directory level between the `compiled`
> > directory and its contents, with the directory named with the Racket
> > version number with which the code is compiled.  Only that Racket version
> > will run that compiled code.
> >
> > (When I've been using Racket version A to run a code tree, and then use
> > Racket version B with the same code tree, I don't mind waiting for a
> > recompile of the code tree, and I'd really prefer it not disturb the code
> > compiled with Racket version A.  I also don't mind cluttering disk space
> > with compiled code for Racket versions that I no longer use; I can always
> > make a script to clean those up.)
> >
> > Neil V.
> >
> >
> 
> 
> -- 
> Dan Liebgold[dan.liebg...@gmail.com]
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] internal error during gc

[Matthew Flatt]

I think the root of the problem is that `raco setup` gets anywhere
close to the 1 GB limit. Also, since the "images.scrbl" document
involves images, the problem may be related to using foreign libraries
when close to the memory limit (where the foreign library can't force a
Racket GC to recover from an allocation failure).

Rendering "images.scrbl" by itself peaks somewhere between 600 MB and
700 MB in a 64-bit build, so I wouldn't expect it to be anywhere close
to 1GB in 32-bit mode. Building within `raco setup` adds some extra
overhead, and there may be a leak or finalizer interaction that makes
`raco setup` carry along too much memory by the time it gets to
"images.scrbl".

Of course, 700 MB is a lot of memory to build a 100-page manual. It
takes only 700 MB instead of many GB because I periodically hunt down
inefficiencies and leaks in an attempt to keep `raco setup` under
control. It's past time for another hunt.

At Tue, 30 Dec 2014 08:00:12 -0500, Philippe Meunier wrote:
> Juan Francisco Cantero Hurtado wrote:
> >Sincerely, I don't know if the bug is in racket or in openbsd.
> 
> Okay...  Well, is anyone interested in trying to locate the cause of
> the problem?  I can recompile the GC with debugging options turned
> on, etc., if more information is needed, but I don't know how the GC's
> code actually works and I don't have the time to dive into that...
> 
> Philippe
> 
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] support for arm64 / aarch64

[Matthew Flatt]

It looks like this patch was submitted for v6.1. Version 6.1.1 (the
current release), uses SGC instead of Boehm's GC during the build
process by default. So, it at least avoids this immediate problem.

I can't think of any other problem that would turn up in v6.1.1, but
I'm not sure it will work. We'd definitely welcome feedback on whether
Racket 6.1.1 builds on AArch64, or where it gets stuck if not.

Thanks!

At Mon, 29 Dec 2014 11:09:16 +0100, David Bremner wrote:
> 
> A user submitted the attached patch for partial support for arm64 on
> Debian/Ubuntu.  It only enables cgc, 3m still hangs during compilation.
> If the patch makes sense to you, perhaps you'd like to apply it
> upstream.
> 
> d
> 
> From 7b5acfba9a1df00f0427d1d2e1a92570da3ab2d1 Mon Sep 17 00:00:00 2001
> From: Yvan Roux 
> Date: Wed, 23 Jan 2013 23:15:57 +0400
> Subject: [PATCH] Add AArch64 (64-bit ARM) target support
> 
> * include/private/gcconfig.h (AARCH64): New macro (defined only if
> __aarch64__).
> * include/private/gcconfig.h (mach_type_known): Update comment adding
> ARM AArch64 target.
> * include/private/gcconfig.h (NOSYS, mach_type_known, CPP_WORDSZ,
> MACH_TYPE, ALIGNMENT, HBLKSIZE, OS_TYPE, LINUX_STACKBOTTOM,
> DYNAMIC_LOADING, DATASTART, DATAEND, STACKBOTTOM): Define for AArch64.
> ---
>  include/private/gcconfig.h | 37 +
>  1 file changed, 37 insertions(+)
> 
> --- a/src/racket/gc/include/private/gcconfig.h
> +++ b/src/racket/gc/include/private/gcconfig.h
> @@ -76,6 +76,13 @@
>  # endif
>  
>  /* Determine the machine type: */
> +# if defined(__aarch64__)
> +#define AARCH64
> +#if !defined(LINUX)
> +#  define NOSYS
> +#  define mach_type_known
> +#endif
> +# endif
>  # if defined(__arm__) || defined(__thumb__)
>  #define ARM32
>  #if !defined(LINUX) && !defined(NETBSD) && !defined(OPENBSD)
> @@ -249,6 +256,10 @@
>  #define IA64
>  #define mach_type_known
>  # endif
> +# if defined(LINUX) && defined(__aarch64__)
> +#define AARCH64
> +#define mach_type_known
> +# endif
>  # if defined(LINUX) && defined(__arm__)
>  #define ARM32
>  #define mach_type_known
> @@ -529,6 +540,7 @@
>   /*  running Amdahl UTS4 */
>  /* S390   ==> 390-like machine  */
>   /*  running LINUX   */
> + /* AARCH64==> ARM AArch64   */
>   /* ARM32  ==> Intel StrongARM   */
>   /* IA64   ==> Intel IPF */
>   /*(e.g. Itanium)*/
> @@ -1818,6 +1830,31 @@
>  #   endif
>  # endif
>  
> +# ifdef AARCH64
> +#   define CPP_WORDSZ 64
> +#   define MACH_TYPE "AARCH64"
> +#   define ALIGNMENT 8
> +#   ifndef HBLKSIZE
> +# define HBLKSIZE 4096
> +#   endif
> +#   ifdef LINUX
> +# define OS_TYPE "LINUX"
> +# define LINUX_STACKBOTTOM
> +# define DYNAMIC_LOADING
> +  extern int __data_start[];
> +# define DATASTART ((ptr_t)__data_start)
> +  extern char _end[];
> +# define DATAEND ((ptr_t)(&_end))
> +#   endif
> +#   ifdef NOSYS
> +  /* __data_start is usually defined in the target linker script.   */
> +  extern int __data_start[];
> +# define DATASTART ((ptr_t)__data_start)
> +  extern void *__stack_base__;
> +# define STACKBOTTOM ((ptr_t)__stack_base__)
> +#   endif
> +# endif
> +
>  # ifdef ARM32
>  #   define CPP_WORDSZ 32
>  #   define MACH_TYPE "ARM32"
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [racket/typed-racket] 3e45f2: Adjust TR test package dependencies

[Matthew Flatt]

At Wed, 17 Dec 2014 10:38:59 -0500, Asumu Takikawa wrote:
> On 2014-12-17 08:22:33 -0700, Matthew Flatt wrote:
> > Commit 134f793 moved files out of a directory named "tests". By
> > default, a directory named "tests" is treated specially by the
> > dependency checker, reflecting that the directory name is treated
> > specially for the creation of "binary" and "binary library" variants of
> > a package: Unless the "info.rkt" file says otherwise, anything in
> > "tests" is treated as "build-time" code, and so the directory's content
> > creates build dependencies, not general dependencies.
> >
> > That's why it was sufficient to list "unstable-flonum-lib" in
> > `build-deps` before commit 134f793, and why "unstable-flonum-lib" had
> > to be moved to `deps` afterward.
> 
> Thanks Matthew, that clears it up. Is this documented anywhere in the
> pkg docs? I looked in section 5 of the package docs which says that
> "tests" folders are pruned for binary packages, but I'm not sure if that
> implies that the dependency checker treats it as "build-time" code.

The `build-deps` field for "info.rkt" is defined in terms of binary
packages. The documentation there could draw out some of the
implications, such as the possibility of listing packages in
`build-deps` instead of `deps` when they're used only within files in
"tests" or "scribblings" subdirectories and those subdirectories are
treated in the default way. Is that what you had in mind?

Or maybe the issue is that the dependency checking performed by `raco
setup` is in some sense unspecified. The `deps` and `build-deps` fields
are defined in terms of how the package system uses them for installing
additional packages. The checking performed by `raco setup` is separate
and `lint`-like: it find apparent mismatches between dependency
declarations and implementation references, but there's no guarantee
that all problems are found or that apparent mismatches are actual. The
`raco setup` docs could use a section that describes more precisely how
dependencies are checked.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [racket/typed-racket] 3e45f2: Adjust TR test package dependencies

[Matthew Flatt]

At Tue, 16 Dec 2014 16:32:59 -0500, Asumu Takikawa wrote:
> On 2014-12-16 13:26:38 -0800, Asumu Takikawa wrote:
> >   Branch: refs/heads/master
> >   Home:   https://github.com/racket/typed-racket
> >   Commit: 3e45f258bed22d16b1f7ab1cac701d20c5f57e06
> >   
> https://github.com/racket/typed-racket/commit/3e45f258bed22d16b1f7ab1cac701d20c5
> f57e06
> >   Author: Asumu Takikawa 
> >   Date:   2014-12-16 (Tue, 16 Dec 2014)
> > 
> >   Changed paths:
> > M typed-racket-test/info.rkt
> > 
> >   Log Message:
> >   ---
> >   Adjust TR test package dependencies
> 
> I made this change to satisfy the package dependency checker, but it
> seemed like the dependency should've been detected prior to Vincent's
> recent commit (that reduced a level of nesting).
> 
> Is it possible that the dependency checker was failing to detect some
> dependencies?

Commit 134f793 moved files out of a directory named "tests". By
default, a directory named "tests" is treated specially by the
dependency checker, reflecting that the directory name is treated
specially for the creation of "binary" and "binary library" variants of
a package: Unless the "info.rkt" file says otherwise, anything in
"tests" is treated as "build-time" code, and so the directory's content
creates build dependencies, not general dependencies.

That's why it was sufficient to list "unstable-flonum-lib" in
`build-deps` before commit 134f793, and why "unstable-flonum-lib" had
to be moved to `deps` afterward.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] collects search order

[Matthew Flatt]

At Thu, 11 Dec 2014 17:00:14 -0800, Dan Liebgold wrote:
> If I use the -X and -S command line parameters to Racket to make my local
> collects dir the first one searched, it makes it so I can't do (require
> srfi/1).

Yes, this is subtle and confusing.

> Is there something special about the srfi package? (it's layout seems
> different)

The `srfi/1` library is installed as a package, so it's not in the main
"collects" tree. Package paths are registered in "etc/config.rktd".
When that file isn't present or doesn't explicitly list the location of
packages, then the default location is within the "share" directory.
The default location of the "share" directory, in turn, is "../share".

(I'm using paths for an in-place installation; the details vary for a
Unix-style installation, or I may have a detail slightly wrong, but
it's about the same.)

The "../share" path is relative... to what? It turns out that it's
relative to the main "collects" directory. So, when you provide `-X`,
you're also changing the location of package installations.

It may seem odd that the default location of packages (and even the
configuration file) is derived from the main "collects" path, but it's
part of a delicate balance of configuration options that keeps many old
things working, enables `make` in the top-level of the Racket
repository, and so on. The default paths are convenient for some
purposes and inconvenient for others.


To take control of the various paths, create a configuration file as
"config.rktd", and then point Racket at that file's directory using
`-G`. For example, if I copy the "collects" directory of my
installation to "/tmp/collects", then

  racket -X /tmp/collects -l srfi/1

fails, as you say. But if I create "/tmp/etc/config.rktd" with the
content

 #hash((share-dir . "/original/path/to/racket/share"))

then

  racket -X /tmp/collects -G /tmp/etc -l srfi/1

works.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Multiple 'raco make' processes

[Matthew Flatt]

We do protect against those kinds of problems (even on Windows, where
it isn't trivial).

At Tue, 9 Dec 2014 19:00:09 -0600, Robby Findler wrote:
> I think even stranger things can happen because of race-conditions at
> the filesystem level (altho we could probably protect against that if
> we wanted).
> 
> Robby
> 
> 
> On Tue, Dec 9, 2014 at 6:22 PM, Jay McCarthy  wrote:
> > Specifically, you can get errors like "Called export 72 and expected
> > set->list but got list->set. Chaos!"
> >
> > Jay
> >
> > On Tue, Dec 9, 2014 at 3:09 PM, Matthew Flatt  wrote:
> >> Unfortunately, the bytecode compiler is not completely deterministic.
> >> Generating the same ".zo" file from the same source is likely to
> >> produce different bytes each time. The root causes are various counters
> >> and hash orders, and I hope to fix that eventually. For now, since the
> >> generated bytecode is different, there can be inconsistencies instead
> >> of just duplicated work.
> >>
> >> At Tue, 9 Dec 2014 11:54:16 -0800, Dan Liebgold wrote:
> >>> If the multiple 'raco make's would produce the same results, then there
> >>> should be no inconsistency, just duplicate work, right?
> >>>
> >>> And my use case is a bit different... I need to spawn the multiple 'raco
> >>> makes', rather than have raco make spawn multiple racket instances (via 
> >>> the
> >>> -j flag). Although this particular setup will bear some examination.
> >>>
> >>> On Tue, Dec 9, 2014 at 11:47 AM, Robby Findler 
> >>> 
> >>> wrote:
> >>>
> >>> > Ah sorry: meant to add: did you try the -j flag?
> >>> >
> >>> >
> >>> > On Tuesday, December 9, 2014, Robby Findler 
> >>> > 
> >>> > wrote:
> >>> >
> >>> >> I think they can stomp on each other and you can get inconsistent
> >>> >> results, theoretically.
> >>> >>
> >>> >> Robby
> >>> >>
> >>> >> On Tuesday, December 9, 2014, Dan Liebgold 
> >>> >> wrote:
> >>> >>
> >>> >>> If I have multiple instances of raco make running and some of the 
> >>> >>> files
> >>> >>> they are checking/rebuilding are shared across the instances... what
> >>> >>> happens?  Does raco make have lock to ensure no contention? Or does 
> >>> >>> each
> >>> >>> process potentially redo some work?
> >>> >>>
> >>> >>> --
> >>> >>> Dan Liebgold[dan.liebg...@gmail.com]
> >>> >>>
> >>> >>
> >>>
> >>>
> >>> --
> >>> Dan Liebgold[dan.liebg...@gmail.com]
> >>> _
> >>>   Racket Developers list:
> >>>   http://lists.racket-lang.org/dev
> >> _
> >>   Racket Developers list:
> >>   http://lists.racket-lang.org/dev
> >
> >
> >
> > --
> > Jay McCarthy
> > http://jeapostrophe.github.io
> >
> >"Wherefore, be not weary in well-doing,
> >   for ye are laying the foundation of a great work.
> > And out of small things proceedeth that which is great."
> >   - D&C 64:33
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Multiple 'raco make' processes

[Matthew Flatt]

Unfortunately, the bytecode compiler is not completely deterministic.
Generating the same ".zo" file from the same source is likely to
produce different bytes each time. The root causes are various counters
and hash orders, and I hope to fix that eventually. For now, since the
generated bytecode is different, there can be inconsistencies instead
of just duplicated work.

At Tue, 9 Dec 2014 11:54:16 -0800, Dan Liebgold wrote:
> If the multiple 'raco make's would produce the same results, then there
> should be no inconsistency, just duplicate work, right?
> 
> And my use case is a bit different... I need to spawn the multiple 'raco
> makes', rather than have raco make spawn multiple racket instances (via the
> -j flag). Although this particular setup will bear some examination.
> 
> On Tue, Dec 9, 2014 at 11:47 AM, Robby Findler 
> wrote:
> 
> > Ah sorry: meant to add: did you try the -j flag?
> >
> >
> > On Tuesday, December 9, 2014, Robby Findler 
> > wrote:
> >
> >> I think they can stomp on each other and you can get inconsistent
> >> results, theoretically.
> >>
> >> Robby
> >>
> >> On Tuesday, December 9, 2014, Dan Liebgold 
> >> wrote:
> >>
> >>> If I have multiple instances of raco make running and some of the files
> >>> they are checking/rebuilding are shared across the instances... what
> >>> happens?  Does raco make have lock to ensure no contention? Or does each
> >>> process potentially redo some work?
> >>>
> >>> --
> >>> Dan Liebgold[dan.liebg...@gmail.com]
> >>>
> >>
> 
> 
> -- 
> Dan Liebgold[dan.liebg...@gmail.com]
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] The repository is now split

[Matthew Flatt]

At Fri, 5 Dec 2014 14:04:47 -0800, John Clements wrote:
> 1) compilation failed because it couldn't find the 'racket' collection, but
> I noticed that it was referring to a nonexistent path, presumably because I
> had moved the root of the installation.  Has that always been a bad idea?

Ok, yes, it has always been a bad idea to do that without re-running
`configure`.

When I tried moving my checkout and using `make`, I got

 ../racket/racket3m -G /Users/mflatt/splt/build/config -cqu \
"../../gracket/../mac/rename-app.rkt" \
"/Users/mflatt/plt/racket/lib/GRacket.app" "GRacket3m" "GRacket"
 standard-module-name-resolver: collection not found
   for module path: (submod racket/base reader)
   collection: "racket"
   in collection directories: [...]

That's because a makefile generated by `configure` includes an absolute
path. Getting rid of absolute paths in the makefiles is difficult and
probably not worthwhile.

In contrast, a `raco setup` or `raco pkg update` should work after
moving a checkout, because an in-place build is supposed to be movable
at the Racket level.


Continuing with your report at step 2, you deleted "racket/src/build",
which should allow `make` to work, because it re-runs `configure`. You
ran into trouble, but the error message told you how to recover --- and
that won't happen again, since the way `make` links packages has
stabilized.

At steps 3 and 4, `make` didn't do enough work, but I've since changed
the makefile to fix that problem.

At step 5, you ran `make setup`, and it still failed -- but it should
have worked, as noted above. I'm still stumped about that failure,
because `raco setup` (which is now run by `make`) worked for me at that
point. I'll just have to keep an eye out for similar problems.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] The repository is now split

[Matthew Flatt]

I think this problem wasn't due to an interrupted `make`.

The "data-lib" package changed today in a way that made "data.scrbl"
run for a very long time. The problem has been fixed, which is why your
fresh clone worked.

Even though `raco setup` showed a status line for "search.scrbl", I
imagine that a separate place was still running "data.scrbl" (with an
earlier status line), and that was the one that hadn't completed.

At Fri, 5 Dec 2014 19:05:41 -0500, Stephen Chang wrote:
> One more issue,
> 
> If I resume an interrupted "make" with "make" again, the compile takes
> around ~5hrs to complete on my (modern, lots of memory, desktop)
> machine. It gets stuck around "search.scrbl", as Vincent mentioned on
> IRC (but eventually finishes).
> 
> I just grabbed a fresh clone though and it compiled in normal time, so
> it has something to do with interrupted makes.
> 
> On Fri, Dec 5, 2014 at 5:36 PM, Matthew Flatt  wrote:
> > Sorry --- I've figure out how I misread your message, and I'm no longer
> > interested in the output of `raco show`.
> >
> > The log shown for step 3 makes sense, in that `make` doesn't currently
> > run `raco setup` at the end. So, if "main-distribution" is already
> > installed, then nothing more happens after the build of minimal Racket.
> > I think that aspect of the makefile should change.
> >
> > Meanwhile, I'm not clear on why moving your directory caused the
> > package system to lose track of "at-exp-lib", and I'll look into that
> > further.
> >
> > At Fri, 5 Dec 2014 15:22:03 -0700, Matthew Flatt wrote:
> >> Can you run `raco pkg show`? It looks like `raco pkg install` thinks
> >> you have "main-distribution" and "main-distribution-test" installed
> >> already.
> >>
> >> At Fri, 5 Dec 2014 14:04:47 -0800, John Clements wrote:
> >> > Urg... more interesting problems. I pulled and tried to rebuild, and 
> >> > things
> >> > went pear-shaped.
> >> >
> >> > 1) compilation failed because it couldn't find the 'racket' collection, 
> >> > but
> >> > I noticed that it was referring to a nonexistent path, presumably 
> >> > because I
> >> > had moved the root of the installation.  Has that always been a bad idea?
> >> > Dunno.
> >> > 2) I removed the racket/src/build directory, and built again. This time 
> >> > it
> >> > failed because it detected anomalies in my racket/etc/config.rktd:
> >> >
> >> > cp "../COPYING-libscheme.txt" "../COPYING_LESSER.txt" "../COPYING.txt"
> >> > "/Users/clements/plt/racket/share"/
> >> > /Applications/Xcode.app/Contents/Developer/usr/bin/make pkgs-catalog
> >> > racket/bin/racket -U -G build/config racket/src/pkgs-catalog.rkt --link
> >> > racket/share/pkgs-catalog pkgs
> >> > Finding packages
> >> >  Cataloging package at-exp-lib
> >> >  Cataloging package racket-doc
> >> >  Cataloging package racket-lib
> >> >  Cataloging package plt-services
> >> >  Cataloging package base
> >> >  Cataloging package racket-test
> >> >  Cataloging package racket-benchmarks
> >> >  Cataloging package racket-index
> >> > racket/bin/racket -U -G build/config racket/src/pkgs-config.rkt
> >> > pkgs-catalog: config file exists, but does not have a definition of
> >> > `catalogs' that starts as expected
> >> >   config file: racket/etc/config.rktd
> >> >   expected initial element: "../share/pkgs-catalog"
> >> >   possible solution: delete the config file
> >> >   context...:
> >> >/Users/clements/plt/racket/src/pkgs-config.rkt: [running body]
> >> > make[2]: *** [pkgs-catalog] Error 1
> >> > make[1]: *** [plain-in-place] Error 2
> >> > make: *** [in-place] Error 2
> >> >
> >> > 3) I deleted the file, and ran again. This time, compilation finished
> >> > almost instantly, and nothing was compiled (Log appears at bottom).
> >> >
> >> > 4) Then, I tried running make again, and it stopped even faster.
> >> >
> >> > 5) Then, I tried running raco setup, and I got a build with literally
> >> > thousands of errors of the form
> >> >
> >> > standard-module-name-resolver: collection not found
> >> >   for module path: (s

Re: [racket-dev] The repository is now split

[Matthew Flatt]

Sorry --- I've figure out how I misread your message, and I'm no longer
interested in the output of `raco show`.

The log shown for step 3 makes sense, in that `make` doesn't currently
run `raco setup` at the end. So, if "main-distribution" is already
installed, then nothing more happens after the build of minimal Racket.
I think that aspect of the makefile should change.

Meanwhile, I'm not clear on why moving your directory caused the
package system to lose track of "at-exp-lib", and I'll look into that
further.

At Fri, 5 Dec 2014 15:22:03 -0700, Matthew Flatt wrote:
> Can you run `raco pkg show`? It looks like `raco pkg install` thinks
> you have "main-distribution" and "main-distribution-test" installed
> already.
> 
> At Fri, 5 Dec 2014 14:04:47 -0800, John Clements wrote:
> > Urg... more interesting problems. I pulled and tried to rebuild, and things
> > went pear-shaped.
> > 
> > 1) compilation failed because it couldn't find the 'racket' collection, but
> > I noticed that it was referring to a nonexistent path, presumably because I
> > had moved the root of the installation.  Has that always been a bad idea?
> > Dunno.
> > 2) I removed the racket/src/build directory, and built again. This time it
> > failed because it detected anomalies in my racket/etc/config.rktd:
> > 
> > cp "../COPYING-libscheme.txt" "../COPYING_LESSER.txt" "../COPYING.txt"
> > "/Users/clements/plt/racket/share"/
> > /Applications/Xcode.app/Contents/Developer/usr/bin/make pkgs-catalog
> > racket/bin/racket -U -G build/config racket/src/pkgs-catalog.rkt --link
> > racket/share/pkgs-catalog pkgs
> > Finding packages
> >  Cataloging package at-exp-lib
> >  Cataloging package racket-doc
> >  Cataloging package racket-lib
> >  Cataloging package plt-services
> >  Cataloging package base
> >  Cataloging package racket-test
> >  Cataloging package racket-benchmarks
> >  Cataloging package racket-index
> > racket/bin/racket -U -G build/config racket/src/pkgs-config.rkt
> > pkgs-catalog: config file exists, but does not have a definition of
> > `catalogs' that starts as expected
> >   config file: racket/etc/config.rktd
> >   expected initial element: "../share/pkgs-catalog"
> >   possible solution: delete the config file
> >   context...:
> >/Users/clements/plt/racket/src/pkgs-config.rkt: [running body]
> > make[2]: *** [pkgs-catalog] Error 1
> > make[1]: *** [plain-in-place] Error 2
> > make: *** [in-place] Error 2
> > 
> > 3) I deleted the file, and ran again. This time, compilation finished
> > almost instantly, and nothing was compiled (Log appears at bottom).
> > 
> > 4) Then, I tried running make again, and it stopped even faster.
> > 
> > 5) Then, I tried running raco setup, and I got a build with literally
> > thousands of errors of the form
> > 
> > standard-module-name-resolver: collection not found
> >   for module path: (submod at-exp reader)
> >   collection: "at-exp"
> >   in collection directories:
> >/Users/clements/Library/Racket/development/collects
> >/Users/clements/plt/racket/collects
> >... [189 additional linked and package directories]
> >   context...:
> >show-collection-err
> >standard-module-name-resolver
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:353:23
> >/Users/clements/plt/racket/collects/syntax/modcode.rkt:62:2: reader
> >/Users/clements/plt/racket/collects/syntax/modcode.rkt:264:5: compile-one
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:315:0: compile-zo*
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:519:26
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:511:42
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:476:0:
> > maybe-compile-zo
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:591:2: do-check
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:630:15
> > 
> > /Users/clements/plt/racket/collects/compiler/../racket/private/map.rkt:113:23:
> > loop
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:591:2: do-check
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:630:15
> > 
> > /Users/clements/plt/racket/collects/compiler/../racket/private/map.rkt:113:23:
> > loop
> >/Users/clements/plt/racket/collects/compiler/cm.rkt:591:2: do-check...
> > 
> > 6) at this point, I gave up and checked out a fresh copy.
> > 
> > John
> > 
> > 
> > pcp

Re: [racket-dev] The repository is now split

[Matthew Flatt]

Can you run `raco pkg show`? It looks like `raco pkg install` thinks
you have "main-distribution" and "main-distribution-test" installed
already.

At Fri, 5 Dec 2014 14:04:47 -0800, John Clements wrote:
> Urg... more interesting problems. I pulled and tried to rebuild, and things
> went pear-shaped.
> 
> 1) compilation failed because it couldn't find the 'racket' collection, but
> I noticed that it was referring to a nonexistent path, presumably because I
> had moved the root of the installation.  Has that always been a bad idea?
> Dunno.
> 2) I removed the racket/src/build directory, and built again. This time it
> failed because it detected anomalies in my racket/etc/config.rktd:
> 
> cp "../COPYING-libscheme.txt" "../COPYING_LESSER.txt" "../COPYING.txt"
> "/Users/clements/plt/racket/share"/
> /Applications/Xcode.app/Contents/Developer/usr/bin/make pkgs-catalog
> racket/bin/racket -U -G build/config racket/src/pkgs-catalog.rkt --link
> racket/share/pkgs-catalog pkgs
> Finding packages
>  Cataloging package at-exp-lib
>  Cataloging package racket-doc
>  Cataloging package racket-lib
>  Cataloging package plt-services
>  Cataloging package base
>  Cataloging package racket-test
>  Cataloging package racket-benchmarks
>  Cataloging package racket-index
> racket/bin/racket -U -G build/config racket/src/pkgs-config.rkt
> pkgs-catalog: config file exists, but does not have a definition of
> `catalogs' that starts as expected
>   config file: racket/etc/config.rktd
>   expected initial element: "../share/pkgs-catalog"
>   possible solution: delete the config file
>   context...:
>/Users/clements/plt/racket/src/pkgs-config.rkt: [running body]
> make[2]: *** [pkgs-catalog] Error 1
> make[1]: *** [plain-in-place] Error 2
> make: *** [in-place] Error 2
> 
> 3) I deleted the file, and ran again. This time, compilation finished
> almost instantly, and nothing was compiled (Log appears at bottom).
> 
> 4) Then, I tried running make again, and it stopped even faster.
> 
> 5) Then, I tried running raco setup, and I got a build with literally
> thousands of errors of the form
> 
> standard-module-name-resolver: collection not found
>   for module path: (submod at-exp reader)
>   collection: "at-exp"
>   in collection directories:
>/Users/clements/Library/Racket/development/collects
>/Users/clements/plt/racket/collects
>... [189 additional linked and package directories]
>   context...:
>show-collection-err
>standard-module-name-resolver
>/Users/clements/plt/racket/collects/compiler/cm.rkt:353:23
>/Users/clements/plt/racket/collects/syntax/modcode.rkt:62:2: reader
>/Users/clements/plt/racket/collects/syntax/modcode.rkt:264:5: compile-one
>/Users/clements/plt/racket/collects/compiler/cm.rkt:315:0: compile-zo*
>/Users/clements/plt/racket/collects/compiler/cm.rkt:519:26
>/Users/clements/plt/racket/collects/compiler/cm.rkt:511:42
>/Users/clements/plt/racket/collects/compiler/cm.rkt:476:0:
> maybe-compile-zo
>/Users/clements/plt/racket/collects/compiler/cm.rkt:591:2: do-check
>/Users/clements/plt/racket/collects/compiler/cm.rkt:630:15
> 
> /Users/clements/plt/racket/collects/compiler/../racket/private/map.rkt:113:23:
> loop
>/Users/clements/plt/racket/collects/compiler/cm.rkt:591:2: do-check
>/Users/clements/plt/racket/collects/compiler/cm.rkt:630:15
> 
> /Users/clements/plt/racket/collects/compiler/../racket/private/map.rkt:113:23:
> loop
>/Users/clements/plt/racket/collects/compiler/cm.rkt:591:2: do-check...
> 
> 6) at this point, I gave up and checked out a fresh copy.
> 
> John
> 
> 
> pcp067879pcs:~/plt (git)-[master]- clements> make
> if [ "" = "" ] ; \
>  then /Applications/Xcode.app/Contents/Developer/usr/bin/make
> plain-in-place PKGS="main-distribution main-distribution-test" ; \
>  else /Applications/Xcode.app/Contents/Developer/usr/bin/make
> cpus-in-place CPUS="" PKGS="main-distribution main-distribution-test" ; fi
> /Applications/Xcode.app/Contents/Developer/usr/bin/make base
> mkdir -p build/config
> echo '#hash((links-search-files . ()))' > build/config/config.rktd
> mkdir -p racket/src/build
> /Applications/Xcode.app/Contents/Developer/usr/bin/make
> racket/src/build/Makefile
> make[3]: `racket/src/build/Makefile' is up to date.
> cd racket/src/build;
> /Applications/Xcode.app/Contents/Developer/usr/bin/make reconfigure
> /Applications/Xcode.app/Contents/Developer/usr/bin/make Makefile
> make[4]: `Makefile' is up to date.
> cd racket/src/build;
> /Applications/Xcode.app/Contents/Developer/usr/bin/make
> SELF_RACKET_FLAGS="-G `cd ../../../build/config; pwd`"
> /Applications/Xcode.app/Contents/Developer/usr/bin/make 3m
> cd racket; /Applications/Xcode.app/Contents/Developer/usr/bin/make 3m
> /Applications/Xcode.app/Contents/Developer/usr/bin/make cgc
> /Applications/Xcode.app/Contents/Developer/usr/bin/make common
> /Applications/Xcode.app/Contents/Developer/usr/bin/make g-c
> cd sgc; /Applications/Xcode.app/Contents/Develope

Re: [racket-dev] The repository is now split

[Matthew Flatt]

In cooperation with Sam, I've pushed a change to the way that `make`
links the content of "pkgs". If you run `make` again, it should tell
you to delete your old

  racket/etc/config.rktd 


Also, the "native-pkgs" submodule is gone. The problem that John saw
with "libintl.8.dylib" has been fixed by updating the native-library
packages on "pkgs.racket-lang.org". You'll need to use `raco pkg
update` to get those updates!

For now, use `git pull` to update core Racket and the handful of
packages in "pkgs". Use `raco pkg update` to update anything else. We
expect to make `raco pkg update` apply to everything (i.e., to imply a
`git pull` in the main repository's directory), but we're not there,
yet.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Splitting the Racket repository

[Matthew Flatt]

At Sat, 29 Nov 2014 22:00:44 -0500, Eli Barzilay wrote:
> On Sat, Nov 29, 2014 at 8:30 PM, Sam Tobin-Hochstadt
>  wrote:
> > On Sat, Nov 29, 2014 at 8:16 PM, Eli Barzilay  wrote:
> >> On Sat, Nov 29, 2014 at 7:14 PM, Sam Tobin-Hochstadt
> >>  wrote:
> >>> To clone individual repositories, use the new `--clone` option for
> >>> `raco pkg`, such as:
> >>> [...]
> >>
> >> This is very obscure.  Is there a compact description of what to do
> >> when you want to make a change in a file that is not on the main
> >> repo?
> >
> > You just need to edit and change the relevant repository -- nothing
> > else is required. You don't have to use --clone or any other
> > mechanism.  [...]
> 
> This doesn't answer my question -- so I think that I wasn't clear
> enough.  Say that I have some change to a specific file.  I want to know
> how to do it in the following two ways:
> 
> [...]
> 
> 2. The thing that is more relevant for others (and that's the important
>one): I have a normal installation, I find a bug -- what do I do now?
>[...]

A strategy is explained in the "Developing Packages with Git" section
of the package-manager documentation. To elaborate for the running
example:


If you have "remote-shell" installed via "pkgs.racket-lang.org" and you
want to modify it, then

 raco pkg update --clone <...>/remote-shell

creates a clone at `<...>/remote-shell'. You can then use just
`remote-shell` to make a subdirectory of the current directory.

If you do that with the version of `raco pkg` as of this morning, it
will suggest that you use `--multi-clone ask`. (As of just now,
`--multi-clone ask` is the default.) When you're in `--multi-clone ask`
mode, `raco pkg` will suggest that "remote-shell", "remote-shell-lib",
and "remote-shell-doc", are all cloned together.


The argument after `--clone` is a directory path, but if no other
arguments are provided to `raco pkg update`, a package name is inferred
from the directory path (and a Git repo is inferred from that package
name).

In the example above, if you want to clone "remote-shell" to the
current directory instead of a subdirectory of the current directory,
supply the package name explicitly after "." as the directory path:

 raco pkg update --clone . remote-shell

That is, a clone of the package "remote-shell" is made in the current
directory as linked as the package installation.



If you installed a Git-based package via "pkgs.racket-lang.org" using a
version of `raco pkg` that's older than a week or so, then `raco pkg`
didn't record that the package came from a Git-repo URL, and so the
simple `--clone` form above doesn't work. More significantly, if you
install a snapshot of Racket, then the simple `--clone` form won't work
with a package that is provided in built form by the snapshot's
catalog. I'm working on a refinement to better address those
situations.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system and versions

[Matthew Flatt]

Although I object to some of your characterizations of the difference
between PLaneT and the new package system, it's fair to say that PLaneT
provides better support to package authors for creating new APIs that
are intended to replace (but coexist with) old APIs.

I think the answer to that problem in the new system is better tools
for package authors, along with better catalog-display mechanisms for
package users. That is, I think "javascript" through "javascript9" are
fine as package names (and even collection names), but I think those
packages should be grouped together as a listing in the catalog.
Meanwhile, we need tools to help the author of "javascript3" introduce
"javascript4", and so on.

Creating an external tool might not be necessary if the notion of
major-version-compatibility were built into the package system in a
different way. Unfortunately, it's not easy to add more version support
into the current design and implementation. It's the same as designing
a more traditional programming language: choices interact with all
sorts of trade-offs, both conceptually and in the implementation.

So, if the question is "will the core package system's treatment of
versions change?", then I think the answer is "no". If the question is
"will the package versioning experience improve for package authors and
users?", then I hope the answer is "yes" --- but I intend to leave the
actual answer to others (yes, you!), for now. Until we find otherwise,
I can't help feeling that the current design and implementation has all
the primitives that we need[*] for building better things on top.

 [*] Of course, it has way too many primitives. Again, like any design,
 it's a process to figure out what the true core could be.

At Sun, 30 Nov 2014 10:43:19 -0500, Neil Van Dyke wrote:
> Any chance of revisiting the new package system's stances on versions -- 
> specifically, on the two issues:
> 1. Can subsequent versions of a named package (which has an identity)  
> be non-backward-compatible?
> 2. Can a Racket setup (and even an individual program) have multiple 
> versions of a package at the same time?
> 
> Regarding #1, the requirement to never make a non-backward-compatible 
> version of a package without giving up package identity is a burden, and 
> a deterrent to third-party package releases.
> 
> For a real-world example, see "http://planet.racket-lang.org/";, where it 
> looks like most of the familiar names (who were going to a good amount 
> of trouble to release already), managed to release packages marked as 
> non-backward-compatible (i.e., PLaneT major version number other than 
> 1).  Wouldn't requiring them to never break backward compatibility be a 
> deterrent to releasing at all? Or, if they still released, would 
> `dherman`, following the instructions of Racket to create a new package 
> identity for any backward-incompatible version, have given us packages 
> `javascript`, `javascript2`, ... `javascript9`, rather than `javascript` 
> versions `1.x` through `9.x`.
> 
> No-backward-incompatibility might make sense for core Racket (although 
> even core Racket has broken this multiple times), but it's a big 
> deterrent to researchers, and to industry developers who are willing to 
> open source components of their larger systems in lightweight ways.
> 
> Regarding #2, I suggest that should be revisited if #1 is revisited.
> 
> I think PLaneT got both of these a lot closer to right, at least for 
> third-party packages.
> 
> Neil V.
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] new package system collections and conflicts

[Matthew Flatt]

We should change that example. It would indeed be strange for package
named "tic-tac-toe" would introduce a `data/matrix` module, and the
documentation really shouldn't suggest that it makes sense for a
package to introduce overlaps that are not reasonably expected from the
package name.


There are plenty of real examples where it's sensible for different
packages to introduce modules in overlapping collections, though.
Sometimes, it's because different packages implement different facets
of a conceptual whole, such as "math-lib", "math-doc", and "math-test"
all extending the "math" collection. Sometimes, it's more a mater of
history, such as "compatibility-lib" extending several different
collections. And then there are example that you might classify as
either of those or as a third kind, depending on how you look at it:
"draw-lib" adds `racket/draw`, and "gui-package-manager" adds
`pkg/gui`.


The "data" collection is among those that were intended to be extended
by multiple packages. Whether that's really a good idea remains to be
seen, but that's why "data" shows up as an example in the manual.
(Again, though, no one would expect a "tic-tac-toe" package to
introduce a new module in "data".)


At Sun, 30 Nov 2014 10:22:49 -0500, Neil Van Dyke wrote:
> Given the example from the documentation, of the `tic-tac-toe` package 
> and "conflicts" (quoted at end of this email), instead, why isn't the 
> norm to do:
> 
>  (require tic-tac-toe)
> 
> Or, when necessary:
> 
>  (require tic-tac-toe/matrix)
> 
> Why, when one installs a package named `tic-tac-toe`, would one be 
> referencing modules of the package as `data/matrix`?
> 
> I don't recall ever seeing a situation in which I wanted different 
> third-party Racket packages to be stomping over each other's 
> module-naming namespaces.
> 
> For people who want to release lots of well-behaved reusable packages, 
> using good practices, with minimal effort and head-scratching, things 
> like this are a problem.
> 
> > 2 Package Concepts
> >
> > A package is a set of modules in some number of collections. Modules 
> > installed using the Racket package manager are required like any other 
> > modules. For example, if the package tic-tac-toe contains the module 
> > "matrix.rkt" in a "data" collection, then after tic-tac-toe is installed,
> >
> > (require data/matrix)
> [...]
> > 2.5 Package Conflicts
> >
> > Two packages are in conflict if they contain the same module. For 
> > example, if the package tic-tac-toe contains the module file 
> > "data/matrix.rkt" and the package factory-optimize contains the module 
> > file "data/matrix.rkt", then tic-tac-toe and factory-optimize are in 
> > conflict.
> >
> > A package may also be in conflict with Racket itself, if it contains a 
> > module file that is part of the base Racket implementation. For 
> > example, any package that contains "racket/list.rkt" is in conflict 
> > with Racket.
> >
> > For the purposes of conflicts, a module is a file that ends in ".rkt", 
> > ".ss", or ".scrbl".
> 
> Neil V.
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Line editing in the default REPL

[Matthew Flatt]

Do you have in mind making "xrepl" intended to be part of Minimal
Racket? If not, what's the mechanism for `racket` using "xrepl" when
it's available?

A similar question applies to "libeditline". Currently, for Linux and
other Unix platforms (not counting "natipkg" variants), our convention
is that native libraries are either part of the `[g]racket` executable
or they are installed separately by users through the OS's package
manager. We can't link to "libreadline" by default in a Racket
distribution, and since "libeditline" isn't typically included with
Linux distributions (as far as I can tell), it seems like we haven't
solved any problem unless we provide "libeditline". Should
"libeditline" become not only part of the Minimal Racket distribution,
but even part of the Racket executable? Or should our convention and/or
distribution infrastructure change?


At Mon, 24 Nov 2014 18:02:45 -0500, Sam Tobin-Hochstadt wrote:
> My understanding of the licensing issues is that if the code works with
> both "libeditline" and "libreadline" then it isn't a derived work of
> readline, and therefore could be licensed under the LGPL, like the rest of
> Racket. Furthermore, turning use of "libeditline" on by default wouldn't be
> linking to any GPL code, meaning that we could do that by default.
> 
> I think we should split up the `readline` collection into `readline` and
> `readline/base` which would be what's compatible with editline, and have
> xrepl in a `readline/base` mode on by default.
> 
> Sam
> 
> On Mon, Nov 24, 2014 at 5:57 PM, Leif Andersen 
> wrote:
> 
> > Hello,
> >
> > When a user first starts the racket repl, it doesn't do line editing due
> > to licensing issues. For example, it would be nice if the default editor
> > would give the previous line if you hit up arrow, rather than writing
> > "^[[A".
> >
> > I have now pointed out xrepl to several users, and every time they always
> > ask me why it's not the default repl. Apparently the problem is that xrepl
> > uses GNU Readline, which is GPL. However, Asumu found that if we replace
> > requiring readline with BSD's libedit (not libeditline), everything works
> > fine due to libedit's readline compatibility. It doesn't have all of the
> > features of readline, but it does have some of the biggest ones (such as
> > being able to use arrow keys)
> >
> > What do you all think of having `(require editline)` that works for the
> > default repl, so that it gets line editing features. This would allow us to
> > also keep `(require readline)` as before, maintaining backwards
> > compatibility.
> >
> > If we do do this, this leads to the question of distribution. Would we
> > want to include libedit inside Racket distributions, or should we just link
> > to whatever the user has on their system?
> >
> > ~Leif Andersen
> >
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> >
> >
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] DrRacket PF1 Search Bug?

[Matthew Flatt]

At Sat, 22 Nov 2014 14:16:32 -0500, Eli Barzilay wrote:
> On Fri, Nov 21, 2014 at 4:26 PM, Robby Findler
>  wrote:
> > The two candidates are the trampoline approach and the "just move the
> > documentation files over into the user space as if a package had been
> > installed".
> 
> Oh, *that*'s the other solution?  That sounds pretty bad not only
> because it complicates file installation which is already very complex,
> but also because it's a significant weight.  (For example, the current
> size I see is about 180M, multiply that for a department with a thousand
> users and you get unhappy admins.)

Just the documentation "index" (main page, search page, etc.) would get
generated in the user's space. It's on the order of 1 MB.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Errors compiling on cygwin

[Matthew Flatt]

Yes, our Cygwin support has rotted in a variety of small but exotic
ways.

I'm in the process of fixing the problems. The enclosed patch applies
to the development branch or the v6.1.1 release candidate, and it might
work for you, but I'm still checking it.

The fixed-up implementation will be limited in various ways, such as
not supporting places or futures, but I should be able to make all the
basics work.

At Fri, 31 Oct 2014 15:31:32 +, Saurabh T wrote:
> 
> >  Hi,
> > 
> > I downloaded racket-6.1-src-builtpkgs.tgz. According to src/README, this 
> > should be compilable on cygwin with --enable-shared. I did not have 
> > success doing so.
> > 
> > The first problem was dynsrc/mzdyn.c failed to 
> > compile due to "expected specifier-qualifier-list before ‘(’ token" 
> > error at schemex.h:1004. The line here reads
> > (*scheme_jit_find_code_end)(void *p);
> > I grepped some and stuck a void* in front of it, and compilation went ahead.
> > 
> > But then it failed with "'WINDOWS_DEFAULT_STACK_SIZE' undeclared (first use 
> in this function)" at eval.c:546
> > intptr_t sz = WINDOWS_DEFAULT_STACK_SIZE;
> 
> To update, I went ahead and copied the macro from where it was defined for 
> windows over to the cygwin section. The next errors were due to three 
> consecutive assignments in schemex.inc that were accessing undefined 
> functions 
> (ones related to "freeze" or "frozen" - unfortunately I don't have the code 
> right now), so I commented those lines out. Now I have a successful link, 
> except it segfaults during make install. This is because this target runs the 
> racketcgc (spelling?) executable, which immediately seg faults (I tried 
> running 
> it in the shell to be sure). 
> 
> I hope someone has more clues about this. Thank you.
>  
> > I
> >  am at a loss at this point. Is racket not expected to compile on 
> > cygwin? If so, can someone remove the cygwin parts in src/README?
> > 
> > Thank you.
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

0001-fixups-for-compiling-on-Cygwin.patch
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Understanding the performance of raco make

[Matthew Flatt]

I think you probably want to get information directly from
`compiler/cm`, probably `compiler/cm` doesn't provide the right
information right now, and probably some adjustments to `compiler/cm`
could get you useful information through the logging API. Also,
`parallel-compile-files` might need to change to propagate logging from
from a separate place to the original one.

Part of the reason that `compiler/cm` doesn't already provide good
information is that it has a `manager-trace-handler` parameter that
pre-dates the logging API. The default trace handler logs trace strings
as 'debug events, but it's easy to imagine that structured information
as 'info events would be more helpful for your purposes.

In other words, I'm suggesting that you change `compiler/cm` and
probably `parallel-compile-files` --- if that sounds reasonable to you.

Coincidentally, I'm right now adjusting the way that logging is used
for `compiler/cm-accomplice`, but I don't think those changes would
interfere with changes you might make.

At Tue, 28 Oct 2014 08:45:09 -0700, Eric Dobson wrote:
> I'm trying to improve the compilation speed of some modules, and to do that
> I need to understand how raco make is spending its time.
> 
> I decided to call parallel-compile-files myself with a handler that gets
> the time of start and done messages, and then turn those into a timeline
> plot.
> 
> I have a attached a screen shot of what that produces.​
>  Screen Shot 2014-10-28 at 8.07.42 AM.png
> 
> ​
> The problem I have found is that when one builder is building a module I
> don't get insight into what it is doing while compiling that modules
> dependencies that were not prefetched by a different builder. This causes
> two problems. The first is that compilation time for some modules looks
> very long, even though most of it is in compiling their dependencies. The
> second is that I cannot tell what is going on when only the original
> threads are doing processing, and so it isn't obvious why other threads
> start up again.
> 
> Is there a better way to instrument raco make?
> 
> Hacky code:
> 
> Instrumented compile:
> #lang racket/base
> 
> (require setup/parallel-build racket/match)
> (define times (make-hash))
> (parallel-compile-files (list
> 
> "/Users/endobson/proj/racket/plt/pkgs/typed-racket-pkgs/typed-racket-lib/typed-r
> acket/core.rkt")
>   #:handler
> (λ (id type path-string msg out err)
>   (define time (current-inexact-milliseconds))
>   (define ps (if (path? path-string) (path->string path-string)
> path-string))
>   (cond
> [(equal? type 'start)
>  (hash-set! times (list ps id) time)]
> [(equal? type 'done)
>  (hash-set! times (list ps id) (list (hash-ref times (list ps id))
> time))])))
> (for ([(name time) times])
>   (match-define (list start stop) time)
>   (match-define (list file id) name)
>   (write (list id file start stop))
>   (newline))
> 
> 
> 
> Plotter:
> #lang racket/base
> 
> (require plot racket/match racket/list)
> (plot-new-window? #t)
> 
> (define work-units
>   (call-with-input-file* "data.rktd"
> (lambda (port)
>   (let loop ([items empty])
> (define item (read port))
> (if (eof-object? item)
> items
> (loop (cons item items)))
> 
> (define converted-work-units
>   (for/list ([work-unit (in-list work-units)])
> (match-define (list index file-name start stop) work-unit)
> (list
>   (rectangles
> #:alpha 0
> (list (list (ivl start stop) (ivl (- index 1/3) (+ index 1/3)
>   #;
>   (point-label
> (list (/ (+ start stop) 2) index)
> file-name
> #:anchor 'center
> #:point-size 0
> 
> 
> (plot
>   #:width 1500
>   converted-work-units)
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Release Announcement for v6.1.1

[Matthew Flatt]

At Mon, 27 Oct 2014 12:25:22 -0400, Ryan Culpepper wrote:
> mflatt:
>   - optimizations (most from Gustavo Massaccesi) (82ffd405, 25c05d66,
> a7a912ee, 1f2f7a1d, d14b4a80, 769c5b6e, 35eb6562, 15423988)
>   - add replace-evt (as suggested by Jan Dvořák) (bc69a9b0)
>   - fixing letrec updates? (eg 926e64f5?)
>   - Mac OS X Yosemite Pango repair (76f1ebde)
>   - DPI-aware racket/gui on Windows (a64a1cb1)
>   - raco pkg add '--binary-lib' (05523a0b, b2b00010)

Plus the 32-bit Windows repair:

 * Mac OS X Yosemite: fixed compatibility problems, mainly by patching
   the Pango text-drawing library that is bundled with Racket.

 * Windows, 32-bit version: fixed window-update crashes by patching the
   Cairo drawing library that is bundled with Racket.

 * Windows: made the GUI library DPI-aware.

 * Added a "binary library" installation mode to install packages
   without source or documentation. Use the `--binary-lib` option with
   `raco pkg install`.

 * Repaired the compiler's use-before-defined analysis for certain
   forms of nested `letrec`, some `let` forms, and some uses of `set!`
   or `with-continuation-mark`.

 * Added bytecode optimizations (thanks to Gustavo Massaccesi).

 * Added a `replace-evt` event constructor (as suggested by Jan Dvořák).


>   - performance tuning (c570a862, 1809df45)
>   - windows: use native api for dates (135ccf09)
>   - allow mixing exceptions with ffi/unsafe/alloc (from Jan Dvořák)
>(8bd5aa38)
>   - senora gc (2916fc34, a312f499, 881990ed)
>   - throw out latex back-end for picts ? (77ddf71b)
>   - chaperones w/o redirections (1f1a10db, a8d0534e)
>   - Windows: fix handling of junctions as links (cf7c0134)
>   - behavior of numpad Enter (7d388a07, a41cc0c3)
>   - UDP improvements (2a387ace)
>   - natipkg (40f5ec07)

These seem too minor for the announcement.


_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Pre-Release Checklist for v6.1.1, Second Call

[Matthew Flatt]

Ad, I hadn't guessed that you use the "Installer Package". It isn't one
of our normal distribution mode (i.e., there's no link to it on the
eventual download page), so far, and so I forgot about it, but it is
certaily prominent on the pre-release page.

I'm not yet sure of the right repair, but I'll work on it.

At Thu, 23 Oct 2014 15:11:03 -0400, Matthias Felleisen wrote:
> 
> Yes, I did and expected this to be the problem. Can't we add a line to the 
> Mac 
> package installer that removes left-over stuff? 
> 
> I deleted the old installation and re-installed and drracket starts up fine. 
> 
> -- Matthias
> 
> 
> 
> 
> 
> On Oct 23, 2014, at 2:26 PM, Matthew Flatt  wrote:
> 
> > Is it possible that you installed on top of an existing v6.0.900.900
> > installation (for the previous release's candidate)?
> > 
> > The file
> > 
> >  share/pkgs/drracket/drracket/private/compiled/rectangle-intersect_rkt.zo
> > 
> > existed in v6.0.900.900, but it does not exist in v6.1.0.900, because
> > the library moved to a different package, "drracket-tool-lib". If you
> > installed v6.1.0.900 on top of v6.0.900.900, then the file would not
> > get replaced, and the module search would potentially find the old file
> > instead of the replacement that is in a different location.
> > 
> > At Thu, 23 Oct 2014 13:24:08 -0400, Matthias Felleisen wrote:
> >> 
> >> DrRacket on Mac OS X, 64-bit, Installer Package won't start on 
> >> double-click. 
> >> 
> >> At the command line, I get this error message: 
> >> 
> >> [:~] matthias% /Applications/Racket/DrRacket.app/Contents/MacOS/DrRacket 
> >> 
> /Applications/Racket/share/pkgs/drracket/drracket/private/compiled/rectangle-int
> >> ersect_rkt.zo::1: read (compiled): wrong version for compiled code
> >>  compiled version: 6.0.900.900
> >>  expected version: 6.1.0.900
> >>  context...:
> >>   standard-module-name-resolver
> >> 
> >> 
> /Applications/Racket/share/pkgs/drracket/drracket/private/module-browser.rkt: 
> >> [traversing imports]
> >>   /Applications/Racket/share/pkgs/drracket/drracket/private/link.rkt: 
> >> [traversing imports]
> >>   /Applications/Racket/share/pkgs/drracket/drracket/tool-lib.rkt: 
> [traversing 
> >> imports]
> >> 
> >> 
> /Applications/Racket/share/pkgs/drracket/drracket/private/drracket-normal.rkt:
>  
> >> [running body]
> >>   /Applications/Racket/share/pkgs/drracket/drracket/drracket.rkt: [running 
> >> body]
> >> 
> >> 
> >> 
> >> 
> >> 
> >> On Oct 23, 2014, at 12:48 PM, Ryan Culpepper  wrote:
> >> 
> >>> Checklist items for the v6.1.1 release
> >>> (using the v6.1.0.900 release candidate build)
> >>> 
> >>> Search for your name to find relevant items, reply when you finish an
> >>> item (please indicate which item/s is/are done).  Also, if you have any
> >>> commits that should have been picked, make sure that the changes are in.
> >>> 
> >>> Important: new builds are created without announcement, usually whenever
> >>> I pick a few commits.  If you need to commit changes, please make sure
> >>> you tell me to pick it into the release branch.
> >>> 
> >>> --> Release candidates are at
> >>> -->   http://pre-release.racket-lang.org
> >>> 
> >>> Please use these installers (or source bundles) -- don't test from
> >>> your own git clone (don't test the `master' branch by mistake!).  To
> >>> get the tests, you can do this:
> >>> 
> >>> cd ...racket-root...
> >>> ./bin/raco pkg install -i main-distribution-test
> >>> 
> >>> --
> >>> 
> >>> * Matthias Felleisen 
> >>> - Teachpacks Tests: check that new teachpacks are addable
> >>> - Teachpack Docs: check teachpack docs in the bundles
> >>> - Try teaching-languages testing framework (check-expect)
> >>> Updates:
> >>> - Teachpack Updates: update HISTORY
> >>> (updates should show v6.1.1 as the most current version; email me
> >>> to pick the changes when they're done, or tell me if there are no such
> >>> changes.)
> >>> 
> >>> * Eli Barzilay 
> >>> - Swindle Tests
> >>> - XREPL Tests
> 

Re: [racket-dev] Pre-Release Checklist for v6.1.1, Second Call

[Matthew Flatt]

Is it possible that you installed on top of an existing v6.0.900.900
installation (for the previous release's candidate)?

The file

  share/pkgs/drracket/drracket/private/compiled/rectangle-intersect_rkt.zo

existed in v6.0.900.900, but it does not exist in v6.1.0.900, because
the library moved to a different package, "drracket-tool-lib". If you
installed v6.1.0.900 on top of v6.0.900.900, then the file would not
get replaced, and the module search would potentially find the old file
instead of the replacement that is in a different location.

At Thu, 23 Oct 2014 13:24:08 -0400, Matthias Felleisen wrote:
> 
> DrRacket on Mac OS X, 64-bit, Installer Package won't start on double-click. 
> 
> At the command line, I get this error message: 
> 
> [:~] matthias% /Applications/Racket/DrRacket.app/Contents/MacOS/DrRacket 
> /Applications/Racket/share/pkgs/drracket/drracket/private/compiled/rectangle-int
> ersect_rkt.zo::1: read (compiled): wrong version for compiled code
>   compiled version: 6.0.900.900
>   expected version: 6.1.0.900
>   context...:
>standard-module-name-resolver
>
> /Applications/Racket/share/pkgs/drracket/drracket/private/module-browser.rkt: 
> [traversing imports]
>/Applications/Racket/share/pkgs/drracket/drracket/private/link.rkt: 
> [traversing imports]
>/Applications/Racket/share/pkgs/drracket/drracket/tool-lib.rkt: 
> [traversing 
> imports]
>
> /Applications/Racket/share/pkgs/drracket/drracket/private/drracket-normal.rkt:
>  
> [running body]
>/Applications/Racket/share/pkgs/drracket/drracket/drracket.rkt: [running 
> body]
> 
> 
> 
> 
> 
> On Oct 23, 2014, at 12:48 PM, Ryan Culpepper  wrote:
> 
> > Checklist items for the v6.1.1 release
> >  (using the v6.1.0.900 release candidate build)
> > 
> > Search for your name to find relevant items, reply when you finish an
> > item (please indicate which item/s is/are done).  Also, if you have any
> > commits that should have been picked, make sure that the changes are in.
> > 
> > Important: new builds are created without announcement, usually whenever
> > I pick a few commits.  If you need to commit changes, please make sure
> > you tell me to pick it into the release branch.
> > 
> > --> Release candidates are at
> > -->   http://pre-release.racket-lang.org
> > 
> > Please use these installers (or source bundles) -- don't test from
> > your own git clone (don't test the `master' branch by mistake!).  To
> > get the tests, you can do this:
> > 
> >  cd ...racket-root...
> >  ./bin/raco pkg install -i main-distribution-test
> > 
> > --
> > 
> > * Matthias Felleisen 
> >  - Teachpacks Tests: check that new teachpacks are addable
> >  - Teachpack Docs: check teachpack docs in the bundles
> >  - Try teaching-languages testing framework (check-expect)
> >  Updates:
> >  - Teachpack Updates: update HISTORY
> >  (updates should show v6.1.1 as the most current version; email me
> >  to pick the changes when they're done, or tell me if there are no such
> >  changes.)
> > 
> > * Eli Barzilay 
> >  - Swindle Tests
> >  - XREPL Tests
> >  - Verify PL language
> >  - Racket Tree: compare new distribution tree to previous one
> >  - Run the unix installer tests
> >  - Run zsh completions tests
> >(". .../racket-completion.zsh; _racket --self-test")
> > 
> > * Stephen Bloch 
> >  - Picturing Programs Tests
> > 
> > * Jon Rafkind 
> >  Release tests for (one of the) linux releases:
> >  - Test that the `racket' and `racket-textual' source releases
> >compile fine (note that they're still called `plt' and `mz' at
> >this stage).
> >  - Test that the binary installers for both work, try each one in
> >both normal and unix-style installation modes. (just ubuntu)
> >  [Note: get the release candidates from the URL in this email. Use
> >   the 'static table' link to see a list of all tar files available]
> > 
> > * David Van Horn 
> >  - EoPL Tests
> > 
> > * Neil Toronto 
> >  - Plot Tests
> >  - Images Tests
> >  - Inspect icons
> >  - Math tests
> 
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] crash running raco setup with racket 6.1

[Matthew Flatt]

I can confirm the crash with a Cairo 1.14 build on 64-bit Mac OS X.
I've submitted a bug report for Cairo (Bug 85372).


For the record, here's how I assembled the report:

The crash happened for me when building the "plot" documentation. By
successively pruning the document's source, I whittled the crashing
expression down to just

 (plot3d (surface3d (λ (x y) (* (cos x) (sin y)))
(- pi) pi (- pi) pi))

Next, I put the generated picture in "datum" form via `plot3d/dc` and
`record-dc%`. That gave me a 35k-line S-expression that corresponds to
drawing commands. I performed a binary search on that sequence to find
a small segment that still causes a crash. I end up with

 #lang racket
 (require racket/draw)

 (define W 242) ; size must be big enough to trigger the crash
 (define H 242)
 ; must use the bitmap (non-platform-specific) back-end:
 (define dc (send (make-bitmap W H) make-dc))

 (define picture
   '((set-smoothing smoothed)
 (do-set-pen! ((0 0 0 1.0) 1/3 solid round round #f))
 (draw-lines
  ((94.49384481799765 . 241.40423862491832)
   (97.92538321881572 . 237.25698103165843)
   (103.86884481799765 . 235.02180530906503)
   (100.43730641717958 . 239.0632764810762)
   (94.49384481799765 . 241.40423862491832))
  0.0
  0.0)))

 ((recorded-datum->procedure picture) dc)

To make a Cairo bug report easier to assemble, I instrumented the Cairo
FFI library to log all calls. There were 30 or so calls, many of which
I could tell were redundant or irrelevant. I produced this C program
that crashes with a C-level stack trace like the original one:

 #include 

 int main () {
   cairo_t *cr;

   cr = cairo_create(cairo_image_surface_create(0, 242, 242));

   cairo_set_antialias(cr, 2);
   cairo_set_line_width(cr, 1.0/3.0);

   cairo_new_path(cr);
   cairo_move_to(cr, 94.49384481799765, 241.40423862491832);
   cairo_line_to(cr, 97.92538321881572, 237.25698103165843);
   cairo_line_to(cr, 103.86884481799765, 235.02180530906503);
   cairo_line_to(cr, 100.43730641717958, 239.0632764810762);
   cairo_line_to(cr, 94.49384481799765, 241.40423862491832);
   cairo_stroke(cr);
 }

I'm pretty sure that program shouldn't crash, and it doesn't crash for
me with Cairo 1.12.

I've spelled out my strategy above in the hope that someone else will
be able to file a Cairo bug report the next time there's a similar
problem. :)

At Thu, 23 Oct 2014 12:46:55 +0200, David Bremner wrote:
> David Bremner  writes:
> >
> > As a point of information, I can duplicate the crash with yesterdays
> > snapshot (20141022-d9f2a84).  I didn't bother getting a backtrace there,
> > but I can if it would help.
> 
> I verified that the version of libcairo2 is what makes a difference.
> Installing Debian version 1.12.16-5 made the racket build work again
> both for 6.1 and for the snapshot. Alas that's not going to be a
> feasible strategy for the official Debian builds.
> 
> d
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Strange issue with identifier-binding being wrong for lexical variables

[Matthew Flatt]

At Wed, 22 Oct 2014 10:25:51 -0400, Sam Tobin-Hochstadt wrote:
> On Wed, Oct 22, 2014 at 10:20 AM, Matthew Flatt  wrote:
> > Expansions that produce this bad `identifier-binding` result probably
> > happen up all the time. They don't bother the bytecode compiler,
> > because the compiler uses `free-identifier=?` to compare bindings in
> > expanded code, keeping track of all of the bindings that are in the
> > environment of a given expression. Depending on your use case, that
> > might be the way to go for now.
> 
> If I understand correctly, I should maintain an environment of
> lexically-bound identifiers at every point, and if a given identifier
> is in that environment, it should be treated as lexical, even if
> `identifier-binding` says that it's a module-bound variable. Only if
> an identifier isn't currently bound should I treat it as a
> module-bound variable.
> 
> Does that sound right? This would be easy enough for me to do.

Yes, that's right.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Strange issue with identifier-binding being wrong for lexical variables

[Matthew Flatt]

I agree that this is broken, but I'd like to put it on hold, because
its another basic problem with the way the current macro expander
represents lexical context.

Adjusting the context of the expressions changes the result, because
its the macro-introduced nature of the `main` definition that triggers
the bad result from `identifier-binding`.

Expansions that produce this bad `identifier-binding` result probably
happen up all the time. They don't bother the bytecode compiler,
because the compiler uses `free-identifier=?` to compare bindings in
expanded code, keeping track of all of the bindings that are in the
environment of a given expression. Depending on your use case, that
might be the way to go for now.

At Tue, 21 Oct 2014 22:26:26 -0400, Sam Tobin-Hochstadt wrote:
> I've found what I think is a bug in the expander where lexical
> references can get an `identifier-binding` result that suggests that
> they're module-bound.
> 
> In particular, you need these three files:
> 
> bugtest.rkt:
> 
> (module bugtest "wraptest.rkt")
> 
> bugtest.scm:
> 
> (define (gcbench)
>   (define main #f)
>   main)
> 
> (define main #f)
> 
> wraptest.rkt:
> 
> #lang racket/base
> (provide (rename-out (module-begin #%module-begin)))
> 
> (require racket/include
>  (for-syntax racket/base))
> 
> (define-syntax (module-begin stx)
>   (define name (syntax-property stx 'enclosing-module-name))
>   #`(#%module-begin
>  (include #,(format "~a.scm" name
> 
> Then run the macro stepper with macro hiding off on "bugtest.rkt".
> Click on the reference to `main` inside `gcbench`. You'll see that it
> says that it's a module-bound variable named `main.2` which is defined
> in this module.
> 
> Then try changing the name of the top-level definition in bugtest.scm
> to `main2`. Re-run the macro stepper, and you'll see that the
> identifier-binding is now lexical.
> 
> I tried a few things to get this to go away (such as using
> `#%plain-module-begin`) which didn't work. Reducing it to two files,
> ie doing the include directly in bugtest.rkt, made the problem
> disappear.
> 
> Changing the body of the `module-begin` macro to:
> 
> (define-syntax (module-begin stx)
>   (define name (syntax-property stx 'enclosing-module-name))
>   #`(#%module-begin
>  #,(datum->syntax stx `(include ,(format "~a.scm" name)))
>  #,(datum->syntax stx '(main
> 
> and then providing a bunch of extra stuff made the issue go away.
> Because there's the workaround, the issue isn't urgent.
> 
> Sam (and Tobias, who found this bug)

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] advice on the 6.x build system.

[Matthew Flatt]

At Fri, 17 Oct 2014 17:56:51 +0200, David Bremner wrote:
> Matthew Flatt  writes:
> 
> >
> > Meanwhile, I haven't answered your original question. Can you remind me
> > of the specific steps that I'd need to follow to try the script that
> > you sent before?
> 
> With your indulgence, I'll just answer this part now. I have re-included
> the Makefile as an attachment, since make is fussy about whitespace.
> 
> Save the attached file to e.g. /tmp/test.mk
> 
> In a recent checkout of the release branch, 
> 
> % make -f /tmp/test.mk build-indep-stamp
> 
> the resulting build of racket will be in test-dest, in the top level
> directory of racket. For me, test-dest/usr/bin only has racket and raco
> in it.

The problem is that `make install` works with a configuration whose
paths are based on DESTDIR, but as its final act it rewrites
"config.rktd" and other files to strip away DESTDIR. That rewrite tells
the later use of `raco pkg install` to put binaries in the DESTDIRless
path (i.e., "/usr/bin"), and so on. It's a kind of bad luck that `raco
pkg install` runs at all, since it could depend on the DESTDIRless pass
in any number of ways, such as an embedded path in a shared library.

If you really want to go this way, I could extend the makefile to let
you supply a command that runs before the path-fixup step of `make
install`. Then, your "test.mk" would look more like this:

build-arch-stamp: ${base_build_dir}/Makefile
$(MAKE) -C ${base_build_dir} 
$(MAKE) -C ${base_build_dir} DESTDIR=${destdir} install \
  FINAL_PREP_CMD="cd $(CURDIR) \
  && $(MAKE) RACKET="'"'"${PRERACKET}"'"'" \
 local-source-catalog \
  && ${PRERACKET} -N raco -l- \
  pkg install ${raco_args} \
  main-distribution racket-lib"
touch $@

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

[racket-dev] package source as a Git repo

[Matthew Flatt]

The development version of the package manager now supports Git
repository references using "http[s]://" and sites other than GitHub.
That is, a package source can have the form

 git://host/[...]

for a host other than "github.com", and a source can have one of the
forms

 http://[...].git
 https://[...].git

which use the Git protocol over a "smart" HTTP(S) transport. In
particular, the "HTTPS" URL that appears on a GitHub repository page
will work directly as a package source.


Since generalized Git repository references are not supported in the
upcoming v6.1.1 release, it will be a while before you'll want to
publish package sources using the new forms. 

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Pre-Release Checklist for v6.1.1

[Matthew Flatt]

At Thu, 16 Oct 2014 09:13:54 -0400, Ryan Culpepper wrote:
> * Matthew Flatt 
>- Racket Tests
>- Languages Tests
>- GRacket Tests (Also check that `gracket -z' and `gracket-text' still
>  works in Windows and Mac OS X)
>- mzc --exe tests
>- .plt-packing Tests
>- Games Tests
>- Unit Tests
>- Syntax Color Tests
>- R6RS Tests
>- JPR's test suite
>- Create an executable from a BSL program
>- Run COM tests
>- Embed-in-c test
>- Try compiling with -funsigned-char
>- Try compiling with TEST_ALTERNATE_TARGET_REGISTER

Passed.

>Updates:
>- Racket Updates: update HISTORY

Done.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] advice on the 6.x build system.

[Matthew Flatt]

At Fri, 17 Oct 2014 07:43:17 +0200, David Bremner wrote:
> Matthew Flatt  writes:
> 
> >
> > That said, is there a particular reason that basing the build on the
> > git repo would be better?
> >
> 
> One reason is that I need I need to track from release to release the
> files that are removed from the racket source by debian for
> licensing-related reasons. Currently this looks like:
> 
> ╰─ (git)-[new-master]-% git diff --stat dfsg..upstream
>  .../drdr/static/jquery-1.6.2.min.js|   18 +
>  .../resources/js/libs/gumby.min.js |1 +
>  .../js/libs/jquery-1.9.1.min.js|5 +
>  .../libs/jquery.mobile.custom.min.js   |3 +
>  .../js/libs/modernizr-2.6.2.min.js |4 +
>  .../resources/js/plugins.js|8 +
>  .../racket/benchmarks/common/maze.sch  |  680 ++
>  .../racket/benchmarks/common/maze2.sch |  695 ++
>  .../common/psyntax-input.txt   | 4296 
>  .../benchmarks/common/typed/maze2.rktl |  772 ++
>  .../racket-test/tests/xml/xmltest.zip  |  Bin 0 -> 107060 bytes

Happily, none of those are the main-distribution bundles. I don't think
that's a coincidence, and in any case, it should be a goal that we
don't include troublesome files in the distribution.

> A second reason is that I want to be able to able to backport patches to
> older releases of racket running on Debian.  This is much easier if I
> can just use "git cherry-pick".

I can see that `git cherry-pick` is more convenient than creating patch
files, at least until we split the Racket repo. In the near future,
when the main distribution is spread out over 60-90 repos, then it
sounds less convenient.

> A third reason (related) is that from time to time I need to test the
> Debian packaging of an as yet unreleased racket version, e.g. to check
> if a build failure is fixed in the upcoming 6.1.1 branch.

The daily snapshot builds have the same shape as a release, as do the
test builds at http://pre-release.racket-lang.org/. So, I think a source
build will be available when you need one.

Also, you can always create your own source bundle from the git repo by
using `make installer SOURCE_MODE=--source`. That begs the question of
what it means to use a source distribution instead the repo, but the
point is that `make installer` is likely to change shape in the near
future, while the source distribution is not (so a build based on the
source distribution may be a more reusable component.)


You can add a "FWIW" in front of all of those, since you know the
constraints and problems of Debian packaging better. I just worry about
the mismatch between the idea of "get a Racket distribution from a
single git repo" versus the more distributed and package-based
direction that we're heading.


Meanwhile, I haven't answered your original question. Can you remind me
of the specific steps that I'd need to follow to try the script that
you sent before?


_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] advice on the 6.x build system.

[Matthew Flatt]

Hi David,

If I understand correctly, you're trying to base the build on a
checkout of the Racket git repository. I think it's better to base it
on a release source distribution, instead:

 * The source distribution embeds a reference to a release-specific
   package catalog, which effectively freezes certain packages to that
   release. (Which packages and exactly how they should freeze will be
   an ongoing experiment, but the source distribution will be set up
   the right way, in any case).

 * We plan to split up the current Racket repository into many
   package-specific repositories. The current structure was intended as
   a stop-gap, and it's not clear that anything like `make
   local-source-catalog` will continue to exist.

That said, is there a particular reason that basing the build on the
git repo would be better?

Matthew

At Thu, 16 Oct 2014 23:23:37 +0200, David Bremner wrote:
> 
> I've been been trying to rework the debian racket packaging, and to
> understand the new racket build system.  I need to have the two seperate
> targets, which most of the package installation is done in the the
> "build-indep-stamp" target.
> 
> The following makefile snippet is _almost_ working, except that I'm
> missing a launcher for drracket. After installing debian/tmp, running
> /usr/lib/racket/gracket and (require drracket) seems to work ok to start
> drracket, so I take that as indicating most of the packages / collects
> are in the right place.  Can anyone see what I'm doing wrong here?
> 
> I should say that I tried to make better use of the top level makefile,
> but I ended up with wrong default collects paths (probably just a
> different error on my part).
> 
> I tried all of this (most recently) with the current release branch
> (c326c21b73356e)
> 
> destdir:=$(CURDIR)/debian/tmp
> base_build_dir:=$(CURDIR)/racket/src/build
> PRERACKET:=${destdir}/usr/bin/racket  -X ${destdir}/usr/share/racket/collects
> 
> raco_args:=--catalog build/local/catalog --auto -i --skip-installed
> 
> 
> ${base_build_dir}/Makefile: 
>   mkdir -p ${base_build_dir}
>   cd ${base_build_dir} && $(CURDIR)/racket/src/configure --prefix=/usr
> 
> build-arch-stamp: ${base_build_dir}/Makefile
>   $(MAKE) -C ${base_build_dir} 
>   $(MAKE) -C ${base_build_dir} DESTDIR=${destdir} install
>   touch $@
> 
> build-indep-stamp: build-arch-stamp
>   $(MAKE) RACKET="${PRERACKET}" \
>   local-source-catalog 
>   ${PRERACKET} -N raco -l- \
>   pkg install ${raco_args}  main-distribution racket-lib
>   touch $@
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Release for v6.1.1 is about to begin

[Matthew Flatt]

The latest should work. Please try a snapshot:

  http://pre.racket-lang.org/


At Thu, 2 Oct 2014 12:55:50 -0700, Nick Sivo wrote:
> Figured I'd wait until OS X Yosemite was closer to release before
> mentioning this, but DrRacket 6.1 doesn't run on it. Now that Yosemite
> has reached Release Candidate status is there anyone I can invite to
> my Apple Developer Team so they can look into it? I'd try to track it
> down myself but don't have time :(
> 
> The issues seems to be Pango related:
> 
> Process:   DrRacket [2824]
> Path:  /Users/USER/*/DrRacket.app/Contents/MacOS/DrRacket
> Identifier:org.racket-lang.DrRacket
> Version:   6.1 (6.1)
> Code Type: X86-64 (Native)
> Parent Process:??? [1]
> Responsible:   DrRacket [2824]
> User ID:   501
> 
> Date/Time: 2014-10-02 12:53:47.687 -0700
> OS Version:Mac OS X 10.10 (14A379a)
> Report Version:11
> Anonymous UUID:A0B5077B-6D6B-8413-7993-274AB8C3DD72
> 
> Sleep/Wake UUID:   09950D45-3120-4F71-8E32-9D1C1BC16A8F
> 
> Time Awake Since Boot: 6900 seconds
> Time Since Wake:   5800 seconds
> 
> Crashed Thread:0  Dispatch queue: com.apple.main-thread
> 
> Exception Type:EXC_BAD_ACCESS (SIGSEGV)
> Exception Codes:   KERN_INVALID_ADDRESS at 0x
> 
> VM Regions Near 0:
> -->
> __TEXT 0001-00018000 [   32K]
> r-x/rwx SM=COW  /Users/USER/*/DrRacket.app/Contents/MacOS/DrRacket
> 
> Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
> 0   libpangocairo-1.0.0.dylib 0x000106227942 basic_engine_shape + 530
> 1   libpango-1.0.0.dylib   0x00010616e95f
> _pango_engine_shape_shape + 47
> 2   libpango-1.0.0.dylib   0x0001061826c2 pango_shape_full + 194
> 3   libpango-1.0.0.dylib   0x00010617a52a shape_run + 714
> 4   libpango-1.0.0.dylib   0x000106179b8a process_item + 250
> 5   libpango-1.0.0.dylib   0x00010617319f
> pango_layout_check_lines + 2351
> 6   libpango-1.0.0.dylib   0x000106175283
> pango_layout_get_unknown_glyphs_count + 195
> 7   Racket 0x00010034907c ffi_call_unix64 + 76
> 8   Racket 0x0001003498ec ffi_call + 860
> 9   Racket 0x00010033a35f ffi_do_call + 2143
> 10  Racket 0x000100339adc
> ffi_do_call_after_stack_check + 268
> 11  ???   0x0001005f9b4e 0 + 4301232974
> 12  ???   0x00010fd8d97d 0 + 4560836989
> 13  ???   0x000101df60ed 0 + 4326383853
> 14  Racket 0x00010009ba5b _apply_native + 267
> 15  Racket 0x00010009c562
> scheme_apply_chaperone + 2674
> 16  Racket 0x000100069cd1 scheme_do_eval + 3537
> 17  Racket 0x00010006c656
> _scheme_apply_multi_from_native + 342
> 18  ???   0x000101df553c 0 + 4326380860
> 19  Racket 0x00010006bb25 scheme_do_eval + 11301
> 20  Racket 0x0001000a383f scheme_dynamic_wind + 
> 1119
> 21  Racket 0x000100094349 dynamic_wind + 329
> 22  ???   0x0001005f1114 0 + 4301197588
> 23  ???   0x000101df55f9 0 + 4326381049
> 24  ???   0x000101df55f9 0 + 4326381049
> 25  Racket 0x00010006bb25 scheme_do_eval + 11301
> 26  Racket 0x0001000a383f scheme_dynamic_wind + 
> 1119
> 27  Racket 0x000100094349 dynamic_wind + 329
> 28  ???   0x0001005f1114 0 + 4301197588
> 29  Racket 0x00010006bb25 scheme_do_eval + 11301
> 30  Racket 0x00010009fa90
> scheme_finish_apply_for_prompt + 624
> 31  Racket 0x00010009fe00
> scheme_apply_for_prompt + 112
> 32  Racket 0x000100090baf call_with_prompt + 1743
> 33  ???   0x0001005f1114 0 + 4301197588
> 34  Racket 0x00010006bb25 scheme_do_eval + 11301
> 35  Racket 0x00010006c656
> _scheme_apply_multi_from_native + 342
> 36  ???   0x000101dcac5c 0 + 4326206556
> 37  ???   0x000101dca85d 0 + 4326205533
> 38  ???   0x000101dca85d 0 + 4326205533
> 39  ???   0x000101dca85d 0 + 4326205533
> 40  ???   0x000101dca85d 0 + 4326205533
> 41  ???   0x000101dca85d 0 + 4326205533
> 42  ???   0x000101dca85d 0 + 4326205533
> 43  ??? 

Re: [racket-dev] Intermittent Segfault on DrDr

[Matthew Flatt]

I think that commit b946d4639e fixes this crash.

It's nice of you to ask the day after the repair. :) I had spent a lot
of time trying to replicate that crash, with no success, but yesterday
was my lucky day. Claire ran a pessimization experiment on the bytecode
compiler that made "image.scrbl" crash reliably. I can't say for
certain that the bug that I fixed caused this crash, but the details
are a close match for the test content and crash reports.

The bug was introduced in 2009 (commit bc47db42e4).

At Thu, 02 Oct 2014 15:00:25 -0400, Vincent St-Amour wrote:
> The file listed below has been segfaulting intermittently for about a
> month, but I can't reproduce the crash on any of my machines (Linux or
> Mac OS, both 64 bits).
> 
> Walking backwards through the commit log from the first occurrence of
> the crash on DrDr, the first commit that looks like it may be related to
> segfaults is this one:
> 
> https://github.com/plt/racket/commit/15423988220ce1fa5dea60917c5c7e83dde941de
> 
> Any ideas?
> 
> Can anyone reproduce this crash?
> 
> Vincent
> 
> 
> At Wed,  1 Oct 2014 15:56:01 -0400 (EDT),
> d...@racket-lang.org wrote:
> > 
> > DrDr has finished building push #29301 after 3.32h.
> > 
> > http://drdr.racket-lang.org/29301/
> > 
> > A file you are responsible for has a condition that may need inspecting.
> >   unclean:
> > 
> http://drdr.racket-lang.org/29301/pkgs/typed-racket-pkgs/typed-racket-test/tests
> /typed-racket/succeed/mandelbrot.rkt
> > 
> >   stderr:
> > 
> http://drdr.racket-lang.org/29301/pkgs/typed-racket-pkgs/typed-racket-test/tests
> /typed-racket/succeed/mandelbrot.rkt
> > 
> > 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Problem with planet and Windows shortcuts

[Matthew Flatt]

To catch up the list: Antonio sent me more information, and the problem
is that Racket was handling symbolic links but not junctions (which are
a different kind of link on Windows).

Antonio: I've pushed a repair to handle junctions, so let me know if
it works for you.

At Thu, 18 Sep 2014 12:32:48 +0100, Antonio Menezes Leitao wrote:
> Hi,
> 
> On Thu, Sep 18, 2014 at 12:00 PM, Matthew Flatt  wrote:
> 
> > I think there must be a problem with v6.1's support for soft links on
> > Windows, while previous versions of Racket were oblivious to links on
> > Windows.
> >
> > What does
> >
> >  (resolve-path "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> >
> > report?
> >
> >
> >  (resolve-path "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> "C:\\Users\\aml\\AppData\\Roaming\\Racket"
> 
> 
> Best,
> António.
> 
> 
> At Wed, 17 Sep 2014 17:31:55 +0100, Antonio Menezes Leitao wrote:
> > > Hi,
> > >
> > > Recent versions of Racket for Windows (at least, starting from 6.1.0.3)
> > > cannot install packages from planet when Racket's AppData/Roaming/ folder
> > > is a shortcut. I'm sure Racket could do it in previous versions.
> > >
> > > The output I get is:
> > >
> > > ..\..\Program Files\Racket-6.1.0.3\collects\racket\path.rkt:66:14:
> > > normalize-path: element within the input path is not a directory or does
> > > not exist
> > >   element: C:\Users\aml\AppData\Roaming\Racket
> > >
> > > Here are some additional tests:
> > >
> > > > (directory-exists? "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> > > #f
> > > > (file-exists? "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> > > #f
> > > > (link-exists? "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> > > #t
> > >
> > > The reason why I use a shortcut is simply because I use several different
> > > machines which synchronize that folder using Dropbox.
> > >
> > > Is there a workaround (besides making
> > C:\Users\aml\AppData\Roaming\Racket a
> > > regular folder)?
> > >
> > > Best,
> > > António.
> > > _
> > >   Racket Developers list:
> > >   http://lists.racket-lang.org/dev
> >
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> >

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Problem with planet and Windows shortcuts

[Matthew Flatt]

I think there must be a problem with v6.1's support for soft links on
Windows, while previous versions of Racket were oblivious to links on
Windows.

What does

 (resolve-path "C:\\Users\\aml\\AppData\\Roaming\\Racket")

report?

At Wed, 17 Sep 2014 17:31:55 +0100, Antonio Menezes Leitao wrote:
> Hi,
> 
> Recent versions of Racket for Windows (at least, starting from 6.1.0.3)
> cannot install packages from planet when Racket's AppData/Roaming/ folder
> is a shortcut. I'm sure Racket could do it in previous versions.
> 
> The output I get is:
> 
> ..\..\Program Files\Racket-6.1.0.3\collects\racket\path.rkt:66:14:
> normalize-path: element within the input path is not a directory or does
> not exist
>   element: C:\Users\aml\AppData\Roaming\Racket
> 
> Here are some additional tests:
> 
> > (directory-exists? "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> #f
> > (file-exists? "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> #f
> > (link-exists? "C:\\Users\\aml\\AppData\\Roaming\\Racket")
> #t
> 
> The reason why I use a shortcut is simply because I use several different
> machines which synchronize that folder using Dropbox.
> 
> Is there a workaround (besides making C:\Users\aml\AppData\Roaming\Racket a
> regular folder)?
> 
> Best,
> António.
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] GNU lightning version

[Matthew Flatt]

At Wed, 03 Sep 2014 13:07:05 +0400, Yaroslav Tsarko wrote:
> are there any 
> reasons why Racket currently uses very old
> version of GNU Lightning? According to sources, Racket 6.1 uses GNU 
> Lightning version 1.2 which originates from 2004
> [...]
> Is it manpower problems (there is nobody who can upgrade this component 
> in Racket), technical ones or something else?

There are a few technical issues, any of which might be addressed by
contributing back to Lighting, but mostly it's a question of manpower.

We've modified GNU Lighting in various ways. Some of those ways turned
out to be uninteresting, and we'd do just as well to re-sync with the
latest Lighting. Other changes involved the conventions for register
and stack usage, where where re-syncing might be more difficult, and
Racket depends on some implementation details instead of treating
Lighting as a black box. Some changes should have been wrappers around
GNU Lighting, in retrospect, so re-syncing would be good, but it's a
lot of work.

I don't know GNU Lightning's current stance on jump distances on
various platforms, but for Racket, we adapted it to handle a mixture of
short jumps (faster when the JIT knows that it will work), long jumps
(otherwise), and very long jumps (on x86-64 when the memory allocated
for code becomes further apart than fit in 32-bit values). Adding the
jump-distance generalizations to 2.0 would take some work, and it's
messy enough that I'm not sure the Lighting maintainers would be happy
to take it.

One other change might be tricky: the x86 build uses both SSE and x87
instructions to support a mixture of flonums and extflonums, and
Lighting originally worked with only one or the other --- but that
might have changed.


> and current version of 
> GNU Lightning is 2.0 which supports many new back-ends, particularly MIPS.

I back-ported ARM support from and almost-2.0 version of Lighting, and
that process was relatively easy. I had to know about and adapt
Racket's assumptions about Lightning, though.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] variable wrong procedure or structure-type shape error?

[Matthew Flatt]

At Sat, 30 Aug 2014 10:25:53 -0700, Kevin Forchione wrote:
> 
> On Aug 30, 2014, at 7:09 AM, Matthew Flatt  wrote:
> 
> > Just for the record, this is my fault for not incrementing the version
> > with a change to the compiler's optimizer.
> > 
> > I thought of the optimization as "local" and having no effect on a
> > module's interface to other modules. There's no such thing, though,
> > since optimizer-inferred properties of a function's body are exported
> > with a function, and those properties can change with most any
> > optimizer adjustment.
> 
> 
> Just to fill in another hole in my swiss-cheese education, what do we
> mean by “shape” with regard to procedure/structure type?

The term "shape" in the error message isn't meant as any particular,
accepted concept outside the optimizer. In this context, "shape" just
means information about the value that the optimizer finds worthwhile
to track, such as the function's arity and whether it adjusts the
continuation-mark stack.

> It was the require of a class 
> that was producing the error, and I’ve gathered that classes are implemented 
> with structures. It was baffling to my why the module containing the class 
> compiled without error, but compiling the module requiring it failed. It was 
> also interesting that the module did compile fine with the latest version, 
> until I’d tossed the compiled folder in the trash. Running race make on the 
> top-level file resolved the problem, but I’m wondering why  the “run” in 
> DrRacket didn’t do the same.

By default, DrRacket compiles the files that it loads, but it compiles
and uses them in "errortrace" mode (the bytecode is kept in a separate
subdirectory), so that may have something to do with it. I can't say
exactly how it went in your case, but things go wrong in various
baroque ways when the optimizer changes and the version number doesn't.


_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] BUG: busy-waiting

[Matthew Flatt]

Thanks! I've pushed a repair.

At Thu, 4 Sep 2014 01:56:07 -0400, Marc Burns wrote:
> Yes. I forgot that, in the case I was debugging, `unzip` was wrapping
> its input pipe with `make-limited-input-port`. The user port created by
> `make-limited-input-port` is causing the loop. Here's an example:
> 
> #lang racket/base
> (require racket/port)
> 
> (define-values (in out) (make-pipe))
> (define p (make-limited-input-port in 10 #t))
> 
> (write-bytes #"hello" out)
> 
> (define buffer (make-bytes 1))
> (peek-bytes-avail! buffer 6 #f p 0 1)
> 
> The `do-peek` function in port.rkt:1027 makes a nonblocking attempt to
> peek from the wrapped port. When the peek is unsuccessful, it returns an
> event that is ready when the wrapped port is ready and whose value is
> zero.
> 
> Thus, when the wrapped port has greater than zero and fewer than `skip`
> bytes available, `do-peek` is nonblocking.
> 
> The call to `peek-bytes-avail!` then loops calling `do-peek`.
> 
> I think that this could be resolved by returning
> 
> (choice-evt
>   (wrap-evt progress-evt (lambda(x) #f))
>   (wrap-evt
> (peek-bytes-evt skip 0 #f port)
> (lambda(x) 0)))
> 
> instead of the wrapped port event at port.rkt:1027
> 
> Then again, couldn't `do-peek` just return the appropriate
> `peek-bytes-evt` when `count` is nonzero?
> 
> Marc
> 
> On Thu, Sep 04, 2014 at 05:49:17AM +0200, Matthew Flatt wrote:
> > Can you provide an example?
> > 
> > Here's what I tried, but it blocks without busy-waiting:
> > 
> > 
> > #lang racket
> > (require file/gunzip file/gzip)
> > 
> > (define dest (open-output-bytes))
> > (deflate (open-input-bytes #"hello") dest)
> > 
> > (define bstr (get-output-bytes dest))
> > 
> > (define-values (i o) (make-pipe))
> > 
> > ;; Omit the last byte:
> > (write-bytes bstr o 0 (- (bytes-length bstr) 1))
> > 
> > 'inflating...
> > (inflate i (open-output-bytes))
> > 
> > 
> > 
> > At Wed, 3 Sep 2014 21:02:06 -0400, Marc Burns wrote:
> > > The offending function in my case seems to be peek-bytes-avail!
> > > 
> > > The busy wait is entered on line 284 of collects/file/gunzip.rkt. Here's
> > > what happens when I add some tracing to gunzip.rkt and then try to
> > > inflate a Racket pipe:
> > > 
> > > ...
> > > +   (displayln "before read-bytes!")
> > > (read-bytes! buffer input-port 0 (max 0 (- buf-max 
> MAX-LOOKAHEAD)))
> > > +   (displayln "after read-bytes!")
> > > ;; Even though we won't actually use bytes that we "unwind",
> > > ;; setting `buf-pos' to the number of unwound bytes lets us
> > > ;; keep track of how much to not actually read at the end.
> > > (set! buf-pos (min MAX-LOOKAHEAD buf-max)))
> > >   ;; Peek (not read) some available bytes:
> > > + (displayln "before peek-bytes-avail!")
> > >   (let ([got (peek-bytes-avail! buffer buf-pos #f input-port 
> buf-pos 
> > > BUFFER-SIZE)])
> > > +   (displayln "after peek-bytes-avail!")
> > > ...
> > > +  (trace READBITS)
> > > 
> > > Output:
> > > 
> > > ...
> > > >(READBITS 9)
> > > <#
> > > >(READBITS 9)
> > > before read-bytes!
> > > after read-bytes!
> > > before peek-bytes-avail!
> > > 
> > > That's it. Racket is busy-waiting in peek-bytes-avail!
> > > 
> > > On Wed, Sep 03, 2014 at 10:52:18PM +0200, Jan Dvořák wrote:
> > > > Hello,
> > > > 
> > > > I am hitting a rather uncomfortable bug that causes runtime to start
> > > > internal busy-waiting at around:
> > > > 
> > > > #0  scheme_block_until(_f=,
> > > > fdf=)
> > > >at ../src/thread.c:5199
> > > > #1  do_sync (name="sync", with_break=0, with_timeout=0)
> > > >at ../src/thread.c:7109
> > > > 
> > > > It goes through ../src/thread.c:5190, which Matthew mentions in one of 
> > > > his
> > > > recent patches.
> > > > 
> > > > The code that manages to trigger this have been written under NDA and 
> cannot
> > > > be published. I have not yet managed to reproduce the issue separately, 
> but
> > > > it seems that this might not be the only instance:
> > > > 
> > > >(09:28:36 PM) m4burns: Mordae: I'm having a similar problem 
> > > > somewhere 
> in
> > > >   `inflate`. Haven't applied a debugger yet, but
> > > >   there's certainly no busy waiting in the 
> > > > script.
> > > > 
> > > > I am stuck and would like to ask for your help.
> > > > 
> > > > Best regards,
> > > > Jan Dvorak
> > > > 
> > > > 
> > > > 
> > > > _
> > > >  Racket Developers list:
> > > >  http://lists.racket-lang.org/dev
> > > > 
> > > _
> > >   Racket Developers list:
> > >   http://lists.racket-lang.org/dev
> > 

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] BUG: busy-waiting

[Matthew Flatt]

Can you provide an example?

Here's what I tried, but it blocks without busy-waiting:


#lang racket
(require file/gunzip file/gzip)

(define dest (open-output-bytes))
(deflate (open-input-bytes #"hello") dest)

(define bstr (get-output-bytes dest))

(define-values (i o) (make-pipe))

;; Omit the last byte:
(write-bytes bstr o 0 (- (bytes-length bstr) 1))

'inflating...
(inflate i (open-output-bytes))



At Wed, 3 Sep 2014 21:02:06 -0400, Marc Burns wrote:
> The offending function in my case seems to be peek-bytes-avail!
> 
> The busy wait is entered on line 284 of collects/file/gunzip.rkt. Here's
> what happens when I add some tracing to gunzip.rkt and then try to
> inflate a Racket pipe:
> 
> ...
> +   (displayln "before read-bytes!")
> (read-bytes! buffer input-port 0 (max 0 (- buf-max 
> MAX-LOOKAHEAD)))
> +   (displayln "after read-bytes!")
> ;; Even though we won't actually use bytes that we "unwind",
> ;; setting `buf-pos' to the number of unwound bytes lets us
> ;; keep track of how much to not actually read at the end.
> (set! buf-pos (min MAX-LOOKAHEAD buf-max)))
>   ;; Peek (not read) some available bytes:
> + (displayln "before peek-bytes-avail!")
>   (let ([got (peek-bytes-avail! buffer buf-pos #f input-port buf-pos 
> BUFFER-SIZE)])
> +   (displayln "after peek-bytes-avail!")
> ...
> +  (trace READBITS)
> 
> Output:
> 
> ...
> >(READBITS 9)
> <#
> >(READBITS 9)
> before read-bytes!
> after read-bytes!
> before peek-bytes-avail!
> 
> That's it. Racket is busy-waiting in peek-bytes-avail!
> 
> On Wed, Sep 03, 2014 at 10:52:18PM +0200, Jan Dvořák wrote:
> > Hello,
> > 
> > I am hitting a rather uncomfortable bug that causes runtime to start
> > internal busy-waiting at around:
> > 
> > #0  scheme_block_until(_f=,
> > fdf=)
> >at ../src/thread.c:5199
> > #1  do_sync (name="sync", with_break=0, with_timeout=0)
> >at ../src/thread.c:7109
> > 
> > It goes through ../src/thread.c:5190, which Matthew mentions in one of his
> > recent patches.
> > 
> > The code that manages to trigger this have been written under NDA and cannot
> > be published. I have not yet managed to reproduce the issue separately, but
> > it seems that this might not be the only instance:
> > 
> >(09:28:36 PM) m4burns: Mordae: I'm having a similar problem somewhere in
> >   `inflate`. Haven't applied a debugger yet, but
> >   there's certainly no busy waiting in the script.
> > 
> > I am stuck and would like to ask for your help.
> > 
> > Best regards,
> > Jan Dvorak
> > 
> > 
> > 
> > _
> >  Racket Developers list:
> >  http://lists.racket-lang.org/dev
> > 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] variable wrong procedure or structure-type shape error?

[Matthew Flatt]

Just for the record, this is my fault for not incrementing the version
with a change to the compiler's optimizer.

I thought of the optimization as "local" and having no effect on a
module's interface to other modules. There's no such thing, though,
since optimizer-inferred properties of a function's body are exported
with a function, and those properties can change with most any
optimizer adjustment.

At Fri, 29 Aug 2014 16:22:10 -0700, Kevin Forchione wrote:
> 
> On Aug 29, 2014, at 2:17 PM, Matthias Felleisen  wrote:
> 
> > 
> > You need to clean out the cached compiled code and remake those 
> > collections. 
> Remove the compiled directories, and run raco make again. -- Matthias
> > 
> > 
> > 
> > On Aug 29, 2014, at 5:10 PM, Kevin Forchione  wrote:
> > 
> >> H…. something changed between Racket 6.1 and the latest release that 
> >> is 
> producing this mysterious (to me at least!) error in a project of mine that 
> compiles fine with 6.1, but not with the latest version, producing: 
> >> 
> >> Welcome to DrRacket, version 6.1.0.5--2014-08-28(a1f5340/a) [3m].
> >> Language: racket [custom]; memory limit: 512 MB.
> >> link: bad variable linkage;
> >> reference to a variable that has the wrong procedure or structure-type 
> >> shape
> >> reference phase level: 0
> >> variable module: "/Users/lysseus/Racket/games/lib/matrix.rkt"
> >> variable phase: 0
> >> reference in module: "/Users/lysseus/Racket/games/2048/2048-obj.rkt" in: 
> matrix-row
> >> 
> >> Any idea wheat this error means and why 6.1.0.5 is getting the error? It 
> appears to have been introduced sometime after version 
> 6.1.0.5--2014-08-25(32ae3f8/a) [3m]., which compiled fine.
> 
> That sorted it. And that’s got me on to learning more about race. :)
> 
> -Kevin
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [racket] Performance. Higher-order function

[Matthew Flatt]

I didn't change the "main-distribution" package. I changed the Utah
snapshot's configuration to make the Racket distribution have
"main-distribution" plus "optimization-coach".

Changing "main-distribution" would interfere with a top-level `make`
and other things that are still built around the current transitional
model (i.e., still having a main Racket repository for the content of
the main distribution).

To really get "optimization-coach" into the main distribution, I think
we have two options:

 * Split the main repo and shift everything into that mode.

 * Move "optimization-coach" into the main repo, for now, even though
   we expect to split the main repo in the near future.


At Wed, 27 Aug 2014 13:44:34 -0400, Vincent St-Amour wrote:
> Great, thanks!
> 
> I didn't see changes to the main repo to reflect that addition. Are the
> contents of the main distribution not part of the repo anymore?
> 
> Vincent
> 
> 
> 
> At Wed, 27 Aug 2014 08:46:02 -0600,
> Matthew Flatt wrote:
> > 
> > The Racket snapshots at
> > 
> >  http://www.cs.utah.edu/plt/snapshots/
> > 
> > now include the Optimization Coach package.
> > 
> > At Tue, 12 Aug 2014 18:32:05 +0100, Matthew Flatt wrote:
> > > Our build system can create a distribution from any set of packages,
> > > and so we can easily switch over one of the snapshots to use that mode
> > > and include the OC. The Utah snapshot machine fell off the network
> > > (that's why we have more than snapshot site...), but I can try adding
> > > OC when I can reset it next week.
> > 
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [racket] Performance. Higher-order function

[Matthew Flatt]

The Racket snapshots at

 http://www.cs.utah.edu/plt/snapshots/

now include the Optimization Coach package.

At Tue, 12 Aug 2014 18:32:05 +0100, Matthew Flatt wrote:
> Our build system can create a distribution from any set of packages,
> and so we can easily switch over one of the snapshots to use that mode
> and include the OC. The Utah snapshot machine fell off the network
> (that's why we have more than snapshot site...), but I can try adding
> OC when I can reset it next week.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] internal-definition-context-seal doesn't seem to do anything

[Matthew Flatt]

If you put the code below in a module, you don't get an error, because
the syntax object that has the unsealed context doesn't appear in the
fully-expanded module. The syntax object is in a local macro binding
that disappears.

If you remove the `let' around

  (bind h (define q 5))
  (define q 8)
  (nab h)

then the relevant syntax object remains in the fully expanded result,
and you get an error.

At Fri, 15 Aug 2014 14:33:35 -0400, Stephen Chang wrote:
> The docs say that if I make an internal definition context with
> syntax-local-make-definition-context, I have to seal it with
> internal-definition-context-seal, otherwise an exception gets raised.
> But I can't get this to happen. Does someone have an example that
> causes the exception to get thrown?
> 
> For example, this program from the racket tests produces the same
> result if I comment out the seal line. Other tests behave similarly.
> 
> (define-syntax (bind stx)
>   (syntax-case stx ()
> [(_ handle def)
>  (let ([def-ctx (syntax-local-make-definition-context)]
>[ctx (cons (gensym 'intdef)
>   (let ([orig-ctx (syntax-local-context)])
> (if (pair? orig-ctx)
> orig-ctx
> null)))]
>[kernel-forms (list #'define-values)])
>(let ([def (local-expand #'def ctx kernel-forms def-ctx)])
>  (syntax-case def ()
>[(define-values (id) rhs)
> (begin
>   (syntax-local-bind-syntaxes (list #'id) #f def-ctx)
>   (internal-definition-context-seal def-ctx)
>   #'(begin
>   (define-values (id) rhs)
>   (define-syntax handle (quote-syntax id]
>[_ (error "no")])))]))
> 
> (define-syntax (nab stx)
>   (syntax-case stx ()
> [(_ handle)
>  (syntax-local-value #'handle)]))
> 
> (let ()
>   (bind h (define q 5))
>   (define q 8)
>   (nab h))
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] SGC as default

[Matthew Flatt]

Good point!

I assumed (based on earlier conversations?) that Sam knows about
`--enable-racket=...`. I thought that he was commenting on how the
build process would create things that are needed only for `racketcgc`,
even when `racketcgc` itself isn't needed.

I've now pushed changes that should skip all `racketcgc`-specific
pieces when `--enable-racket=...` is provided.

At Wed, 13 Aug 2014 11:08:22 +0200, Tobias Hammer wrote:
> What about ./configure --enable-racket=`which racket` ?
> Already needed and used for cross-compilation.
> 
> Tobias
> 
> 
> On Tue, 12 Aug 2014 18:00:21 +0200, Matthew Flatt   
> wrote:
> 
> > I'm not sure how difficult it will be. It's tedious enough that the
> > last time I thought about it, I just left a note next to
> > "no-cgc-needed" in "racket/src/racket/Makefile.in", but maybe it's
> > worth pursuing now.
> >
> > At Tue, 12 Aug 2014 04:39:55 -0700, Sam Tobin-Hochstadt wrote:
> >> How difficult would it be to allow the bootstrap process to use a
> >> preexisting Racket installation? This would alleviate some of the
> >> performance loss, for example in rebuilds by developers or in continuous
> >> integration.
> >>
> >> Sam
> >> On Aug 11, 2014 11:16 PM, "Matthew Flatt"  wrote:
> >>
> >> > I've changed the Racket CGC implementation --- which is mostly used
> >> > only to build the normal Racket variant --- to use SGC by default,
> >> > instead of the Boehm GC. The intent of the switch is to make the more
> >> > portable GC the default.
> >> >
> >> > If you have an existing build in a repo checkout, then `make` is  
> >> likely
> >> > to fail, because the makefile dependencies are not precise enough to
> >> > deal with the switch. You can discard your old build directory, or it
> >> > might work to simply delete
> >> >
> >> >   /racket/libmzgc.a
> >> >
> >> > If you're using CGC and want to continue using the Boehm GC, then
> >> > provide `--disable-sgc` to `configure`. I've tuned SGC to bring its
> >> > performance closer to the Boehm GC, but it's still slower.
> >> >
> >> > _
> >> >   Racket Developers list:
> >> >   http://lists.racket-lang.org/dev
> >> >
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> 
> 
> -- 
> Tobias Hammer
> DLR / Robotics and Mechatronics Center (RMC)
> Muenchner Str. 20, D-82234 Wessling
> Tel.: 08153/28-1487
> Mail: tobias.ham...@dlr.de
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] [racket] Performance. Higher-order function

[Matthew Flatt]

We discussed the possibility git submodules last year, and I think the
consensus was that it wouldn't work well. We did attach the
native-library repo (for Mac OS X and Windows) as a submodule to the
main Racket repo, and that worked well enough, but I think it has
worked only because the native-library repo changes rarely.

Our build system can create a distribution from any set of packages,
and so we can easily switch over one of the snapshots to use that mode
and include the OC. The Utah snapshot machine fell off the network
(that's why we have more than snapshot site...), but I can try adding
OC when I can reset it next week.

Including OC in a build from a checkout of the main repo seems harder.
The main repo in its current shape was always intended to be an
intermediate step before we split it up into separate, package-specific
repos; that still seems like the right direction to me.

At Tue, 12 Aug 2014 12:33:01 -0400, Leif Andersen wrote:
> > but my guess is that that will require some work internally to make
> not be a pain for people who want to build from git
> 
> Wouldn't this be the ideal case for git submodules?
> 
> 
> ~Leif Andersen
> 
> 
> On Tue, Aug 12, 2014 at 7:55 AM, Robby Findler 
> wrote:
> 
> > I think you'd have to add a dependency to the 'main-distribution' pkg,
> > but my guess is that that will require some work internally to make
> > not be a pain for people who want to build from git. If you have the
> > inclination, you could give it a try locally and let us know how it
> > goes?
> >
> > Robby
> >
> > On Tue, Aug 12, 2014 at 6:49 AM, Vincent St-Amour 
> > wrote:
> > > Ok, let's try to do that. Is there currently a way to include packages
> > > from 3rd party repos to the main distribution?
> > >
> > > Vincent
> > >
> > >
> > > At Tue, 12 Aug 2014 00:03:04 -0400,
> > > Greg Hendershott wrote:
> > >>
> > >> > Being in the main repo is different from being in the distribution
> > (and thus automatically installed). I think that OC should be there when
> > you download the full bundle.
> > >>
> > >> Definitely.
> > >>
> > >> 1. It's very useful.
> > >> 2. Its existence says, Racket optimization is a thing.
> > >> 3. It's used with one of Racket's most appealing and unique facets,
> > DrRacket.
> > >> &c
> > > _
> > >   Racket Developers list:
> > >   http://lists.racket-lang.org/dev
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> >
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] SGC as default

[Matthew Flatt]

I'm not sure how difficult it will be. It's tedious enough that the
last time I thought about it, I just left a note next to
"no-cgc-needed" in "racket/src/racket/Makefile.in", but maybe it's
worth pursuing now.

At Tue, 12 Aug 2014 04:39:55 -0700, Sam Tobin-Hochstadt wrote:
> How difficult would it be to allow the bootstrap process to use a
> preexisting Racket installation? This would alleviate some of the
> performance loss, for example in rebuilds by developers or in continuous
> integration.
> 
> Sam
> On Aug 11, 2014 11:16 PM, "Matthew Flatt"  wrote:
> 
> > I've changed the Racket CGC implementation --- which is mostly used
> > only to build the normal Racket variant --- to use SGC by default,
> > instead of the Boehm GC. The intent of the switch is to make the more
> > portable GC the default.
> >
> > If you have an existing build in a repo checkout, then `make` is likely
> > to fail, because the makefile dependencies are not precise enough to
> > deal with the switch. You can discard your old build directory, or it
> > might work to simply delete
> >
> >   /racket/libmzgc.a
> >
> > If you're using CGC and want to continue using the Boehm GC, then
> > provide `--disable-sgc` to `configure`. I've tuned SGC to bring its
> > performance closer to the Boehm GC, but it's still slower.
> >
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> >
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] SGC as default

[Matthew Flatt]

The gcc 4.9 release notes warn about this optimization:

 https://gcc.gnu.org/gcc-4.9/porting_to.html

I'm surprised that this change hasn't caused more trouble for us.

At Tue, 12 Aug 2014 08:39:01 +0100, Matthew Flatt wrote:
> Apparently, when gcc 4.9.1 sees
> 
>  memcpy(x, y, n);
>  if (y)
>
> 
> then it believes `y` can be assumed to be NULL --- even if `n` turns
> out to be zero --- and so the conditional can be optimized away.
> 
> I'm surprised by that rule, but it's easy enough to move the test
> before the memcpy().
> 
> At Tue, 12 Aug 2014 02:00:04 -0400, Asumu Takikawa wrote:
> > On 2014-08-12 06:06:51 +0100, Matthew Flatt wrote:
> > > What platform are you using?
> > > 
> > > I imagine that running `./racketcgc` within the "racket" subdirectory
> > > of your build directory will similarly crash. Can you get any
> > > information from running `gdb racketcgc`?
> > 
> > This is on Linux. Running the built executable immediately produces a 
> segfault.
> > 
> > Here's a backtrace from gdb:
> >   (gdb) bt
> >   #0  0x005e78db in free_managed (s=s@entry=0x0) at 
> > ../../../racket/sgc/sgc.c:1535
> >   #1  0x005e7f63 in GC_add_roots (start=start@entry=0x881b70 
> > , end=end@entry=0x881b79 ) at 
> > ../../../racket/sgc/sgc.c:1657
> >   #2  0x00576f9a in scheme_register_static (ptr=ptr@entry=0x881b70 
> > , size=size@entry=8) at ../../../racket/src/salloc.c:720
> >   #3  0x0045b87f in scheme_set_collects_path 
> (p=p@entry=0x77f701a0) 
> > at ../../../racket/src/file.c:6841
> >   #4  0x00436673 in run_from_cmd_line (mk_basic_env= > out>, 
> > cont_run=0x435a10 , _argv=, argc=0) at 
> > ../../racket/cmdline.inc:1444
> >   #5  main_after_stack (data=data@entry=0x7fffdc30) at 
> > ../../racket/main.c:450
> >   #6  0x0057694d in do_main_stack_setup (data=0x7fffdc30, 
> > _main=0x435a80 , no_auto_statics=1) at 
> > ../../../racket/src/salloc.c:198
> >   #7  scheme_main_stack_setup (no_auto_statics=no_auto_statics@entry=1, 
> > _main=_main@entry=0x435a80 , 
> > data=data@entry=0x7fffdc30)
> >   at ../../../racket/src/salloc.c:310
> >   #8  0x004346ee in main_after_dlls (argv=, 
> > argc=) at ../../racket/main.c:381
> >   #9  main (argc=, argv=) at 
> > ../../racket/main.c:341
> > 
> > Cheers,
> > Asumu
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] SGC as default

[Matthew Flatt]

Apparently, when gcc 4.9.1 sees

 memcpy(x, y, n);
 if (y)
   

then it believes `y` can be assumed to be NULL --- even if `n` turns
out to be zero --- and so the conditional can be optimized away.

I'm surprised by that rule, but it's easy enough to move the test
before the memcpy().

At Tue, 12 Aug 2014 02:00:04 -0400, Asumu Takikawa wrote:
> On 2014-08-12 06:06:51 +0100, Matthew Flatt wrote:
> > What platform are you using?
> > 
> > I imagine that running `./racketcgc` within the "racket" subdirectory
> > of your build directory will similarly crash. Can you get any
> > information from running `gdb racketcgc`?
> 
> This is on Linux. Running the built executable immediately produces a 
> segfault.
> 
> Here's a backtrace from gdb:
>   (gdb) bt
>   #0  0x005e78db in free_managed (s=s@entry=0x0) at 
> ../../../racket/sgc/sgc.c:1535
>   #1  0x005e7f63 in GC_add_roots (start=start@entry=0x881b70 
> , end=end@entry=0x881b79 ) at 
> ../../../racket/sgc/sgc.c:1657
>   #2  0x00576f9a in scheme_register_static (ptr=ptr@entry=0x881b70 
> , size=size@entry=8) at ../../../racket/src/salloc.c:720
>   #3  0x0045b87f in scheme_set_collects_path 
> (p=p@entry=0x77f701a0) 
> at ../../../racket/src/file.c:6841
>   #4  0x00436673 in run_from_cmd_line (mk_basic_env=, 
> cont_run=0x435a10 , _argv=, argc=0) at 
> ../../racket/cmdline.inc:1444
>   #5  main_after_stack (data=data@entry=0x7fffdc30) at 
> ../../racket/main.c:450
>   #6  0x0057694d in do_main_stack_setup (data=0x7fffdc30, 
> _main=0x435a80 , no_auto_statics=1) at 
> ../../../racket/src/salloc.c:198
>   #7  scheme_main_stack_setup (no_auto_statics=no_auto_statics@entry=1, 
> _main=_main@entry=0x435a80 , data=data@entry=0x7fffdc30)
>   at ../../../racket/src/salloc.c:310
>   #8  0x004346ee in main_after_dlls (argv=, 
> argc=) at ../../racket/main.c:381
>   #9  main (argc=, argv=) at 
> ../../racket/main.c:341
> 
> Cheers,
> Asumu
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] SGC as default

[Matthew Flatt]

At Tue, 12 Aug 2014 00:43:04 -0400, Asumu Takikawa wrote:
> On 2014-08-12 05:16:21 +0100, Matthew Flatt wrote:
> > If you have an existing build in a repo checkout, then `make` is likely
> > to fail, because the makefile dependencies are not precise enough to
> > deal with the switch. You can discard your old build directory, or it
> > might work to simply delete
> >
> >   /racket/libmzgc.a
> 
> The build didn't work for me in an existing checkout, so I tried making a 
> fresh
> one and still got this failure:
> 
>   make xsrc/precomp.h
>   make[7]: Entering directory 
> '/home/asumu/plt/racket-fresh/racket/src/build/racket/gc2'
>   env XFORM_PRECOMP=yes ../racketcgc -G 
> /home/asumu/plt/racket-fresh/build/config -cqu ../../../racket/gc2/xform.rkt 
> --setup . --cpp "gcc -E -I./.. -I../../../racket/gc2/../include -pthread   
> -DUSE_SENORA_GC   "  --keep-lines -o xsrc/precomp.h 
> ../../../racket/gc2/precomp.c
>   Segmentation fault
>   Makefile:202: recipe for target 'xsrc/precomp.h' failed
>   make[7]: *** [xsrc/precomp.h] Error 139
> 
> Anything I should try to debug this?

What platform are you using?

I imagine that running `./racketcgc` within the "racket" subdirectory
of your build directory will similarly crash. Can you get any
information from running `gdb racketcgc`?

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

[racket-dev] SGC as default

[Matthew Flatt]

I've changed the Racket CGC implementation --- which is mostly used
only to build the normal Racket variant --- to use SGC by default,
instead of the Boehm GC. The intent of the switch is to make the more
portable GC the default.

If you have an existing build in a repo checkout, then `make` is likely
to fail, because the makefile dependencies are not precise enough to
deal with the switch. You can discard your old build directory, or it
might work to simply delete

  /racket/libmzgc.a

If you're using CGC and want to continue using the Boehm GC, then
provide `--disable-sgc` to `configure`. I've tuned SGC to bring its
performance closer to the Boehm GC, but it's still slower.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Seg fault

[Matthew Flatt]

Thanks for the report. Unfortunately, I'm not able to replicate the
crash, so I'm not sure what to recommend.

If you have Visual Studio installed, then it may offer to open a
debugger, or you could attach to the process while a "program has
crashed" dialog is showing. A C-level backtrace extracted from the
debugger might be helpful.

If you don't have Visual Studio installed, does the OS offer any
additional information at all?

At Thu, 31 Jul 2014 16:54:15 +0700 (NOVT), "Evgeny Odegov" wrote:
> Today some conditions met on my machine and DrRacket began to crash almost
> immediately after run. So I reproduced it many times.
> 
> The results of last several crashes:
> 
> *  Seg fault (internal error) at 31a3a4c
>Seg fault (internal error) at 31a3a4c
>Seg fault (internal error) at 31a3a4c
> 
> *  Seg fault (internal error) at 25863a4c
>Seg fault (internal error) at 25863a4c
>Seg fault (internal error) at 25863a4c
> 
> *  Seg fault (internal error) at 29e93a4c
>Seg fault (internal error) at 29e93a4c
>Seg fault (internal error) at 29e93a4c
> 
> *  Seg fault (internal error) at 4223a4c
>Seg fault (internal error) at 4223a4c
>Seg fault (internal error) at 4223a4c
> 
> *  Seg fault (internal error) at 1c55392ac
>Seg fault (internal error) at 1c55392ac
>Seg fault (internal error) at 1c55392ac
> 
> *  Seg fault (internal error) at 256b41fc
>Seg fault (internal error) at 256b41fc
>Seg fault (internal error) at 256b41fc
> 
> Racket version: (32 bit x86) 6.1.0.2--2014-07-14(c7dd72a/a)
> OS: Windows 7 Prof 64 bit
> 
> I also decided to try current snapshot 6.1.0.3--2014-07-30(f3a8883/a).
> The crashes are repeated:
> 
> *  Seg fault (internal error) at 2b9c3a4c
>Seg fault (internal error) at 2b9c3a4c
>Seg fault (internal error) at 2b9c3a4c
> 
> *  Seg fault (internal error) at 28b53a4c
>Seg fault (internal error) at 28b53a4c
>Seg fault (internal error) at 28b53a4c
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Unable to expand cross-phase persistent module

[Matthew Flatt]

Sorry that I lost track of this one. I've pushed a repair.

At Tue, 29 Jul 2014 16:26:21 -0700, Sam Tobin-Hochstadt wrote:
> Here's a simpler version of this problem:
> 
> #lang racket
> (parameterize ([current-namespace (make-base-namespace)])
>   (expand (datum->syntax
>#f
>'(module m '#%kernel
>   (#%declare #:cross-phase-persistent)
> 
> Sam
> 
> On Wed, Jul 16, 2014 at 12:59 PM, Sam Tobin-Hochstadt
>  wrote:
> > Running `expand` on the module defined in `racket/tcp` errors.
> >
> > In transcript form:
> >
> > -> (define p (open-input-file
> > "/home/samth/sw/plt/racket/collects/racket/tcp.rkt"))
> > -> (define mod (read-syntax (object-name p) p))
> > -> (parameterize ([current-namespace (make-base-namespace)])
> >  (expand (namespace-syntax-introduce mod)))
> > ; /home/samth/sw/plt/racket/collects/racket/tcp.rkt::2: module: cannot be
> > ;   cross-phase persistent due to required modules
> > ;   in: (#%module-begin (#%require (all-except (quote #%network) 
> tcp-addresses)
> > ; (rename (quote #%network) c:tcp-addresses tcp-addresses)) (#%provide
> > ; tcp-connect tcp-connect/enable-break tcp-listen tcp-close
> > ; tcp-accept-ready? tcp-accept tcp-accept-evt tcp-accept...
> > ; [,bt for context]
> >
> > I don't know why it would do this. The same thing happens with the
> > minimal module that's declared cross-phase persistent.
> >
> > Sam
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Release Announcement for v6.1

[Matthew Flatt]

At Tue, 29 Jul 2014 06:31:21 -0700, Sam Tobin-Hochstadt wrote:
> Plumbers look like a fundamental new runtime system concept, and so I think
> we should mention them, even though most people won't use them.

At Tue, 29 Jul 2014 13:47:58 -0400, Matthias Felleisen wrote:
> +1 on plumbers, if only for the word :-) 

Ok, here's a draft bullet:

 * Plumbers generalize the flush-on-exit capability of primitive output
   ports to enable arbitrary flushing actions and to give programmers
   control over the timing of flushes (i.e., a composable `atexit`).
   New functions include `current-plumber`, `plumber-add-flush!`, and
   `plumber-flush-all`.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Release Announcement for v6.1

[Matthew Flatt]

At Mon, 28 Jul 2014 14:33:07 -0400, Ryan Culpepper wrote:
> mflatt:
> - ARM JIT: fix software floating-point (ffb0dd52)
> - add plumbers (d5b42f8c)
> - raco make: improve parallelism (9e3b9844)
> - deprecate 3-arg module name resolver calls (8aaa3fc5)
> - win32: support symbolic links (3e3cb716, 9fed5b58)
> - drawing, bounding boxes, picts, scribble
>(970b040d, 37af1c8e, c4a58dc4, 05760a12, dac8ba28)

These seem too minor to mention in a release announcement. (FWIW, I
think the first one was included in 6.0.1.)

Possibly, it's worth noting the upgraded native libraries on Windows
and Mac OS X:

 * Upgraded and normalized versions of graphics libraries and
   dependencies (Pango, Cairo, GLib, etc.) that are bundled with Racket
   on Windows and Mac OS X. For example, FreeType support is
   consistently enabled.

Like Robby's bullets, though, feel free to leave that one out.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] A tricky chaperone puzzle

[Matthew Flatt]

Unless I still have it wrong, the implementation of 2 was
straightforward.

I would have overlooked the need to restrict `chaperone-struct` to
chaperones of accessors and mutators if you hadn't mentioned it.

At Thu, 24 Jul 2014 15:45:18 -0400, Sam Tobin-Hochstadt wrote:
> Consider the following module:
> 
> (module m racket
>   (struct x [a])
>   (define v1 (x 'secret))
>   (define v2 (x 'public))
>   (provide v1 v2)
>   (provide/contract [x-a (-> x? (not/c 'secret))]))
> 
> It appears that this ensures that you can't get 'secret. But, it turns
> out that I can write a function outside of `m` that behaves like `x-a`
> without the contract:
> 
> (require (prefix-in m: 'm))
> 
> (define (x-a v)
>   (define out #f)
>   (with-handlers ([void void])
> (m:x-a (chaperone-struct v m:x-a (λ (s v) (set! out v) v
>   out)
> 
> Now this works:
> 
> (displayln (x-a m:v1)) ;; => 'secret
> 
> The problem is that `m:x-a` is treated as a
> `struct-accessor-procedure?`, which is a capability for accessing the
> a field, even though it's a significantly restricted capability.
> 
> There are a couple possible solutions I've thought of:
> 
> 1. Require a non-chaperoned/impersonated accessor.
> 2. Actually use the chaperoned/impersonatored accessor to get the
> value out instead of the underlying accessor.
> 
> 1 is a little less expressive. But note that 2 means that you have to
> only allow chaperoned procedures with `chaperone-struct`, and imposes
> significant complication on the runtime.
> 
> I favor 1.
> 
> Sam
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] strange top-level binding for module-defined identifiers

[Matthew Flatt]

I'll push a repair.

The problem is in the representation of syntax objects and the flaky
way that it was generalized to support identifiers that move across
submodule boundaries (but the problem didn't just affect programs with
submodules, in this case).

At Thu, 24 Jul 2014 16:24:42 -0400, Sam Tobin-Hochstadt wrote:
> Ok, here's a much simpler example:
> 
> #lang racket
> 
> (module foo racket
>   (provide def-wrap)
>   (define-syntax-rule (def-wrap)
> (begin  (define y 1) y)))
> 
> (module bar racket
>   (require (submod ".." foo))
>   (def-wrap))
> 
> In the fully-expanded syntax, the macro stepper suggests that `y` has
> no apparent binding.
> 
> At this point it seems unlikely that it's a bug, since it happens all
> the time, but I still don't understand.
> 
> Sam
> 
> On Thu, Jul 24, 2014 at 4:08 PM, Sam Tobin-Hochstadt
>  wrote:
> > If you take this program (which is a lot like the implementation of
> > `racket/fixnum`):
> >
> > #lang racket/base
> >
> > (require '#%flfxnum
> >  racket/private/vector-wraps
> >  racket/unsafe/ops
> >  (for-syntax racket/base))
> >
> > (define-vector-wraps "fxvector"
> >   "fixnum?" fixnum?
> >   fxvector? fxvector-length fxvector-ref fxvector-set! make-fxvector
> >   unsafe-fxvector-ref unsafe-fxvector-set! unsafe-fxvector-length
> >   in-fxvector*
> >   in-fxvector
> >   for/fxvector
> >   for*/fxvector
> >   fxvector-copy
> >   0)
> >
> > And run it in the macro stepper with macro hiding off, at the end you
> > get a fully-expanded module where the first definition is
> > `:fXvector-gen`. However, if you click on this identifier, either in
> > the definition or in a subsequent use, it says "Apparent identifier
> > binding: none" (which means that `identifier-binding` returns #f,
> > which can be confirmed on the expanded syntax).
> >
> > How can this happen? Is it a bug in something, or a special case that
> > I didn't expect?
> >
> > Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] A tricky chaperone puzzle

[Matthew Flatt]

Nice example. Offhand, I think that #2 is right, but I'll have to look
at it more to be sure.

At Thu, 24 Jul 2014 15:45:18 -0400, Sam Tobin-Hochstadt wrote:
> Consider the following module:
> 
> (module m racket
>   (struct x [a])
>   (define v1 (x 'secret))
>   (define v2 (x 'public))
>   (provide v1 v2)
>   (provide/contract [x-a (-> x? (not/c 'secret))]))
> 
> It appears that this ensures that you can't get 'secret. But, it turns
> out that I can write a function outside of `m` that behaves like `x-a`
> without the contract:
> 
> (require (prefix-in m: 'm))
> 
> (define (x-a v)
>   (define out #f)
>   (with-handlers ([void void])
> (m:x-a (chaperone-struct v m:x-a (λ (s v) (set! out v) v
>   out)
> 
> Now this works:
> 
> (displayln (x-a m:v1)) ;; => 'secret
> 
> The problem is that `m:x-a` is treated as a
> `struct-accessor-procedure?`, which is a capability for accessing the
> a field, even though it's a significantly restricted capability.
> 
> There are a couple possible solutions I've thought of:
> 
> 1. Require a non-chaperoned/impersonated accessor.
> 2. Actually use the chaperoned/impersonatored accessor to get the
> value out instead of the underlying accessor.
> 
> 1 is a little less expressive. But note that 2 means that you have to
> only allow chaperoned procedures with `chaperone-struct`, and imposes
> significant complication on the runtime.
> 
> I favor 1.
> 
> Sam
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Racket hash tables vs. Python dicts - Performance

[Matthew Flatt]

Well, I couldn't leave this alone --- especially after I realized that
some of the dispatching overhead has to do with C functions that
cooperate specially with the GC for more complex cases.

I've streamlined various paths and cut about 1/3 of Racket's time on
the example.

At Thu, 24 Jul 2014 11:09:11 +0100, Matthew Flatt wrote:
> In Racket, it looks like a lot of the time is still in generic
> dispatch: directing hash operations to `equal?`-based hashing,
> directing `equal?`-based hashing operation to the string-specific
> operation, directing equality comparison to string-specific equality
> comparison, and so on.
> 
> I'm not sure if that's the full story, but my guess is that such paths
> are more streamlined in Python, given the central role that
> dictionaries play there.
> 
> At Wed, 23 Jul 2014 16:54:47 +0100, Pedro Ramos wrote:
> > Hi,
> > 
> > I've been developing an implementation of Python in Racket, where I'm
> > implementing Python's dictionaries over Racket custom hash tables.
> > 
> > While my occasional benchmarks typically show better performance on Racket
> > programs than their Python equivalents, Racket's hash tables generally seem
> > to be slower than Python's dicts.
> > 
> > I've set up this benchmark in Racket:
> > 
> > 
> > #lang racket
> > 
> > (define alphabet "abcdefghijklmnopqrstuvwxyz")
> > (define (random-word n)
> >   (build-string n (lambda (x) (string-ref alphabet (random 23)
> > 
> > (define words (for/list ([k 100])
> > (random-word 3)))
> > (define d (make-hash))
> > 
> > (time (for ([w words])
> > (if (hash-has-key? d w)
> > (hash-set! d w (add1 (hash-ref d w)))
> > (hash-set! d w 1
> > 
> > 
> > And its equivalent in Python:
> > 
> > 
> > import random
> > import time
> > 
> > alphabet = "abcdefghijklmnopqrstuvwxyz"
> > def random_word(n):
> >   return ''.join([random.choice(alphabet) for i in range(n)])
> > 
> > words = [random_word(3) for k in xrange(100)]
> > d = {}
> > 
> > a = time.time()
> > for w in words:
> >   if w in d:
> > d[w] = d[w] + 1
> >   else:
> > d[w] = 1
> > b = time.time()
> > print b-a, 'seconds'
> > 
> > 
> > The Racket example yields running times of around 500 ms (running on Racket
> > v6.0.1) while the Python example yields running times of around 330 ms
> > (running on Python 2.7.3).
> > 
> > I find this unusual because Python is somewhat more dynamic than Racket,
> > since
> > (a) Python's equality and hashing functions have to dispatched at runtime
> > for each key;
> > (b) referencing and setting values in a Python dict is done using a very
> > general operator, [], whose behaviour also has to be dispatched at runtime,
> > unlike the more specific hash-ref and hash-set! Racket functions.
> > 
> > Is there something I'm missing about Racket's hash tables which explains
> > this slower speed?
> > 
> > Thanks,
> > Pedro Ramos
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Racket hash tables vs. Python dicts - Performance

[Matthew Flatt]

In Racket, it looks like a lot of the time is still in generic
dispatch: directing hash operations to `equal?`-based hashing,
directing `equal?`-based hashing operation to the string-specific
operation, directing equality comparison to string-specific equality
comparison, and so on.

I'm not sure if that's the full story, but my guess is that such paths
are more streamlined in Python, given the central role that
dictionaries play there.

At Wed, 23 Jul 2014 16:54:47 +0100, Pedro Ramos wrote:
> Hi,
> 
> I've been developing an implementation of Python in Racket, where I'm
> implementing Python's dictionaries over Racket custom hash tables.
> 
> While my occasional benchmarks typically show better performance on Racket
> programs than their Python equivalents, Racket's hash tables generally seem
> to be slower than Python's dicts.
> 
> I've set up this benchmark in Racket:
> 
> 
> #lang racket
> 
> (define alphabet "abcdefghijklmnopqrstuvwxyz")
> (define (random-word n)
>   (build-string n (lambda (x) (string-ref alphabet (random 23)
> 
> (define words (for/list ([k 100])
> (random-word 3)))
> (define d (make-hash))
> 
> (time (for ([w words])
> (if (hash-has-key? d w)
> (hash-set! d w (add1 (hash-ref d w)))
> (hash-set! d w 1
> 
> 
> And its equivalent in Python:
> 
> 
> import random
> import time
> 
> alphabet = "abcdefghijklmnopqrstuvwxyz"
> def random_word(n):
>   return ''.join([random.choice(alphabet) for i in range(n)])
> 
> words = [random_word(3) for k in xrange(100)]
> d = {}
> 
> a = time.time()
> for w in words:
>   if w in d:
> d[w] = d[w] + 1
>   else:
> d[w] = 1
> b = time.time()
> print b-a, 'seconds'
> 
> 
> The Racket example yields running times of around 500 ms (running on Racket
> v6.0.1) while the Python example yields running times of around 330 ms
> (running on Python 2.7.3).
> 
> I find this unusual because Python is somewhat more dynamic than Racket,
> since
> (a) Python's equality and hashing functions have to dispatched at runtime
> for each key;
> (b) referencing and setting values in a Python dict is done using a very
> general operator, [], whose behaviour also has to be dispatched at runtime,
> unlike the more specific hash-ref and hash-set! Racket functions.
> 
> Is there something I'm missing about Racket's hash tables which explains
> this slower speed?
> 
> Thanks,
> Pedro Ramos
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Pre-Release Checklist for v6.1

[Matthew Flatt]

At Sun, 20 Jul 2014 22:37:36 -0700, Jon Rafkind wrote:
> 
> On 07/17/2014 05:03 PM, Ryan Culpepper wrote:
> >
> > * Jon Rafkind 
> >   Release tests for (one of the) linux releases:
> >   - Test that the `racket' and `racket-textual' source releases
> > compile fine (note that they're still called `plt' and `mz' at
> > this stage).
> >   - Test that the binary installers for both work, try each one in
> > both normal and unix-style installation modes. (just ubuntu)
> >   [Note: get the release candidates from the URL in this email. Use
> >the 'static table' link to see a list of all tar files available]
> 
> Done, but I saw an error once during a build of 
> racket-minimal-6.0.900.900-src.tgz during 'make install'. I accidentally 
> destroyed the terminal and wasn't able to reproduce the error after two 
> additional tries, so I'm not sure what went wrong but it was something like
> 
> raco: building scribblings/framework.rkt
>   sqlite3: unable to read from database
> 
> I'll try once or twice more to see if it happens again.

A minimal Racket build would not include documentation or "framework".
Was something installed in user scope with a previous v6.0.900.900
installation and maybe not removed before the Racket installation was
itself removed?

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] flatten-begin

[Matthew Flatt]

At Fri, 18 Jul 2014 09:52:26 -0500, Robby Findler wrote:
> Unless someone knows why it is a bad idea, how about adding a #:all?
> argument that flattens all the way down?
> 
> I don't see many uses of flatten-begin in our tree, but the one in
> compatibility/package sure looks like it could use the #:all?
> argument.

I don't think so. Eagerly flattening would break examples like

   (begin
(define begin +)
(begin 1 2)))

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Help with build failed error message

[Matthew Flatt]

"Killed" means that the OS terminated the `racket/racket3m` process
from the outside. For example, the process may have exceeded a
memory-use limit.

At Sat, 19 Jul 2014 20:25:24 -0400, "Alexander D. Knauth" wrote:
> I made a fork of the racket repo and committed some changes in a topic 
> branch, 
> but I got this:
> make[8]: Leaving directory 
> `/home/travis/build/AlexKnauth/racket/racket/src/build'
> make[7]: Leaving directory 
> `/home/travis/build/AlexKnauth/racket/racket/src/build'
> make[6]: Leaving directory 
> `/home/travis/build/AlexKnauth/racket/racket/src/build'
> racket/racket3m -X "/home/travis/build/AlexKnauth/racket/racket/collects" -G 
> "/home/travis/build/AlexKnauth/racket/racket/etc" -G 
> /home/travis/build/AlexKnauth/racket/build/config -N "raco" -l- setup  
> --no-user -j 2  
> raco setup: bootstrapping from source...
> Killed
> make[5]: *** [install-3m] Error 137
> make[5]: Leaving directory 
> `/home/travis/build/AlexKnauth/racket/racket/src/build'
> make[4]: *** [install] Error 2
> make[4]: Leaving directory 
> `/home/travis/build/AlexKnauth/racket/racket/src/build'
> make[3]: *** [base] Error 2
> make[3]: Leaving directory `/home/travis/build/AlexKnauth/racket'
> make[2]: *** [plain-in-place] Error 2
> make[2]: Leaving directory `/home/travis/build/AlexKnauth/racket'
> make[1]: *** [cpus-in-place] Error 2
> make[1]: Leaving directory `/home/travis/build/AlexKnauth/racket'
> make: *** [in-place] Error 2
> The command "make CPUS="2" PKGS="racket-test db-test unstable-flonum-lib 
> net-test"" exited with 2.
> (lines 812-830 of https://travis-ci.org/AlexKnauth/racket/jobs/30368892)
> 
> What does this mean?
> 
> What does the “Killed” mean?
> And what is error 137?
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Pre-Release Checklist for v6.1

[Matthew Flatt]

At Thu, 17 Jul 2014 20:03:12 -0400, Ryan Culpepper wrote:
> * Matthew Flatt 
>- Racket Tests
>- Languages Tests
>- GRacket Tests (Also check that `gracket -z' and `gracket-text' still
>  works in Windows and Mac OS X)
>- mzc --exe tests
>- .plt-packing Tests
>- Games Tests
>- Unit Tests
>- Syntax Color Tests
>- R6RS Tests
>- JPR's test suite
>- Create an executable from a BSL program
>- Run COM tests
>- Embed-in-c test
>- Try compiling with -funsigned-char
>- Try compiling with TEST_ALTERNATE_TARGET_REGISTER

Done.

>Updates:
>- Racket Updates: update HISTORY
>(updates should show v6.1 as the most current version)

Done.

>- Update man pages in racket/man/man1: racket.1, gracket.1, raco.1

No change.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] src-id in identifier-binding for same-module definitions

[Matthew Flatt]

Does `identifier-binding` not give you the symbol that you need?

At Wed, 16 Jul 2014 23:32:46 -0400, Sam Tobin-Hochstadt wrote:
> Ok, I thought I had figured this out, but I was wrong.
> 
> Here's what I want to be able to do:
> 
>  - take an identifier in a fully-expanded source file
>  - translate that identifier to some symbol in a predictable way
>  - so that other references to that same (free-identifier=?)
> identifier get translated to the same symbol
> 
> It's pretty easy to do this in a single module -- just keep a
> free-id-table of all the identifiers mapping to gensyms. But I want to
> be able to do this across modules, and across invocations of this
> program. IOW, when I run my program on one source file, I'd like to
> get a symbol for a provided definition that's the same symbol I get
> when I run my program on a different source file containing a
> reference to that definition.
> 
> Clearly this is possible, since Racket manages, but is there a way
> that I can do it?
> 
> Sam
> 
> On Wed, Jul 16, 2014 at 7:55 AM, Matthew Flatt  wrote:
> > Yes, it can be ".2", etc. The numbers are generated as needed to create
> > distinct names --- deterministically for a given module compilation,
> > assuming that all macros used by expansion are deterministic.
> >
> > At Wed, 16 Jul 2014 07:36:50 -0400, Sam Tobin-Hochstadt wrote:
> >> Does that mean that I can/should just drop the .1 to get the defined name?
> >> Can it also be .2 etc?
> >>
> >> Sam
> >> On Jul 16, 2014 4:34 AM, "Matthew Flatt"  wrote:
> >>
> >> > That `posn1.1` is a unreadable symbol that stands for the symbol
> >> > `posn1` plus some marks that distinguish it.
> >> >
> >> > In other words, `posn1.1` bridges (in an ugly way) the symbol-based
> >> > world of module environments and the identifier-based world of syntax.
> >> > In the future, I hope to shift module environments to be
> >> > identifier-based to avoid these unreadable symbols.
> >> >
> >> > At Tue, 15 Jul 2014 09:10:26 -0400, Sam Tobin-Hochstadt wrote:
> >> > > If you take this program and fully-expand it in the macro stepper:
> >> > >
> >> > > #lang racket
> >> > > (struct posn (x y))
> >> > > (define p1 (posn 1 2))
> >> > >
> >> > > You see that the residual program has an application of the `posn1`
> >> > > function, which is the hidden constructor. And indeed, the
> >> > > fully-expanded program has a definition of `posn1`. However, if you
> >> > > click on the use of `posn1`, the macro stepper will tell you that it's
> >> > > defined in this module as `posn1.1`, and provided as `posn1.1` as
> >> > > well. If you write program to grovel through the fully-expanded
> >> > > syntax, you get these same results as the `src-id` and
> >> > > `nominal-src-id` from `identifier-binding`.
> >> > >
> >> > > Why is this? And is there a way to get from `posn1.1` to `posn1`
> >> > reliably?
> >> > >
> >> > > Sam
> >> >
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] src-id in identifier-binding for same-module definitions

[Matthew Flatt]

Yes, it can be ".2", etc. The numbers are generated as needed to create
distinct names --- deterministically for a given module compilation,
assuming that all macros used by expansion are deterministic.

At Wed, 16 Jul 2014 07:36:50 -0400, Sam Tobin-Hochstadt wrote:
> Does that mean that I can/should just drop the .1 to get the defined name?
> Can it also be .2 etc?
> 
> Sam
> On Jul 16, 2014 4:34 AM, "Matthew Flatt"  wrote:
> 
> > That `posn1.1` is a unreadable symbol that stands for the symbol
> > `posn1` plus some marks that distinguish it.
> >
> > In other words, `posn1.1` bridges (in an ugly way) the symbol-based
> > world of module environments and the identifier-based world of syntax.
> > In the future, I hope to shift module environments to be
> > identifier-based to avoid these unreadable symbols.
> >
> > At Tue, 15 Jul 2014 09:10:26 -0400, Sam Tobin-Hochstadt wrote:
> > > If you take this program and fully-expand it in the macro stepper:
> > >
> > > #lang racket
> > > (struct posn (x y))
> > > (define p1 (posn 1 2))
> > >
> > > You see that the residual program has an application of the `posn1`
> > > function, which is the hidden constructor. And indeed, the
> > > fully-expanded program has a definition of `posn1`. However, if you
> > > click on the use of `posn1`, the macro stepper will tell you that it's
> > > defined in this module as `posn1.1`, and provided as `posn1.1` as
> > > well. If you write program to grovel through the fully-expanded
> > > syntax, you get these same results as the `src-id` and
> > > `nominal-src-id` from `identifier-binding`.
> > >
> > > Why is this? And is there a way to get from `posn1.1` to `posn1`
> > reliably?
> > >
> > > Sam
> >
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] src-id in identifier-binding for same-module definitions

[Matthew Flatt]

That `posn1.1` is a unreadable symbol that stands for the symbol
`posn1` plus some marks that distinguish it.

In other words, `posn1.1` bridges (in an ugly way) the symbol-based
world of module environments and the identifier-based world of syntax.
In the future, I hope to shift module environments to be
identifier-based to avoid these unreadable symbols.

At Tue, 15 Jul 2014 09:10:26 -0400, Sam Tobin-Hochstadt wrote:
> If you take this program and fully-expand it in the macro stepper:
> 
> #lang racket
> (struct posn (x y))
> (define p1 (posn 1 2))
> 
> You see that the residual program has an application of the `posn1`
> function, which is the hidden constructor. And indeed, the
> fully-expanded program has a definition of `posn1`. However, if you
> click on the use of `posn1`, the macro stepper will tell you that it's
> defined in this module as `posn1.1`, and provided as `posn1.1` as
> well. If you write program to grovel through the fully-expanded
> syntax, you get these same results as the `src-id` and
> `nominal-src-id` from `identifier-binding`.
> 
> Why is this? And is there a way to get from `posn1.1` to `posn1` reliably?
> 
> Sam
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] request: replace-evt

[Matthew Flatt]

Added!

At Tue, 8 Jul 2014 05:36:55 +0100, Matthew Flatt wrote:
> Hi Jan,
> 
> That's a nice idea. Something similar --- but in a restricted form ---
> is used internally to implement various primitive events. I think I see
> how to generalize it to work with more arbitrary events and non-atomic
> replace functions.
> 
> There could easily be a catch that I'm overlooking, but I'll give it a
> try.
> 
> Matthew
> 
> At Mon, 07 Jul 2014 22:41:55 +0200, Jan Dvořák wrote:
> > Hi,
> > 
> > I would like to humbly request a new `replace-evt` procedure that would
> > work as `wrap-evt` except that when it produces another `evt?`, `sync`
> > replaces the old event with the newly produced one and restarts.
> > 
> > It will be a good replacement for threads and channels when filtering
> > events or addition of bookkeeping handlers is desired.
> > 
> > I peeked at the syncing code and it seems doable, but the code is too
> > dense for me to grok it. :-(
> > 
> > What do you think?
> > 
> > Best regards,
> > Jan Dvorak
> > 
> > 
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Questions about the GC code

[Matthew Flatt]

The "gc" directory is Boehm's GC. We've modified it a little (grep for
"PLTSCHEME"), but we try not to maintain the GC itself, except to
upgrade every once in a while.

See

  http://www.hboehm.info/gc/

for the latest version, the mailing list, etc.

I should look again at making Racket work with a stock build of Boehm's
GC, so we can get out of the business of upgrading and modifying it. So
far, it has been easier to upgrade occasionally than to sort that out.

Meanwhile, Racket includes a slower and more portable "SGC" for
building the conservative-GC variant of Racket. You can enable it with
`--enable-sgc` as a flag to `configure`. Probably we should switch to
SGC as the default, since RacketCGC is now mainly used just to build
normal Racket.

At Tue, 15 Jul 2014 04:37:45 +0200, Juan Francisco Cantero Hurtado wrote:
> I've been reading the code of the files gc/os_dep.c and 
> gc/include/private/gcconfig.h. I've some questions related to OpenBSD.
> 
> ---
> 
> In the X86_64 config, the file defines STACKBOTTOM and HEURISTIC2 
> (unused because racket picks STACKBOTTOM). What's the preferred?. I'm 
> asking because I don't know if the adding of HEURISTIC2 was an error or 
> really there a good reason to select one or other.
> 
> ---
> 
> There is some recommended benchmark to test the performance of MMAP vs 
> sbrk? I've tried both with generic code and I don't see the difference.
> 
> ---
> 
> OpenBSD only supports MIPS64. Should I add ELFCLASS64 to the MIPS block 
> within gcconfig.h?
> 
> 
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] current packages' docs, errors, and conflicts

[Matthew Flatt]

Yes, `raco setup` with no arguments would succeeded and should fix
things up at this point.

When you use `raco pkg update`, it effectively passes the `--tidy` flag
to `raco setup`. That is, `raco setup --tidy rackjure` would avoid the
problem, and it should also fix things up at this point.

It's tempting to conclude that `--tidy` mode should be the default for
`raco setup`, and that may be right. I'm not yet sure; `raco setup` is
trying to provide a "do only things that I've requested" interface when
collections are specified, and that competes with its "do the right
thing" job.

At Wed, 9 Jul 2014 20:29:52 -0400, Greg Hendershott wrote:
> On Wed, Jul 9, 2014 at 8:21 PM, Greg Hendershott
>  wrote:
> > On the next raco setup I get:
> >
> > raco setup: --- building documentation ---
> > raco setup: WARNING: duplicate tag: (def ((lib "rackjure/alist.rkt") alist))
> > raco setup:  in: 
> > raco setup:  in:
> > /Users/greg/src/scheme/collects/rackjure/rackjure/rackjure.scrbl
> > raco setup: WARNING: duplicate tag: (def ((lib "rackjure/alist.rkt")
> > current-curly-dict))
> > raco setup:  in: 
> > raco setup:  in:
> > /Users/greg/src/scheme/collects/rackjure/rackjure/rackjure.scrbl
> > ... and many more...
> 
> To clarify, that was when I tried `raco setup rackjure`.
> 
> Maybe a full `raco setup` would have succeeded?  But even if so, what
> will `raco pkg update rackjure` do -- the full `raco setup` or just
> the `raco setup rackjure`?
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] racket/fasl allows sandbox escape

[Matthew Flatt]

I've pushed a repair. To double-check it, change 1140 to something like
1340, since the table of primitives changed as part of the repair.

Thanks for the report!

At Wed, 9 Jul 2014 09:39:50 -0400, Sam Tobin-Hochstadt wrote:
> The following exchange with rudybot, which is running the programs in
> a sandbox, demonstrates the issue:
> 
> 09:35  rudybot: eval (let () (local-require compiler/zo-marshal
> compiler/zo-structs racket/fasl) (fasl->s-exp (zo-marshal
> (compilation-top 3 (prefix 0 '() '()) (let-void 1 #t (install-value 1
> 0 #t (primval 1140) (localref #t 0 #f #f #f )))
> 09:35  samth: ; Value: #
> 09:36  rudybot: eval ((let () (local-require
> compiler/zo-marshal compiler/zo-structs racket/fasl) (fasl->s-exp
> (zo-marshal (compilation-top 3 (prefix 0 '() '()) (let-void 1 #t
> (install-value 1 0 #t (primval 1140) (localref #t 0 #f #f #f )))
> vector-ref vector-ref)
> 09:36  samth: ; Value: 32681168
> 
> The relevant program is:
> 
> (let ()
>   (local-require compiler/zo-marshal compiler/zo-structs racket/fasl)
>   (fasl->s-exp (zo-marshal
>   (compilation-top 3
>  (prefix 0 '() '())
>  (let-void 1 #t (install-value 1 0 #t (primval
> 1140) (localref #t 0 #f #f #f )))
> 
> Sam
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] current packages' docs, errors, and conflicts

[Matthew Flatt]

At Tue, 8 Jul 2014 11:49:49 -0400, Sam Tobin-Hochstadt wrote:
> On Tue, Jul 8, 2014 at 11:35 AM, Matthew Flatt  wrote:
> > At Tue, 8 Jul 2014 10:15:10 -0400, Sam Tobin-Hochstadt wrote:
> >>  - I think we need to support planet packages -- there are some people
> >> still releasing new ones, and there are old ones would take
> >> non-trivial work to port.
> >
> > Supporting Planet packages is a lot of work. Overcoming constrained
> > network access in the sandbox is the most obvious problem and probably
> > easy to solve. A more subtle and important piece of the puzzle is the
> > notion of "built" packages, which can be quickly installed for
> > dependent packages or for assembling documentation at the end. Planet
> > packages don't have a "built" concept, and a package that depends on a
> > Planet package won't have the right "built" properties: it will
> > install, but not quickly.
> >
> > I think we're much better off moving Planet packages to supported
> > packages in the new package system --- at least, for use by packages in
> > the new package system.
> 
> i was mostly thinking of handling packages from
> planet-compat.racket-lang.org, which would avoid (some of) the network
> access issues and perhaps also the "built" package issues.
> 
> But maybe the problem then just reappears for the regular Planet
> packages that the planet-compat packages depend on?

I think the planet-compat packages have been converted to depend on
other planet-compat packages. Maybe they'll work out, but it doesn't
seem worth the effort to me. Support for those packages to date has
been good and valuable, but we could reasonably draw the line at this
point.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] current packages' docs, errors, and conflicts

[Matthew Flatt]

At Tue, 8 Jul 2014 10:15:10 -0400, Sam Tobin-Hochstadt wrote:
>  - I wonder if using Docker instead of VirtualBox could make
> incrementality easier, since that's one of things that they focus on.

I don't think it would be easier, but it might be more efficient and
even easier to set up, so it seems worth trying.

If you're interested in trying that, the information below should help
you get oriented.


>  - I wanted to be able to see which of my packages had problems, so I
> wrote this PR: https://github.com/plt/racket/pull/721 but I'm not sure
> how to test it.

I've merged and tweaked that change, but maybe you want to make more
changes... I don't have a good way to try it other than to run a full
build.


I've enclosed a script that will build based on the latest snapshot at
Northwestern (but running the script with the latest from the Racket
repo). The current directory is used as working space. After a first
run, changing `just-summary?` to #t can help you run and fine-tune the
summary part even faster.


To run the script as-is, you'll need a Virtual Box VM named "Ubuntu
Server 14.04" that is accesible from its host at 192.168.56.107 and
that has a snapshot named "build" that is the starting point.

You could use this image:

 build.ova

Boot that image, save its state, and take a snapshot named "build". You
can make things faster by cloning the VM, adjusting each to have its
own static IP, and adding the new VMs in the `#:vms` argument --- and I
bet Docker is better at that kind of thing.


>  - I think we need to support planet packages -- there are some people
> still releasing new ones, and there are old ones would take
> non-trivial work to port.

Supporting Planet packages is a lot of work. Overcoming constrained
network access in the sandbox is the most obvious problem and probably
easy to solve. A more subtle and important piece of the puzzle is the
notion of "built" packages, which can be quickly installed for
dependent packages or for assembling documentation at the end. Planet
packages don't have a "built" concept, and a package that depends on a
Planet package won't have the right "built" properties: it will
install, but not quickly.

I think we're much better off moving Planet packages to supported
packages in the new package system --- at least, for use by packages in
the new package system.


nwu-docs.rkt
Description: Binary data
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] current packages' docs, errors, and conflicts

[Matthew Flatt]

At Tue, 8 Jul 2014 09:08:34 -0400, Sam Tobin-Hochstadt wrote:
> On Tue, Jul 8, 2014 at 8:14 AM, Matthew Flatt  wrote:
> > At Tue, 08 Jul 2014 14:08:27 +0200, Jan Dvořák wrote:
> >> On Tue, 2014-07-08 at 12:46 +0100, Matthew Flatt wrote:
> >> > The rightmost column of the table may need some explanation. The column
> >> > highlights conflicts among names of package-installed executables,
> >> > foreign libraries, and documents. Currently, all the conflicts are
> >> > document names, because several packages have a documents named simply
> >> > "manual" or "main".
> >>
> >> Can you provide some guidelines on docs naming?
> >> I am responsible for half of the conflicts. :-)
> >
> > A package "X" that provides a collection "X" of the same name should
> > probably also call its documentation "X".
> >
> > If the package provides both "guide" and "reference" documentation,
> > then "X-guide" and "X-reference" are good choices.
> >
> >
> > Thanks for looking into this! I've recently updated the Racket
> > documentation, but I expect that more is needed. As far as I can tell,
> > nothing in our documentation previously suggested that documentation
> > names needs to be distinct.
> 
> Is there a reason we can't just make this work with duplicate names
> instead? Perhaps disambiguiating by collection name?
> 
> I think this would change all the current URLs on
> docs.racket-lang.org, so we'd have to fix that, but it seems better to
> make us do a fixed amount of work rather than push this coordination
> problem onto everyone who writes packages in the future.

I think that documentation names could be disambiguated in that way,
but it's a significant chunk of work to switch. To pick one non-obvious
interaction, the way that "local-redirect" plugs in to link documents
would have to be adjusted.

A more compatible and probably easier direction would be a variant of
`scribblings` that tends to distinguish document names and that becomes
the preferred form.

Since it's not a huge burden on package authors or an especially big
obstacle for existing packages, I'd prefer to put this problem aside
for now.


_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] current packages' docs, errors, and conflicts

[Matthew Flatt]

At Tue, 08 Jul 2014 14:08:27 +0200, Jan Dvořák wrote:
> On Tue, 2014-07-08 at 12:46 +0100, Matthew Flatt wrote:
> > The rightmost column of the table may need some explanation. The column
> > highlights conflicts among names of package-installed executables,
> > foreign libraries, and documents. Currently, all the conflicts are
> > document names, because several packages have a documents named simply
> > "manual" or "main".
> 
> Can you provide some guidelines on docs naming?
> I am responsible for half of the conflicts. :-)

A package "X" that provides a collection "X" of the same name should
probably also call its documentation "X".

If the package provides both "guide" and "reference" documentation,
then "X-guide" and "X-reference" are good choices.


Thanks for looking into this! I've recently updated the Racket
documentation, but I expect that more is needed. As far as I can tell,
nothing in our documentation previously suggested that documentation
names needs to be distinct.


_
  Racket Developers list:
  http://lists.racket-lang.org/dev

[racket-dev] current packages' docs, errors, and conflicts

[Matthew Flatt]

I've been working on a service that builds all packages listed at
"pkgs.racket-lang.org". The idea is to run builds regularly (at least
once a day) and link to documentation and build-status information from
"pkgs.racket-lang.org".

Here's a table showing the current results for each package:

  http://www.cs.utah.edu/~mflatt/tmp/pkg-build/

Overall, there's still some work to do before this service can can go
live, but I think it's getting close.

Currently, a package fails to install if it depends on a PLaneT package
--- even a compatibility variant from "planet-compats.racket-lang.org".
That limitation could be removed with more work, but I think it's
probably better to get all packages that we care about onto
"pkgs.racket-lang.org".

The rightmost column of the table may need some explanation. The column
highlights conflicts among names of package-installed executables,
foreign libraries, and documents. Currently, all the conflicts are
document names, because several packages have a documents named simply
"manual" or "main". Overlapping documentation names cause no trouble
among packages in user scope, which is why this problem has gone mostly
unnoticed. Packages in installation scope, however, must have distinct
document names, because installation-scope documentation goes to a
single directory. So, to support installation scope, various packages
need to be updated to give a more specific name to generated
documentation.


The implementation of the build service is `meta/pkg-build/main` in the
Racket repo. The build service relies on VirtualBox MV instances to
sandbox package builds, and the complexity is related to handling
failure and making the build incremental (so that a small amount of
work can be done if a small number of packages need to be rebuilt).
It's still a work in progress, but I'm happy to provide more details if
anyone wants to try it out. A full package build takes about an hour on
my laptop, in addition to the 45 minutes or so to build a snapshot
distribution from scratch.

_
  Racket Developers list:
  http://lists.racket-lang.org/dev