Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-10 Thread Jay McCarthy
FWIW, a zip file may not be a good format because the file index is just a O(n) access list (no matter how many sub-directories the zip file contains.) I didn't find any standard archive formats with O(log n) or better performance, which is why DrDr uses a custom format: http://jeapostrophe.github

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-10 Thread Dan Liebgold
On Wed, Oct 10, 2012 at 7:19 AM, Matthew Flatt wrote: > > People who know about network filesystems tell me that the difference > is in how the filesystems are tuned. They think a single file to hold > bytecode will likely help by trading meta-data operations on many files > for many read operati

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-10 Thread Eli Barzilay
Just now, Matthew Flatt wrote: > > I guess we should try this, but I think we should keep it specific > to bytecode loading. That is, I don't think we should try to splice > a .zip-based filesystem into Racket's core filesystem operations. That's what I had in mind, though splicing a zip at the l

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-10 Thread Sam Tobin-Hochstadt
On Wed, Oct 10, 2012 at 10:19 AM, Matthew Flatt wrote: > > I guess we should try this, but I think we should keep it specific to > bytecode loading. That is, I don't think we should try to splice a > .zip-based filesystem into Racket's core filesystem operations. I would think that something alon

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-10 Thread Matthew Flatt
At Tue, 9 Oct 2012 12:22:59 -0700, Dan Liebgold wrote: > On Tue, Oct 9, 2012 at 11:32 AM, Matthew Flatt wrote: > > > At Tue, 9 Oct 2012 18:00:14 + (UTC), Dan Liebgold wrote: > > It still seems strange to me that we should implement our own > > filesystem of some sort to get decent filesystem

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-10 Thread Eli Barzilay
Yesterday, Matthew Flatt wrote: > At Tue, 9 Oct 2012 18:00:14 + (UTC), Dan Liebgold wrote: > > Now is there a way to archive the bytecode files in general > > (thinking .jar files), and the standard collects in particular? > > Right now our app will stat and load thousands of .zo files during >

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-09 Thread Matthias Felleisen
So are you looking for a tree-shaker in the old Lisp sense that leaves modules alone but eliminates irrelevant ones? Or are you looking for something in the IDE that tells you when you are requiring more files than needed? Always leaving modules alone. -- Matthias On Oct 9, 2012, at 5:37

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-09 Thread Dan Liebgold
This sounds good, but I'm looking for something that operates orthogonally to the rest of the module machinery, so that everything else can be the same and this can just be something like an archiving layer that can be introduced or removed with little effort. It's a production requirement driving

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-09 Thread Jay McCarthy
One option may be to use the new "demodularizer" that my student Blake Johnson is working on. Basically, you give it the root of your application (main.rkt) and it creates a new zo (compiled/main_rkt.zo) that has no dependencies but "copies" [*] all the code in that would otherwise be required onc

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-09 Thread Dan Liebgold
On Tue, Oct 9, 2012 at 11:32 AM, Matthew Flatt wrote: > At Tue, 9 Oct 2012 18:00:14 + (UTC), Dan Liebgold wrote: > It still seems strange to me that we should implement our own > filesystem of some sort to get decent filesystem performance. > > Indeed. I think in game development we tend to b

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-09 Thread Matthew Flatt
At Tue, 9 Oct 2012 18:00:14 + (UTC), Dan Liebgold wrote: > Now is there a way to archive the bytecode files in general (thinking .jar > files), and the standard collects in particular? Right now our app will stat > and load thousands of .zo files during it's normal operation. In performance

Re: [racket-dev] PLTCOMPILEDROOTS

2012-10-09 Thread Dan Liebgold
Matthew Flatt writes: > > In v5.3.0.24, the new `current-compiled-file-roots' parameter and > associated PLTCOMPILEDROOTS environment variable give you more control > over where bytecode files are written. > (replying over gmane because I joined the mailing list later) Excellent. Thanks for t

[racket-dev] PLTCOMPILEDROOTS

2012-09-11 Thread Matthew Flatt
In v5.3.0.24, the new `current-compiled-file-roots' parameter and associated PLTCOMPILEDROOTS environment variable give you more control over where bytecode files are written. The new parameter holds a list of paths. When looking for a compiled file, the default `load/use-compiled' handler checks