Re: [racket-dev] dependencies for `racket-test` package
I'm pretty sure that racket-test needs to be split up. (For the initial cut, I just put the whole tests collection there.) I'm not sure about compiling tests for the purposes of dependency checking. It sounds ok, but I have a feeling that I'm forgetting some reason that we disabled compilation for tests. At Tue, 2 Jul 2013 12:20:57 -0400, Sam Tobin-Hochstadt wrote: Currently, the `racket-test` package has very few declared dependencies, but this is wrong. The checking in 'raco setup' doesn't catch this because the files aren't compiled, but at a minimum it depends on `unstable/debug` (currently a hidden part of `typed-racket-lib`) and some portions of htdp that aren't declared. This brings up (at least) 3 questions - What is it ok for `racket-test` to depend on? I'd like this set to be small, so we can test the core without needing to build everything. - Should we compile the contents of `racket-test` so that we get dependency checking? - Should parts of `racket-test` be split up? I'd like to figure this out, but I don't have answers yet. Sam _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies for `racket-test` package
I believe the reason we disabled compilation was that tests weren't constrained to depend only on the core. Some tests may use planet packages, etc. Originally, this was mostly for rackunit, which was a planet package. This suggests to me that we should (a) remove the test packages from the default distribution, and (b) re-enable compilation for them. That way the dependencies will be computed properly. It also suggests that if they aren't already included, perhaps we should add planet packages to the list of things we consider among dependencies? Carl Eastlund On Tue, Jul 2, 2013 at 12:42 PM, Matthew Flatt mfl...@cs.utah.edu wrote: I'm pretty sure that racket-test needs to be split up. (For the initial cut, I just put the whole tests collection there.) I'm not sure about compiling tests for the purposes of dependency checking. It sounds ok, but I have a feeling that I'm forgetting some reason that we disabled compilation for tests. At Tue, 2 Jul 2013 12:20:57 -0400, Sam Tobin-Hochstadt wrote: Currently, the `racket-test` package has very few declared dependencies, but this is wrong. The checking in 'raco setup' doesn't catch this because the files aren't compiled, but at a minimum it depends on `unstable/debug` (currently a hidden part of `typed-racket-lib`) and some portions of htdp that aren't declared. This brings up (at least) 3 questions - What is it ok for `racket-test` to depend on? I'd like this set to be small, so we can test the core without needing to build everything. - Should we compile the contents of `racket-test` so that we get dependency checking? - Should parts of `racket-test` be split up? I'd like to figure this out, but I don't have answers yet. Sam _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies for `racket-test` package
On Jul 2, 2013 12:42 PM, Matthew Flatt mfl...@cs.utah.edu wrote: I'm pretty sure that racket-test needs to be split up. (For the initial cut, I just put the whole tests collection there.) Sorry, I should have been more specific; 'tests/racket/stx.rktl' depends on htdp. I expect that the tests in that directory depend on other packages as well, such as sandbox and compiler. I'm not sure about compiling tests for the purposes of dependency checking. It sounds ok, but I have a feeling that I'm forgetting some reason that we disabled compilation for tests. At Tue, 2 Jul 2013 12:20:57 -0400, Sam Tobin-Hochstadt wrote: Currently, the `racket-test` package has very few declared dependencies, but this is wrong. The checking in 'raco setup' doesn't catch this because the files aren't compiled, but at a minimum it depends on `unstable/debug` (currently a hidden part of `typed-racket-lib`) and some portions of htdp that aren't declared. This brings up (at least) 3 questions - What is it ok for `racket-test` to depend on? I'd like this set to be small, so we can test the core without needing to build everything. - Should we compile the contents of `racket-test` so that we get dependency checking? - Should parts of `racket-test` be split up? I'd like to figure this out, but I don't have answers yet. Sam _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies for `racket-test` package
At Tue, 2 Jul 2013 12:53:02 -0400, Sam Tobin-Hochstadt wrote: On Jul 2, 2013 12:42 PM, Matthew Flatt mfl...@cs.utah.edu wrote: I'm pretty sure that racket-test needs to be split up. (For the initial cut, I just put the whole tests collection there.) Sorry, I should have been more specific; 'tests/racket/stx.rktl' depends on htdp. I expect that the tests in that directory depend on other packages as well, such as sandbox and compiler. Depending on sandbox and compiler seems ok, while the tests that depend on htdp can be changed to use different modules. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies for `racket-test` package
On Tue, Jul 2, 2013 at 12:01 PM, Matthew Flatt mfl...@cs.utah.edu wrote: At Tue, 2 Jul 2013 12:53:02 -0400, Sam Tobin-Hochstadt wrote: On Jul 2, 2013 12:42 PM, Matthew Flatt mfl...@cs.utah.edu wrote: I'm pretty sure that racket-test needs to be split up. (For the initial cut, I just put the whole tests collection there.) Sorry, I should have been more specific; 'tests/racket/stx.rktl' depends on htdp. I expect that the tests in that directory depend on other packages as well, such as sandbox and compiler. Depending on sandbox and compiler seems ok, while the tests that depend on htdp can be changed to use different modules. Or maybe, if they are actually testing htdp, move into the htdp pkg. But another issue we're forgetting here is that many of the tests are not in #lang but are load-based. Robby _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies for `racket-test` package
On Tue, Jul 2, 2013 at 6:15 PM, Robby Findler ro...@eecs.northwestern.edu wrote: On Tue, Jul 2, 2013 at 12:01 PM, Matthew Flatt mfl...@cs.utah.edu wrote: At Tue, 2 Jul 2013 12:53:02 -0400, Sam Tobin-Hochstadt wrote: On Jul 2, 2013 12:42 PM, Matthew Flatt mfl...@cs.utah.edu wrote: I'm pretty sure that racket-test needs to be split up. (For the initial cut, I just put the whole tests collection there.) Sorry, I should have been more specific; 'tests/racket/stx.rktl' depends on htdp. I expect that the tests in that directory depend on other packages as well, such as sandbox and compiler. Depending on sandbox and compiler seems ok, while the tests that depend on htdp can be changed to use different modules. Or maybe, if they are actually testing htdp, move into the htdp pkg. The tests I was looking were testing the syntax system, so I've modified them. But another issue we're forgetting here is that many of the tests are not in #lang but are load-based. Right, those tests can't be compiled. There are lots of tests in `racket-test` (for, eg, json and pkg and xml) that are thankfully not load-based, so we should decide what to do with them. Sam _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies and racket
Another variation, if you're thinking about cloud infrastructure today: you could pretty easily make your own faux PLaneT server that either is for a single app or takes the identity/profile of the app as part of the URL the app uses to access the PLaneT server. The faux server can be a tiny HTTP server process, as it only has to perform a single, simple REST operation. This faux PLaneT server would *enforce* the same profile of PLaneT package versions for the app, and also serve the packages from a trusted cache at app compile time. Or you can use the careful-crafted ~/.racket/ directory approach. I suppose the choice depends on which is easier to do well in your existing infrastructure. It's good to see interest in Racket from cloud people. Feel free to ask lots of questions on the Racket users and dev email lists as you work through your solution. -- http://www.neilvandyke.org/ _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies and racket
On Fri, Dec 30, 2011 at 9:45 AM, Neil Van Dyke n...@neilvandyke.org wrote: It's good to see interest in Racket from cloud people. Feel free to ask lots of questions on the Racket users and dev email lists as you work through your solution. Well, this a side-interest for personally, but *anyone* can make a 'build pack' for Heroku, and they can be simple. Take the Python build pack for instance, which is little more than three bash files in the 'bin' directory: https://github.com/heroku/heroku-buildpack-python And one for Go: https://github.com/kr/heroku-buildpack-go The Go one, in particular, is not baked into the platform (one must set BUILDPACK_URL to use it, and it downloads its own preferred version of the compiler) and yet does what it needs to do in about 50 lines of not-very-dense bash -- downloads and installs the compiler and dependencies , and that's it. There is nothing particularly 'cloudy' about these, and the most simple Racket buildpack would do little more than download Racket, or maybe run raco distribute. I came to this list because it appears to me that such a distributed binary would download dependencies on the fly per invocation of the Racket program, and that would be a lousy property for the software being written, and could also have a negative impact on PlaneT as an infrastructure (it also might just not work, depending if writes were attempted to a read-only part of the file system). I also wanted to seek out idioms already in the wild for handling this problem. I do not see this as a cloud problem, but rather a tooling or communication gap in packaging and distributing Racket applications that make use of anything except libraries included in the Racket distribution. -- fdr _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies and racket
On Thu, Dec 29, 2011 at 8:07 PM, Robby Findler ro...@eecs.northwestern.edu wrote: I think that a minor variation on Neil's strategy should not be too difficult to do (but I'll certainly agree that it is not an ideal situation). What you'd do is first just download (via the Bam method below :) the planet packages you want. Then, if you look inside this directory: (build-path (find-system-path 'addon-dir) planet 300) you should find some files with extensions .plt. Re-build that same portion of the directory structure to contain those files in the deployment environment, and then set the PLTPLANETURL environment variable to something bogus. That looks like a pretty reasonable way to get started. Thanks for telling me about the build-path and find-system-path operators, and the bogusifying of the PLT URL to prevent accidental hammering. Also, FWIW, I maintain, in the weakest sense of the word, the current planet service. (All I really do is react to emergencies that break things.) Jay and Eli are looking into a whole new thing and have been talking about it for more than a year, I believe, so hopefully plans are firming up. Their system should be able to support this kind of use-case much more effectively. I think that kind of thing is designed in for them. I'd be interested in reading their current thoughts. Having used now a small stable of dependency management systems, my favorite, by far, is bundler for Ruby. It can handle upgrades, version pinning, allowing for auditing in upgrades of packages, preventing an executed program from using libraries that are not in its manifest (to prevent 'accidental' dependency), bin-directory generation for packages that install command line programs, and it will transitively consider all version requirements of all libraries to try and select the latest version -- should any one version exist -- that can satisfy all the constraints. Above all, it does that without being burdensome to use or understand. One could do worse than implementing the same thing, but for Racket. -- fdr _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] dependencies and racket
Daniel Farina wrote at 12/29/2011 07:59 PM: The goal is that a program written, say, three years ago should be able to run the same way it did when it was written, so it's really useful to freeze all the dependencies into the file system somehow and preserve it. Someone else can comment on any automated ways of ``compiling in'' PLaneT packages, and solutions like that. A so-so solution I *don't* recommend... If you preserved the exact PLaneT linkage table, and were very careful, then I think you could preserve the exact versions of PLaneT modules used (required directly, and indirectly). This assumes either: (1) integrity of the PLaneT server, and trust in your connection to it; or (2) preservation of the pertinent parts of ~/.racket. Then you'd want to disable attempts to access the PLaneT server. Perhaps, if you were to automate this, it could be done safely and reliably. Something that has been done for a large Racket-based Web app on production servers is to manually determine all the PLaneT packages that are used, downloaded and extract these packages (in a directory scheme that preserves naming and version info), check it all into their configuration management system, change all the PLaneT module references to filesystem ones, check it into CM again, and audit. PLaneT server access is always blocked, except for the manual download. You can make simple tools to help automate the more mechanical parts of this (I don't know of any such tools publicly available at this time), but the audit is mostly human expertise-intensive. This cloning of PLaneT packages is not something that most people need to do, and it totally spoils the buzz of ``I'll just add a (require (planet ...)) and bam!,'' but it's not too hard to do if you choose to. PLaneT isn't my baby (I think Jay and Eli are leading it right now), but I'm looking into funding so that I can put solid chunks of time into helping with ways to improve PLaneT security. Security is usually a constant consideration in my Internet consulting work. Part of this tentative work will address your concerns about stability, and (in my current thinking) about having the option of removing PLaneT server dependencies for an app. Much of this work has implications for some other network software distribution methods, so PLaneT is both a research testbed and an early practical beneficiary (much like Racket in general). Stay tuned (I have a lot more experience in solving problems than in writing research grant proposals). -- http://www.neilvandyke.org/ _ Racket Developers list: http://lists.racket-lang.org/dev
