Re: [racket-dev] racket virtual machine out of memory error
On May 16, 2011, at 11:25 AM, Matthew Flatt wrote: > At Mon, 16 May 2011 14:19:05 -0400, Stephen Chang wrote: >>> This question is somewhat vague but I've been able to replicate it >>> pretty reliably so I'll ask anyways. I'm running redex-check for a >>> long time and after about 6-8 hours, drracket crashes with a "racket >>> out of virtual memory" error. This is on Windows 7. The machine has >>> lots of physical ram but no swap memory. Would this cause the crash? >>> As another data point, I successfully ran the same redex-check'ing on >>> a linux machine with less physical memory for several days. I've >>> attached the relevant events from the windows log. >> >> Sorry, I meant to say, the error is "Racket virtual machine has run >> out of memory", as in the crash log. > > Is this a 32-bit or 64-bit build? I ask because a 32-bit Racket build > under Windows is restricted to 2 GB of virtual address space. I've just seen this error when running a racket web server on a VPS running linux--not Windows---with 4GB of ram. Is this likely the same error? In this case, the process itself shouldn't be using more than 100MB or so of live data, but perhaps it just never collects and constantly increases the virtual memory address space? Forgive my ignorance, John Clements smime.p7s Description: S/MIME cryptographic signature _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
Yes, just changing the default. "raco test -r" will behave like "racket -t" unless there is a test submodule, in which case, it will actually run the tests. Jay On Fri, Mar 9, 2012 at 10:04 PM, Sam Tobin-Hochstadt wrote: > On Fri, Mar 9, 2012 at 9:54 PM, Jay McCarthy wrote: >> >> I intend to change DrDr so that it always uses "raco test -r" rather >> than "racket -t". > > I assume you mean just changing the defaults. Will this resulting in > running any less code? > > -- > sam th > sa...@ccs.neu.edu -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
On Fri, Mar 9, 2012 at 9:54 PM, Jay McCarthy wrote: > > I intend to change DrDr so that it always uses "raco test -r" rather > than "racket -t". I assume you mean just changing the defaults. Will this resulting in running any less code? -- sam th sa...@ccs.neu.edu _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
I've renamed module* to slice and removed its ability to specify a language other than #f (maybe it could be a #:keyword later) I've removed when-testing because (slice test ...) is shorter than (begin-for-testing test ...) [which, btw Carl, is the best name.] I've expanded 'raco test' to support running a different submodule or running the default module (what's a good name for it?) if the submodule isn't there. I intend to change DrDr so that it always uses "raco test -r" rather than "racket -t". Jay On Fri, Mar 9, 2012 at 5:58 PM, Eli Barzilay wrote: > 30 minutes ago, Matthew Flatt wrote: >> >> How about `facet', with the terminology that "facets" are >> implemented as "submodules"? > > I like using a different term with just that explanation (ie, "a facet > is a particular kind of a submodule"). > > But I still don't like `facet' -- not only because of my non-native > point, but also because it sounds to me like facets are distinct. (So > it's weird that one facet can require another.) > > How about just `slice'? It sounds good on two levels: you split a > module into several slices, and also you split each of this by small > slices of code. You'd read code as > > (slice #|of|# main > ...) > > -- > ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: > http://barzilay.org/ Maze is Life! -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
30 minutes ago, Matthew Flatt wrote: > > How about `facet', with the terminology that "facets" are > implemented as "submodules"? I like using a different term with just that explanation (ie, "a facet is a particular kind of a submodule"). But I still don't like `facet' -- not only because of my non-native point, but also because it sounds to me like facets are distinct. (So it's weird that one facet can require another.) How about just `slice'? It sounds good on two levels: you split a module into several slices, and also you split each of this by small slices of code. You'd read code as (slice #|of|# main ...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
I'll reply to the rest later, but two quick ones: Just now, Matthew Flatt wrote: > > How about just (submodule foo ...) be a more memorable syntax for > > (module* foo #f ...)? > > That was Jon's suggestion, and my objection is that "submodule" > means something more general than those nested modules that are > declared with `submodule'. I just realized that the main use of this (the specific combination of `module*' with #f) is for sectioning the file -- so how about going in the direction of `subsection', or to avoid the obvious problem with scribble: `subpart'? > > (BTW, there's an obvious question here of why not do that for all > > paths, so that `foo/bar/baz' can access a `bar/baz' submodule in > > `foo' or a `baz' in `foo/bar'... > > Search paths cause lots of trouble and should be avoided when > possible. This particular two-step search seem to be just barely > tolerable for `#lang', and I still worry about it; I wouldn't > suggest it if I saw a better way to accommodate existing code. I completely agree -- specifically, the "mess" that I referred to at the end of that comment was due to how things ended: I inevitably would resort to grepping text to find where some change needs to be made, I created code mostly via copy-paste, and macros would get very confusing to deal with. It was a kind of an experiment that demonstrated why this particular generalization looks really cute on paper, but in practice it is a total failure. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
At Fri, 9 Mar 2012 15:58:11 -0700, Jay McCarthy wrote: > I just pushed... > > - module** > > Like module* but combines multiple occurrences of the same submodule > name into one module* I like this direction --- and like everyone, I wish for a better name. I don't like `submodule' or `sub' for this, because it's a particular way of creating submodules, not a general form for submodules. How about `facet', with the terminology that "facets" are implemented as "submodules"? For example, a module with a `main' facet could be #lang racket/base (facet main (require racket/cmdline) (command-line .)) but you could also use `(facet main )' multiple times to add parts to the `main' submodule. > - when-testing > > An abbreviation of module** with the name test and the #f language I agree with Eli that if we have a good replacement for `submodule**', then we may not need this one. For example, I like (facet test ) better than (when-testing ) > - raco test > > Finds all the files in a directory and requires their test module Good! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
At Fri, 9 Mar 2012 16:35:25 -0500, Eli Barzilay wrote: > Two days ago, Matthew Flatt wrote: > > > > Given the term "submodule", the first thing that you're likely to try > > will work as expected: > > > > #lang racket/base > > > > (module zoo racket/base > > (provide tiger) > > (define tiger "Tony")) > > > > (require 'zoo) > > > > tiger > > > > Within `module', a module path of the form `(quote id)' refers to > > the submodule `id', if any. If there's no such submodule, then > > `(quote id)' refers to an interactively declared module, as before. > > This is *really* nice in how it's building a scope for module names, > but OTOH, it makes the quote seem more like a syntactic hack... Agreed. > > [...later...] > > > > (require (submod 'zoo monkey-house)) > > > > [...] a shorthand for `(submod "." zoo)' [...] > > ...especially here -- why is there a quote only on the first and not > on the second? That jumped out at me, too. But the first is a module path and the second is a submodule name. For example, (submod zoo monkey-house) would mean the `monkey-house' submodule of "main.rkt" from the "zoo" collection, not a `monkey-house' subsubmodule in a `zoo' submodule. > Is there any meaning for (submod foo) without the > quote (or a sting)? We could allow (submod foo) as equivalent to foo that is, as a reference to `foo' with an empty submodule path. > Perhaps a more obvious question is why aren't all `submod' uses get > "." as an implicit first expression? I don't see how lets you reach a submodule from the top level or from within a different top-level module. > > The 'zoo module path above is really a shorthand for `(submod "." > > zoo)', where "." means the enclosing module and `zoo' is its > > submodule. You could write `(submod "." zoo monkey-house)' in place > > of `(submod 'zoo monkey-house)'. > > I also love the analogy to paths -- but using strings for them doesn't > look so nice, since strings are begging to be combined... (I can > already see myself wishing for "./../foo".) Other than the obvious > reader issue, would there be a problem in using `|.|' and `..' for > these (and making them into special module names)? Submodule identifiers are not currently constrained, and I think we should avoid constraining them. If we refrain from constraining submodule names, then `..' could be the name of a submodule. > How about just (submodule foo ...) be a more > memorable syntax for (module* foo #f ...)? That was Jon's suggestion, and my objection is that "submodule" means something more general than those nested modules that are declared with `submodule'. > taking what Jay did (I think, didn't look at the code or the > followups, yet), and making this: > > (submodule E ...) > > be something that can appear anywhere in the code, expands to what > (module* main #f E ...) is doing now, *and* is allowed to be used more > than once, with all uses being collected into a single submodule. Yes, I think this is a promising direction, and we just need to find a better name than `module**'. I'll leave that discussion to the other subthread. > > The `#lang' reader form was previously defined as a shorthand for > > `#reader' where the name after the `#lang' is mangled by adding > > "/lang/reader". With submodules, `#lang' first tries using the name > > as-is and checking for a `reader' submodule; if it is found, then > > the submodule is used instead of mangling the name with > > "/lang/reader", otherwise it falls back to the old behavior. > > (BTW, there's an obvious question here of why not do that for all > paths, so that `foo/bar/baz' can access a `bar/baz' submodule in `foo' > or a `baz' in `foo/bar'... Search paths cause lots of trouble and should be avoided when possible. This particular two-step search seem to be just barely tolerable for `#lang', and I still worry about it; I wouldn't suggest it if I saw a better way to accommodate existing code. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
Some more comments: * There's probably a better way to collect the expression lists than set!-ing it. * `sub' is probably a bad name, though `submodule' looks too long for this. [ * I should have said earlier that if there's a `when-testing' then the bad result is adding a `when-in-main' -- even when it begs an example of: #lang racket (when-in-main (displayln "Go see Matthias.")) ] -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
A few minutes ago, Eli Barzilay wrote: > [...] Here's what I have in mind: (Other things were broken there...) Here's the code that works now: (sub tests (require tests/eli-tester)) ; make it known before main (sub main (printf "Welcome to MY library!\n")) (define (plus x y) (+ x y)) (sub tests (test (plus 1 2) => 3)) (sub main (require (submod "." ".." tests)) (printf "Goodbye.\n")) There is the obvious output order here where there main printouts come after the test message because `tests' is instantiated before the printout forms in `main'. If the test is changed to a failure, it throws an error and you won't see that printout. Complete code below, with the same example done with rackunit. --- #lang racket/base (require (for-syntax racket/base)) (define-for-syntax subs '()) (define-syntax (sub stx) (syntax-case stx () [(_ name E ...) (identifier? #'name) (let ([sub (assq (syntax-e #'name) subs)]) (if sub (begin ((cdr sub) #'(begin E ...)) #'(begin)) (with-syntax ([(submod) (generate-temporaries #'(name))]) (let ([bodies (list (syntax-local-introduce #'(begin E ...)))]) (set! subs (cons (cons (syntax-e #'name) (case-lambda [(E) (set! bodies (cons E bodies))] [() (reverse bodies)])) subs))) (syntax-local-lift-module-end-declaration #'(submod)) #'(define-syntax (submod stx) (with-syntax ([(E* (... ...)) ((cdr (assq 'name subs)))]) (syntax-local-introduce #'(module* name #f E* (... ...])) (sub tests (require rackunit)) ; make it known before main (sub main (printf "Welcome to MY library!\n")) (define (plus x y) (+ x y)) (sub tests (check-equal? (plus 1 2) 3) (check-equal? (plus 1 2) 33)) (sub main (require (submod "." ".." tests)) (printf "Goodbye.\n")) --- -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
Just now, Jay McCarthy wrote: > I just pushed... > > - module** > > Like module* but combines multiple occurrences of the same submodule > name into one module* But... it adds more stars... I don't think that there's any need for a convenience macro that does just the combining, since that feature is for macro writers anyway. Here's what I have in mind: (sub main (printf "Welcome to MY library!\n")) (sub tests (require tests/eli-tester) (printf "Testing...")) (define (plus x y) (+ x y)) (sub tests (printf ">>> ~s\n" shuffle)) (sub main (printf "Running tests...\n") (require 'tests) (printf "Goodbye.\n")) I have it almost working -- I just have one problem left (that last nested require doesn't work). > - when-testing > > An abbreviation of module** with the name test and the #f language (... As I wrote the above, I realized that `tests' is a better name for this.) In any case, I think that the above is lightweight enough that there is no need for any `when-something'. (I dislike specific names because there's more than just testing.) > - raco test > > Finds all the files in a directory and requires their test module (And it works for the specified path(s) if given?) How about generalizing this too? Something like raco run tests This way, you get the others for free since `tests' is not hard wired: raco run extensive-tests raco run docs ... -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
On Fri, Mar 9, 2012 at 5:58 PM, Jay McCarthy wrote: > I just pushed... > > - module** > > Like module* but combines multiple occurrences of the same submodule > name into one module* > > - when-testing > > An abbreviation of module** with the name test and the #f language > > - raco test > > Finds all the files in a directory and requires their test module > > I like the name 'raco test'; I'm meh on 'when-testing', I didn't want > to clash with eli-tester/test or something, but I think whatever we go > with (and I think there should be such a convenience macro) should be > similar to what happens with main; I don't know a better name for > module**. I thought of something using an @ to suggest the splicing, > but didn't like anything when I tried it. > How about "begin-for-testing"? And "extend-submodule"? --Carl _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
I just pushed... - module** Like module* but combines multiple occurrences of the same submodule name into one module* - when-testing An abbreviation of module** with the name test and the #f language - raco test Finds all the files in a directory and requires their test module I like the name 'raco test'; I'm meh on 'when-testing', I didn't want to clash with eli-tester/test or something, but I think whatever we go with (and I think there should be such a convenience macro) should be similar to what happens with main; I don't know a better name for module**. I thought of something using an @ to suggest the splicing, but didn't like anything when I tried it. Jay On Fri, Mar 9, 2012 at 11:56 AM, Jay McCarthy wrote: > I'll pushed an improved version of the test macro shortly. > > Jay > > On Fri, Mar 9, 2012 at 11:04 AM, Matthew Flatt wrote: >> At Wed, 7 Mar 2012 10:14:35 -0700, Matthew Flatt wrote: >>> I've added "submodules" to a version of Racket labeled v5.2.900.1 >> >> Submodules are now pushed to the Racket git repo. >> >> >> I haven't yet added a syntactic form to simplify >> >> (module* main #f >> ) >> >> My first idea was `main', as in >> >> (main >> ) >> >> but that seems too quiet and likely to create collisions. (It does >> create a collision in part of the contract library, which imports >> `racket/base' for syntax and defines a for-syntax `main' function.) >> >> Jon suggests `submodule': >> >> (submodule main >> ...) >> >> This suggestion has the advantage of replacing all `(module* #f >> )' combinations, and mostly I like this direction. However, >> `submodule' would be only one way to define a "submodule", while >> `submod' in a module path corresponds to "submodule" in the more >> general sense. It could be that `submodule' is on the right path and >> other terminology should change, or maybe there's a more specific word >> to use for a `(module* #f )' replacement instead of `submodule'. >> >> _ >> Racket Developers list: >> http://lists.racket-lang.org/dev > > > > -- > Jay McCarthy > Assistant Professor / Brigham Young University > http://faculty.cs.byu.edu/~jay > > "The glory of God is Intelligence" - D&C 93 -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Racket web server doesn't handle POST requests with 'Transfer-Encoding: chunked' ?
On Mar 9, 2012, at 1:29 PM, Jay McCarthy wrote: > Yes. I should definitely fix that, whether or not I implemented > chunked transfers. FWIW, chunked transfers *are* implemented on the receiving side; there are tests in net/url/tests (sp?) that explicitly check the ability to reassemble chunked data when it comes back as a response, so this may be as simple as making the right call in the right place. John > > Jay > > On Fri, Mar 9, 2012 at 2:18 PM, John Clements > wrote: >> >> On Mar 9, 2012, at 12:18 PM, Jay McCarthy wrote: >> >>> There is nothing going wrong with the supported features of the Web >>> server. It's just that the Web server doesn't support any >>> Transfer-Encoding other than the default of no encoding. >> >> From RFC2616: >> >> "A server which receives an entity-body with a transfer-coding it does not >> understand SHOULD return 501 (Unimplemented), and close the connection. A >> server MUST NOT send transfer-codings to an HTTP/1.0 client. " >> >> Am I right in thinking that we're falling afoul of this "should", making us >> a "conditionally compliant" implementation of HTTP/1.1? >> >> John >> > > > > -- > Jay McCarthy > Assistant Professor / Brigham Young University > http://faculty.cs.byu.edu/~jay > > "The glory of God is Intelligence" - D&C 93 smime.p7s Description: S/MIME cryptographic signature _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
[Some of these points may have come up earlier, I just didn't want to lose comments so reply as I read it.] [To be clear in advance, the following should be qualified by an "I love it".] Two days ago, Matthew Flatt wrote: > > Given the term "submodule", the first thing that you're likely to try > will work as expected: > > #lang racket/base > > (module zoo racket/base > (provide tiger) > (define tiger "Tony")) > > (require 'zoo) > > tiger > > Within `module', a module path of the form `(quote id)' refers to > the submodule `id', if any. If there's no such submodule, then > `(quote id)' refers to an interactively declared module, as before. This is *really* nice in how it's building a scope for module names, but OTOH, it makes the quote seem more like a syntactic hack... > [...later...] > > (require (submod 'zoo monkey-house)) > > [...] a shorthand for `(submod "." zoo)' [...] ...especially here -- why is there a quote only on the first and not on the second? Is there any meaning for (submod foo) without the quote (or a sting)? If not, then having (submod zoo monkey-house) seems beter in building up an intuition of `submod' implicitly "quoting" the following name. Perhaps a more obvious question is why aren't all `submod' uses get "." as an implicit first expression? IIUC, this means that it works nicely without quotes, and there's only a single magical ".." thing. > The 'zoo module path above is really a shorthand for `(submod "." > zoo)', where "." means the enclosing module and `zoo' is its > submodule. You could write `(submod "." zoo monkey-house)' in place > of `(submod 'zoo monkey-house)'. I also love the analogy to paths -- but using strings for them doesn't look so nice, since strings are begging to be combined... (I can already see myself wishing for "./../foo".) Other than the obvious reader issue, would there be a problem in using `|.|' and `..' for these (and making them into special module names)? Or if there was always an implicit first ".", then there's only need for a single `..' magic token thing... (It even makes syntactic sense that (submod) is something that you can't require.) Reading the later "Design Issues" section, I see that these are indeed arbitrary. So to clarify, I propose a `submod' form that always begins with a "." (under its current semantics), with no need to use a quote, and with `..' being a special name that goes up the hierarchy. To make things connect nicely to toplevel requires, (submod foo) could also resort to the repl-entered module thing, which means that we now have: (require (submod foo)) is very common, so it can be written a bit more conveniently as (require 'foo) (But this might be pushing the description into showing them as closer than they are.) > Instead of `require'ing its enclosing module, a `module*' form can > use `#f' as its language, in which case its lexical context starts > with all of the bindings of the enclosing module (implicitly > imported) instead of with an empty lexical context. As a result, the > submodule can access bindings of the enclosing module that are not > exported: > > #lang racket/base > > (module aquarium racket/base > (define fish '(1 2)) > > (module* book #f > (append fish '(red blue > > (require (submod 'aquarium book)) The explanation for this looks reasonable, but the result looks unfortunate since the common case requires you to remember to add two special things (the `*' and `#f'). But I also really don't like to abuse `module' for both uses. How about just (submodule foo ...) be a more memorable syntax for (module* foo #f ...)? Later, I see: > As things stand, the ugly pattern `(module* main #f ...)' would be > common. Probably we should have a macro that expands to `(module* main > #f ...)'. Should the macro be called `main'? So above is my suggestion. But maybe even make it more generally convenient so it can cover the `test' use without needing a library -- taking what Jay did (I think, didn't look at the code or the followups, yet), and making this: (submodule E ...) be something that can appear anywhere in the code, expands to what (module* main #f E ...) is doing now, *and* is allowed to be used more than once, with all uses being collected into a single submodule. This way I can easily interleave regular code, main code, test code, sanity assertion code, documentation code etc. IOW, I see this as something that can be used straightforwardly with any testing tool, or made into a macro by one, or being used in the obvious way by in-line documentation tools like Neil V's mcfly thing (which is more straightforward for this than srcdoc). > A common use of `module*' is likely to be with `main', since > `racket' will load a `main' submodule (after `require'ing its > enclosing module) for a module named on its command line. For > example, if you run this program via `racket': > > #lang racket/base > >
Re: [racket-dev] Racket web server doesn't handle POST requests with 'Transfer-Encoding: chunked' ?
Yes. I should definitely fix that, whether or not I implemented chunked transfers. Jay On Fri, Mar 9, 2012 at 2:18 PM, John Clements wrote: > > On Mar 9, 2012, at 12:18 PM, Jay McCarthy wrote: > >> There is nothing going wrong with the supported features of the Web >> server. It's just that the Web server doesn't support any >> Transfer-Encoding other than the default of no encoding. > > From RFC2616: > > "A server which receives an entity-body with a transfer-coding it does not > understand SHOULD return 501 (Unimplemented), and close the connection. A > server MUST NOT send transfer-codings to an HTTP/1.0 client. " > > Am I right in thinking that we're falling afoul of this "should", making us a > "conditionally compliant" implementation of HTTP/1.1? > > John > -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Racket web server doesn't handle POST requests with 'Transfer-Encoding: chunked' ?
On Mar 9, 2012, at 12:18 PM, Jay McCarthy wrote: > There is nothing going wrong with the supported features of the Web > server. It's just that the Web server doesn't support any > Transfer-Encoding other than the default of no encoding. From RFC2616: "A server which receives an entity-body with a transfer-coding it does not understand SHOULD return 501 (Unimplemented), and close the connection. A server MUST NOT send transfer-codings to an HTTP/1.0 client. " Am I right in thinking that we're falling afoul of this "should", making us a "conditionally compliant" implementation of HTTP/1.1? John smime.p7s Description: S/MIME cryptographic signature _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Racket web server doesn't handle POST requests with 'Transfer-Encoding: chunked' ?
There is nothing going wrong with the supported features of the Web server. It's just that the Web server doesn't support any Transfer-Encoding other than the default of no encoding. Jay On Fri, Mar 9, 2012 at 1:44 AM, John Clements wrote: > My students are working on an assignment that has them CPS and > defunctionalize a small evaluator so that they can ship computations from one > compute server to another: > > http://www.brinckerhoff.org/clements/csc430-wi12/Assignments/ass7.html > > As part of this assignment, some of the students are writing servers in > node.js, and one of them drew my attention to what appears to be a small bug > in Racket's web server. Specifically, it appears that when a request is made > with POST data using the 'Transfer-Encoding: chunked" style, that the result > of calling request-post-bytes/raw on the request is #f. I can't tell whether > the web server is actually reading the POST data correctly and discarding it > when the request is constructed, or actually failing to read the data. > > To see this happen, run this racket program: > > #lang racket > > (require web-server/servlet-env > web-server/http/xexpr > web-server/http) > > ; the function that handles requests: > (define (start request) > (define post-bytes (request-post-data/raw > request)) > (cond [(bytes? post-bytes) (response/xexpr `(html "yep"))] > [else (response/xexpr `(html "nope"))])) > > ; start serving the 'start' function on port 8054: > (serve/servlet start > #:servlet-regexp #px".*" > #:port 8054 > #:launch-browser? #f) > > ... and then ping it with curl: > > curl -v -X POST -d "hello" http://localhost:8054 -H 'Transfer-Encoding: > chunked' > * About to connect() to localhost port 8054 (#0) > * Trying 127.0.0.1... connected > * Connected to localhost (127.0.0.1) port 8054 (#0) >> POST / HTTP/1.1 >> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 >> OpenSSL/0.9.8r zlib/1.2.3 >> Host: localhost:8054 >> Accept: */* >> Transfer-Encoding: chunked >> Content-Type: application/x-www-form-urlencoded >> >> 5 > < HTTP/1.1 200 Okay > < Date: Fri, 09 Mar 2012 08:41:55 GMT > < Last-Modified: Fri, 09 Mar 2012 08:41:55 GMT > < Server: Racket > < Content-Type: text/html; charset=utf-8 > < Connection: close > < > * Closing connection #0 > nope > > > The header lines shown by curl here are exactly what they should be, but the > server is getting #f for the post-bytes. > > Trying this without the 'Transfer-Encoding: chunked' returns the expected > byte-string: > > oiseau:~ clements$ curl -v -X POST -d "hello" http://localhost:8054* About to > connect() to localhost port 8054 (#0) > * Trying 127.0.0.1... connected > * Connected to localhost (127.0.0.1) port 8054 (#0) >> POST / HTTP/1.1 >> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 >> OpenSSL/0.9.8r zlib/1.2.3 >> Host: localhost:8054 >> Accept: */* >> Content-Length: 5 >> Content-Type: application/x-www-form-urlencoded >> > < HTTP/1.1 200 Okay > < Date: Fri, 09 Mar 2012 08:43:25 GMT > < Last-Modified: Fri, 09 Mar 2012 08:43:25 GMT > < Server: Racket > < Content-Type: text/html; charset=utf-8 > < Connection: close > < > * Closing connection #0 > yep > > So, is this a bug, or a known problem, or am I just doing something silly? I > spent (more time than I want to admit) poking around in the source tree, but > I couldn't find the spot where the request was constructed. > > John > > > _ > Racket Developers list: > http://lists.racket-lang.org/dev > -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Compiling tests
Does anyone mind avoiding compilation of `collects/tests/*'? I'm counting now about 10% of the tree build time spent on compiling stuff in collects/tests. Probably a large part of that is the eopl tests, since most of the other big tests already disable compilation. I could just add the info file to avoid compiling the eopl tests, but I prefer to just disable compilation of all of collects/tests, so that future additions will not need to be dealt with individually. In the past there were objections to doing that because compiling the tests has some testing value too -- but it seems that drdr is now completely filling that role. -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Intro to 3m?
Hello, I'm optimizing some code and have used gc-info logging to correlate substantial application pauses with GC occurrences. This make sense. The obvious solution is to allocate less memory, but tracking down where it's coming from isn't easy. In the short term I plan to inject (current-memory-use) based logging all over the code I'm optimizing, but would like to develop and contribute something more generic and re-usable if there's interest in it. My ideal goal is to add logging that reveals which code is allocating memory (how often, how much), and also, for each GC pass, the origin(s) and final resting place of what was collected. So now I'm trying to learn more about Racket's 3m garbage collector so I can instrument it. I've scoured the guide and reference, but found nothing that gets into the details of the runtime/collector interaction and in which code files that lives. Before I start grepping through the source and trying to piece it all together on my own, I figured I'd ask. Some pointers in the right direction, even vague hints of where to start, would be awesome :) Even a description of the build process - where xform comes into play - would be infinitely helpful. Please also feel free to tell me I'm crazy and point out something obviously simpler I'm missing. I just want faster code with fewer, shorter GC pauses - I have no vested interest in how it happens, but would love to contribute a useful tool if that makes sense. Regards, Nick _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
30 minutes ago, Jay McCarthy wrote: > I'll pushed an improved version of the test macro shortly. Please don't put it in `racket/test', since it's not a testing thing. (I haven't got to reply to that yet, but it looks like a more general convenience macro for pulling some expressions into a submodule. In any case, `racket/test' being a wrapper test code but not being the test code is going to be very confusing.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
What about specially treating the #:main keyword when it appears in a
module top-level. Everything that follows it gets wrapped in a
(module* main #f), perhaps with #:niam as an end delimiter. Or maybe
#:submodule , if it appears at the top-level of a module could
mean that. Or maybe all new syntax:
#submodule {
submodule body for goes here ...
}
Robby
On Fri, Mar 9, 2012 at 12:04 PM, Matthew Flatt wrote:
> At Wed, 7 Mar 2012 10:14:35 -0700, Matthew Flatt wrote:
>> I've added "submodules" to a version of Racket labeled v5.2.900.1
>
> Submodules are now pushed to the Racket git repo.
>
>
> I haven't yet added a syntactic form to simplify
>
> (module* main #f
> )
>
> My first idea was `main', as in
>
> (main
> )
>
> but that seems too quiet and likely to create collisions. (It does
> create a collision in part of the contract library, which imports
> `racket/base' for syntax and defines a for-syntax `main' function.)
>
> Jon suggests `submodule':
>
> (submodule main
> ...)
>
> This suggestion has the advantage of replacing all `(module* #f
> )' combinations, and mostly I like this direction. However,
> `submodule' would be only one way to define a "submodule", while
> `submod' in a module path corresponds to "submodule" in the more
> general sense. It could be that `submodule' is on the right path and
> other terminology should change, or maybe there's a more specific word
> to use for a `(module* #f )' replacement instead of `submodule'.
>
> _
> Racket Developers list:
> http://lists.racket-lang.org/dev
_
Racket Developers list:
http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
I'll pushed an improved version of the test macro shortly. Jay On Fri, Mar 9, 2012 at 11:04 AM, Matthew Flatt wrote: > At Wed, 7 Mar 2012 10:14:35 -0700, Matthew Flatt wrote: >> I've added "submodules" to a version of Racket labeled v5.2.900.1 > > Submodules are now pushed to the Racket git repo. > > > I haven't yet added a syntactic form to simplify > > (module* main #f > ) > > My first idea was `main', as in > > (main > ) > > but that seems too quiet and likely to create collisions. (It does > create a collision in part of the contract library, which imports > `racket/base' for syntax and defines a for-syntax `main' function.) > > Jon suggests `submodule': > > (submodule main > ...) > > This suggestion has the advantage of replacing all `(module* #f > )' combinations, and mostly I like this direction. However, > `submodule' would be only one way to define a "submodule", while > `submod' in a module path corresponds to "submodule" in the more > general sense. It could be that `submodule' is on the right path and > other terminology should change, or maybe there's a more specific word > to use for a `(module* #f )' replacement instead of `submodule'. > > _ > Racket Developers list: > http://lists.racket-lang.org/dev -- Jay McCarthy Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Any notion of ".jar" files for Racket?
Two days ago, Matthew Flatt wrote: > > If you don't need this functionality enabled by default, you could > install a `load/use-compiled' handler that similarly recognized > paths into a ".zip" file. The problem with that is that it's a hook that gets used only for loading code -- a lower level hook would allow loading any content from whatever (an archive, a cache divert, or a url). A possibly nice place for it would be hooking onto security guards -- in a similar way that parameter guards can change the actual value. For example, a security guard for openning a file could be expected to return a port, which a user-installed guard can now create from any of these sources. It would also be nice for the sandboxing aspect of security guards: instead of throwing an error, a FS guard could divert the requests to a local "disk file" similarly to what VMs do, a network guard could be used to provide a fake connection that can be used to test network code. As a concrete example, consider the problem of not being able to have default-sandboxed code do anything with preferences. In its current role, that's the *right* decision for the sandbox library, but right now your only choices are either to go with the default and forbid it, or allow access to your own preference. With such a security-guard- as-IO-hooks facility, the sandbox library could instead divert the preferences file into a private per-sandbox file (and with some more work, an in-memeory file representation, or a cheap way of achieving it with a shm filesystem from the OS). Another concrete example: imagine a homework assignment to implement a specific server, with sandbox tests that execute the code in an environment that arranges to have testing code talk to the sandbox and test it, with no danger of "leaking out" a student server. > I must admit, though, I'm disappointed if we can't rely on an > OS-supplied filesystem to provide a good filesystem abstraction. > Maybe there's some OS-level extension you could use to mount a file > in an existing filesystem as a new filesystem (like Mac OS X's > mounting of ".dmg" files)? I have the same opinion in general -- but in reality we have the same problem in NEU also. The releases are all stored on NFS (in "/proj/racket", which you might know as its former name), and to ensure that they always have the recently released version, they made the /usr/bin executables be symlinks to that directory[*]. I measured running drracket on a random machine -- it took about 40 seconds to start the first time, and subsequent runs took <5 seconds. I know that it'll still take a bunch of work to go from a hook as I imagine above to running it from a zip file, so that part is not really related to this problem, and I think that the proper solution is to simply have them use the PPA to install racket locally on all of the machines. Danny: I think that the same solution should be suggested in your case. ([*] But they didn't update the names, which means that students often try to run "drracket" and when it doesn't work assume that it's not installed...) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] submodules
At Wed, 7 Mar 2012 10:14:35 -0700, Matthew Flatt wrote: > I've added "submodules" to a version of Racket labeled v5.2.900.1 Submodules are now pushed to the Racket git repo. I haven't yet added a syntactic form to simplify (module* main #f ) My first idea was `main', as in (main ) but that seems too quiet and likely to create collisions. (It does create a collision in part of the contract library, which imports `racket/base' for syntax and defines a for-syntax `main' function.) Jon suggests `submodule': (submodule main ...) This suggestion has the advantage of replacing all `(module* #f )' combinations, and mostly I like this direction. However, `submodule' would be only one way to define a "submodule", while `submod' in a module path corresponds to "submodule" in the more general sense. It could be that `submodule' is on the right path and other terminology should change, or maybe there's a more specific word to use for a `(module* #f )' replacement instead of `submodule'. _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] Gnats & UTF8
Just in case: if you use gmail, make sure your encoding is set to UTF-8. Thanks, Eli! Robby On Fri, Mar 9, 2012 at 2:46 AM, Eli Barzilay wrote: > In preparation for a move to github, I've finished a very long and > tedious[*] scan of the complete gnats db, and everything is now > properly utf8-ized. In addition, the web interface declares a utf8 > charset which means that the texts are fine there too. There is one > problem that is still left: incoming emails are added to the bug > history without attention to charset (or encoding, but that's not as > important). > > To make a long story short: please try to use UTF-8 in your emailers. > Probably the biggest example of needing to do this is gmail, which for > some reason doesn't default to UTF-8. (On the first settings page > there's a checkbox for "Use Unicode (UTF-8) encoding for outgoing > messages".) > > > ([*] I thought that much of this could be automated... which extremely > naive. There were some examples of insanely bad encoding byproducts, > in many cases I resorted to detective methods of googling names of > people, using translate, grepping for bits of texts in other bugs, > using Emacs to guess encodings, using iconv, and more. In an extreme > case (PR8719) the file was so broken that I had to figure out the > quoted text from the string-constants file of the time and the commit > that fixed it, then figure out how it was broken and write some code > to unbreak it... Emacs was absolutely essential for all of this.) > > -- > ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: > http://barzilay.org/ Maze is Life! > _ > Racket Developers list: > http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] odd error message in race setup
Robby Findler wrote at 03/08/2012 08:00 PM: I think that the issue probably does not predate Kevin's recent push (distributed places). If you'd like to audit the push security concerns, I'm sure that'd be welcome. I meant that I might need to take a look at it because the example we saw would raise eyebrows with some of my consulting clients. Sounds like it was just a documentation example that is being removed before release, so a non-issue. Separate from that, I wouldn't mind taking a look at security implications of whatever this mechanism, but I doubt I'd notice anything that a PLTer who designed the mechanism hadn't already identified. -- http://www.neilvandyke.org/ _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Gnats & UTF8
In preparation for a move to github, I've finished a very long and tedious[*] scan of the complete gnats db, and everything is now properly utf8-ized. In addition, the web interface declares a utf8 charset which means that the texts are fine there too. There is one problem that is still left: incoming emails are added to the bug history without attention to charset (or encoding, but that's not as important). To make a long story short: please try to use UTF-8 in your emailers. Probably the biggest example of needing to do this is gmail, which for some reason doesn't default to UTF-8. (On the first settings page there's a checkbox for "Use Unicode (UTF-8) encoding for outgoing messages".) ([*] I thought that much of this could be automated... which extremely naive. There were some examples of insanely bad encoding byproducts, in many cases I resorted to detective methods of googling names of people, using translate, grepping for bits of texts in other bugs, using Emacs to guess encodings, using iconv, and more. In an extreme case (PR8719) the file was so broken that I had to figure out the quoted text from the string-constants file of the time and the commit that fixed it, then figure out how it was broken and write some code to unbreak it... Emacs was absolutely essential for all of this.) -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] Racket web server doesn't handle POST requests with 'Transfer-Encoding: chunked' ?
My students are working on an assignment that has them CPS and defunctionalize a small evaluator so that they can ship computations from one compute server to another: http://www.brinckerhoff.org/clements/csc430-wi12/Assignments/ass7.html As part of this assignment, some of the students are writing servers in node.js, and one of them drew my attention to what appears to be a small bug in Racket's web server. Specifically, it appears that when a request is made with POST data using the 'Transfer-Encoding: chunked" style, that the result of calling request-post-bytes/raw on the request is #f. I can't tell whether the web server is actually reading the POST data correctly and discarding it when the request is constructed, or actually failing to read the data. To see this happen, run this racket program: #lang racket (require web-server/servlet-env web-server/http/xexpr web-server/http) ; the function that handles requests: (define (start request) (define post-bytes (request-post-data/raw request)) (cond [(bytes? post-bytes) (response/xexpr `(html "yep"))] [else (response/xexpr `(html "nope"))])) ; start serving the 'start' function on port 8054: (serve/servlet start #:servlet-regexp #px".*" #:port 8054 #:launch-browser? #f) ... and then ping it with curl: curl -v -X POST -d "hello" http://localhost:8054 -H 'Transfer-Encoding: chunked' * About to connect() to localhost port 8054 (#0) * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 8054 (#0) > POST / HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 > OpenSSL/0.9.8r zlib/1.2.3 > Host: localhost:8054 > Accept: */* > Transfer-Encoding: chunked > Content-Type: application/x-www-form-urlencoded > > 5 < HTTP/1.1 200 Okay < Date: Fri, 09 Mar 2012 08:41:55 GMT < Last-Modified: Fri, 09 Mar 2012 08:41:55 GMT < Server: Racket < Content-Type: text/html; charset=utf-8 < Connection: close < * Closing connection #0 nope The header lines shown by curl here are exactly what they should be, but the server is getting #f for the post-bytes. Trying this without the 'Transfer-Encoding: chunked' returns the expected byte-string: oiseau:~ clements$ curl -v -X POST -d "hello" http://localhost:8054* About to connect() to localhost port 8054 (#0) * Trying 127.0.0.1... connected * Connected to localhost (127.0.0.1) port 8054 (#0) > POST / HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 > OpenSSL/0.9.8r zlib/1.2.3 > Host: localhost:8054 > Accept: */* > Content-Length: 5 > Content-Type: application/x-www-form-urlencoded > < HTTP/1.1 200 Okay < Date: Fri, 09 Mar 2012 08:43:25 GMT < Last-Modified: Fri, 09 Mar 2012 08:43:25 GMT < Server: Racket < Content-Type: text/html; charset=utf-8 < Connection: close < * Closing connection #0 yep So, is this a bug, or a known problem, or am I just doing something silly? I spent (more time than I want to admit) poking around in the source tree, but I couldn't find the spot where the request was constructed. John smime.p7s Description: S/MIME cryptographic signature _ Racket Developers list: http://lists.racket-lang.org/dev
