Thanks for your comments. The only legal files to upload in this case are plain text, so I'm not too worried about size. I'm relying on the web-server libraries to deal with any malicious attempts to send overwhelmingly large files (if that's a bad idea, I'd definitely appreciate hearing it!). Other parts of the application are implemented in #lang web-server, including some access control logic surrounding the requests that are proxied to the external service.
With other requests, the post-data/raw field of the request struct has been #f only when the method field is #"GET": with POST requests, it has otherwise (and I thought it always would) contained the raw POST data e.g. # "corpus=austen&tool=corpus.CorpusMetadata". I thought the bindings from the bindings/raw-promise field were simply an abstraction over the post-data/raw (and/or query part of the uri field), which is why I'm confused that this POST request has bindings, but has #f for its post-data/raw. -Philip On Thu, Jun 29, 2017 at 9:44 PM, Neil Van Dyke <n...@neilvandyke.org> wrote: > I don't know the answer to your particular questions with `web-server` > (I've made my own implementations of this in the past), and these comments > might not apply to your particular application, but I'll mention here for > whomever is interested... > > It sounds like you're using this, which might preempt your question: > > post-data/raw : (or/c false/c bytes?) >> > > Does your application permit a large file upload (an uploaded DVD-ROM > ".iso" file, like for a Linux distro install disc 1, is typically a few > gigabytes, and video files can also get huge), and is your program > (including libraries it uses) going to try to allocate gigabytes at a time > just for one HTTP request? > > If the `POST` data is potentially huge, you might want to think about > doing stream reading of it (i.e., not sucking it all into memory before you > do something with it), and sending blocks out your proxy approximately as > soon as they come in (without buffering too much). That can make your > program more robust, lower latency, and maybe even improve overall speed. > > Or, if you want to keep getting a convenient byte string out of the MIME > parser, and you plan to reject huge `POST` data before it > accidentally/intentionally DoS's your server, that will probably happen > either as the HTTP request is being read, or in the MIME multipart parser > (when the request is in MIME multipart, which `POST` isn't always, and if > the HTTP code hands off a pretty raw input port to multipart parsing code, > which it should). This is because you can't assume that HTTP or part > headers will tell you the content size before you read the content -- > sometimes you have to read to find the EOF or the MIME boundary string > kludge. > > I think streaming algorithms are usually the way to go for potentially > huge data. (Well, until you then get into what I'll call "poetic license" > situations, in which you know how to do it in streaming, and you know why > you don't have to stream in this case.) > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
