Hey Matthew, 2011/2/8 Matthew Sherborne <[email protected]>: > I concur with the need for wstream resource. I need it for serving objects > straight out of Mongolia gridfs.
I would be interested in hearing how it works out? > I would also like agile upload to support chunking straight into a function > or istream, that would mean I could put large uploads straight into gridfs, > without the need for a large intermediate file on the is. Noble as it is, to achieve this, two problems need to be solved: The connector libraries currently cache an incoming request until it is entirely received. The underlying idea is that connector libraries are entirely async I/O (well, only ISAPI and wthttpd currently are) and thus act as buffers for talking to a potentially slow browser. Only after a whole request is received, the CGI parser is used to parse its body, and spool uploaded files to the file system. (See for example src/http/WtReply.C constructor, where a memory or file buffer is chosen, and consumeRequestBody() where the contents is written). So, somehow, we need to make an exception for certain types of requests (e.g. those belonging to file uploads, for which we already have something in place for monitoring their progress), so that we asynchronously perform CGI parsing while the data is being received and skip the first buffering. Two challenges here: - refactor CgiParser so that it can be used asynchronously (that should actually not be too hard since it already works in chunks of fixed sized buffers) - link that to consumeRequestBody() with a clean API Secondly, you want to indicate to the CgiParser that is should not create a spool file in the file system, but that you already have std::ostream objects waiting for the incoming file. Again this somewhat cuts through layers but this is probably not the hardest part. So, it's definitely some work and so far we never saw a real (performance) need to do it. > I might have another go in a few months, if king Koen doesn't beat me ;) Careful with that. It might sound like King Kong when mispronounced :-) Regards, koen ------------------------------------------------------------------------------ The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb _______________________________________________ witty-interest mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/witty-interest
