On Tue, 09 Aug 2011 18:58:22 +0000, dsimcha wrote: > > Is there any interest in a ParallelByChunk and ParallelByLine in > std.stdio.File? These would have a set of some user specified size > nBuffers of buffers, which would be recycled. A background thread > would read ahead until all buffers are full, then wait on a > condition variable. When popFront() has been called several times > and a large number of buffers are available, the background thread > would be woken to fill the newly available buffers.
Interesting. A possible use-case would be reading from a set of files, say, a directory full of log files which need to be analyzed line-by-line. So a reader that accepted a range of files might be handy. (I guess that the file concatenation could be handled at the OS level, with "cat" and a pipe, but a range-of-files would be more convenient.) On a tangent, the first thing that came to mind when you mentioned this was Tim Bray's WideFinder benchmark: http://wikis.sun.com/display/WideFinder/Wide+Finder+Home ParallelByLine would make for a great WideFinder implementation. Best, Graham
