Hi all, I created an interesting derivative of function/objects/Stream.st called PipeStream.st. It uses the POSIX <ucontext.h> functions to implement fully deterministic coroutine-based streams whose producer is a BlockClosure. Any attempt to read the stream transfers control to the producer coroutine (running on the same OS thread) where it attempts to provide enough data to fill the request. From the producer's point of view, they just run to completion.
It's like other languages' "generators" feature, but without having to do an explicit "yield". The PipeStream machinery just detects when you write enough objects back to it (or write a few then exit), then resumes control in the reader. I'm quite happy with it, and will be using it to separate my C preprocessor into multiple pipelined parsing stages so that each stage is independent and easier to debug. The only downside I can see is that <ucontext.h> is not all that common. I don't know if something like it exists for Windows (actually, NT Fibers could be abused to provide the same thing, I think) or MacOS. GNU Pth provides similar functionality, so it may be possible to learn how to reimplement the context-switching features on other platforms. You also have to assign some stack space to the BlockClosure, and eventually, it will be best to allocate that space from the caller's stack so that it has the correct permissions. I don't think there's anything too crazy in this patch so it could probably be easily backported to jolt-burg as well. I've just attached my change to Stream.st, and all the files I used for testing. In the future, I'll incorporate this into my public "ocean" repository, but first I want to do some more work on the preprocessor and benchmark the monolithic vs. coroutine approaches. Have fun, -- Michael FIG <[EMAIL PROTECTED]> //\ http://michael.fig.org/ \//
diff -r 742206fd6931 function/objects/Stream.st --- a/function/objects/Stream.st Fri May 02 01:47:36 2008 -0600 +++ b/function/objects/Stream.st Fri May 02 06:51:31 2008 -0600 @@ -67,13 +67,16 @@ ReadStream refill [ ^false ] ReadStream flush [ - position > (collection size // 2) - ifTrue: - [| newLimit | - newLimit := readLimit - position. - collection replaceFrom: 0 to: newLimit - 1 with: collection startingAt: position. - readLimit := newLimit. - position := 0] + position >= ((collection size + 1) // 2) + ifTrue: [self flushFrom: position. position := 0] +] + +ReadStream flushFrom: aPosition +[ + | newLimit | + newLimit := readLimit - aPosition. + collection replaceFrom: 0 to: newLimit - 1 with: collection startingAt: aPosition. + readLimit := newLimit. ] ReadStream grow [ self grow: collection size ]
Makefile
Description: Binary data
u.k
Description: Binary data
PipeStream.st
Description: Binary data
_______________________________________________ fonc mailing list [email protected] http://vpri.org/mailman/listinfo/fonc
