Been doing a little reading on the Windows Pipe and I/O functions. and it
appears you could also encounter a problem, though in a slightly different
way. If you are still writing data to the pipe and the buffer is full,
WriteFile() will block until the other end of the pipe reads enough data
from the pipe to complete your write. Unfortunately, the other process may
have already filled the output pipe and is sitting in the same WriteFile
wait situation waiting for you to read from the output pipe.

It is possible to create a non-blocking pipe by using CreateNamedPipe()
rather than CreatePipe(), but I'm not sure this fixes the problem since the
non-blocking behavior applies to both read and write operations. This means
that the other process will see unexpected errors if either pipe's buffer
is full, and likely terminate the command. I suspect for now, we may need
to see how this plays out in practice. I see it is possible to specify a
buffer size on the CreatePipe() call. Perhaps creating a larger output
buffer would prevent the other process from filling it before the input
data is all written.


On Thu, Apr 5, 2018 at 8:29 PM, Gil Barmwater <>

> Let me start by saying that I know next to nothing about *ix but perhaps
> the sequence in Windows may shed some light here.  The example I used for
> my package, and which is the basis of what I'm doing for ADDRESS...WITH on
> Windows, does the following: 1) create the pipes and pass the handles to
> the child process at creation, 2) with both processes now executing
> asynchronously, the main process writes all the data to the input pipe,
> possibly waiting should it fill the pipe, and closes it, 3) the child
> process reads the data, waiting if necessary, until it reaches the end and
> writes the data to the output and error pipes, 4) the main process,
> meanwhile, having finished writing the input pipe data, waits for the child
> process to terminate, 5) the child process finishes writing the output and
> error pipe data, closes them and terminates, and 6) the main process
> resumes after the child process has terminated, and reads the output and
> error pipes.  I don't see a blocking problem on the input side but there
> could be one on the output side if the pipes fill up and the OS can't
> "extend" them in some way.  Then the child will block, unable to write (and
> therefore to complete and terminate) and the main program will wait
> forever.  I'm hoping the OS can detect that somehow and notify one or both
> processes but without lots of research, I don't know the answer.
> Gil
> On 4/5/2018 5:58 PM, Erich Steinböck wrote:
> I have a working redirecting context on Ubuntu (no tests run yet).
> But I guess I need to think a little more about the blocking issue.
> What we're doing here is, fork a child (which runs the command) and
> connect us to the child through at most three pipes. A stdin pipe, a stdout
> pipe, and an error pipe. We write to the stdin pipe, which the child then
> reads, and the child writes to both the stdout and the error pipe, which we
> than will read back in.
> Now pipes are something that the OS provides and it does some buffering
> for us: you can write stuff to a pipe, before the child consumes it. But
> it's just a small buffer, 4K or 64K. After that the pipe may block (or
> break, which I may have experienced)
> So if we're eager to send stuff through the stdin pipe for the child to
> use it, and the child is slow, or, worse, doesn't even expect anything on
> stdin, the pipe blocks and we're stalled. In the meantime we could do a
> little reading on the output or error pipe, because the child may be slow,
> because it itself has already filled up, say, its output pipe and reading
> that would be a relief (break the stall), but I'm not sure how to actually
> do this.
> When do we write, when read? Which pipes? How large pipe read/write blocks
> do we use? Do we sleep a little bit in between?
> I also need to figure out how to do non-blocking reads.
> We don't yet have ADDRESS WITH tests for a real Unix redirecting command
> context yet, have we?
> On Wed, Apr 4, 2018 at 4:04 PM, Erich Steinböck <
>> wrote:
>> From the description in (very bottom) it
>> seems "trivial" to do.
>> Not a Linux guru at all, but I'd give it a try if no one else volunteers.
>> The blog also contains a discussion of an interesting quirk with fork(),
>> which is why the sample code uses POSIX spawn to execute the command.
>> POSIX spawn seems to be available on all the platforms ooRexx supports.
>> Not on Android though.
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites,!
> _______________________________________________
> Oorexx-devel mailing 
> listOorexx-devel@lists.sourceforge.net
> --
> Gil Barmwater
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites,!
> _______________________________________________
> Oorexx-devel mailing list
Check out the vibrant tech community on one of the world's most
engaging tech sites,!
Oorexx-devel mailing list

Reply via email to