On Thu, Apr 5, 2018 at 8:29 PM, Gil Barmwater <gbarmwa...@alum.rpi.edu>
wrote:

> Let me start by saying that I know next to nothing about *ix but perhaps
> the sequence in Windows may shed some light here.  The example I used for
> my package, and which is the basis of what I'm doing for ADDRESS...WITH on
> Windows, does the following: 1) create the pipes and pass the handles to
> the child process at creation, 2) with both processes now executing
> asynchronously, the main process writes all the data to the input pipe,
> possibly waiting should it fill the pipe, and closes it, 3) the child
> process reads the data, waiting if necessary, until it reaches the end and
> writes the data to the output and error pipes, 4) the main process,
> meanwhile, having finished writing the input pipe data, waits for the child
> process to terminate, 5) the child process finishes writing the output and
> error pipe data, closes them and terminates, and 6) the main process
> resumes after the child process has terminated, and reads the output and
> error pipes.  I don't see a blocking problem on the input side but there
> could be one on the output side if the pipes fill up and the OS can't
> "extend" them in some way.  Then the child will block, unable to write (and
> therefore to complete and terminate) and the main program will wait
> forever.  I'm hoping the OS can detect that somehow and notify one or both
> processes but without lots of research, I don't know the answer.
>

It sounds like that's the basics of what's going on with the Linux version
as well. However, on Linux the pipes are fairly limited in size. So Erich's
situation is basically this:

1) He writes as much data to the input pipe, but manages to write the data
faster than the process is consuming it, so he gets an error indicating the
pipe is full.
2) He can't just wait a little bit and try again, because the other process
might have the same problem on the output pipes.
3) If he switches to reading from the output pipes to clear those out, the
other process will resume reading from the input pipe because it has not
been closed yet, it will block.
4) Meanwhile, Erich is also blocked because he's also looking for data on a
pipe.

A non-blocking read would fix that problem, since it would allow him to
switch back-and-forth between reading and writing without creating a stall
situation. Hopefully, Windows will handle this for out, but I'm pretty sure
non-blocking reads are possible on Windows.

Rick




> Gil
>
> On 4/5/2018 5:58 PM, Erich Steinböck wrote:
>
> I have a working redirecting context on Ubuntu (no tests run yet).
> But I guess I need to think a little more about the blocking issue.
> What we're doing here is, fork a child (which runs the command) and
> connect us to the child through at most three pipes. A stdin pipe, a stdout
> pipe, and an error pipe. We write to the stdin pipe, which the child then
> reads, and the child writes to both the stdout and the error pipe, which we
> than will read back in.
>
> Now pipes are something that the OS provides and it does some buffering
> for us: you can write stuff to a pipe, before the child consumes it. But
> it's just a small buffer, 4K or 64K. After that the pipe may block (or
> break, which I may have experienced)
>
> So if we're eager to send stuff through the stdin pipe for the child to
> use it, and the child is slow, or, worse, doesn't even expect anything on
> stdin, the pipe blocks and we're stalled. In the meantime we could do a
> little reading on the output or error pipe, because the child may be slow,
> because it itself has already filled up, say, its output pipe and reading
> that would be a relief (break the stall), but I'm not sure how to actually
> do this.
>
> When do we write, when read? Which pipes? How large pipe read/write blocks
> do we use? Do we sleep a little bit in between?
> I also need to figure out how to do non-blocking reads.
>
>
> We don't yet have ADDRESS WITH tests for a real Unix redirecting command
> context yet, have we?
>
>
> On Wed, Apr 4, 2018 at 4:04 PM, Erich Steinböck <
> erich.steinbo...@gmail.com> wrote:
>
>> From the description in https://blog.uxul.de/e?e=mt-160 (very bottom) it
>> seems "trivial" to do.
>> Not a Linux guru at all, but I'd give it a try if no one else volunteers.
>>
>> The blog also contains a discussion of an interesting quirk with fork(),
>> which is why the sample code uses POSIX spawn to execute the command.
>> POSIX spawn seems to be available on all the platforms ooRexx supports.
>> Not on Android though.
>>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Oorexx-devel mailing 
> listOorexx-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
>
> --
> Gil Barmwater
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Reply via email to