Gil,

ReadInput returns just the data for the next line, with no line terminator.
You can include the line terminator by doing a second write for just the
terminator.

I am also working on a ReadInputBuffer() function that will allow you to
obtain the entire redirected output as a single buffer. Your question about
the linends does raise a question though. I've written this code to use the
platform EOL marker (i.e., CRLF on Windows). Should I be using '\n'
everywhere?

Rick

On Fri, Apr 6, 2018 at 10:36 AM, Gil Barmwater <gbarmwa...@alum.rpi.edu>
wrote:

> Rick,
>
> Thanks for that info.  I agree that we should see how it plays out with
> the default pipe buffer sizes and address it later if we encounter problems
> in practice.
>
> I'm working on the code to write the data to the input pipe.  I've looked
> at the testCommandHandler() and found the ReadInput() method (haven't done
> an SVN update recently so you may have changed that) but wasn't sure what
> the interface was.  It appears to read a line at a time but does it return
> the \n as part of the data?  I need to include the \n characters when I
> write the data to the pipe.  And is there a maximum size for a line of
> input?
>
> Gil
> On 4/6/2018 9:43 AM, Rick McGuire wrote:
>
> Gil,
>
> Been doing a little reading on the Windows Pipe and I/O functions. and it
> appears you could also encounter a problem, though in a slightly different
> way. If you are still writing data to the pipe and the buffer is full,
> WriteFile() will block until the other end of the pipe reads enough data
> from the pipe to complete your write. Unfortunately, the other process may
> have already filled the output pipe and is sitting in the same WriteFile
> wait situation waiting for you to read from the output pipe.
>
> It is possible to create a non-blocking pipe by using CreateNamedPipe()
> rather than CreatePipe(), but I'm not sure this fixes the problem since the
> non-blocking behavior applies to both read and write operations. This means
> that the other process will see unexpected errors if either pipe's buffer
> is full, and likely terminate the command. I suspect for now, we may need
> to see how this plays out in practice. I see it is possible to specify a
> buffer size on the CreatePipe() call. Perhaps creating a larger output
> buffer would prevent the other process from filling it before the input
> data is all written.
>
> Rick
>
> On Thu, Apr 5, 2018 at 8:29 PM, Gil Barmwater <gbarmwa...@alum.rpi.edu>
> wrote:
>
>> Let me start by saying that I know next to nothing about *ix but perhaps
>> the sequence in Windows may shed some light here.  The example I used for
>> my package, and which is the basis of what I'm doing for ADDRESS...WITH on
>> Windows, does the following: 1) create the pipes and pass the handles to
>> the child process at creation, 2) with both processes now executing
>> asynchronously, the main process writes all the data to the input pipe,
>> possibly waiting should it fill the pipe, and closes it, 3) the child
>> process reads the data, waiting if necessary, until it reaches the end and
>> writes the data to the output and error pipes, 4) the main process,
>> meanwhile, having finished writing the input pipe data, waits for the child
>> process to terminate, 5) the child process finishes writing the output and
>> error pipe data, closes them and terminates, and 6) the main process
>> resumes after the child process has terminated, and reads the output and
>> error pipes.  I don't see a blocking problem on the input side but there
>> could be one on the output side if the pipes fill up and the OS can't
>> "extend" them in some way.  Then the child will block, unable to write (and
>> therefore to complete and terminate) and the main program will wait
>> forever.  I'm hoping the OS can detect that somehow and notify one or both
>> processes but without lots of research, I don't know the answer.
>> Gil
>>
>> On 4/5/2018 5:58 PM, Erich Steinböck wrote:
>>
>> I have a working redirecting context on Ubuntu (no tests run yet).
>> But I guess I need to think a little more about the blocking issue.
>> What we're doing here is, fork a child (which runs the command) and
>> connect us to the child through at most three pipes. A stdin pipe, a stdout
>> pipe, and an error pipe. We write to the stdin pipe, which the child then
>> reads, and the child writes to both the stdout and the error pipe, which we
>> than will read back in.
>>
>> Now pipes are something that the OS provides and it does some buffering
>> for us: you can write stuff to a pipe, before the child consumes it. But
>> it's just a small buffer, 4K or 64K. After that the pipe may block (or
>> break, which I may have experienced)
>>
>> So if we're eager to send stuff through the stdin pipe for the child to
>> use it, and the child is slow, or, worse, doesn't even expect anything on
>> stdin, the pipe blocks and we're stalled. In the meantime we could do a
>> little reading on the output or error pipe, because the child may be slow,
>> because it itself has already filled up, say, its output pipe and reading
>> that would be a relief (break the stall), but I'm not sure how to actually
>> do this.
>>
>> When do we write, when read? Which pipes? How large pipe read/write
>> blocks do we use? Do we sleep a little bit in between?
>> I also need to figure out how to do non-blocking reads.
>>
>>
>> We don't yet have ADDRESS WITH tests for a real Unix redirecting command
>> context yet, have we?
>>
>>
>> On Wed, Apr 4, 2018 at 4:04 PM, Erich Steinböck <
>> erich.steinbo...@gmail.com> wrote:
>>
>>> From the description in https://blog.uxul.de/e?e=mt-160 (very bottom)
>>> it seems "trivial" to do.
>>> Not a Linux guru at all, but I'd give it a try if no one else volunteers.
>>>
>>> The blog also contains a discussion of an interesting quirk with fork(),
>>> which is why the sample code uses POSIX spawn to execute the command.
>>> POSIX spawn seems to be available on all the platforms ooRexx supports.
>>> Not on Android though.
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Oorexx-devel mailing 
>> listOorexx-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>
>>
>> --
>> Gil Barmwater
>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Oorexx-devel mailing list
>> Oorexx-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>
>>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Oorexx-devel mailing 
> listOorexx-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
> --
> Gil Barmwater
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Reply via email to