Thanks for the quick reply.  I may wait for the ReadInputBuffer() as it may make it more like my package code which takes an array, converts it to a string with embedded linends, and writes it in chunks to the pipe.

On the subject of the "proper" linends, I would think that since this is Windows-specific code, you wouldn't need to use the platform EOL marker.

Gil
On 4/6/2018 10:56 AM, Rick McGuire wrote:
Gil,

ReadInput returns just the data for the next line, with no line terminator. You can include the line terminator by doing a second write for just the terminator.

I am also working on a ReadInputBuffer() function that will allow you to obtain the entire redirected output as a single buffer. Your question about the linends does raise a question though. I've written this code to use the platform EOL marker (i.e., CRLF on Windows). Should I be using '\n' everywhere?

Rick

On Fri, Apr 6, 2018 at 10:36 AM, Gil Barmwater <gbarmwa...@alum.rpi.edu <mailto:gbarmwa...@alum.rpi.edu>> wrote:

    Rick,

    Thanks for that info.  I agree that we should see how it plays out
    with the default pipe buffer sizes and address it later if we
    encounter problems in practice.

    I'm working on the code to write the data to the input pipe.  I've
    looked at the testCommandHandler() and found the ReadInput()
    method (haven't done an SVN update recently so you may have
    changed that) but wasn't sure what the interface was.  It appears
    to read a line at a time but does it return the \n as part of the
    data?  I need to include the \n characters when I write the data
    to the pipe.  And is there a maximum size for a line of input?

    Gil

    On 4/6/2018 9:43 AM, Rick McGuire wrote:
    Gil,

    Been doing a little reading on the Windows Pipe and I/O
    functions. and it appears you could also encounter a problem,
    though in a slightly different way. If you are still writing data
    to the pipe and the buffer is full, WriteFile() will block until
    the other end of the pipe reads enough data from the pipe to
    complete your write. Unfortunately, the other process may have
    already filled the output pipe and is sitting in the same
    WriteFile wait situation waiting for you to read from the output
    pipe.

    It is possible to create a non-blocking pipe by using
    CreateNamedPipe() rather than CreatePipe(), but I'm not sure this
    fixes the problem since the non-blocking behavior applies to both
    read and write operations. This means that the other process will
    see unexpected errors if either pipe's buffer is full, and likely
    terminate the command. I suspect for now, we may need to see how
    this plays out in practice. I see it is possible to specify a
    buffer size on the CreatePipe() call. Perhaps creating a larger
    output buffer would prevent the other process from filling it
    before the input data is all written.

    Rick

    On Thu, Apr 5, 2018 at 8:29 PM, Gil Barmwater
    <gbarmwa...@alum.rpi.edu <mailto:gbarmwa...@alum.rpi.edu>> wrote:

        Let me start by saying that I know next to nothing about *ix
        but perhaps the sequence in Windows may shed some light
        here.  The example I used for my package, and which is the
        basis of what I'm doing for ADDRESS...WITH on Windows, does
        the following: 1) create the pipes and pass the handles to
        the child process at creation, 2) with both processes now
        executing asynchronously, the main process writes all the
        data to the input pipe, possibly waiting should it fill the
        pipe, and closes it, 3) the child process reads the data,
        waiting if necessary, until it reaches the end and writes the
        data to the output and error pipes, 4) the main process,
        meanwhile, having finished writing the input pipe data, waits
        for the child process to terminate, 5) the child process
        finishes writing the output and error pipe data, closes them
        and terminates, and 6) the main process resumes after the
        child process has terminated, and reads the output and error
        pipes.  I don't see a blocking problem on the input side but
        there could be one on the output side if the pipes fill up
        and the OS can't "extend" them in some way. Then the child
        will block, unable to write (and therefore to complete and
        terminate) and the main program will wait forever. I'm hoping
        the OS can detect that somehow and notify one or both
        processes but without lots of research, I don't know the answer.

        Gil

        On 4/5/2018 5:58 PM, Erich Steinböck wrote:
        I have a working redirecting context on Ubuntu (no tests run
        yet).
        But I guess I need to think a little more about the blocking
        issue.
        What we're doing here is, fork a child (which runs the
        command) and connect us to the child through at most three
        pipes. A stdin pipe, a stdout pipe, and an error pipe. We
        write to the stdin pipe, which the child then reads, and the
        child writes to both the stdout and the error pipe, which we
        than will read back in.

        Now pipes are something that the OS provides and it does
        some buffering for us: you can write stuff to a pipe, before
        the child consumes it. But it's just a small buffer, 4K or
        64K. After that the pipe may block (or break, which I may
        have experienced)

        So if we're eager to send stuff through the stdin pipe for
        the child to use it, and the child is slow, or, worse,
        doesn't even expect anything on stdin, the pipe blocks and
        we're stalled. In the meantime we could do a little reading
        on the output or error pipe, because the child may be slow,
        because it itself has already filled up, say, its output
        pipe and reading that would be a relief (break the stall),
        but I'm not sure how to actually do this.

        When do we write, when read? Which pipes? How large pipe
        read/write blocks do we use? Do we sleep a little bit in
        between?
        I also need to figure out how to do non-blocking reads.


        We don't yet have ADDRESS WITH tests for a real Unix
        redirecting command context yet, have we?


        On Wed, Apr 4, 2018 at 4:04 PM, Erich Steinböck
        <erich.steinbo...@gmail.com
        <mailto:erich.steinbo...@gmail.com>> wrote:

            From the description in https://blog.uxul.de/e?e=mt-160
            <https://blog.uxul.de/e?e=mt-160> (very bottom) it seems
            "trivial" to do.
            Not a Linux guru at all, but I'd give it a try if no one
            else volunteers.

            The blog also contains a discussion of an interesting
            quirk with fork(), which is why the sample code uses
            POSIX spawn to execute the command.
            POSIX spawn seems to be available on all the platforms
            ooRexx supports. Not on Android though.




        
------------------------------------------------------------------------------
        Check out the vibrant tech community on one of the world's most
        engaging tech sites, Slashdot.org!http://sdm.link/slashdot


        _______________________________________________
        Oorexx-devel mailing list
        Oorexx-devel@lists.sourceforge.net
        <mailto:Oorexx-devel@lists.sourceforge.net>
        https://lists.sourceforge.net/lists/listinfo/oorexx-devel
        <https://lists.sourceforge.net/lists/listinfo/oorexx-devel>

-- Gil Barmwater


        
------------------------------------------------------------------------------
        Check out the vibrant tech community on one of the world's most
        engaging tech sites, Slashdot.org! http://sdm.link/slashdot
        _______________________________________________
        Oorexx-devel mailing list
        Oorexx-devel@lists.sourceforge.net
        <mailto:Oorexx-devel@lists.sourceforge.net>
        https://lists.sourceforge.net/lists/listinfo/oorexx-devel
        <https://lists.sourceforge.net/lists/listinfo/oorexx-devel>




    
------------------------------------------------------------------------------
    Check out the vibrant tech community on one of the world's most
    engaging tech sites, Slashdot.org!http://sdm.link/slashdot


    _______________________________________________
    Oorexx-devel mailing list
    Oorexx-devel@lists.sourceforge.net
    <mailto:Oorexx-devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/oorexx-devel
    <https://lists.sourceforge.net/lists/listinfo/oorexx-devel>

-- Gil Barmwater


    
------------------------------------------------------------------------------
    Check out the vibrant tech community on one of the world's most
    engaging tech sites, Slashdot.org! http://sdm.link/slashdot
    _______________________________________________
    Oorexx-devel mailing list
    Oorexx-devel@lists.sourceforge.net
    <mailto:Oorexx-devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/oorexx-devel
    <https://lists.sourceforge.net/lists/listinfo/oorexx-devel>




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

--
Gil Barmwater

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Reply via email to