I hate it when code compiles and runs on the first attempt :-) Just moved
the pipe reading code to before wait for process termination and the big
data test case started working. Erich, What I did for Windows with the
separate thread should be easily adapted for Linux.
Rick
On Fri, Apr 13, 2018 at 6:58 AM, Rick McGuire <object.r...@gmail.com> wrote:
> Ok, I have things working with a separate thread filling the input pipe,
> but I'm able to get a hang with a simple example (see below). As I
> suspected, the problem is caused by waiting for the process to complete
> before reading from the output pipe. The command process stalls when the
> pipe gets filled and waits until data is read from the pipe. So, the
> process never terminates because it is waiting for the pipe to be cleared
> out. The attached programs work fine with much smaller amounts of data.
>
> test.rex
>
> a1 = .array~new(100000)a1~fill('This is a test')
> address command 'rexx flipit.rex' with input using (a1) output stem a.say a.0
>
> flipit.rex
>
> signal on notready
> loop forever say reverse(linein())end
> notready:
>
> Rick
>
> On Thu, Apr 12, 2018 at 7:59 PM, Gil Barmwater <gbarmwa...@alum.rpi.edu>
> wrote:
>
>> You are correct on the placement of the APPEND|REPLACE keywords. I
>> somehow misread the BNF :-( . As for the required parenthesis around
>> the variable name, this will be a difference, I suspect, between ooRexx and
>> Regina that we need to clearly document, probably by including an example
>> showing that usage. And I have no problem with the required dot after stem
>> objects (as in using (foo.)) but 'output stem s' seems to me unambiguous as
>> the keyword 'stem' indicates the name 's' is the name of a stem. Just my
>> opinion.
>>
>> I access the pipes the same way the sample code I found did, with the
>> ReadFile() and WriteFile() functions. As I said earlier, I hadn't tested
>> large amounts of redirected output data so the problem didn't arise during
>> my development and testing. The sample code didn't actually do a wait so
>> that was something I added without thinking about the full pipe scenario.
>> I believe a loop with a timed wait and pipe reads (and buffered output
>> writes) might address this but I haven't had time to experiment with it
>> yet. However, even that approach won't cover the case where the code
>> blocks writing to the input pipe since we wouldn't get to the wait loop
>> until the input data is all written. Sounds like the thread approach might
>> be the most straightforward way to go.
>> Gil
>>
>> On 4/12/2018 5:39 PM, Rick McGuire wrote:
>>
>> It looks like spinning off a thread to do this is pretty easy using the
>> classes we already have. Gil, what commands have you been using to test
>> both ends of the pipe?
>>
>> Also, I notice that the code waits for the process to complete before
>> reading the output pipes. Shouldn't it be reading before the process
>> completes? Otherwise, a command that creates a large amount of output will
>> fill the pipe up.
>>
>> Rick
>>
>> On Thu, Apr 12, 2018 at 5:03 PM, Rick McGuire <object.r...@gmail.com>
>> wrote:
>>
>>> I decided to post a question to StackOverflow to see if anybody has an
>>> suggestions. I'm sort of thinking the solution might be to spin off a
>>> thread to handle the writes to the buffer so that the main thread can
>>> immediately start emptying the output pipes. I think I'll try taking a stab
>>> at this for the Windows version.
>>>
>>> Rick
>>>
>>> On Thu, Apr 12, 2018 at 2:56 PM, Gil Barmwater <gbarmwa...@alum.rpi.edu>
>>> wrote:
>>>
>>>> Found and fixed the problem and have been through all my tests that I
>>>> developed for my proof of concept package. In the process, I found several
>>>> things I wasn't expecting.
>>>>
>>>> 1) The parser seems to have reversed the expected positions of the
>>>> target type - STEM, STREAM, or USING - and the "mode" - APPEND or REPLACE.
>>>> I believe the syntax per the ANSI standard should be e.g. WITH OUTPUT STEM
>>>> APPEND someStem. and not WITH OUTPUT APPEND STEM someStem.
>>>>
>>>> 2) There seems to be a requirement for the stream/stem name to be
>>>> either a literal - "abc.txt" - or if it is a variable, enclosed in
>>>> parenthesis - (someFile). I expected the () for USING but not for STEM or
>>>> STREAM.
>>>>
>>>> 3) My package allowed for stem "names" to be specified with or without
>>>> the trailing . but my testing shows the trailing . is required or an error
>>>> message is generated. I'm OK with that as long as it is documented.
>>>>
>>>> 4) I get an error (unable to open ... for output) when I specify
>>>>
>>>> output stream "abc2.txt" error replace stream "abc2.txt"
>>>> which was testing the redirection of both output and error to the same
>>>> stream.
>>>>
>>>> 5) Using a non-existent file for redirected input generates an error
>>>> (unable to open ... for reading) while my package treated this as a null
>>>> file. Probably not a problem as the error message would flag a file name
>>>> typo :-)
>>>>
>>>> As for checking in the code, I am not a committer so I plan to generate
>>>> a patch file and attach it to RFE 4.
>>>>
>>>> One other item that I discovered is that I can get a deadlock in
>>>> Windows as well. I had not tested sending a lot of data to redirected
>>>> output but when I did, my package locked up, presumably due to the output
>>>> pipe being full. I plan to experiment with ways around this but will
>>>> generate the patch file with the code as it stands now.
>>>>
>>>> Gil
>>>>
>>>> On 4/11/2018 2:32 PM, Rick McGuire wrote:
>>>>
>>>>
>>>> On Wed, Apr 11, 2018 at 2:26 PM Gil Barmwater <gbarmwa...@alum.rpi.edu>
>>>> wrote:
>>>>
>>>>> Just a quick update on my progress so far. I've gotten all the code
>>>>> in
>>>>> (hopefully) the right places and I have a clean build. My initial
>>>>> tests
>>>>> are working but I've just hit my first problem which I need to debug.
>>>>> Hopefully, it will not be too much longer before I'm done.
>>>>
>>>> checking the code in as it is is also an option...it gets more eyeballs
>>>> on the problem
>>>>
>>>> Rick
>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> --
>>>>> Gil Barmwater
>>>>>
>>>>>
>>>>> ------------------------------------------------------------
>>>>> ------------------
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> Oorexx-devel mailing list
>>>>> Oorexx-devel@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Oorexx-devel mailing
>>>> listOorexx-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>
>>>> --
>>>> Gil Barmwater
>>>>
>>>>
>>>> ------------------------------------------------------------
>>>> ------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> Oorexx-devel mailing list
>>>> Oorexx-devel@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>
>>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Oorexx-devel mailing
>> listOorexx-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>
>> --
>> Gil Barmwater
>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Oorexx-devel mailing list
>> Oorexx-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>
>>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
