Andrew Haines wrote:
Michalis Kamburelis wrote:
Andrew Haines wrote:
[...]
I am not extremely familiar with how streams work, but using
LoadFromStream(Output) may not "Read" the data so it can be replaced
with new data. Also as I said I think S.LoadFromStream is blocking
until the end of the stream is reached, which can't happen until the
program has ended possibly resulting in lost data. Someone please
correct me if I am wrong ;)
That's a good point, S.LoadFromStream *should* be blocking but
*currently it isn't*. I reproduced Bobby's problem with
TStringList.LoadFromStream (on Linux, so it's not specific to Win32).
Looking at implementation in rtl/objpas/classes/stringl.inc, there's a
test
Until BytesRead<>BufDelta
that should probably be changed to
Until BytesRead = 0
Then it should be possible to call S.LoadFromStream just once, and
read the whole output of a process. There should be no need to do the
partial reads, because TStringList.LoadFromStream should already
handle this. Generally, example "Reading large output" on
[http://wiki.lazarus.freepascal.org/index.php/Executing_External_Programs]
will not need to use memory stream anymore. One should just construct
process without poWaitOnExit (to prevent deadlock), and then be able
to read it's whole output using single call
AStringList.LoadFromStream(AProcess.Output);
So right now I think that what Bobby is observing is a result of a bug
in FPC's implementation of TStringList.LoadFromStream. I'll try to fix
it and submit to FPC devels.
Anyway, for now: the method of example "Reading large output" in the
wiki still works OK. In other words: Bobby, for now you should use
memory streams to read the pipe, like presented on the wiki (and like
Andrew suggests). Then, after grabbing everything to TMemoryStream,
you can safely convert it to e.g. TStringList by
AStringList.LoadFromStream(AMemoryStream);
This Should Work :)
I think that TProcess.Output cannot be larger than a certain size on
some platforms, so this is why it is read in increments rather than
waiting until it is done. Because the program will stop until some
output is read to make room for new output.
Yes, what you say is true, and it doesn't contradict what I said :)
Internal buffer inside the pipe behind TProcess.Output cannot be larger
than certain size. That's why you can't just execute the process, wait
until it stops, and then read the whole Output -- because if the child
process will write a lot of data to the pipe, at some time the write
operation will block and will wait for someone to read from the pipe. So
you have to read from the pipe *while the child process is still
running* (and possibly writing something to the pipe).
Stating it simply, it means that one shouldn't use both poWaitOnExit and
poUsePipes flags, because this can cause AProcess.Execute to hang.
What I meant is that implementation TStringList.LoadFromStream should do
(in fact, it already does, but not completely) proper reading of Stream,
by parts. Consider this:
AProcess.Options := [poUsePipes];
AProcess.Execute;
{ At this point, the AProcess may be still running }
{ Now, AStringList.LoadFromStream is implemented by calling
AProcess.Output.Read repeatedly until stream AProcess.Output
ends (see it's implementation in FPC sources in
rtl/objpas/classes/stringl.inc). This means that while
AStringList.LoadFromStream works, the pipe is read
(and this also means that child process may write something to
the pipe). Some calls to AProcess.Output.Read will return
immediately, some will block until child process will write
something.
AStringList.LoadFromStream below will
finish work when child process's stdout will end and
when everything from pipe has been read to AStringList. }
AStringList.LoadFromStream(AProcess.Output);
{ Most probably child process ended by now
(although there is no 100% guarantee for that,
AFAIK process may close it's stdout before it ends.)
In any case, it's not important whether
child process still works now. It's important that it's
stdout was closed, so we have everything we need in
AStringList. }
Michalis
_________________________________________________________________
To unsubscribe: mail [EMAIL PROTECTED] with
"unsubscribe" as the Subject
archives at http://www.lazarus.freepascal.org/mailarchives
