Vincent Snijders wrote:
I have looked a bit more why outputfilter is so slow at parsing a lot
of compiler output (e.g. compiling a simple LCL app with -va). The
pipe buffer is rather small, NumBytesAvailable is not bigger than 1280
bytes. So OnAsyncReadData reads only 1280 bytes at a time.
OnAsyncReadData is only called after an application.handlemessage. So
on line 319 of outputfilter.pas, the buffer is most times empty, but
NumBytesAvailable = 1280, and we are sleeping until
Application.HandleMessage is called.
I had some success with the following patch. It does a sleep(0) to
give the compiler the chance to write to the pipe again and tries to
read it again.
--- ide/outputfilter.pas
+++ ide/outputfilter.pas
@@ -1206,9 +1206,12 @@ var
Count: LongWord;
begin
if fProcess=nil then exit;
- Count:=TAsyncProcess(fProcess).NumBytesAvailable;
- if Count>0 then
- FAsyncOutput.Push(TStream(TAsyncProcess(fProcess).Output),Count);
+ repeat
+ Count:=TAsyncProcess(fProcess).NumBytesAvailable;
+ if Count>0 then
+ FAsyncOutput.Push(TStream(TAsyncProcess(fProcess).Output),Count);
+ sleep(0);
+ until Count=0;
end;
Makes sense. Why the sleep(0)?
Maybe the buffersize can also be increased?
In fact if you make everything eventdriven, then you could introduce
some method to mark events as thread-save, and the event-loop could
automatically deploy threads, if a lot of events occur => that would
be cool.
It is not clear to me why we can't check NumBytesAvailable (available
since fpc 2.2.2 or earlier) on unix and windows and just forget about
TAsyncProcess for compiling?
NumButesAvailable seems to block on TProcess.
From my understanding on a none blocking socket/pipe you can call all
the operations like read/write/peek and they will always return immediately.
On a blocking socket they will always wait until data is there.
On a blocking socket one has to call select (fpselect), which should
indicate the availabilty of data without blocking.
But Imho we should avoid sitting in a loop and sleeping.
That is what TASyncProcess.OnReadData is there fore. Read and handle all
data in the event, exit the event and wait for the event to fire again.
Then you can even to find in files in parallel with compiling (it wil be
slow, since both access the disk) and it will work.
Martin
--
_______________________________________________
Lazarus mailing list
Lazarus@lists.lazarus.freepascal.org
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
