Re: [fpc-pascal] Using Serial in a TCP/RS232 gateway, how to set buffer sizes?

[wkitty42]

On 09/08/2017 04:34 AM, Bo Berglund wrote:

Next I unchecked the checkbox before doing the long transfer, and
lo-and-behold(!) now there are no losses anymore!

So the action of writing text to the listbox in the LogHex function
was actually causing the application to lose incoming serial data!



this tells you right off that you should have at least XON/XOFF flow control so 
the receiver can tell the sender to hold up a second and let it get the buffer 
worked down... at least this with good buffer level monitoring...



--
 NOTE: No off-list assistance is given without prior approval.
   *Please keep mailing list traffic on the list unless*
   *a signed and pre-paid contract is in effect with us.*
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Using Serial in a TCP/RS232 gateway, how to set buffer sizes?

[Bo Berglund]

On Fri, 8 Sep 2017 13:06:16 +0100, el es
 wrote:

>
>I would decouple the log display from the serial handling routines -
>by delegating the serial task to a thread, only display of log is handled in
>the main thread (this is true whether your application is cli-only or gui - 
>have the tasks that should not be interrupted, like bulk serial communication, 
>handled by a thread, and whatever the thread wants to display, use a buffer 
>(like TThreadList) to
>communicate to the main thread.
>

Yes, when I saw this I figured that the data handling should be done
in a thread.
However, I am not at all sure how this can be accomplished.
Basically all of the activity is handled inside my RelayData()
function. This is where the Tcp and RS232 data are crossfed and also
where I call the logging function.

But if I create a thread and put this function into it, how would that
work concerning access to the IdTcpClient and Serial objects? They
were set up and connected in the main application and nothing but the
actual data handling needs to be put inside the thread if it can
communicate the logging info in a way that makes it non-disturbed.

But how?
I have 4 TStaticText controls that show the accumulated byte counts in
the Rx and Tx channels of the socket and comport. These are updated in
RelayData(). And of course I have the listbox displaying the data
going back and forth. I think that the problem is in the listbox,
which will get more and more data as time goes by...
Probably I should have some other logging/display system here.

I have very limited experience with threads and then these did not
interact at all with any GUI functions.

-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Using Serial in a TCP/RS232 gateway, how to set buffer sizes?

[el es]

On 08/09/17 09:34, Bo Berglund wrote:

> 6) Then I rebuilt my application, but I have no idea if this actually
> caused any buffer change. I know too little about the inner workings
> of the Lazarus/FPC system...
> 
> So it might still use the tiny 2K buffer???
> 
> 

I would decouple the log display from the serial handling routines -
by delegating the serial task to a thread, only display of log is handled in
the main thread (this is true whether your application is cli-only or gui - 
have the tasks that should not be interrupted, like bulk serial communication, 
handled by a thread, and whatever the thread wants to display, use a buffer 
(like TThreadList) to
communicate to the main thread.

-L.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] Using Serial in a TCP/RS232 gateway, how to set buffer sizes?

[Bo Berglund]

On Thu, 07 Sep 2017 13:48:27 +0200, Bo Berglund
 wrote:

>I think I will have to create a bogus image file where I have a
>regular pattern of bytes so I can see when the hickups occur in the
>log. In a normal memory file most of the data is zero, so there are
>long stretches of zero bytes sent. Therefore it is not easy to see
>where the losses occur, I only see that the last part of the file
>(which contains some data) has been displaced to lower addresses.

I did so and discovered that loss of data started after some 48-50
Kbytes had been transferred. At that point a few bytes were missing.
But the losses got progressively worse as the transfer went on and in
the end there was about 50 K lost.

Then I tested a few things (like adding delays etc) but with no real
difference.

FINALLY FOUND A SOLUTION...

Finally I realized that the relayer was very busy during the transfer
since it was displaying the relayed data in a log window (a listbox) I
had put on the screen!

In the RelayData procedure there are two calls like this for the two
data directions:

  LogHex('Rx', Buf); //Incoming TCP/IP data
and
  LogHex('Tx', Buf); //Incoming RS232 data

The LogHex procedure does this:

procedure TfrmMain.LogHex(Prefix: AnsiString; var Buf: TIdBytes);
var
  i, l: integer;
  sLine,
  sTime: AnsiString;
begin
  sTime := FormatDateTime('-mm-dd hh:nn:ss.zzz', Now()) + '  ';
  l := Length(Buf);
  sLine := Prefix + ': ';
  for i := 0 to l-1 do
  begin
sLine := sLine + IntToHex(Buf[i],2);
  end;
  lbxLog.Items.Add(sTime + ' ' + sLine);
  lbxLog.ItemIndex := lbxLog.Items.Count-1;
end;

Then I added a checkbox on the application form to switch on/off this
logging to the listbox.
I put this check as the first line in the LogHex function:

  if not ckbShowLogData.Checked then exit;

Next I unchecked the checkbox before doing the long transfer, and
lo-and-behold(!) now there are no losses anymore!

So the action of writing text to the listbox in the LogHex function
was actually causing the application to lose incoming serial data!

Serial buffer size?
---
I thought that upping the serial buffers to 60 K would have helped but
apparently not. Maybe the change I did was not used after all?

What I did regarding buffer change:
1) Rightclicked SerSetParams() and selected "Find declaration"
2) This brought up the serial.pp unit at the declaration
3) Used Ctrl-Shft-downarrow to find the implementation
4) This had to be repeated for SerSetParamsPrivate()
5) In this function body the first line was this:
const   bufSize= 2048;

which I changed to:
const   bufSize= 61440;

6) Then I rebuilt my application, but I have no idea if this actually
caused any buffer change. I know too little about the inner workings
of the Lazarus/FPC system...

So it might still use the tiny 2K buffer???


-- 
Bo Berglund
Developer in Sweden

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal