Re: [twsocket] udp packet loss

2011-03-01 Thread emanuele bizzarri
Hi Francois, hi all

 I cannot reproduce the packet loss on localhost.

Strange. I always lose some packets on localhost (at least setting
Interval=0 on the client).

 I can reproduce on different computers, but I found a flaw in your
 design: remember TWSocket use non blocking. SendTo will fail if winsock
 is not able to receive data and you don't check that condition.

Yes you're right. Now I've modified the code like this:

procedure TForm1.OnTimer(aSender:TObject);
begin
  if fWS.State=wsConnected then
  begin
move(fCounter,fData^,4);
if (fWS.SendTo(fPeerSrc,fPeerSrcLen,fData,fDataSize)0)or
   (WSocket_WSAGetLastError=WSAEWOULDBLOCK)
then
  inc(fSent);
  end;
end;

procedure TForm1.WSocketDataSent(Sender: TObject; ErrCode: Word);
begin
  if fSent0 then
  begin
dec(fSent);
inc(fCounter);
inc(fBR,fDataSize);
Restart;
  end;
end;

Is it right? This is like I do in my real application.

 I'm not sure you correctly checked with wireshark that all packets where
 sent actually because their aren't when SendTo fails.

I'm pretty sure yes.
I've set internal packet number equal to wireshark packet number. Inside
wireshark, if I select one lost packet by the server application, I can
see that internal packet number corresponds.
So I think that this client side bug is not the cause of packet loss.
However I've fixed it.

Yesterday I have done some tests. I've tried different computers.
I noticed that:
On localhost I can reach very different max bitrates. From about 90Mbps
under winXp (cpu=T2300@1.66GHz), until 250Mbps under win7
(cpu=I72630QM@2.00GHz), but always some packet are lost.

On different machines, connected with a cross cable and Gbps ethernet
cards (I've installed the udp server over Q9300 2,5GHz machine, using
different operating systems), max bitrate is always about 160Mbps but:
under winXp a lot of packets are lost.
under win7 less packets are lost than using winXp.
under ubuntu+wine less packets are lost than using winXp and about the
same as win7.

If I open wireshark many more packets are lost by the application (not
by wireshark).

I noticed that if I set:
SetPriorityClass (GetCurrentProcess, HIGH_PRIORITY_CLASS);
and
fUDPServer.Priority:=tpNormal

less packets are lost (no improvement if I set also
fUDPServer.Priority:=tpTimeCritical);

I've tried to compile a third party example of udp server that use
winsock in a different way:
http://www.tenouk.com/Winsock/Winsock2example9.html
under windows and under linux, but the results are like the same.

I've also tried Indy project, but no improvement has been obtained.


I don't know if my tests are completely correct, but
my conclusion is that the mechanism of messaging notification used by
windows can create a udp rx bottleneck in some circumstances (system
wide and not only application wide).

In my real application (where bitrates are two orders of magnitude
lower) I've create a thread pool to manage incoming and outcoming data.
Data are transferred by threads using async queues (that use win
messages) with skip data mechanism where possible.
The result is that, however, udp rx is very (too much) sensible to any
action done on the system.
Now I'm working to reduce this sensibility. I can accept any other kind
of compromise, but I'd like that if udp packets phisically arrive on the
machine they were not discarded.


 var
  lBuffer:array[0..1500] of AnsiChar;
 I'd advice you not to allocate static buffer inside a method because
it is placed inside the stack every time method is called.


yes you're right. In my real application buffers are statically allocated.
I'm going to modify also the example like you say.


Thankyou for help,
Emanuele
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-03-01 Thread Francois PIETTE

Is it right? This is like I do in my real application.


IMO it isn't. If sending full speed, forget the timer and only use the 
event.
When using the timer, check a flag you set in OnDataSent. if falg not set, 
do not send anything, just exit the timer event handler, data will be sent 
on next tick.



Yesterday I have done some tests. I've tried different computers.
I noticed that:
On localhost I can reach very different max bitrates. From about 90Mbps
under winXp (cpu=T2300@1.66GHz), until 250Mbps under win7
(cpu=I72630QM@2.00GHz), but always some packet are lost.


In my opinion, sending back to back UDP packet will almost always result in 
packet lost. This is because the thread receiving data could be suspended 
for at last 20 mS or even much much more (Windows is not a real time OS). 
Winsock buffer must be large enough to buffer all data while thread is 
suspended and even larger since it has to somehow empty the buffer (Remember 
UDP has no flow control). You can set winsock to use a larger buffer 
(default is 8KB is memory serve me well).



If I open wireshark many more packets are lost by the application
(not by wireshark).


Wireshark do not loose packet because it has a Windows Driver running in 
kernel mode and working using interrupt, packets are buffered in memory and 
that buffer is displayed by the GUI independently (just like you are 
browsing a database). Of course you can do that as well, but not with ICS 
which only works in usermode.


--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be



- Original Message - 
From: emanuele bizzarri e.bizza...@e-works.it

To: ICS support mailing twsocket@elists.org
Sent: Tuesday, March 01, 2011 10:11 AM
Subject: Re: [twsocket] udp packet loss



Hi Francois, hi all


I cannot reproduce the packet loss on localhost.


Strange. I always lose some packets on localhost (at least setting
Interval=0 on the client).


I can reproduce on different computers, but I found a flaw in your
design: remember TWSocket use non blocking. SendTo will fail if winsock
is not able to receive data and you don't check that condition.


Yes you're right. Now I've modified the code like this:

procedure TForm1.OnTimer(aSender:TObject);
begin
 if fWS.State=wsConnected then
 begin
   move(fCounter,fData^,4);
   if (fWS.SendTo(fPeerSrc,fPeerSrcLen,fData,fDataSize)0)or
  (WSocket_WSAGetLastError=WSAEWOULDBLOCK)
   then
 inc(fSent);
 end;
end;

procedure TForm1.WSocketDataSent(Sender: TObject; ErrCode: Word);
begin
 if fSent0 then
 begin
   dec(fSent);
   inc(fCounter);
   inc(fBR,fDataSize);
   Restart;
 end;
end;

Is it right? This is like I do in my real application.


I'm not sure you correctly checked with wireshark that all packets where
sent actually because their aren't when SendTo fails.


I'm pretty sure yes.
I've set internal packet number equal to wireshark packet number. Inside
wireshark, if I select one lost packet by the server application, I can
see that internal packet number corresponds.
So I think that this client side bug is not the cause of packet loss.
However I've fixed it.

Yesterday I have done some tests. I've tried different computers.
I noticed that:
On localhost I can reach very different max bitrates. From about 90Mbps
under winXp (cpu=T2300@1.66GHz), until 250Mbps under win7
(cpu=I72630QM@2.00GHz), but always some packet are lost.

On different machines, connected with a cross cable and Gbps ethernet
cards (I've installed the udp server over Q9300 2,5GHz machine, using
different operating systems), max bitrate is always about 160Mbps but:
under winXp a lot of packets are lost.
under win7 less packets are lost than using winXp.
under ubuntu+wine less packets are lost than using winXp and about the
same as win7.

If I open wireshark many more packets are lost by the application (not
by wireshark).

I noticed that if I set:
SetPriorityClass (GetCurrentProcess, HIGH_PRIORITY_CLASS);
and
fUDPServer.Priority:=tpNormal

less packets are lost (no improvement if I set also
fUDPServer.Priority:=tpTimeCritical);

I've tried to compile a third party example of udp server that use
winsock in a different way:
http://www.tenouk.com/Winsock/Winsock2example9.html
under windows and under linux, but the results are like the same.

I've also tried Indy project, but no improvement has been obtained.


I don't know if my tests are completely correct, but
my conclusion is that the mechanism of messaging notification used by
windows can create a udp rx bottleneck in some circumstances (system
wide and not only application wide).

In my real application (where bitrates are two orders of magnitude
lower) I've create a thread pool to manage incoming and outcoming data.
Data are transferred by threads using async queues (that use win
messages) with skip data mechanism where possible.
The result is that, however, udp rx is very (too much

Re: [twsocket] udp packet loss

2011-02-28 Thread Anton S.
var
  lBuffer:array[0..1500] of AnsiChar;
I'd advice you not to allocate static buffer inside a method because it is 
placed inside the stack every time method is called.

-- 
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-27 Thread emanuele bizzarri
Hi,

 I'll assume you use a direct cable or put wireshark at the receiving
 computer because sending a packet is not a problem. Having the packet
 reaching the network card of the receiving computer is a different thing
 if there is something in between.

Yes, I use a cross cable and two laptop with 1gbps ethernet cards
(CPU1=T8100@2.10GHz CPU2=T2300@1.66GHz).
I use wireshark on the udp server machine in this way:
1. Start wireshark on port 9000 (udp server is listening on it).
2. Start udp server (expected packet number is set to 1)
3. Start udp client on the client machine (start sending packets, first
packet number=1)
4. When udp server signal packet loss stop the client.
5. If packet loss is not signalled try to resize the udp server form, or
reduce time interval on the client (min=0 max speed).
5. Search inside wireshark the expected lost packet: I can see the
packet (the first 4 bytes correspond to the wireshark packet number).

If I set Interval=0 (max speed) on the client, the bitrate exceeds
100Mbps, but some packets are lost.

 It is not the messages which interfere, but GUI message processing can
 be very slow compared to network messages and while a GUI message is
 processed, no other message of the same thread can be processed.
 
 So I've created a worker thread that
 manage socket in its own message loop.
 
 That's perfect. But be sure to avoid using Synchronize or to block your
 thread while the GUI handle your messages. You really have the thread
 pumping messages as fast as possible. Just read the UDP socket and put
 the message in a queue for the GUI thread, then signal the GUI that the
 queue has a new message and let the thread pump the next message.

 Look at how your worker thread handle messages. As I said above, it has
 to be as fast as possible. Just receive the data into some kind of queue
 and signal it to the GUI thread without ever waiting for the GUI thread
 to handle the data.

 Probably not. At least you have to pass data to the GUI thread.


This is the windows procedure of the udp server thread:

procedure TUDPServer.ThreadWndProc(var aMessage:TMessage);
begin
 if aMessage.Msg=WM_UDP_LISTEN then
  begin
fExpected:=1;
fWS.Listen;
  end;
  if aMessage.Msg=WM_UDP_CLOSE then
  begin
fWS.Close;
  end;
  inherited ThreadWndProc(aMessage);
end;


and this is the DataAvailable of the socket:

procedure TUDPServer.WSocketDataAvailable(Sender: TObject; Error: Word);
var
  lBuffer:array[0..1500] of AnsiChar;
  lLen:integer;
  lSrc:TSockAddrIn;
  lSrcLen:integer;
  lRx:integer;
begin
  lSrcLen:=SizeOf(lSrc);
  lLen:=fWS.ReceiveFrom(@lBuffer,SizeOf(lBuffer),lSrc,lSrcLen);
  if lLen=0 then
  begin
inc(pBR,lLen);
move(lBuffer[0],lRx,4);
if lRx fExpected then
  PostMessage(self.fHandle,WM_UDP_DATA,lRx,fExpected);
fExpected:=lRx+1;
if lLenfExpectedSize then
  PostMessage(self.fHandle,WM_UDP_SIZE,lLen,fExpectedSize);
  end;
end;

If the thread detects a packet loss or a packet size error (size error
never happened) it sends a message to the GUI thread (self.fHandle).
No other operation is done.

 One last note: Disable any firewall and security product to do your
 testing. Many of those security products are trapping network traffic
 and can slow down thruput and may have bugs. So in case of difficulties
 like you have, it is better to disable everything and use a bare bone
 clean computer setup. Of course later you'll turn security back on.

Yes I have no firewall.

Another thing:
I have tried using udp client/server on the same computer, and in this
case the packet loss is reached only trasmitting at the max speed
possible, but:
1. Wireshark doesn't capture packets on the same machine.
2. The cpu goes very high.
So I think that packet loss is possible in this situation. I prefere to
use two separated laptops.

My client/server project size is only 12KB, is it possible to attach it
to the mail? Someone could try it...
If you think that could be a good example in order to test network udp
performance (and it is not buggy), it could be added to the user made
section of the ics site.

Thank you,

Emanuele
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-27 Thread Francois PIETTE

My client/server project size is only 12KB, is it possible to attach it
to the mail? Someone could try it...


It don't primize, but you may send the source code to me (no executable 
please) I will try to test it. Include complete projects so that I can test 
within a few minutes, and instructions to reproduce the issue.


--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-27 Thread emanuele bizzarri
Hi,
now I've made a console application for the UDP server, but nothing changes.
I see packet loss inside application, but all packets are correctly
captured by wireshark.

Thankyou,
Emanuele

Il 27/02/2011 15.09, Francois PIETTE ha scritto:
 My client/server project size is only 12KB, is it possible to attach it
 to the mail? Someone could try it...
 
 It don't primize, but you may send the source code to me (no executable
 please) I will try to test it. Include complete projects so that I can
 test within a few minutes, and instructions to reproduce the issue.
 
 -- 
 francois.pie...@overbyte.be
 The author of the freeware multi-tier middleware MidWare
 The author of the freeware Internet Component Suite (ICS)
 http://www.overbyte.be
 
 -- 
 To unsubscribe or change your settings for TWSocket mailing list
 please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
 Visit our website at http://www.overbyte.be

-- 
Ing. Emanuele Bizzarri
Software Development Department
e-works s.r.l.
41011 - Campogalliano - Modena - Italy
tel. +39 059 2929081 int. 23
fax +39 059 2925035

e-mail: e.bizza...@e-works.it - http://www.e-works.it
-
La presente comunicazione, che potrebbe contenere informazioni riservate
e/o protette da segreto professionale, è indirizzata esclusivamente ai
destinatari della medesima qui indicati. Le opinioni, le conclusioni e
le altre informazioni qui contenute, che non siano relative alla nostra
attività caratteristica, devono essere considerate come non inviate né
avvalorate da noi. Tutti i pareri e le informazioni qui contenuti sono
soggetti ai termini ed alle condizioni previsti dagli accordi che
regolano il nostro rapporto con il cliente. Nel caso in cui abbiate
ricevuto per errore la presente comunicazione, vogliate cortesemente
darcene immediata notizia, rispondendo a questo stesso indirizzo di
e-mail, e poi procedere alla cancellazione di questo messaggio dal
Vostro sistema. E' strettamente proibito e potrebbe essere fonte di
violazione di legge qualsiasi uso, comunicazione, copia o diffusione dei
contenuti di questa comunicazione da parte di chi la abbia ricevuta per
errore o in violazione degli scopi della presente.
-
This communication, that may contain confidential and/or legally
privileged information, is intended solely for the use of the intended
addressees. Opinions, conclusions and other information contained in
this message, that do not relate to the official business of this firm,
shall be considered as not given or endorsed by it. Every opinion or
advice contained in this communication is subject to the terms and
conditions provided by the agreement governing the engagement with such
a client. If you have received this communication in error, please
notify us immediately by responding to this email and then delete it
from your system. Any use, disclosure, copying or distribution of the
contents of this communication by a not-intended recipient or in
violation of the purposes of this communication is strictly prohibited
and may be unlawful.

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-27 Thread Francois PIETTE

Hi,

I have received your code and looking at it.
I cannot reproduce the packet loss on localhost.
I can reproduce on different computers, but I found a flaw in your design: 
remember TWSocket use non blocking. SendTo will fail if winsock is not able 
to receive data and you don't check that condition.

Here is how I found the problem:
procedure TForm1.OnTimer(aSender:TObject);
begin
 if fWS.State=wsConnected then
 begin
   move(fCounter,fData^,4);
   if fWS.SendTo(fPeerSrc,fPeerSrcLen,fData,fDataSize) = fDataSize then 
 test changed

   begin
 inc(fCounter);
 inc(fBR,fDataSize);
   end
   else
   ShowMessage('send failed'); 
message displayed

   Restart;
 end;
end;

You can't call SendTo as fast as you like when the socket is using async 
mode.

Before sending, you should check if the event OnDataSent has been triggered.
I'm not sure you correctly checked with wireshark that all packets where 
sent actually because their aren't when SendTo fails.


--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be



- Original Message - 
From: emanuele bizzarri e.bizza...@e-works.it

To: twsocket@elists.org
Sent: Sunday, February 27, 2011 7:05 PM
Subject: Re: [twsocket] udp packet loss


Hi,
now I've made a console application for the UDP server, but nothing changes.
I see packet loss inside application, but all packets are correctly
captured by wireshark.

Thankyou,
Emanuele

Il 27/02/2011 15.09, Francois PIETTE ha scritto:

My client/server project size is only 12KB, is it possible to attach it
to the mail? Someone could try it...


It don't primize, but you may send the source code to me (no executable
please) I will try to test it. Include complete projects so that I can
test within a few minutes, and instructions to reproduce the issue.

--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be


--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-26 Thread Francois PIETTE

I've a problem with udp.
Often my application detects packet losses.


At first glance, this is expected with UDP. By construction UDP is an 
unreliable transport.



In order to systematically reproduce the problem, I've used the
following configuration.
I use 2 pc connected with a cross cable.
The client sends udp numbered packets to the server.
Packet size is 1460. One packet is sent every 1 ms (about 11 Mbps).

I use wireshark to analyze the traffic on the server machine.
I see that all the packets arrive on the server, but not inside the
application.

Packet loss increases if I resize the form, so I've made a worker thread
in order to manage udp server socket messages, but the problem remains.


This is normal. If the application is not able to read UDP packets as fast 
as packets are incomming, they are simply dropped. There is no flow control 
with UDP. This is how UDP is working.



Have you any suggestion to resolve this problem?


The easiest way is to use TCP which is a reliable protocol, with flow 
control, retries and everything needed to make it reliable.



Maybe the socket configuration should be changed?


Won't change anything.
If you still want to use UDP, you may use a dedicated thread to handle 
communication and give it a high priority so that it will receive CPU before 
the user interface.




I attach the code of server, client and worker thread:


Sorry no time to read your code. i just browsed quickly and see you don't 
create TWSocket in thread's execute method, so all events are handled by the 
main thread. This is a flaw in your design, at least.


--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-26 Thread emanuele bizzarri
Hi Francois,
thank you for your response.
I've created the socket inside DoInit, that is an override method of the
virtual DoInit of the worker thread.
The DoInit method is called inside Execute method, so the socket is
created by the thread and its messages are handled by it (try to use
GetThreadID inside socket callbacks).
As you can see in my code I've also try to set tpTimeCritical to thread
priority, but without any success.
The worker thread check if expected packet number (first four bytes)
corresponds to expected packet number.
In my example I think that the gui should not interfere with worker thread.

I can't use tcp in my application, I need udp.

Any ideas?

Thank you,
Emanuele



Il 26/02/2011 16.29, Francois PIETTE ha scritto:
 I've a problem with udp.
 Often my application detects packet losses.
 
 At first glance, this is expected with UDP. By construction UDP is an
 unreliable transport.
 
 In order to systematically reproduce the problem, I've used the
 following configuration.
 I use 2 pc connected with a cross cable.
 The client sends udp numbered packets to the server.
 Packet size is 1460. One packet is sent every 1 ms (about 11 Mbps).

 I use wireshark to analyze the traffic on the server machine.
 I see that all the packets arrive on the server, but not inside the
 application.

 Packet loss increases if I resize the form, so I've made a worker thread
 in order to manage udp server socket messages, but the problem remains.
 
 This is normal. If the application is not able to read UDP packets as
 fast as packets are incomming, they are simply dropped. There is no flow
 control with UDP. This is how UDP is working.
 
 Have you any suggestion to resolve this problem?
 
 The easiest way is to use TCP which is a reliable protocol, with flow
 control, retries and everything needed to make it reliable.
 
 Maybe the socket configuration should be changed?
 
 Won't change anything.
 If you still want to use UDP, you may use a dedicated thread to handle
 communication and give it a high priority so that it will receive CPU
 before the user interface.
 
 
 I attach the code of server, client and worker thread:
 
 Sorry no time to read your code. i just browsed quickly and see you
 don't create TWSocket in thread's execute method, so all events are
 handled by the main thread. This is a flaw in your design, at least.
 
 -- 
 francois.pie...@overbyte.be
 The author of the freeware multi-tier middleware MidWare
 The author of the freeware Internet Component Suite (ICS)
 http://www.overbyte.be
 
 -- 
 To unsubscribe or change your settings for TWSocket mailing list
 please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
 Visit our website at http://www.overbyte.be

-- 
Ing. Emanuele Bizzarri
Software Development Department
e-works s.r.l.
41011 - Campogalliano - Modena - Italy
tel. +39 059 2929081 int. 23
fax +39 059 2925035

e-mail: e.bizza...@e-works.it - http://www.e-works.it
-
La presente comunicazione, che potrebbe contenere informazioni riservate
e/o protette da segreto professionale, è indirizzata esclusivamente ai
destinatari della medesima qui indicati. Le opinioni, le conclusioni e
le altre informazioni qui contenute, che non siano relative alla nostra
attività caratteristica, devono essere considerate come non inviate né
avvalorate da noi. Tutti i pareri e le informazioni qui contenuti sono
soggetti ai termini ed alle condizioni previsti dagli accordi che
regolano il nostro rapporto con il cliente. Nel caso in cui abbiate
ricevuto per errore la presente comunicazione, vogliate cortesemente
darcene immediata notizia, rispondendo a questo stesso indirizzo di
e-mail, e poi procedere alla cancellazione di questo messaggio dal
Vostro sistema. E' strettamente proibito e potrebbe essere fonte di
violazione di legge qualsiasi uso, comunicazione, copia o diffusione dei
contenuti di questa comunicazione da parte di chi la abbia ricevuta per
errore o in violazione degli scopi della presente.
-
This communication, that may contain confidential and/or legally
privileged information, is intended solely for the use of the intended
addressees. Opinions, conclusions and other information contained in
this message, that do not relate to the official business of this firm,
shall be considered as not given or endorsed by it. Every opinion or
advice contained in this communication is subject to the terms and
conditions provided by the agreement governing the engagement with such
a client. If you have received this communication in error, please
notify us immediately by responding to this email and then delete it
from your system. Any use, disclosure, copying or distribution of the
contents of this communication by a not-intended recipient or in
violation of the purposes of this communication is strictly prohibited
and may be unlawful.

--
To unsubscribe or 

Re: [twsocket] udp packet loss

2011-02-26 Thread Angus Robertson - Magenta Systems Ltd
 As you can see in my code I've also try to set tpTimeCritical to 
 thread priority, but without any success.

Try changing the application priority:

SetPriorityClass (GetCurrentProcess, HIGH_PRIORITY_CLASS); 

or even REALTIME_PRIORITY_CLASS, but this last one can be dangerous since
Windows may lock up if your application goes wild. 

Angus


--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-26 Thread Francois PIETTE

As you can see


Sorry, no time to examine it in details.
Priority is given to my business customers...


in my code I've also try to set tpTimeCritical to thread
priority, but without any success.



The worker thread check if expected packet number (first four bytes)
corresponds to expected packet number.
I can't use tcp in my application, I need udp.


You software, when using UDP, must be preapred to packet loss, duplicate 
packets and packet in incorrect order. If it is not possible in your case, 
you can't use UDP. UDP is a simple packet protocol with only best delivery 
effort. This is not an issue with ICS, it is simple how UDP works.


In my example I think that the gui should not interfere with worker 
thread.

Any ideas?


If you loose packet, then your software, or the whole computer is not fast 
enough to handle all packets as they are delivered. Buy a faster computer or 
optimize the software.


Of course also check if your network card is OK and setup for 100 Mbps.


I can't use tcp in my application, I need udp.


Why do you think you must use UDP ?
Why isn't TCP good for you ?

--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be


Re: [twsocket] udp packet loss

2011-02-26 Thread emanuele bizzarri
Hi,
I need to use udp because my application is an h323/sip endpoint.
RTP uses udp sockets to transmit/receive audio/video data.
RTCP can be used to manage packet loss.

But in this case packets are not lost by the network. If I use wireshark
I can see all the packets.

I think that there is a problem in my code. I know that gui messages can
interfere with socket messages. So I've created a worker thread that
manage socket in its own message loop.

Could be the problem in the way I create/use the message loop in worker
thread?
This is the execute method (DoInit create the socket):

procedure TWorkerThread.Execute;
begin
  DoInit;
  MessageLoop;
end;

Inside the message loop method, I allocate fThreadHandle and then begin
dispatch messages.

procedure TWorkerThread.MessageLoop;
var
  lMsg:TMsg;
begin
  fThreadHandle:=AllocateHWnd(ThreadWndProc);
  PeekMessage(lMsg,0,0,0,PM_NOREMOVE);
  SetEvent(fResumeEvent);
  //GetMessage return false on WM_QUIT
  while (GetMessage(lMsg,0,0,0))and(not self.Terminated) do
  begin
TranslateMessage(lMsg);
DispatchMessage(lMsg);
  end;
  DeallocateHWnd(fThreadHandle);
end;

The DoInit method is called before AllocateHWnd, so the socket is
created before. Could be here the problem?

It seems that the thread is not fast enought to elaborate socket
messages. Or that the gui slow the worker thread in some way. If I call
GetCurrentThreadID inside the DataAvailable the result is the id of the
worker thread.
The only thing done inside the DataAvailable callback is check packet
number.
The cpu is used about 12%.

I've tried to set the process priority (thank you Angus), but nothig
changes.

What kind of performance do you reach using udp sockets? I think that my
performance are not satisfactory.

Thank you for your help.

Emanuele




Il 26/02/2011 21.15, Francois PIETTE ha scritto:
 As you can see
 
 Sorry, no time to examine it in details.
 Priority is given to my business customers...
 
 in my code I've also try to set tpTimeCritical to thread
 priority, but without any success.
 
 The worker thread check if expected packet number (first four bytes)
 corresponds to expected packet number.
 I can't use tcp in my application, I need udp.
 
 You software, when using UDP, must be preapred to packet loss, duplicate
 packets and packet in incorrect order. If it is not possible in your
 case, you can't use UDP. UDP is a simple packet protocol with only best
 delivery effort. This is not an issue with ICS, it is simple how UDP works.
 
 In my example I think that the gui should not interfere with worker
 thread.
 Any ideas?
 
 If you loose packet, then your software, or the whole computer is not
 fast enough to handle all packets as they are delivered. Buy a faster
 computer or optimize the software.
 
 Of course also check if your network card is OK and setup for 100 Mbps.
 
 I can't use tcp in my application, I need udp.
 
 Why do you think you must use UDP ?
 Why isn't TCP good for you ?
 
 -- 
 francois.pie...@overbyte.be
 The author of the freeware multi-tier middleware MidWare
 The author of the freeware Internet Component Suite (ICS)
 http://www.overbyte.be
 
 -- 
 To unsubscribe or change your settings for TWSocket mailing list
 please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
 Visit our website at http://www.overbyte.be

-- 
Ing. Emanuele Bizzarri
Software Development Department
e-works s.r.l.
41011 - Campogalliano - Modena - Italy
tel. +39 059 2929081 int. 23
fax +39 059 2925035

e-mail: e.bizza...@e-works.it - http://www.e-works.it
-
La presente comunicazione, che potrebbe contenere informazioni riservate
e/o protette da segreto professionale, è indirizzata esclusivamente ai
destinatari della medesima qui indicati. Le opinioni, le conclusioni e
le altre informazioni qui contenute, che non siano relative alla nostra
attività caratteristica, devono essere considerate come non inviate né
avvalorate da noi. Tutti i pareri e le informazioni qui contenuti sono
soggetti ai termini ed alle condizioni previsti dagli accordi che
regolano il nostro rapporto con il cliente. Nel caso in cui abbiate
ricevuto per errore la presente comunicazione, vogliate cortesemente
darcene immediata notizia, rispondendo a questo stesso indirizzo di
e-mail, e poi procedere alla cancellazione di questo messaggio dal
Vostro sistema. E' strettamente proibito e potrebbe essere fonte di
violazione di legge qualsiasi uso, comunicazione, copia o diffusione dei
contenuti di questa comunicazione da parte di chi la abbia ricevuta per
errore o in violazione degli scopi della presente.
-
This communication, that may contain confidential and/or legally
privileged information, is intended solely for the use of the intended
addressees. Opinions, conclusions and other information contained in
this message, that do not relate to the official business of this 

Re: [twsocket] udp packet loss

2011-02-26 Thread Francois PIETTE

I need to use udp because my application is an h323/sip endpoint.
RTP uses udp sockets to transmit/receive audio/video data.
RTCP can be used to manage packet loss.


OK, good.


But in this case packets are not lost by the network. If I use wireshark
I can see all the packets.


I'll assume you use a direct cable or put wireshark at the receiving 
computer because sending a packet is not a problem. Having the packet 
reaching the network card of the receiving computer is a different thing if 
there is something in between.


Have you checked your network card is operating at 100 Mbps ?


I think that there is a problem in my code.


Probably. If your network was bad, you would already know it.


I know that gui messages can interfere with socket messages.


It is not the messages which interfere, but GUI message processing can be 
very slow compared to network messages and while a GUI message is processed, 
no other message of the same thread can be processed.



So I've created a worker thread that
manage socket in its own message loop.


That's perfect. But be sure to avoid using Synchronize or to block your 
thread while the GUI handle your messages. You really have the thread 
pumping messages as fast as possible. Just read the UDP socket and put the 
message in a queue for the GUI thread, then signal the GUI that the queue 
has a new message and let the thread pump the next message.



Could be the problem in the way I create/use the message loop in worker
thread?
This is the execute method (DoInit create the socket):


Your code looks OK.


The DoInit method is called before AllocateHWnd, so the socket is
created before. Could be here the problem?


No, it is OK.


It seems that the thread is not fast enought to elaborate socket
messages.


Look at how your worker thread handle messages. As I said above, it has to 
be as fast as possible. Just receive the data into some kind of queue and 
signal it to the GUI thread without ever waiting for the GUI thread to 
handle the data.



Or that the gui slow the worker thread in some way.


If the GUI thread is CPU bound, then it will hog most of the CPU unless the 
workerthread has a higher priority.



If I call GetCurrentThreadID inside the DataAvailable the result
is the id of the worker thread.


So the socket is correctly created.


The only thing done inside the DataAvailable callback is check packet
number.


Probably not. At least you have to pass data to the GUI thread.



The cpu is used about 12%.


Good news.


I've tried to set the process priority (thank you Angus), but nothig
changes.



What kind of performance do you reach using udp sockets? I think that my
performance are not satisfactory.


Depending on your hardware, you should be able to handle 100 Mbps and more. 
If you write data to disk, this may slow down things. If you display data, 
this may slow down as well.


You may try a simple program which do nothing withe the data except checing 
the packet number and see how many packets you can receive.


One last note: Disable any firewall and security product to do your testing. 
Many of those security products are trapping network traffic and can slow 
down thruput and may have bugs. So in case of difficulties like you have, it 
is better to disable everything and use a bare bone clean computer setup. Of 
course later you'll turn security back on.


--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be