[twsocket] Shared handles problem
Hello ICS team, I´m currently using your implementation of shared handles in all my applications (I have actually removed all AllocateHwnd and changed to use your implementation in your to share handles between objects and save some OS resources), and I think that I have found a flaw in the system but I´m not sure how to fix. The problem that I´m facing is that Unsent messages are being received sometimes After a lot of debugging and testing I think that I have came to a conclusion: In my tests, I create many objects and destroy them, so, there are many operations to allocate handles (From your lib) and deallocate them What I could observe is that if I create an object (Allocating a shared message from your system), send a message to it and destroy it (deallocating this message), and creating more objects, the message ID that was allocated to a previous object (already freed) is now allocated to a new object, and the message that was posted is still not received.. so, when the main thread actually process the message and tries to dispatch the message, it will be dispatched to this new object, creating a great mess in the software So I´m not expert in your structure.. but could that scenario occur? Thanks! Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] RES: Shared handles problem
Well
I have seen that you UnregisterMessage routine already removes messages from
the queue:
if FHandle 0 then
while PeekMessage(LMsg, FHandle, Msg, Msg, PM_REMOVE) do {loop} ;
Strange.. must be something else, I will keep digging!
Eric
De: Eric Fleming Bonilha [mailto:e...@digifort.com.br]
Enviada em: sexta-feira, 17 de maio de 2013 17:11
Para: twsocket@elists.org
Assunto: Shared handles problem
Hello ICS team,
I´m currently using your implementation of shared handles in all my
applications (I have actually removed all AllocateHwnd and changed to use
your implementation in your to share handles between objects and save some
OS resources), and I think that I have found a flaw in the system but I´m
not sure how to fix.
The problem that I´m facing is that Unsent messages are being received
sometimes
After a lot of debugging and testing I think that I have came to
a conclusion:
In my tests, I create many objects and destroy them, so, there are many
operations to allocate handles (From your lib) and deallocate them
What I
could observe is that if I create an object (Allocating a shared message
from your system), send a message to it and destroy it (deallocating this
message), and creating more objects, the message ID that was allocated to a
previous object (already freed) is now allocated to a new object, and the
message that was posted is still not received.. so, when the main thread
actually process the message and tries to dispatch the message, it will be
dispatched to this new object, creating a great mess in the software
So
I´m not expert in your structure.. but could that scenario occur?
Thanks!
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] ICS Stable for 64bits?
Hello! I would like to know if ICS is already stable for 64bits development. I`m porting my system to 64bits and I updated to ICS8, but I´m having many issues (ICS is in the core of my system) that looks like memory overflow or overwrite, so, I´m trying to find out the problems and I want to eliminate the possibility of errors on 3th party libraries So, is ICS8 already being used on big systems compiled in 64bits? Thanks! Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] UPNP
Dave, I researched on it, and it is really dangerous, but my application just has to locate some UPnP devices on the network (IP Cameras), so, I don´t have any UPnP server to include security faults in my app, just a simple class that sends UPnP search command to the network to locate the devices :) I made it work and it is working very fine, it locates even more equipments than native microsoft implementation (by using upnp.dll that do not work on windows servers) Regards Eric -- From: Dave Baxter d...@uk-ar.co.uk Sent: Tuesday, January 11, 2011 1:22 PM To: ICS support mailing twsocket@elists.org Subject: Re: [twsocket] UPNP Take care. UPnP is dangerous. It's a very common way for a lot of Malware on one PC to open holes in the border firewall/router for it's own (doubtfull) ends. UPnP was a nice idea for consumer stuff, but it opens a whole tanker full of worms if used, hence many (most now?) windows PC's have that feature blocked, and for very good reason! Your router should have it disabled too. Any resultant wails from your kids about the PSXBox or similar not getting on the web, can be fixed with normal router/firewall Port Forwarding, as per many good sites found with Google. So much safer than letting UPnP work. Regards. Dave B. PS: UPnP is not the same as Plug and Play hardware detection, that is a totally different animal. -Original Message- From: Eric Fleming Bonilha [mailto:e...@digifort.com.br] Sent: 07 January 2011 13:59 To: ICS support mailing Subject: Re: [twsocket] UPNP I thought about that too But I found the problem. The LocalAddr (or addr? don´t remember now) must be changed to local IP address, not 0.0.0.0, or else I don´t receive the responses Thanks! Eric -- From: Markus Humm markus.h...@freenet.de Sent: Friday, January 07, 2011 9:20 AM To: twsocket@elists.org Subject: [twsocket] UPNP Hello Eric, could it be that some other UPNP service is still active on your PC which blocks usage of this port? Greetings Markus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be Nenhum virus encontrado nessa mensagem recebida. Verificado por AVG - www.avgbrasil.com.br Versao: 9.0.872 / Banco de dados de virus: 271.1.1/3364 - Data de Lancamento: 01/06/11 16:34:00 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be Nenhum vírus encontrado nessa mensagem recebida. Verificado por AVG - www.avgbrasil.com.br Versão: 9.0.872 / Banco de dados de vírus: 271.1.1/3372 - Data de Lançamento: 01/10/11 16:34:00 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] UPNP
I thought about that too But I found the problem. The LocalAddr (or addr? don´t remember now) must be changed to local IP address, not 0.0.0.0, or else I don´t receive the responses Thanks! Eric -- From: Markus Humm markus.h...@freenet.de Sent: Friday, January 07, 2011 9:20 AM To: twsocket@elists.org Subject: [twsocket] UPNP Hello Eric, could it be that some other UPNP service is still active on your PC which blocks usage of this port? Greetings Markus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be Nenhum virus encontrado nessa mensagem recebida. Verificado por AVG - www.avgbrasil.com.br Versao: 9.0.872 / Banco de dados de virus: 271.1.1/3364 - Data de Lancamento: 01/06/11 16:34:00 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] UPNP Client
Hi I´m researching on UPnP communication and I found that it relies on multicast messages sent to address 239.255.255.250 on port 1900 I tried to receive messages from this multicast address using ICS but it does not work. Using other multicast addresses it works just fine, but I simply can´t receive messages from multicast address 239.255.255.250:1900. Neither using Indy.. I can send messages to this address, and all UPnP devices answers to my request (I saw by sniffer), but I can´t receive it in my software. Do you know if there is any limitation on the under windows sockets layer? Thanks! Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Hi Anton, I call it directly, in order not to allocate handlers, if you create a class with a handle and post a message to it, it will be the same as using TTimer directly, because you will use as many resources as TTimer would use (1 handle per instance), unless you create a class that is derived from TICSWndControl (Which I´m trying to do, but I found a bug in its message handling implementation and I´m waiting for someone to check it), than you can share some resources. I just created this class because I was using many resources and resources are limited ;) But I was thinking on something, and I would like someone to confirm it or explain me better Using for example, 1.000 instances of TTimer at 1 second triggering, the messages will be received and processed *in a loop* at the main thread, wouldn´t be almost the same as having a loop to trigger the event of all ttimers by the shared timer? Eric -- From: Anton S. an...@rambler.ru Sent: Wednesday, August 18, 2010 10:48 AM To: twsocket@elists.org Subject: Re: [twsocket] ICS Message handlers Eric, talking about your shared timer: do you call OnTimer of all attached objects directly or post WM_TIMER to them? I had though over the same timer class and that was my idea to broadcast timer signal. I think it'll eliminate the issue of blocking during loop since PostMessage is asynchronous. But I'm not sure, maybe it'll overload message queues. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be Nenhum virus encontrado nessa mensagem recebida. Verificado por AVG - www.avgbrasil.com.br Versao: 9.0.851 / Banco de dados de virus: 271.1.1/3079 - Data de Lancamento: 08/18/10 03:35:00 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Suggestion for improvement
Hello I would suggest the changing of constant WH_MAX_MSG in OverbyteIcsWndControl.pas to a global variable to which we can change on our software, without changing ICS code. Doing this will allow us to control the amount of shared resources per handle, giving the ability to create more sockets on a software that already uses many system handles Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Wnd Control - Possible bug
HI Arno
True, I think that will help, I will add this line to my ICS code to check
if it works, but I believe that it will work
I also think that this is something that must be on ICS code, on
UnregisterMessage as you proposed, not custom code.
I will test and come back to you soon
Eric
--
From: Arno Garrels arno.garr...@gmx.de
Sent: Wednesday, August 18, 2010 11:43 AM
To: ICS support mailing twsocket@elists.org
Subject: Re: [twsocket] ICS Wnd Control - Possible bug
Eric,
I guess it helps to remove all pending messages with the given ID
from the queue when the message ID is unregistered.
One could do that either in custom code or maybe it should
be added in TIcsWndHandler.UnregisterMessage as below?
procedure TIcsWndHandler.UnregisterMessage(var Msg: UINT);
var
LMsg: TMsg; // == added
begin
if Msg = 0 then
Exit;
if FMsgLow WM_USER then
raise EIcsException.Create('MsgLow not defined');
if Msg = (FMsgLow + WH_MAX_MSG) then
raise EIcsException.Create('Msg value out of bound');
if not Assigned(FMsgMap[Msg - FMsgLow]) then
raise EIcsException.Create('Msg not registered');
FMsgMap[Msg - FMsgLow] := nil;
Dec(FMsgCnt);
Msg := 0;
while PeekMessage(LMsg, FHandle, Msg, Msg, PM_REMOVE) do {loop}; // ==
added
if FMsgCnt = 0 then
DeallocateHWnd;
end;
--
Arno Garrels
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Nenhum virus encontrado nessa mensagem recebida.
Verificado por AVG - www.avgbrasil.com.br
Versao: 9.0.851 / Banco de dados de virus: 271.1.1/3079 - Data de
Lancamento: 08/18/10 03:35:00
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Wnd Control - Possible bug
Msg := 0;
while PeekMessage(LMsg, FHandle, Msg, Msg, PM_REMOVE) do {loop}; // ==
added
if FMsgCnt = 0 then
DeallocateHWnd;
end;
Just a little correction
while PeekMessage(LMsg, FHandle, Msg, Msg, PM_REMOVE) do {loop}; // ==
added
Msg := 0;
Or else you will pass 0 to PeekMessage
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Suggestion for improvement
That won't work since TIcsWndHandler declares a static array with length WH_MAX_MSG. Instead I suggest that TIcsWndHandlerPool gets a public, thread-safe method which checks whether the value can be changed and would raise an exception if not. This method would succeed only if the pool was empty (when no instance of TWndControl existed) and the value passed was in the range 50-1000, anything beyond makes no sense IMO. Setting the maximum number of messages per handler would be just one line: GWndHandlerPool.SetMaxMsgPerHandler(200); The GWndHandlerPool variable is assigned in OverbyteIcsWndControl's initialization section. That is good too, we just need a method to increase the value, 200 is good ;) Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] ICS WndControl
Hi I´m studying TICSWndControl so I can use it as base class for my app. I´m curious about something, you have the constant WH_MAX_MSG = 100, which means that each Handler created can handle up to 100 different messages, from different objects right? Is there a reason to be 100? If I increase this number to 200 for example, will I be able to create the double amount of sockets? Since, the most limiting fact is the handle (10.000 handlers on the process), if I share 1 handle with more classes, theorically I can create many more sockets Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] ICS Wnd Control - Possible bug
Hello all While I was trying to use TICSWndControl as base class to share handles for message and consume less OS resources I ran into a very strange problem, that took me some nights of sleep, after a lot of researching I have found a problem in WndControl shared handles implementation, and I made an example program that shows the problem I will try to explain the situation The main problem is in the management of registered message queue, when you create a new object it will register its message to the message assignment queue by using AllocateMsgHandlers, so I create I component and it will assign the message 1025 to FMsgRelease, if I create another one, it will assign message 1026 to FMsgRelease of the other control If I Post message 1025 to the first component, just after posting I free it, it will release 1025 from the message queue, making it available for use, and then I create a new component (It will assign 1025 for the new control). When the message is received, it will be received by the third component, but I sent the message to the first one! In my demo program I simulate the following situation: I have two types of classes (TMyControl and TMyControl2) On TMyControl I register a test message TMyControl2 is simple descendent of TICSWndControl, the only thing is that it will call AllocateHWnd on constructor, so it will allocate the release message The path to error: - Create one instance of TMyControl (Messages 1025 and 1026 will be reserved for it (1025 = Release, 1026 = Test) - Create another instance of TMyControl (Messages 1027 and 1028 will be reserved for it) - Post message 1026 to first instance (1026 is a test message) - Free first instance of TMyControl (That was supposed to receive the message, but messages 1025 and 1026 will be released and free to be used for another control) - Create an instance of TMyControl2 (Message 1025 will be assigned to it as FMsgRelease) - Create an instance of TMyControl2 (Message 1026 will be assigned to it as FMsgRelease) Ok, when the handle received message 1026 (That was a message supposed to be received by an instance of TMyControl) it will be actually received by the second instance of TMyControl2, which will release the control since 1026 is now mapped to FMsgRelease message!! Do you have any suggestion of fix? I thought that when releasing a message handler we must in some way clean the message queue if there is any message in windows queue that was directed to that component, but how to do it? The link for test application is: www.intertruco.com.br/download/ICSWndError.rar Thank you very much Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Dear Arno, Did you test that in exactly the same environment, same box, both service or GUI applications, both non or RDP sessions etc.. ? I tested on the same box, GUI and service. Non RDP, all local desktop I will provide much more data, I´m preparing a document for you and a sample program, I´m just curious about this findins, your comments are really appreciated Can I send attachment to this group? If not, how do I send? Eric -- From: Arno Garrels arno.garr...@gmx.de Sent: Thursday, August 12, 2010 2:56 PM To: ICS support mailing twsocket@elists.org Subject: Re: [twsocket] ICS Message handlers Eric, I found a problem in our software that is limiting the total ammount of TCP sockets that I can create. Did you test that in exactly the same environment, same box, both service or GUI applications, both non or RDP sessions etc.. ? Would you please provide a small test application that demonstrates this? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be Nenhum virus encontrado nessa mensagem recebida. Verificado por AVG - www.avgbrasil.com.br Versao: 9.0.851 / Banco de dados de virus: 271.1.1/3066 - Data de Lancamento: 08/12/10 03:34:00 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
François Windows limit the number of windows handles to 10.000. That's why ICS V7 is designed to use only one handler for a lot of TWSocket so that you can create a lot of sockets. Which OS are you using which allows creating 1.000.000 handles ? After more testing, and a clear mind of a new day, I found that I was doing some things wrong.. The limit is really 10.000 handles, and if I create 10.000 handles I can´t create any more TWSocket or anything, so you are right, sorry about that Classes.AllocateHwnd and TIcsWndHandler.AllocateHWnd both use the same API (CreateWindowEx ) to create the handle so both should be able to create the same number of those. You are right here too, it doesn´t matter using your allocatehandle or classes.allocateHwnd, the results are the same What I do found is that windows imposes a lesser limit (Less than 2.000 handles) for service applications that runs under LocalSystem account and do not have desktop interaction, so, by the end of the day, the problem was not related to ICS itself, but to windows service management. If you make a simple service application that tries to create 10.000 handles, it won´t create even 2.000, but if you activate the Deskop Interaction flag in the service, than you can create almost 10.000 Sorry about the confusion 10.000 is a huge number and we will never use that amount of resources, it was just for testing, just to make it clear Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
I'm still stuck with Delphi 7 at the office, so I don't know if this has changed during the years. However, looking at the source code of TIcsWndControl.AllocateHWnd(), it appears that it's still locking the resource in order to create new handles. My guess is, no. I think that they lock in order to threads to be able to create the handlers, because as I understood they share the handles between classes, which is very very clever, I liked the design, to reuse handles, that is why I can create 130.000 instances of TWSocket! They share resources when they can, because each instance will have a different message number for the same messages, that is a very smart idea and I will try to use the same thinking in my software. I have already created a class called TSharedTimer, which allows me to create virtually unlimited number of timers, since they will all share the same resource, and it is working pretty fine. I will try to make something similar to ICS to reduce the amount of handles that I will consume in my software, by sharing the handles and defining different ranges of messages per handle Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Hi SubZero,
I hope you received my reply to you and got everything ok between us. I
have
a question for you:
I got it, don´t worry, I wasn´t mad, I was just thinking that I was asking
something obvious hehe, I thought that it was a silly question, since I
don´t have much knowlege on how TWSocket works internally ;)
I have tried to find this finding before here but nobody seemed to know
it.
So it is very interesting. Did you find this by testing or from a
Microsoft
KB? If so, which one?
Humm, it was just testing, and it is pretty easy to test, and it just
happens in service application, so, I don´t really know why does it happens,
here is a simple code for a service application:
InitializeDebugLog;
for i := 0 to 1 do
begin
if AllocateHWnd(WndProc) = 0 then
DebugMessage('Error creating handle - ' + IntToStr(i));
end;
DebugLog is a routine that I have just to write debug messages into a text
file
The result of this application when running in a service as LocalSystem
account without desktop interaction is:
13/08/2010 - 10:48:41: Error creating handle - 2265
13/08/2010 - 10:48:41: Error creating handle - 2266
13/08/2010 - 10:48:41: Error creating handle - 2267
...
13/08/2010 - 10:48:41: Error creating handle - 1
The result of this application when running in a service as LocalSystem
account WITH desktop interaction is:
13/08/2010 - 10:49:44: Error creating handle - 9982
13/08/2010 - 10:49:44: Error creating handle - 9983
13/08/2010 - 10:49:44: Error creating handle - 9984
...
13/08/2010 - 10:49:44: Error creating handle - 1
So, when running without desktop interaction I can create at most 2264
handles, and with desktop interaction I can create 9981 handles!
That is why I can create 139.733 instances of TWSocket in a service app with
desktop interaction and 30.239 instances of TWSocket in a service app
without desktop interaction
Since TWSocket share the same handle among different instances, it can
create MANY sockets, that was a nice idea
Now I just want to know, WHY does windows limit the handles in a service app
without desktop interaction (Maybe to protect other services from running
out of resources?)
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
François Reusing the design I've done is very easy. Just derive your class from TIcsWndControl and override the methods AllocateMsgHandlers, FreeMsgHandlers and MsgHandlersCount to get your own messages. See how it is done in many ICS components. very easy. It is really easy to use and it is already done and tested! So I will definitely use it ;) I really liked this implementation, congratulations on that Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
TIcsThreadTimer is a good choice for 10,000+ of lazy timers, when accuracy isn't very important. TIcsTimer is a 'normal' timer that requires a TIcsWndControl instance to work. There are demos of both in the MiscDemos directory. What I did was something similar, but not using messages I actually have instances of TSharedTimer, and I have a manager with a list of all timers created, this manager has a TTimer with 10ms loop, and it calls the OnTimer event of the timers when they are ready to be triggered The limitation is that the OnTimer events of SharedTimer will be called from within the main thread only. It is working really good and I can create 10.000+ timers even with high resolution (not THAT high, but as low as 30ms), without using system resources as handles, only memory Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: ICS-based IQ Proxy Server supports 60K concurrent connections on single port now!
SubZero What's new in IQ Proxy Server 3.4.2R - On single port, IQP now supports 60,000 simultaneous connections That is really nice, did you get that by just activating the desktop interaction flag on? Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
One limitation with such kind of implementation (might not be relevant) is that the application blocks until entire list of events is processed. Like iterating over a list of n server clients blocks all clients. Yes, that is true, but I try not to have long processing in timers, just routines that are fast to be executed, but it is true ;) Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] ICS Message handlers
Dear ICS Team, I found a problem in our software that is limiting the total ammount of TCP sockets that I can create. The problem that I found is that I can´t create many many instances of TWSocket, because of its internal message handler (I did many tests), here are my findinds: We use AllocateHWnd from Classes.pas to allocate message handlers to several different objects in our software, so, at one time we can have over 1000 handles allocated across the software I did some tests and using a service application I could create about 170.000 instaces of TWSocket, but if I allocate some thousand handles (By using Classes.AllocateHWnd) than I can´t create ANY instance of TWSocket, in my tests I created 10.000 handles by using classes.allocatehwnd, than I couldn´t create any TWSocket Since my app can have many handles and I need many sockets, I did many tests and found that procedure TIcsWndHandler.AllocateHWnd; create its own windows handle, instead of using classes.allocatehwnd How does this scheme works? To make a test, I changed the ICS implementaion of procedure TIcsWndHandler.AllocateHWnd; to create the handle using Classes.AllocateHWnd, then, I could create over 800.000 instances of TWSocket, and at least in my tests, it was connecting to hosts and receiving data correctly And the better, if I create 1.000.000 of handlers by using CLasses.AllocateHWnd, I could create over 700.000 instaces of TWSocket yet... So, If I change my ICS code to allocate the message handler by using Classes.AllocateHWnd instead of using the implementation of procedure TIcsWndHandler.AllocateHWnd; What could go wrong? Because at preliminary tests, the sockets were working OK, even in multi-threading and I almost have no limit of socket creation Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
It will be interesting to hear an answer to this... Thank you, Why? Is that a silly question? :( I don´t know how ICS works internally, I´m just wondering what is happening ;) Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
François, Windows limit the number of windows handles to 10.000. That's why ICS V7 is designed to use only one handler for a lot of TWSocket so that you can create a lot of sockets. Which OS are you using which allows creating 1.000.000 handles ? Windows 7 64bit with 4GB RAM, and it does allow the creation of 1.000.000 handlers and 800.000 TWSocket instances (After I changed the ICS implementation) Classes.AllocateHwnd and TIcsWndHandler.AllocateHWnd both use the same API (CreateWindowEx ) to create the handle so both should be able to create the same number of those. That is what I was expecting, but not what is happening The number of TWSocket which can be created also depends on the non page memory pool you OS has. Each socket consume some space there and this space is not unlimited. I know it is not unlimited ;) I will give further details and a sample program -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] DNS Problem
Arno Garrels wrote: So multiple calls to DnsLookup from the same thread context are serialized. In other words, DnsLookup always returns immediately, however Win32 API WSAAsyncGetHostByName serializes multiple requests from the same thread context internally. This may trigger OnDnsLookupDone delayed but does not block a thread. A workaround was to write your own name resolution that uses multiple threads. Thank you Arno Now I understand the structure better But, in any case, the thread should not block in my software right? Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] DNS Problem
Correct, the calling thread won't be blocked. Arno, I´m having a problem with a customer, and I sucessfully reproduced the same problem here. I really believe that it is related to DNS Lookup. Here is what is happening: - I have a communication thread that spawns all TCP client sockets, so, all communication messages are carried by this thread, not by the main thread - I can have several instaces of TCP clients spawned by the communication thread The test that I did: - I have some TCP sockets connected to fixed IP addresses and receiving realtime video data (MPEG-4 video stream) - When I try to connect to a DNS address from dnsalias.com, all the data that was receiving from all TCP sockets stops for 2 seconds. I believe that this time is the DNS lookup because if I try to ping the address it takes about 2 seconds to resolve the address You told me that the thread would not blobk, but it stops receiving all TCP data during the DNS lookup? Any idea on why does it happens? If I make my own DNS lookup using threads, would it solve? Thanks! Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] DNS Problem
Correct, the calling thread won't be blocked.
Arno,
I have checked the ICS code, and I found this:
OverbyteIcsWSocket.pas on line 7290 (TCustomWSocket.Connect)
if not FAddrResolved then begin
{ The next line will trigger an exception in case of failure }
sin.sin_addr.s_addr :=
WSocket_Synchronized_ResolveHost(AnsiString(FAddrStr)).s_addr;
FAddrResolved := TRUE;
end;
So, when I first call connect, it will try to resolve the host by using
WSocket_Synchronized_ResolveHost
This is a sync routine, it just quits when it resolves, and in that case,
the thread will be blocked until the address is resolved.
That is the problem that I´m having, because on my customer site it takes 2
seconds to resolve the ip address, and during this time, all TCP
communications are dead, because the .Connect method is called from the
communication thread context
Am I using an old version of ICS? I´m using version 7 with D2010.
If this is the right way of ICS working, then I think I have to make a DNS
lookup thread
Thanks
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] DNS Problem
Sorry I misunderstood. I thought you used method DnsLookup. So in this case it is simple to fix, call method DnsLookup and connect with a dotted IP address from the OnDnsLookupDone event handler. Hi Arno Thanks for the clarification So, if I use the method DnsLookup all the requests will be serialized correct? If I try to connect to several different DNS that takes several seconds to resolve (I tested in my customer site and it actually takes 5 to 6 seconds to resolve), it means that if it is serialized, the last request will take very long to respond Instead, if I spaw a thread (and release after use) for each lookup and use the routine WSocketResolveHost from the new thread context, I can response many addresses all at once, correct? Is it safe to use WSocketResolveHost routine? I can translate the result to dotted IP address easily Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] DNS Problem
Yes, it is safe, however I would consider to use WSocket_gethostbyname as well (more work). What is the differenc? I See that wsocket_gethostbyname returns more information, but is there a reason to use it instead of the other method? Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] DNS Problem
Hello all I believe I have posted the same question some years ago but I can´t remember and find it What could cause the following thing: I have several instances (More than 20) of TWSocket (TCP Client), running all on the same thread (Secondary thread, not main thread). Each instance will connect to a different DNS address from dyndns.org Sometimes, several addresses are offline, or even the internet connection is not available, when we try to connect it locks up the thread for several seconds, and consequently all instances will lock up too, maybe something related to DNS lookup. This doesn´t happen on all windows installations, there are only some machines that already happened such thing, currently I´m having this problem with a machine running Windows 2008 64 bit So, the DNS lookup can lock until it finds the address? If not, why does it happen on some windows installation and some not? Thanks Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS V7 and D2010 mess up with RTTI
I also think that having the NO_EXTENDED_RTTI option is good, but, maybe you
should not define it by default, because some projects that relies on RTTI
can break because of this bug in D2010 compiler, because it is propagating
the flag to all units that are compiled after one unit with the {$RTTI...}
flag..., this is sure a bug in the compiler
Eric
- Original Message -
From: Arno Garrels arno.garr...@gmx.de
To: ICS support mailing twsocket@elists.org
Sent: Thursday, November 05, 2009 4:01 PM
Subject: Re: [twsocket] ICS V7 and D2010 mess up with RTTI
Eric Fleming Bonilha wrote:
So, in theory, your library is correctly programmed, but I don´t know
why, this is propagating to my whole software and disabling RTTI for
all my Record types!!!
Have you ever seen this problem?
Not yet, I haven't had the time to play with the new RTTI.
If it makes problems then we probably should not define NO_EXTENDED_RTTI
by default. EXE size doesn't benefit very much from this directive (tested
with {$WEAKLINKRTTI ON}), however I think this option is nice to have
anyway.
--
Arno Garrels
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] ICS V7 and D2010 mess up with RTTI
Hi,
I had a very strange problem with your library v7 and Delphi 2010.
My software relies too much on RTTI for Record data, and (After loosing a whole
day trying to find out why it was happening) I found that your library sets
NO_EXTENDED_RTTI directive as Default wich sets {$RTTI EXPLICIT METHODS([])
FIELDS([]) PROPERTIES([])}.
But strangely, this $RTTI directive is propagating to the whole project
(Including MY PROJECT FILES...), maybe to some bug in Delphi 2010
I have found the following comments from a blog:
http://wings-of-wind.com/2009/09/03/community-pulse-coderage-4-rtti-reloaded/
The size of the exe file will increase due of new RTTI info. The grow
percentage usually isn't significant, but for the developers which are very
sensible to their application size there are some tools to reduce the code
size: {$WEAKLINKRTTI ON} and {$RTTI EXPLICIT METHODS([]) FIELDS([])
PROPERTIES([])} - we must note that the latter compiler directive disables the
new RTTI only for the types (classes, records etc.) declared in the unit where
you put it, so including it only once at the top of one unit is not enough. You
need to add it to the top of every unit that declare types in addition to
{$WEAKLINKRTTI ON}. Of course, if you want to strip out this info from VCL you
must recompile it.
So, in theory, your library is correctly programmed, but I don´t know why, this
is propagating to my whole software and disabling RTTI for all my Record
types!!!
Have you ever seen this problem?
Thanks
Eric
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] 10K Connections
Hi, I would like to know if anyone here has already developed or worked with any project that uses ICS and handles about 10K Incomming TCP Connections simultaneously? I have a project for video streaming where the server must handle about 10k simultaneous TCP connection (Sending live video feed to the connections). What is the limit of connections that Windows can handle safetly? Thanks! Eric -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] 10K Connections
Thanks I should have searched it before. I can also split into more servers if it is the case, to make a load balancing environment. I currently use ICS6, I just tested ICS-7 on D2009 but I didn´t port my app to D2009 yet. Will ICS-7 work on D2007? Thanks for the hints Eric - Original Message - From: Francois PIETTE francois.pie...@skynet.be To: ICS support mailing twsocket@elists.org Sent: Monday, July 20, 2009 2:48 PM Subject: Re: [twsocket] 10K Connections I have a project for video streaming where the server must handle about 10k simultaneous TCP connection (Sending live video feed to the connections). What is the limit of connections that Windows can handle safetly? This has been frequently asked. You should search the mailing list archive to see the discussions we had. In short: You defenitely need ICS-V7. You need a Windows SERVER operating system. You need 4GB of memory (that is to have the maximum non paged memory pool). You have to very carefully design your application because Windows can run out of resources, specifically handles. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Silly UDP Question
I just worked fine!
Thanks a lot Francois!
Éric
- Original Message -
From: Francois PIETTE [EMAIL PROTECTED]
To: ICS support mailing twsocket@elists.org
Sent: Wednesday, October 22, 2008 2:24 PM
Subject: Re: [twsocket] Silly UDP Question
But how do I fill the TSockAddr structure to pass to SendTo?
Out of my head, not tested:
FDest : TSockAddrIn;
Data : AnsiString;
MyPort : Integer;
MyPort := 1234; // Port number
FillChar(FDest, SizeOf(FDest), 0);
FDest.sin_family := AF_INET;
FDest.sin_port:= WSocket_htons(MyPort);
FDest.sin_addr.s_addr := WSocketResolveHost('www.borland.com').s_addr;
WSocket1.SendTo(FDest, SizeOf(FDest), PAnsiChar(Data), Length(Data));
--
[EMAIL PROTECTED]
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Silly UDP Question
Thanks Francois! But how do I fill the TSockAddr structure to pass to SendTo? Eric - Original Message - From: Francois Piette [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Wednesday, October 22, 2008 4:25 AM Subject: Re: [twsocket] Silly UDP Question I have a simple doubt regarding the TWSocket using UDP protocol. Using the same component can I listen on one port and send data on another one? Yes, you can. Put the instance to listen mode on the port you want and then use SendTo to send datagram to the port you need. -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
