Re: [twsocket] R: ICS for Linux Development
Hello Roberto, hello everyone! On Sun, 2005-09-04 at 13:22 +0200, Roberto Della Pasqua wrote: > Please permit me to suggest you what I consider the best model for highend > scalability under linux: Yes, epoll() seems like a good idea. It seems libevent supports it, too. That makes me wonder if libevent could be used as it has most of the needed event infrastructure in place and it is rather common in modern Linux installations. Best regards, Ari Sundholm [EMAIL PROTECTED] [EMAIL PROTECTED] Author of cddbcomp - THE Delphi component for freedb access Moderator of freedb forums freedb core team Support languages: Finnish, English, German, Swedish DISCLAIMER: Despite being part of the freedb team I only represent myself unless otherwise noted. Freedb is in no way responsible for my actions or opinions. > -Messaggio originale- > Da: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Per > conto di Ari Sundholm > Inviato: domenica 4 settembre 2005 12.36 > A: ICS support mailing > Oggetto: Re: [twsocket] ICS for Linux Development > > On Sat, 2005-09-03 at 18:54 +0200, Francois PIETTE wrote: > > IMO Linux lack both true async socket, and messaging system. > > I wouldn't say so. By using fnctl() one can do the following: set a socket > descriptor F_SETFL - O_ASYNC and possibly F_SETOWN if the process receiving > SIGIO has to be something else than this one and you have truly asynchronous > sockets: You just select() when you receive the signal. That should be the > Linux/POSIX-specific way to do it, with no threads needed. Actually, AFAIK a > threaded implementation is only needed if it is absolutely necessary to > maintain cross-platform support. The real question is if it is desired to > have the rewritten component support other platforms than Linux (and other > POSIX-compliant ones), which would indeed require a decent threaded > implementation. And making multithreaded code right is hard :( > > If anyone spots any errors in the above passage, feel free to correct me. I > haven't done true async for a long time, so I may be a bit rusty. > > > Linux has non-blocking socket but no mechanism to tell the application > > when the socket is ready to send or has received something. You have > > to use the trick I used. > > See above. Signals should make it possible, unless I'm horribly mistaken. > > > Linux has no windows like messaging system, that's why QT built his > > own. But QT implementation is far too slow for anything else than a user > interface. > > And using QT is only possible when X-Window is loaded ! > > Yes, it is true that QT's implementation is dead slow. > > Best regards, > > Ari Sundholm > [EMAIL PROTECTED] > [EMAIL PROTECTED] > Author of cddbcomp - THE Delphi component for freedb access Moderator of > freedb forums freedb core team Support languages: Finnish, English, German, > Swedish > DISCLAIMER: Despite being part of the freedb team I only represent myself > unless otherwise noted. Freedb is in no way responsible for my actions or > opinions. > > > - Original Message - > > From: "Ari Sundholm" <[EMAIL PROTECTED]> > > To: "ICS support mailing" > > Sent: Saturday, September 03, 2005 3:22 PM > > Subject: Re: [twsocket] ICS for Linux Development > > > > > > > Hello Eric, hello everyone! > > > > > >> locks up (threading). In speaking to Ari Sundholm about this, he > > >> mentioned that the design of the TICSSocket components is a bit > > >> awkward and that this sort of thing was bound to happen. I fully > > >> concur with what he has said, and feel that a total rewrite of the > > >> component set under Linux is required. > > >> I don't think that many on this mailing list are using TICSSocket, > > >> or we probably would have seen this issue raise it's head in a > > >> bigger way. I however have a product that is contracted to work > > >> under Windows and under Linux and I must move forward on this now. > > >> I cannot delay any further waiting for this to be addressed. > > > > > > IIRC the reason for the awkwardness was the assumption that async > > > sockets don't exist in Linux (which is simply untrue AFAIK - or am I > > > on > > > crack?) and/or the lack of a messaging system such as in Windows. > > > Therefore an elaborate threaded message passing system was built - > > > with a huge number of subtle as well as obvious bugs (which haven't > > > been debugged and fixed due to my laziness during my scarce free > > > time). I think a completely new approach has to be taken, but I will > > > have to do some reading and thinking to be of any substantial help > > > regarding design. Unfortunately, I have quite serious time problems > > > at the moment due to moving to Japan at the end of September. > > > > > >> Therefore, I would like to re-develop the core component (TWSocket) > > >> set under the Linux platform using Kylix 3. The intention is to > > >> build a version of the core components that is fully compatibly > > >> (from an interface
[twsocket] R: ICS for Linux Development
Please permit me to suggest you what I consider the best model for highend scalability under linux: http://www.xmailserver.org/linux-patches/nio-improve.html use epoll(), a sort of linear select that check the status of all sockets dispatched in kernel (also 100,000 sockets concurrently without do a copy/mem ring3 call); it's a sort of win32 iocompletionport... And NPTL thread pool, for every socket that need operation then start a NPTL thread that close when finished. A modern CPU can dispatch 1 million of NPTL threads for second, so the time spent in create/free is not noticeable. Regards, Roberto Della Pasqua -Messaggio originale- Da: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Per conto di Ari Sundholm Inviato: domenica 4 settembre 2005 12.36 A: ICS support mailing Oggetto: Re: [twsocket] ICS for Linux Development On Sat, 2005-09-03 at 18:54 +0200, Francois PIETTE wrote: > IMO Linux lack both true async socket, and messaging system. I wouldn't say so. By using fnctl() one can do the following: set a socket descriptor F_SETFL - O_ASYNC and possibly F_SETOWN if the process receiving SIGIO has to be something else than this one and you have truly asynchronous sockets: You just select() when you receive the signal. That should be the Linux/POSIX-specific way to do it, with no threads needed. Actually, AFAIK a threaded implementation is only needed if it is absolutely necessary to maintain cross-platform support. The real question is if it is desired to have the rewritten component support other platforms than Linux (and other POSIX-compliant ones), which would indeed require a decent threaded implementation. And making multithreaded code right is hard :( If anyone spots any errors in the above passage, feel free to correct me. I haven't done true async for a long time, so I may be a bit rusty. > Linux has non-blocking socket but no mechanism to tell the application > when the socket is ready to send or has received something. You have > to use the trick I used. See above. Signals should make it possible, unless I'm horribly mistaken. > Linux has no windows like messaging system, that's why QT built his > own. But QT implementation is far too slow for anything else than a user interface. > And using QT is only possible when X-Window is loaded ! Yes, it is true that QT's implementation is dead slow. Best regards, Ari Sundholm [EMAIL PROTECTED] [EMAIL PROTECTED] Author of cddbcomp - THE Delphi component for freedb access Moderator of freedb forums freedb core team Support languages: Finnish, English, German, Swedish DISCLAIMER: Despite being part of the freedb team I only represent myself unless otherwise noted. Freedb is in no way responsible for my actions or opinions. > - Original Message - > From: "Ari Sundholm" <[EMAIL PROTECTED]> > To: "ICS support mailing" > Sent: Saturday, September 03, 2005 3:22 PM > Subject: Re: [twsocket] ICS for Linux Development > > > > Hello Eric, hello everyone! > > > >> locks up (threading). In speaking to Ari Sundholm about this, he > >> mentioned that the design of the TICSSocket components is a bit > >> awkward and that this sort of thing was bound to happen. I fully > >> concur with what he has said, and feel that a total rewrite of the > >> component set under Linux is required. > >> I don't think that many on this mailing list are using TICSSocket, > >> or we probably would have seen this issue raise it's head in a > >> bigger way. I however have a product that is contracted to work > >> under Windows and under Linux and I must move forward on this now. > >> I cannot delay any further waiting for this to be addressed. > > > > IIRC the reason for the awkwardness was the assumption that async > > sockets don't exist in Linux (which is simply untrue AFAIK - or am I > > on > > crack?) and/or the lack of a messaging system such as in Windows. > > Therefore an elaborate threaded message passing system was built - > > with a huge number of subtle as well as obvious bugs (which haven't > > been debugged and fixed due to my laziness during my scarce free > > time). I think a completely new approach has to be taken, but I will > > have to do some reading and thinking to be of any substantial help > > regarding design. Unfortunately, I have quite serious time problems > > at the moment due to moving to Japan at the end of September. > > > >> Therefore, I would like to re-develop the core component (TWSocket) > >> set under the Linux platform using Kylix 3. The intention is to > >> build a version of the core components that is fully compatibly > >> (from an interface perspective anyway) with TWSocket and then to > >> donate it back to this forum. > >> Francois, I would want your blessing to do this, and after that has > >> been given, will invite all of those who are interested and who can > >> commit time on a weekly basis to this project to volunteer to join > >> this project. If a team is not assembled, then I w
