Re: [twsocket] New IcsThread demo shows how to use ICS in a thread (Windows and OS X)
Arno said It's done, I just checked in the new IcsThread demo (ICS-V8 Beta). It's located in folder PlatformDemos and in xSamples.groupproj. Will be also available in next ICS-V8 Beta Snapshot ZIP. http://wiki.overbyte.be/wiki/index.php/ICS_Download Super Arno, thanks for taking the time to do this. I will look through it soon. Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket in thread in FireMonkey HD app for future crossplatform target
Yes. In Windows anyway and in Posix there's Window's messaging emulated (as it's needed to not change ICS' source code too much). In Posix a thread doesn't have to call GetMessage(), PeekMessage() that's the major difference. Unfortunately there's not yet a FMX threaded demo available, I'll try to add one next weekend when I have a few hours. In the meanwhile take look at unit Ics.Posix.Messages and TIcsMessagePump that implements the fake messaging in Posix and OSX. Also take a look at the console demo IcsConSmtp in PlatformDemos directory (has it changed at all?), console applications do work very similar to ICS in a thread. Thanks for the info, Arno. :) Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] TWSocket in thread in FireMonkey HD app for future cross platform target
Hello. I am building some test FireMonkey HD apps in Dephi/RAD Studio XE3 using ICS that I am hoping will eventually build cross platform to Mac. There are two apps, one is a server and the other a client, and they will exchange some large amounts of data. When they are running on the same system, which naturally happens during development and might also occur under certain scenarios on the user's system, large transfers really bog down the main thread in the client side app. This makes the client app VERY unresponsive in that scenario. So, I am thinking to move the client side TCP interfaces to a background thread so that it doesn't slow down the UI so badly. However, being that this thread is in a FireMonkey HD app, I am unsure how to proceed. Do all the threading message pump issues still apply? Knowing that cross platform does away with dependence on platform technologies like PostMessage (Windows only), what should I do regarding this? Are the Windows platform units still relevant with an app that will be built for cross platform? So, just wondering if I still need to provide something in that regard, as far as a message pump in thread and if so, how does that port to Mac later? Or, if I just create the component in the Execute method, do the events just magically fire in FireMonkey apps? Well, hope my question(s) makes sense? I searched the platform demos and didn't see anything that might demonstrate a cross platform approach for this. So, sorry if it is there and I just missed it somehow. J Thanks much. Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Plans for ICS and iOS?
That was my plan last year, but then EMB crippled Delphi for their NextGen compiler IMO. Looking at XE3's source code it becomes clear that there will be no more Pointers, only one string type etc.. in NextGen which seemed to be required for their already announced mobil studio. So I for one gave up that plan, (no Pointers = no Arno coding ICS/Delphi). Wow, boo. Thanks much for the quick response, though. :) Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Plans for ICS and iOS?
Hello. Now that ICS supports both Windows and MAC OS X via FireMonkey (FM2), do you have any plans to support iOS targets in the future as well? Or, are you waiting for the future XE3/4/5.2 or whatever that will eventually include the new native non-Win target compilers? Thanks much. and you guys are awesome! J Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Should next ICS version support anything before Delphi XE ?
Hey... Still maintain some stuff in D2007 with ICS. All new development in XE2. Bug fixes are fine for older D2007, so don't care about new development with D2007. However, DESPERATELY want to see a good socket solution for XE2. :) Regards... Hoby -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of François Piette Sent: Wednesday, March 28, 2012 12:59 PM To: ICS support mailing Subject: [twsocket] Should next ICS version support anything before Delphi XE ? Hi ! Im planning the next ICS version Being unable to use any features added to Delphi in the last 10 years is very restricted. Maybe we need to cease support for old Delphi versions ? Of course ICS V5 and V7 will remains available however, the only changes will be bug fixes. What do you think ? Please keep your answer as short as possible, I just want to have an idea about how many of you are still using an old Delphi version. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Season's greetings
Francois said: Merry Christmas Merry Christmas and Happy New Year to everyone from Denver, Colorado! :) Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL is broke?
Dave says... Seems good old SSL is after all susceptable to some sort of Man In The Middle atack. tireless-OT-ihatetheweb-ranting Thanks for the link Dave. While this issue is in the process of being addressed by the various providers, it is still one of the MANY reasons why I abandoned the web stack a long time ago. For truly secure hosted apps, I developed my own app hosting architecture years ago. My apps are true, native SaaS, and don't use any Web stack technologies, including NO SSL, HTTPX, FTPX, etc. Instead, I use my own app servers, channel security, protocols, client technology, embedded scripting, etc. I am such the radical... ;) And, all done using ICS, of course... :) Anyway, my point is that I am always mystified (stunned) as to why so much of the development world thinks that distributed applications and the Web are synonymous, or that the distributed paradigm started with HTTP, or that SSL is such a great security solution. Distributed apps began much earlier than Web technologies. I was developing them before the Web. In fact, the Web craze set the distributed paradigm back about 20 years. The Web is simply a huge kludge. Oh well, so we have another, oh no, SSL is broke issue. Of course we do. Just wait. This one will be addressed and then the next weakness will emerge. ALL Web stack technologies are broke regarding security and have been since they were innovated. That isn't going to change until we do SOMETHING else. /tireless-OT-ihatetheweb-ranting Regards... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Popup
Hi guys, I have seen the same pop up as well. I have seen them for a while as well. I was quite surprised when I first started seeing them. I use IE 8 and have like 3 popup blockers, but the popups still come up. Specifically, when I first load the site, popups try to load, but my blockers suppress them. Then, when I click on the Support link in the navigation links, a popup comes up, even though I have multiple blockers. How do they do that? Arrgggh. :( I just tested it again and the first time I loaded the site and selected Support, I got a popup with this URL: http://www.travian.us/?ad=10184_1073927120 I closed the browser, reloaded the site, clicked Support and got a popup with this URL: http://minisites.smsgameon.com/default.aspx?siteid=77877126source=zanoxpid =1231330922151062528 I noticed this a while back, but assumed it was excepted behavior. If it is not, then it would be nice if they could go away? :) Regards... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Website hacked?
I have removed Motigo counter. Could you check if you still see the popup (Strangely, I don't see it from here). Hey, Francois... That seems to have done it for me... Thanks much! Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Popup
Francois... This allows me to earn a little bit of money... I don't begrudge that at all! You (and the team) greatly deserve it. Your ICS components have helped me tremendously and made my life much easier. May you receive the prosperity of a million clicks! ... If that is worth anything?... :) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: Was Re: ICS V7 for Delphi and C++ Builder 2010
SZ said... I am quite sure 2 years ago in a roadmap pdf I read mentioning of 64-bit support for BCB which should imply the same for the underlying Delphi. I also recall asking it here and elsewhere and getting answers accordingly. Yes, I distinctly remember it being on a roadmap quite a while back, pre-Embarcadero. I have been watching for it for a while now as well... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Simple SMTP text emails rejected for virus or spam
Francois said... 1- You should try to send the email, with the exact same program, from another computer (or even several computers) using another ISP 2- It is likely that - despite what they say - your IP or hostname is blacklisted. Thanks for the input Francois. I have indeed tested by sending through an alternate gateway and other methods. Same problem. Also, when I attempt to reroute through a gate SMTP, my gateway SMTP logs show that the IP blockage has been removed and that the target SMTP server responds with the virus/spam error instead of host rejection. So, I think they are correct that there is no IP blockage anymore. So, my ultimate conclusion is that, like you said, it is the email itself that they don't like. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Ipv6
Arno said... If ICS shall not die very soon it has to be implemented anyway! Wow! Scared of that statement! What do you mean by that? Is ICS in danger of going away? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Simple SMTP text emails rejected for virus or spam
Well, I finally resolved this issue. What a pain. In the end, the problem was just character data that GoDaddy didn't like. It would have been nice if their SMTP response had said something like, Invalid character data detected, instead of, Rejected due to spam or virus content. But, maybe I am asking too much? ;) Specifically, the problem was that GoDaddy doesn't like parenthesis [( and )] in the from field. I was using parenthesis to designate the email address content from the friendly name, like, Some User (u...@domain.com). I use that format so that users will see a more readable name in their mail client (Outlook, etc). Changing to brackets, like Some User u...@domain.com, fixed the problem. Go figure. Historically, I have used both and never had a problem. In this case, the SMTP profile was stored in an encrypted XML file for my service, so the was giving my problems in the beginning with the XML tags. Hence, I used parenthesis instead. Odd that GoDaddy didn't like it. I have never had a problem with that format in the from field. Oh well, like I said, a more relevant message would have saved two days of my time, as well as their tech's time. BTW, I thought this was rather indicative of GoDaddy's persona. Most servers I have had to log, respond to the SMTP data request with something like, 354 Start mail input; end with CRLF.CRLF, or something informative like that. However, GoDaddy's response is, 354 go ahead punk, make my day. Really. Kind of tells ya something, huh? :) Thanks for the input. Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Simple SMTP text emails rejected for virus or spam
Hello, everyone...
Any input on this would be really appreciated.
I have a processing service that generates very simple SMTP notifications to
customers. Basically, these are system alerts about item due dates, etc. I
am using ICS TSyncSMTPCli v2.5 to send the messages with. I simply set the
envelope parameters and message, then deliver it to the SMTP server. VERY
SIMPLE. I have used this same pattern in numerous apps for years and never
had a problem.
A recent client signed up for the service who is with GoDaddy. For some
very ANNOYING reason, GoDaddy doesn't like something about the emails I am
sending him. It rejects them with 554 The message was rejected because it
contains prohibited virus or spam content. This is of course absurd, since
they are text format emails with one or two sentences of text. I have tried
several different content changes, including just sending, this is a test.
All bounce back with this error.
I just got off the phone with a tech there who is rather clueless. My IP /
host is not blocked and they can't tell me why it is rejecting the emails.
So, currently, my best guess is that something in the way I am creating the
envelope is flagging their spam/virus filter. I have not had this problem
with any other SMTP servers, but that doesn't necessarily mean anything. I
am open to the possibility that I am doing something they don't like, but I
have no clue what that could be. My header is so simple, I am at a loss as
to what they might not like.
Has anyone ever encountered this issue? Any thoughts? I am stumped. I
have been doing this for years without problem. :(
Here is a snippet of code from where I set the email content parameters.
Maybe I am doing something odd? Maybe they don't like the ReplyTo not being
the sender or something? FYI, smtpMain is the TSyncSMTPCli component.
*
Code snippet
*
smtpMain.FromName := fFromAddr;
smtpMain.HdrReplyTo := fReplyTo;
smtpMain.HdrFrom:= fFromName;
smtpMain.HdrTo := aEmail;
smtpMain.HdrCc := '';
smtpMain.HdrSubject := aSubject;
smtpMain.HdrSender := fFromAddr;
smtpMain.RcptName.Clear;
smtpMain.RcptNameAdd(smtpMain.HdrTo,'','');
with smtpMain.MailMessage do
begin
Clear;
if not fNoPrelim then
begin
Add('Per your request, we are notifying you of the following');
Add('event:');
Add('');
Add('');
Add('Account name: ' +
qryNotifyHistory.FieldByName('acct_ref').AsString);
Add(
qryNotifyHistory.FieldByName('proj_label').AsString +
': ' +
qryNotifyHistory.FieldByName('proj_title').AsString);
Add('');
end;
Add(aMessageText);
if not fNoFooter then
begin
Add('');
Add('');
Add('***');
Add('Note: This is an automatically generated email. Replies');
Add('to this email will not be read.');
Add('***');
end;
end;
SetTask('Delivering SMTP message...');
SetTaskProgress(33);
if not smtpMain.MailSync then
begin
Result := False;
LogText(0, 'Err: ' + smtpMain.ErrorMessage, 'Failed');
exit;
end;
*
End Code snippet
*
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] IP V6
Paul for those interested http://blogs.techrepublic.com.com/itdojo/?p=262tag=nl.e101 Sweet! Thanks... :) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] OT RE: EAccessViolations when Posting Data to a HTTPServer
A little OT... :) Brian said... do you know about madExcept No, I didn't. Sweet! Thanks. :) Dod said... or EurekaLog (not free but more powerfull and reliable... Wow, also sweet! Thanks! I got to get out more... :) OT Question: Why is EurekaLog more powerful and reliable than madExcept? Those are somewhat ambiguous classifications. Are your assertions based on empirical usage experience? Or just an observation? Thanks much... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] breakpoint in NTDll
Wilfried said... NTdll.DBGBreakPoint int 3 ret What OS and version? I can't remember exactly what version it was, but I seem to recall that, quite some time ago, MS accidentally released a WinSock (or some related DLL) build into production that still had a hard break in it from one of the testers. It was really annoying, because it was actually in the MS code and would break under certain circumstances when running in debug mode. Like I said, I don't remember what build and all, but seems like it was an NT version some years back. But, from your info, it sure looks like the same issue. Anyway, if that is the case, there is nothing you can do but just live with it or upgrade the OS. :) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Proposal to replace TIniFile by TRegIniFile in allv7 demo applic
@Angus said... I use a CONFIG suffix so the OS does not mess with my files. Angus, can you please elaborate on that? I am not sure what you are referring to. Sounds like a way to keep Vista from virtualizing files? Thanks... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Proposal to replace TIniFile by TRegIniFile in allv7 demo applic
Angus said... and called them .config, but that was probably a red herring, sorry Yeah, virtualization I get. Sorry, thought you were eluding to some super secret Microsoft convention for telling Vista to ignore a file using config in some magical way (name, reg key, etc). That would have been nice. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] OT RE: I wan't to stop the server from listening then start it
Francois... Note that I understand that each one has difference preferences for his development environment. I only explain how I work. Thanks for the info on your package approach, Francois. Interesting approach that I hadn't thought of. Two questions... 1. Would you use this approach across different machines? Meaning, are the packages sufficiently portable across hosts? So, the bpl or whatever form you are using, could just be moved that way? Sorry, but I don't have time to keep up with the package format changes anymore. That got old forever ago. :) 2. How do you handle packages that get frequently updated? For example, I have a specific component provider that FREQUENTLY updates their stuff with bug fixes and feature enhancements. I have others that do the same, but less frequently. Do you rebuild all your packages every time one dependant package gets updated? Or, does D2007 handle the specific inter-dependencies automatically? No hurry on an answer to this. I just thought your approach was interesting and might be a real benefit for me, but I would need to solve those two issues for it to be applicable to my environment(s). Thanks... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Magenta Systems File Transfer Components v2.4 and v3.0 for ICS V6 and V7
Angus... Magenta Systems File Transfer Components are copyrighted software but may be used without cost. Wow... Thanks! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Early web server response [OT]
DZ-Jay said... coercing HTTP as the de facto standard protocol for all communications in the Internet (as a lot of people are trying to make it) is stupid. There are better transfer protocols out there. Preach it! The Commodore 64 implemented better protocols... ;) But this is a rant for another day. I tend to make that rant about five times a day. But, who's counting? ... :) Go DZ! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP client supports Utf-8 in ICSv7 now
Double Ditto! Thanks much... :) From: Francois Piette [EMAIL PROTECTED] Angus and I added UTF-8 support to the TFtpCli V7. I think Angus and Arno deserve a BIG thank from all of us for the huge work they've done supporting internationalisation in FTP component. Thanks guys ! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Low cache size problem and an announcement
We just made our shareware IQ Web/FTP Server freeware! Sweet! Thanks for letting us know... :) Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS in New York, Seattle, San Diego and San Francisco
Starting next week, I start a tour in the USA to visit my customers... Bummer! All too far from me. Actually, with the current price of gas, I think across the street may be too far for me ... ;) Oh well, I would love to have met you. Hope you have a great trip in the states! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Force UDP source port when sending
Hello Dod... Just to be clear about UDP behavior... This only worked because your transmission pattern is based on a single packet exchange. That pattern basically results in a virtual ack/nack process, which orders data by default. UDP is not only unreliable, it is also inherently UNORDERED. So, it is possible that packets could appear out of order, if you were sending multi-packet bursts. This requires you to provide for orderly logic as well. Hence, the queue of some type. IMHO... If you need orderly, reliable transmissions... use TCP. The speed benefit of UDP is only valuable for single packet exchanges. Otherwise, the robust TCP overhead is much better than anything I can provide instead. Regards... Hoby -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dod Sent: Wednesday, July 02, 2008 5:35 AM To: ICS support mailing Subject: Re: [twsocket] Force UDP source port when sending Hello Angus, ARMSL No idea how that worked, maybe the client was using a different port to ARMSL reply. Absolutely, my other programs never needed to re-use same src port so I never got the problem. ARMSL If you have been making use of the TSocketServer client to save ARMSL application data for the reply, you'll need a rethink. Perhaps a FIFO ARMSL queue, I think there's a TList descendent that does that in modern Delphi ARMSL versions. With UDP there's always a risk of lost packets, if your ARMSL transmit conflicts with a new received packet, or at least that's my ARMSL minimal understanding of UDP. UDP packet lost happend as they are sent and without any SYN/ACK dialog between src and dest so UDP is fast but you have to manage packet acknowledge yourself to not loose data. For my application I only receive one packet and send one packet back. regards. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] AN: New e-mail protocol (spam free and more!)
Hello SZ... Astute observation about formats vs. protocols. However, I don't think that David has done away with TCP/IP, the transport protocol used for sending the email with. In a sense, all layer 7 solutions can really be considered a format in a TCP payload. So, the distinction really blurs there to some degree. Regardless, the core issue is still relevant, namely that binary systems think in a binary context. If a driving constraint is efficiency and minimalist approaches, binary formats and protocols are mandated. Regardless of how you slice it, binary makes better sense FROM THE SYSTEMS perspective. As for the TCP hacking attacks you reference, they PALE in comparison to the security flaws that are revealed and exploited on a daily basis that stem from the inherent lack of security in the original web protocol designs. Additionally, those TCP hacks provide no security related exposure, in the sense of data compromise, but rather in systems resiliency. Syn-attacks will not get to your corporate data. XSS attacks, on the other hand, can provide a wonderful mechanism for compromising your systems in a number of ways. Additionally, SSL dependence is at the root of almost ALL BAD web thinking. However, everyone still thinks the Emperor looks great and somehow it's just going to work itself out. If the web protocols had envisioned ANY CONCEPT of user and state, instead of the stupid session-less paradigm it enforces, as well as in-transit security, there would be no need for the web stack paradigm to exist in its currently flawed state. Regarding testing new protocols, new standards emerge EVERY DAY. In the xml related world (given this is a format not a protocol), they emerge every minute. Considering that numerous security hacks attack FORMAT weaknesses in the OS and related software (image files, etc), this issue still applies. Scores of them pass through corporate enterprises without testing every day. Additionally, anything constructed as a layer 7 solution is going to pass through the internet architecture just fine, barring various packet inspector rules, etc. Again, I don't think David has invented a new TCP stack, but rather just different info being passed through TCP. Regards... Hoby -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Thursday, February 07, 2008 10:35 AM To: ICS support mailing Subject: Re: [twsocket] AN: New e-mail protocol (spam free and more!) MP3 is not a protocol but a file format. You are right that TCP/UDP uses binary headers but we are talking about a high level protocol, which if popularized, will be coded by third party coders according to RFC and this I believe will not be so trivial for the binary case! ICS has codes for SMTP/POP/HTTP/FTP (all text) but not for TCP. Wasn't there a need for low level stacks? Think again--we have all lived through syn-attacks and ping-o-deaths. Keep in mind that Microsoft recoded the entire TCP/IP stack in Windows 2008 Server just for this. But since it was something hard to do, through the past decade, Microsoft was being waited to step forward. ALso, for adoption, large enterprises want to test every protocol they use--when they are new. Testing a binary protocol for them would also be hard and this might hammer adoption globally. We have customers who study and stress test for 29 days to understand how our caching and then decide to buy at the end of the trial period! Regards, SZ On 2/7/08, Hoby Smith [EMAIL PROTECTED] wrote: Actually, IMNSHO (In My Not So Humble Opinion), binary protocols make much smarter sense for binary machines. Even with the discrepancies of the various flavors of binary formats (big vs little endian, etc), it is WAY MORE EFFICIENT for binary machines to use binary protocols than to interpret human readable formats (text, xml, etc) into binary formats so that they may be processed appropriately by processor and related software. Only humans think in the context of textual formats; computers do not. The overhead associated with textual translation is a horrible side effect of the web phenomenon. Actually, the statement no popular/common protocol is designed that way is horribly inaccurate. True, the W3C has an inherent conflict of interest in driving the pervasive use of textual formats. However, there are tens of thousands of binary formats that predate the stupid web stack and continue to emerge daily. For example, consider TCP, which is a binary protocol and the actual core technology upon which this mail list and ICS is founded. I would consider TCP to be popular and common. It is only the layer 7 centric, web focused protocols that are textual in nature. Unfortunately, the web has set computing back about 20 years, the effects of which has only recently begun to be realized in all of its awful consequences (slow cumbersome interfaces, security issues galore, etc). Quite
Re: [twsocket] AN: New e-mail protocol (spam free and more!)
Actually, IMNSHO (In My Not So Humble Opinion), binary protocols make much smarter sense for binary machines. Even with the discrepancies of the various flavors of binary formats (big vs little endian, etc), it is WAY MORE EFFICIENT for binary machines to use binary protocols than to interpret human readable formats (text, xml, etc) into binary formats so that they may be processed appropriately by processor and related software. Only humans think in the context of textual formats; computers do not. The overhead associated with textual translation is a horrible side effect of the web phenomenon. Actually, the statement no popular/common protocol is designed that way is horribly inaccurate. True, the W3C has an inherent conflict of interest in driving the pervasive use of textual formats. However, there are tens of thousands of binary formats that predate the stupid web stack and continue to emerge daily. For example, consider TCP, which is a binary protocol and the actual core technology upon which this mail list and ICS is founded. I would consider TCP to be popular and common. It is only the layer 7 centric, web focused protocols that are textual in nature. Unfortunately, the web has set computing back about 20 years, the effects of which has only recently begun to be realized in all of its awful consequences (slow cumbersome interfaces, security issues galore, etc). Quite some time back, before clients of various types were ubiquitous, it was necessary for servers to understand human readable formats (ftp, etc). That need has long been outlived and the perspective that drives it is seriously antiquated. With very few exceptions, the data needs of server systems are way too complex to be constrained by human readable formats. For example, are you going to type the binary values for a MIME image that you want to embed in an email? Try doing that via a textual SMTP interface. The whole need of MIME was to overcome the stupid limitations of a textual email standard. Let's see, how do I represent binary data in a textual format. Yeah, that makes perfect sense. In a web world, I guess it does. How about a word document? Do you want a format that you have to type the font details manually, like HTML? NO, you use a word processor that does all that for you and then stores it in format that the system understands. How about audio? Do you want a media stream protocol that is human readable? Such a media protocol would be too slow be feasible. MP3 is a BINARY solution that attempts to answer the audio data issues. Everyone uses a piece of software that understands MP3. I would consider MP3 to be common and popular, even though it is BINARY. For computing needs, the future reality is surely a return to strong binary representations that can be interpreted into forms that humans can understand as needed, or can be handled by clients as necessary. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Thursday, February 07, 2008 5:43 AM To: ICS support mailing Subject: Re: [twsocket] AN: New e-mail protocol (spam free and more!) I wonder why he chose a binary format instead of text as no popular/common protocol is designed that way (i.e. HTTP, FTP, IMAP--all telnet based)! Regards, SZ On 2/7/08, DZ-Jay [EMAIL PROTECTED] wrote: So, rather than a new protocol, you have created a new e-mail server and client system which communicates in its own proprietary binary format? dZ. On Feb 6, 2008, at 18:50, David A. G. wrote: Dear friends, I have developed a complete and very improved e-mail protocol, highly immune to the SPAM, with data encryption and compression, with sender ID validation, etc. BUT not compatible with the standard email (SMTP). -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] AN: New e-mail protocol (spam free and more!)
Go David. As a disclaimer, I admit that I am seriously biased, since I am probably the only nerd on the planet that thinks Tim Berners Lee should be shot for giving us the stupid web trash we have now, or perhaps it is the large corporate interests to blame that determined for their own personal gain that the web was should be the ULTIMATE vehicle for software facilitation. Either way, I plainly admit my anti-web bias. Nonetheless, the ONLY solution for overcoming the massive amounts of oversights in various areas (security, scalability, state management, etc) in the current protocols (SMTP, FTP, HTTP, etc) is by RETHINKING and REPLACING them with something that inherently thinks smarter. Kudos to David for thinking OUT of the box to ANY DEGREE. Innovation should be supported, not rebuffed for its lack of status quo complicity. Ok, I am deflating my personal soap box now... You may return to your regularly scheduled programming... :) Hoby -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David A. G. Sent: Thursday, February 07, 2008 10:13 AM To: ICS support mailing Subject: Re: [twsocket] AN: New e-mail protocol (spam free and more!) Answering to all... DZ-Jay: So, rather than a new protocol, you have created a new e-mail server and client system which communicates in its own proprietary binary format? That is correct, but according the definition of protocol and thinking about my system runs directly over TCP/IP... DZ-Jay: Also, if it is not compatible with SMTP, how does anybody outside your own mail server network get it? For the moment I have the only server, but this system works in the same way than the standard e-mail ... using Domains and MX-DNS queries. Fastream Technologies: I wonder why he chose a binary format instead of text as no popular/common protocol is designed that way. The reason is simple, using a binary protocol ensures a minimum consumption of bytes through the net. And the most important reason, my protocol uses only 4 steps to send or receive an e-mail: 1- client: connection and authentication 2- server: response 3- client: work identification and e-mail transmision (in a single block!) 4- server: final response (CRC validation made!) 5- client: request to close connection This kind of protocol ensures a very fast interaction client/server and enables a native data compressión. Darin McGee So you lock out 99.9% of the email - no wonder it blocks spam. hehehe ... I added many features that ensures a real spam blocking, not only taking hand on the incompatibility, please read my webpage www.hidens.com.ar I meaning that all known spamming methods are blocked or minimized (from address thefts to mail pumps), except a user sending a real e-mail to another one (obviously). Well you can just denounce that user to the webmaster... Darin McGee That's what I love about standards, everybody has one :) Yes, the only problem here is how to popularize it... hehehe Thanks for your time to every body, you are invited to use it... David Jorge Aguirre Grazio www.djag.com.ar - Original Message - From: Dave Baxter [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Thursday, 07 February, 2008 10:29 AM Subject: Re: [twsocket] AN: New e-mail protocol (spam free and more!) I suspect that's half the point. Only like equipped users can communicate. Guess there could be a use in the financial or military markets, or other intentionaly closed environments... There again, I'd also guess they have such systems implemented already? Servers, nothing to stop you delivering directly, as many corporate systems do already, ours included, so long as you know the IP or domain address of course... I suspect for the above type of users, regular POP/SMTP/IMAP etc incompatability would not be a problem! Have to say though, spam is primeraly user driven from personal experience, from website form filling and so on. And what happens when a spammer gets hold of one of these secure mailer clients etc. Wonder why PGP or Open GPG is not as popular as it could be? There are plugins that integrate OK with the likes of Outlook (ugh!) Thunderbird, Pegasus etc... Ah, of course, the powers that be, like to watch what goes on Silly me... Cheers.. I'll crawl back under my rock, it's a bit too bright out here... Dave B. -Original Message- From: DZ-Jay [mailto:[EMAIL PROTECTED] Sent: Thursday, February 07, 2008 11:56 AM To: ICS support mailing Subject: Re: [twsocket] AN: New e-mail protocol (spam free and more!) Hello: Also, if it is not compatible with SMTP, how does anybody outside your own mail server network get it? And if it does communicate with external SMTP servers in order to inter-operate with other networks (otherwise, what's the point in sending yourself e-mail?) then it *is* susceptible to SPAM and abuse. dZ. -- DZ-Jay [TeamICS]
Re: [twsocket] SMTP Component not ready
I am sure this is not relevant, but just in case... You do realize that you have the address spelled wrong below? I imagine it should be smtp, not smpt? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jonathan M. Freedman Sent: Monday, January 07, 2008 7:06 PM To: twsocket@elists.org Subject: Re: [twsocket] SMTP Component not ready Dear List: I write this code: smtpCaribou-Host = [EMAIL PROTECTED] ; smtpCaribou-Port = 25 ; smtpCaribou-Connect() ; and get a smtp component not ready error message. Question...i don't think I need a twsocket connection first or do i? The server is a timewarner company server. I am a subscriber. I want to send mail directly from a program I have to individual client in my database without going thru Outlook. This is the first step I guess. jf -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hey DZ. Sorry, I didn't mean to drop out of this email thread. I have just been slammed for the last week and didn't have a chance to response to any of the further posts on this (they were buried in very long inbox). From what I see, Wilfried and Arno helped you out more than I would have anyway. Also, sorry I misunderstood your initial post about this. Story of my life... always coming in to the middle of a conversation confused and broke... ;) BTW, the pocket calculator comment was LOL... :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of DZ-Jay Sent: Thursday, November 29, 2007 4:52 AM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer and backlog Wait, I'm sorry, I perhaps did not explain correctly: It was taking 5 to 7 minutes for the server to *process* the client's request to completion, not the connection. My tests, although quick and dirty, are intended to check the behaviour of my application as a whole, not just the connection. For the sake of understanding, here's a brief explanation of my project: Its an e-mail queue server; it has a listening thread running TWSocketServer, which will receive requests from my queue client. The client communicates with a custom line-based protocol, and sends the message data, which will then be stored in the queue (filesystem) by the listening thread. A separate thread periodically scans the queue directories and dispatches the messages to the SMTP server. The client builds and encodes the entire message on the fly, line by line as the data is sent, to avoid having the entire thing on memory at once. But that's not really important to this discussion (I'm just proud of it :). A large message may take a few seconds to transmit. My tests all send the same message: a multi-part message with alternative text and html parts, and a small (4Kb) binary attachment, encoded in MIME. The whole thing was about 14Kb (I think, I'm not sure). I was sending 1000 of these. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) He, he, he. If it was processing 1000 *connections* in 5 minutes, I'd say a pocket calculator! 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? Not the client, but the server is performing some integrity checks, file IO, and eventually storing a record of the transaction in the database. The client does indeed build the message on the fly, even encoding all content as lines are sent to the server (I'm sorry, there I go again, but I think this is pretty nifty :), but it doesn't start doing that until after the connection is established and the message data is going to be sent. Plus both the server and the client were running on the same development machine, along with Delphi-hog-my-memory-2006, in debug mode, with no optimizations. Moreover, the client test app has a TMemo, displaying the progress, and in my rush to make a quick and dirty test, the test app does not free the client objects (all 1000 of them), until it finishes. So the slowness wasn't unexpected. The point of my previous message was to show the difference between two tests, when the only variable was the backlog value: a backlog of 5 took less than half the time to do the exact same work as a backlog of 500. The problem that I see is that the TWSocketServer seems to be taking too long (relatively speaking) to accept the connections. My client seems to be able to send lots of connection requests before a single one is established, thus abusing and exceeding the backlog queue. Of course, it could be my application that is preventing TWSocketServer from doing its work effectively, and if so, then perhaps I should consider using a multi-threaded server. I cringe at that thought, though, because I had so many problems getting TWSocketThrdServer to run properly (due to my own lack of experience with this sort of thing). Any recommendations would be greatly appreciated. dZ. On Nov 28, 2007, at 18:47, Hoby Smith wrote: H... If it is taking your system 5 to 7 MINUTES to process 1000 connect / disconnect cycles, then something is very wrong. I would have to rerun my tests, but I am thinking that I was doing 1K connect / disconnects in about 10 to 15 seconds when running both server and client on a single core P4. Perhaps a little faster using several client instances at the same time, although the performance maxed quickly on a single core CPU. I believe it was much faster on a 4 way Xeon machine I tested it on. I can get more specific stats for you, if you want them. But, whatever my specific results were, 5 to 7 MINUTES is just WAY off. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) 2. Is your client class on the server initiating a bunch of additional processes, like database lookups
Re: [twsocket] Webserver only with local connections
Great additional info, dz. Thanks... Hoby :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Tuesday, December 04, 2007 10:25 AM To: twsocket@elists.org Subject: Re: [twsocket] Webserver only with local connections George, I think Hoby's response is excellent, but I wanted to add a few suggestions based on my interpretation of your problem. 1. If the source address of each of the hosts that will communicate is known, then you can check for this using the GetPeerAddr method from within the SessionAvailable method of TWSocket (I believe this is exposed in HttpCli as the OnClientConnect event), and as Hoby suggested, abort the connection if the address does not match. Obviously, this will bind your application to those specific addresses, so if they ever changed you'll need to make sure to update the validation values. Also, as Hoby mentioned, you must bear in mind that the source IP address can be spoofed, so depending on the criticality or exposure of your application, you may not want to trust it. 2. Authenticate the incoming request by using one of the common mechanisms supported by HttpCli. This still means that the connection needs to be accepted, and actively rejected if it failed authentication. One last note: Under no circumstances trust any information available on the HTTP request/response header for validation or authenticity (IP address, referrer, content-type, etc.); these are very trivial to forge. For most HTTP appliations, this is normally not a problem, as long as they do not expect any of those values to contain critical information that could affect the behaviour of the system. -dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html --- Original Message --- From: Hoby Smith[mailto:[EMAIL PROTECTED] Sent: 12/4/2007 12:05:56 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] Webserver only with local connections George... Fundamentally, there are really only two ways to constrain inbound connections regarding client identification. As I am sure you are aware, keep in mind that neither TCP nor HTTP have any built in mechanisms for facilitating client identification or rejecting connections based on any rules. As a result, it is irrelevant what address / port you listen on, as TCP will always attempt to allow the connection initially, unless a firewall or something else between the host and server prevents the process. It is up to you then to reject any unwanted connect attempts at some point. Given this, you must address the issue in either or both of the following areas: 1. Client origin or source of connection If you wish, you may determine the client's origin and reject the connection based on that origin. The TWSocket components give you this ability at the session level. The standard TCP stack has no mechanism to do this until after the session is actually arbitrated, so by default you must accept the connect (at least partially into the cycle), then reject the connection, once you have determined that you wish to do so. So, if you wanted to constrain the source address of the client to a local or specific address only, you could provide some functionality in the OnClientConnect event that determines the connecting source address and rejects the ones you don't want, such as if it is not in the local address space. Bear in mind that the local address space could originate from the local loopback (127.0.0.1), as well as one of the local NIC addresses as well. For example, you could use, TMyHttpConnection(Client).GetPeerAddr, to get the client's address and then determine if you want to disconnect it. You would have to provide this logic and any rules as you need. Also, bear in mind that the source address can be compromised through various attacks. PPTP and other forms of tunneling attempt to prevent those kinds of issues (such as with VPNs implementations). 2. Some form of secure authentication that should be at least moderately trustable. If I understand your need correctly, you are saying that you DO want to allow connections from other IP subnets, you just want to know if they are from you or something else. To accomplish this, you need to support some form of authentication, because there is NO inherent ability in TCP or HTTP to tell you this. This is what you are really looking for. Regardless of the source of origin, you are really trying to ask the question, Is this me or someone else? Right? If so, the only solution is to use some form of authentication to determine this. I personally don't use HTTP much because it is so weak in this regard; in that, it has no native ability to support authentication. As a result, you must address authentication very high up the stack, on top of protocols that understand nothing about security or authentication. However, there are several mechanisms for performing
Re: [twsocket] TWSocketServer and backlog
FYI... I ran into an issue with some test code I wrote a few months ago, which related to the backlog setting, as well as the annoying issue with Winsock running out of local ports. In my test, I was attempting to see how many connections could be handled by a particular process over a period of time. I believe my results showed that increasing this value can have a very negative effect on performance. Basically, the issue is inherent in how the TCP stack is implemented, not in how a particular app services the stack. I found that surpassing a particular connection rate threshold would result in an exponential gain in processing time on the listening stack. Meaning, the TCP stack performance decreases dramatically as you increase the number of pending connections, when the listening socket is receiving a high rate of connection requests. My assumption is that this is due to the increased overhead in managing the backlog queue. Given this, I made two observations, which may be wrong, but made sense to me. First, this is why the Winsock default is 5. I imagine that the Winsock stack implementation was designed with the perspective that if the backlog is actually filling up enough to reach 5 or more, then something is wrong. Probably, a couple more might be ok, but my results showed that as you increased this value under heavy load, your connection rate was very unpredictable, as well as instable (lots of failed connects). For the TCP/IP stack to be effective, it must be responsive enough to handle the low level connection requests in a timely fashion. If not, then you have a major low level servicing problem or the machine is seriously overloaded with TCP requests. In which case, you want to get connection errors, rather than an overloaded backlog scenario. Second, increasing this value surely creates a greater DOS attack surface, making you more vulnerable to bursts of socket open requests, and surely would make the effects of such an attack even worse. This might also be why the Winsock default is 5. However, as I personally don't think that there is really a practical solution to a well designed DOS attack, then this might not really be relevant. Nonetheless, it might be something you need to consider. So, given that, I personally don't recommend increasing the value. If your app can't service the stack with a backlog setting close to 5, then your system is just overloaded or not responsive for some reason. Anyway, that is what I determined from my testing results. If anyone has found to the contrary, please feel free to correct me... :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 12:58 PM To: twsocket@elists.org Subject: Re: [twsocket] TWSocketServer and backlog Hello: The problem with retrying is that it is not the same as a server full error when the maximum number of clients is reached; 100061 is essentially a port not open error, which is the same error you would get if the server is not running. So there is no real way to know that the listener is currently busy and the backlog full, or if the server is listening on a different port or disabled completely. I will certainly increase the backlog on my server, but will also consider building a number of retries in the connection routine of the client class. Thanks for the help. -dZ. --- Original Message --- From: Wilfried Mestdagh[mailto:[EMAIL PROTECTED] Sent: 11/28/2007 2:26:49 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog Hello dz, a client application should do at least a few (or infinity) retry's if connection fails. so normally not needed to increase it. On the other hand it does no harm to increase it. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Wednesday, November 28, 2007, 18:27, [EMAIL PROTECTED] wrote: Hello: While stress-testing my application, I noticed that I am able to send substantially many more connections in the time it takes the TWSocketServer to handle the incomming requests, causing the default backlog to fill up quickly. Obviously, I can increase the number, but seeing that the default is 5 (which seems rather low to me), I'm thinking that perhaps there may be a concern in setting this too high. Does anybody know what I should take into consideration before changing this value, and if there are any concerns with it being too high? Thanks, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
H... If it is taking your system 5 to 7 MINUTES to process 1000 connect / disconnect cycles, then something is very wrong. I would have to rerun my tests, but I am thinking that I was doing 1K connect / disconnects in about 10 to 15 seconds when running both server and client on a single core P4. Perhaps a little faster using several client instances at the same time, although the performance maxed quickly on a single core CPU. I believe it was much faster on a 4 way Xeon machine I tested it on. I can get more specific stats for you, if you want them. But, whatever my specific results were, 5 to 7 MINUTES is just WAY off. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? 3. Do you have any problems with other apps on your system running slow? Perhaps you have a bad driver or resource conflict with your NIC? Just some thoughts... Hoby -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 4:30 PM To: twsocket@elists.org Subject: Re: [twsocket] TWSocketServer and backlog Hello: Thank you for your very informative response. I was performing some tests on my server application by continually increasing the backlog value with some mixed results, which seem to coincide with your empirical analysis. I kept increasing the backlog value up until I reached 1000, but to my surprise, I noticed that the connections started failing after about 230 requests, out of 1000 clients. These were the first 230 requests, so the backlog queue was significantly less than its maximum. I also thought I noticed that the server was taking longer to respond, but didn't think much of it at the time. However, after reading your post I decided to try once again with a backlog of 5, and set a retry loop every time a connection failed. As expected, the connections started failing almost immediately after the test started. But much to my surprise, the connections were handled quicker -- sometimes orders of magnitude faster than before! As a reference, using my localhost as the server and client, with a test application spawning 1000 clients to connect one right after the other, and re-trying if they failed, it took about 5 to 7 minutes to process the entire lot; while it only took about 2 minutes to process with a backlog of 5. The test with a backlog limit of 5 retried much more times, of course, but when connections were established, they were processed faster. Still, it seems to me that TWSocketServer is taking too long to process incoming connections, as many connections can be queued in the backlog while its instantiating the client and dupping the socket. Any thoughts on this? -dZ. --- Original Message --- From: Hoby Smith[mailto:[EMAIL PROTECTED] Sent: 11/28/2007 5:31:09 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog FYI... I ran into an issue with some test code I wrote a few months ago, which related to the backlog setting, as well as the annoying issue with Winsock running out of local ports. In my test, I was attempting to see how many connections could be handled by a particular process over a period of time. I believe my results showed that increasing this value can have a very negative effect on performance. Basically, the issue is inherent in how the TCP stack is implemented, not in how a particular app services the stack. I found that surpassing a particular connection rate threshold would result in an exponential gain in processing time on the listening stack. Meaning, the TCP stack performance decreases dramatically as you increase the number of pending connections, when the listening socket is receiving a high rate of connection requests. My assumption is that this is due to the increased overhead in managing the backlog queue. Given this, I made two observations, which may be wrong, but made sense to me. First, this is why the Winsock default is 5. I imagine that the Winsock stack implementation was designed with the perspective that if the backlog is actually filling up enough to reach 5 or more, then something is wrong. Probably, a couple more might be ok, but my results showed that as you increased this value under heavy load, your connection rate was very unpredictable, as well as instable (lots of failed connects). For the TCP/IP stack to be effective, it must be responsive enough to handle the low level connection requests in a timely fashion. If not, then you have a major low level servicing problem or the machine is seriously overloaded with TCP requests. In which case, you want to get connection errors, rather than an overloaded backlog scenario. Second, increasing this value surely creates a greater DOS attack surface, making you more
Re: [twsocket] [OT] Writing Vista applications using Delphi
Great document! Thanks! Great timing for me, too. I have been working the last several days on trying to come up to speed on some Vista development issues and that was a great help. BTW, since you brought it up... ;) Can I ask a related question? If it is OT, then perhaps you can direct me to a better place to ask it? I have just got a new Vista Laptop (since you can't get anything else now) and am having the worst time trying to run BDS 2006 on it. I have tried all the approaches I have found on line, but it is just really unstable and often locks up Vista when I close BDS!!! It really surprises me that BDS can lock up Vista?!?!? Am I correct in assuming that my ONLY solution is to move to Delphi 2007 or RAD studio 2007? If anyone knows sometime more than me about trying to run 2006 STABLY on Vista, I would love to know it. All the web references help a bit, but not much. Since I just discovered that CodeGear's stance is that 2006 is not certified for Vista, I assume that means my ONLY choice is to move to a CodeGear 2007 product of some type. Oh well, ANOTHER product I have to buy THANKS to Microsoft. :( Ya know, this development habit is really getting expensive... ;) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Francois PIETTE Sent: Tuesday, November 13, 2007 9:40 AM To: twsocket@elists.org Subject: [twsocket] [OT] Writing Vista applications using Delphi Altough out of topic, I found this document very interesting: http://bdntv.borland.com/pix/fhaglund/VistaUACandDelphi/VistaUACandDelphi.pp t Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] Writing Vista applications using Delphi
Can you supply references to the white papers you mentioned? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dod Sent: Tuesday, November 13, 2007 10:29 AM To: ICS support mailing Subject: Re: [twsocket] [OT] Writing Vista applications using Delphi Hello Francois, I also found two or three white papers about it (and yours also), it is very important to read them if people want to make their application Vista compliant (and compatible). It took me 3 months last year to understand thoses new features that broke my applications. regards. FP Altough out of topic, I found this document very interesting: FP http://bdntv.borland.com/pix/fhaglund/VistaUACandDelphi/VistaUACandDelphi.pp t FP Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html FP -- FP [EMAIL PROTECTED] FP The author of the freeware multi-tier middleware MidWare FP The author of the freeware Internet Component Suite (ICS) FP http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] Writing Vista applications using Delphi
Wow... I hadn't thought of that... Interesting suggestion... Thanks. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Woody Sent: Tuesday, November 13, 2007 10:59 AM To: ICS support mailing Subject: Re: [twsocket] [OT] Writing Vista applications using Delphi From: Hoby Smith [EMAIL PROTECTED] I have just got a new Vista Laptop (since you can't get anything else now) and am having the worst time trying to run BDS 2006 on it. Why not run it in a virtual machine? Load up a VM with XP, install BDS 2006 and there ya go... Woody (TMW) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] Writing Vista applications using Delphi
Thanks, but no it doesn't. I also added rights to standard user account for appropriate subdirectories (bin, ObjResp). -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dod Sent: Tuesday, November 13, 2007 11:09 AM To: ICS support mailing Subject: Re: [twsocket] [OT] Writing Vista applications using Delphi Hello Hoby, Try to run BDS EXE as admin, do it help ? HS Great document! Thanks! Great timing for me, too. I have been working the HS last several days on trying to come up to speed on some Vista development HS issues and that was a great help. HS BTW, since you brought it up... ;) Can I ask a related question? If it is HS OT, then perhaps you can direct me to a better place to ask it? HS I have just got a new Vista Laptop (since you can't get anything else now) HS and am having the worst time trying to run BDS 2006 on it. I have tried all HS the approaches I have found on line, but it is just really unstable and HS often locks up Vista when I close BDS!!! It really surprises me that BDS can HS lock up Vista?!?!? HS Am I correct in assuming that my ONLY solution is to move to Delphi 2007 or HS RAD studio 2007? If anyone knows sometime more than me about trying to run HS 2006 STABLY on Vista, I would love to know it. All the web references help HS a bit, but not much. Since I just discovered that CodeGear's stance is that HS 2006 is not certified for Vista, I assume that means my ONLY choice is to HS move to a CodeGear 2007 product of some type. HS Oh well, ANOTHER product I have to buy THANKS to Microsoft. :( Ya know, HS this development habit is really getting expensive... ;) HS -Original Message- HS From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On HS Behalf Of Francois PIETTE HS Sent: Tuesday, November 13, 2007 9:40 AM HS To: twsocket@elists.org HS Subject: [twsocket] [OT] Writing Vista applications using Delphi HS Altough out of topic, I found this document very interesting: HS http://bdntv.borland.com/pix/fhaglund/VistaUACandDelphi/VistaUACandDelphi.pp HS t HS Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html HS -- HS [EMAIL PROTECTED] HS The author of the freeware multi-tier middleware MidWare HS The author of the freeware Internet Component Suite (ICS) HS http://www.overbyte.be HS -- HS To unsubscribe or change your settings for TWSocket mailing list HS please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket HS Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] Writing Vista applications using Delphi
You are probably correct, at least to some degree. However, I have found that there are a number of Vista bugs that are exacerbated by different applications, though the apps aren't actually the root of them. Specifically, there appears to a number of bugs in the Vista core dlls that are used by 3rd party apps, such as virus software, where the flaw is not the fault of the 3rd party app, but it doesn't occur until you have the 3rd party app. Very frustrating. For example, I have the CA Security Suite and had to download a Microsoft Vista hot fix due to a flaw in Vista that caused a number of apps (such as IE) to perform very poorly or come to a stop, due to the CA firewall encountering some packet filter bug. Oddly, my wife's laptop that has Vista Home and Student had no problem with it, while my laptop with Vista Business did. Ugghhh!! Also, there are known issues with one of the CodeGear studios if you have a logitech web cam driver installed. Go figure?!?!? Don't get me wrong, I have really liked Vista (to my surprise). And the UAC popups don't bother me at all, as they are just like some Linux desktops for acquiring root privileges for various admin funcs, and Linux seems to be more secure as a result (and other things of course). I guess end users might be annoyed by them, but they are probably exactly what I want for constraining admin functions. Evidently, one of the goals of Vista was to next gen some older technologies, such as the TCP/IP stack. I have come across some complaints that the new auto-adjusting TCP/IP window functionality has had issues with some routers and some software. That is obviously to be expected when releasing next generation features. Anyway, the point is that, as I understand it, Vista contains a large number of OS enhancements that can be severely affected by unsupported software, particularly in the debugger arena. While I get that the OS shouldn't allow the lock up to occur, Vista may not be sufficiently aware of these scenarios yet. It appears that Service Pack 1 that will be out next year should fix a large number of them, such as VERY POOR PERFORMANCE when copying files over a network from or to non-Vista machines. It appears from what I have seen that a large number of admins have moved back to XP often, due to these types of issues. Ooops... Sorry for the long OT post... -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Time Bandit Sent: Tuesday, November 13, 2007 3:44 PM To: ICS support mailing Subject: Re: [twsocket] [OT] Writing Vista applications using Delphi On Nov 13, 2007 3:23 PM, Hoby Smith [EMAIL PROTECTED] wrote: Actually, installing is no problem. However, running is a problem. It installs with no problem. It runs really unstable, crashes constantly and sometimes even locks Vista up. Yikes! If it locks up Vista, the problem is not in the program but in the OS in my opinion. No application should be capable of locking a properly implemented OS. But, my opinion is probably based on experience with OS other than Microsoft's one. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Digest, Vol 234, Issue 6
Thanks Francois. I will try to see if I can figure out why my client tester isn't processing messages and remove the ProcessMessages call. I agree, this is obviously a symptom of the underlying problem. However, TWSocketServer does not have what I need for these tests. I am not wanting to know concurrent or contiguous client connections. If I was, then yes, ClientCount would be sufficient. Rather, I am trying to determine how many connection lifecycles can occur over a period of time. This includes connect, build session (TWSOcketServer) and disconnect. This is not something that TWSocketServer will tell me. Ergo, my test is designed to determine this. Thanks much... :) Hoby -Original Message- Date: Sat, 25 Aug 2007 18:53:49 +0200 From: Francois PIETTE [EMAIL PROTECTED] Subject: Re: [twsocket] Problems writing tester with connection counts To: ICS support mailing twsocket@elists.org Message-ID: [EMAIL PROTECTED] Content-Type: text/plain; format=flowed; charset=iso-8859-1; reply-type=original 2. Even though I use local messages to start and stop the connections, the Client application will not process any other messages unless I call Application.Processmessages, which I call in the OnSessionConnected event. That is probably your problem. NEVER call ProcessMessages or any other form of the message pump from an event handler. You'll have events re-entered and this is generally very bad (even with a simple TButton !). btw: Use TWSocketServer for your server program. It already has most of what you need, for example ClientCount. -- [EMAIL PROTECTED] The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Problems writing tester with connection counts
Hello, again, Francois. Thanks for the additional input. Please forgive me, I don't mean to be pesty, but I am confused by how I could be reinventing the wheel?!?! Obviously, the challenge with exchanging information via this email format is that I am quite limited as to the scope of detail I can provide. I think the confusion is that I am trying to explain what I am doing in an abbreviated form, which makes it hard to communicate the actual details of the issue. Let me clarify a few things: As I understand, you are using a single connection to evaluate the time. Actually, the client tester is designed to be run simultaneously on multiple machines, as well as multiple instances on one machine. Yes, this version is single threaded itself and uses one connection. But, obviously, aggregating concurrent connections for a more appropriate result is the goal. I am simply using one single client as a base testing core, but am able to run multiple clients concurrently. This allows me to see the actual and effective performance points in response to seperate client streams on the same CPU, different host, etc. Additionally, just to make sure I am clear, I am not evaluating the time, but rather various types of lifecycles over a period of time. What you do in your code between OnClientConnected and the graceful close, and how you do it will drastically change the results. Fully understanding how Windows handle messages will help interpreting the results. Therein lies the rub. I do get how messages work in Windows; I have been coding in them back to the Borland C++ OWL framework. That is why I was confused as to why the TWSocket internal messages would own the app's thread manager like it is in my tester, especially when I am not calling ProcessMessages. The test code uses out-of-line messages to prevent this very problem. Inserting ProcessMessages is not a fix, but a test of the symptom. My lack of understanding lies in the TWSocket internals, not in Windows messaging, albeit I imagine that I do not have the grasp of it that you do... :) And if you use simultaneous connections, using a thread for the processing of each connection could change the numbers a lot (In both directions !) I agree. The server is built in two modes: One uses a managed thread pool for packet handling, the other uses a single threaded packet handler. By packet, I mean my transport packetizing entities, not IP units. Additionally, the client can be built in either of these two modes as well. These will allow me to compare threaded vs. inline processing. When I ran into this problem, I built both as single threaded handlers, so that I can remove the threading logic from the equation. You do as you like, but believe me, you are reinventing the wheel. The ClientCount property that you are referring to simply returns the number of concurrent connections and has no cummulative bearing over time. I am confused by your recommendation to use the ClientCount property to determine any of the metrics I am looking for. Perhaps, I am greatly confused, which I guess is possible. But, given your component pattern, I do not see the relevance of this property for my test. I have been building TCP/IP apps for quite a number of years, as well as using your great components for years as well, so I don't think I am a complete newbie. However, I definitely do not want to reinvent the wheel, but I was simply stumped by the simplicity of what I was doing and why I was having a problem with it. I am sure the problem lies in my code, not ICS. Anyway, thanks for the input. Perhaps this explains what I am doing more clearly. I did not mean to take so much of your time or generate so many emails. I was hoping there was a quick, short, illuminating answer to my delimma. I will figure this out. Thanks... Hoby -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Francois PIETTE Sent: Sunday, August 26, 2007 7:49 AM To: ICS support mailing Subject: Re: [twsocket] Problems writing tester with connection counts However, TWSocketServer does not have what I need for these tests. I am not wanting to know concurrent or contiguous client connections. If I was, then yes, ClientCount would be sufficient. Rather, I am trying to determine how many connection lifecycles can occur over a period of time. This includes connect, build session (TWSOcketServer) and disconnect. As I understand, you are using a single connection to evaluate the time. This will give a different time tahn if you used several parrallel connections. You must be aware of that. This is because of low level TCP/IP protocol negociation which can take place in parallel for several connections. So having many simultaneous connections result in an overall better result. Setting up a TCP connection and gracefully closing a TCP connection are processes which involve several packets to be exchanged between client and server. There is a
Re: [twsocket] Problems writing tester with connection counts
Thanks much. I will look through it. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Fastream Technologies Sent: Sunday, August 26, 2007 7:54 AM To: ICS support mailing Subject: Re: [twsocket] Problems writing tester with connection counts One additional note from me: If you use single ICS worker thread for the server, then you won't be able to utilize multiple CPU-cores. That's the same for client as well. To see how we cope with this problem on the client side, see the source of my WebStressTester at http://www.fastream.com/ics/Web%20Stress%20Tester%20SRC.zip It's in BCB2006. Best Regards, SZ -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Problems writing tester with connection counts
Hello... I am really stumped. I am attempting to write some testing applications that will test the performance of some TCP/IP transport components that I have written. My components use TWSocket components for the TCP/IP interface and implement an encrypted, packetizing stream over TCP/IP. The test is real simple. I am just trying to see how many connections per second I can make with the components. The problem is that the server and the client report different results for the number of connections and I CAN NOT figure out why?!?!?! I think the problem is my approach to writing the test applications, but I can't figure what where the problem is?!?!? Basically, there are two testing applications, one is a server and one is client. The test is just to see how many connections (sessions) can be generated during a given period, e.g. per second. The test is really simple, actually, which is what is stumping me. The Server works this way: 1. The server starts and immediately enters listening on a port. 2. Whenever the server receives a client connection, I increment a connect count. I do this in the Server's OnClientConnect event. 3. Whenever the server receives a client disconnect, I increment a disconnect count. I do this in the Server's OnClientDisconnect event. That is it for the server, other than a few minor functions, like letting the user manually clear the counts via a button on the form. The Client works this way: 1. The user sets a few controls (length of test, port, etc). 2. The user starts the start via a button. 3. A test flag is set and a system timer is started. 4. The Client initiates a connection to the server. 5. When the client gets a connection, I increment a connect counter and close the connection. I do this in the OnSessionConnected event. I close the connection by posting a message to my main form, which in turn closes the connection. This way, I don't close the connection from the Client Socket's message handler. 6. In the Client's OnSessionClosed event, I check to see if the test has finished. The test is finished when the timer has fired and a flag is cleared. If it has not finished, then I initiate another connection to the server. I do this by posting a message to the main form, which in turn starts the connection, so that I don't start the connection it in the Client Socket's message handler. There are two problems with this: 1. The count of connections on the server and the count of connections on the client will sometimes be off by a few. Not always, but most of the time. The first test will usually be same, say 128 on both ends. But, subsequent tests will be off by two or three, like server = 131 and client = 129. I do not understand this. Is this just an inherent problem with windows socket messages and how I have implemented the test? Perhaps, there are pending messages in the socket que that haven't been processed or something? I am concerned that I am missing an event or something. 2. Even though I use local messages to start and stop the connections, the Client application will not process any other messages unless I call Application.Processmessages, which I call in the OnSessionConnected event. If I do not call ProcessMessages, the timer event never fires and the test just runs forever or until I close the Server, then the Client application processes messages and stops the test. If I am doing the connects and disconnects using posted messages to the form, then why don't other messages get processed as well? Why is it looping in the TWSocket event handlers only? That is rather odd. I hope this makes sense. Is there a better way to implement this test? Do I just not understand the windows socket message handling enough to understand why this isn't working? I am tempted to just ignore the difference in counts, but I am concerned that there is an underlying flaw in how I am using the TWSocket components and I am concerned that I have a bug that I am not aware of in my components. Any input would be appreciated. Thanks much. Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] OT... RE: Register your ICS !
OT... Ok, I comin' clean! I am an evil procrastinator that needs to register my ICS. I repent of my procrastination and (once again) commit to send a post card out shortly... :) In my defense, I intend to join PA (Procrastinators Anonymous) soon, tomorrow maybe?!?! Wait... I'm busy tomorrow, so maybe on Monday, as Friday's are always so hectic! h... Actually, part of the problem is that I have moved to a rather drab geographic area. Over the years, I have lived all over the U.S., in some strikingly beautiful locations. However, while the place I live now is moderately picturesque, it doesn't compare with the other places I have lived and visited. So, actually, I think I have been suffering from topographical inhibitions, where I am embarrassed to send plain, dull postcards of my current locale. However, in the interest of submission, compliance and general camaraderie, I will send you a boring, drab postcard as you have requested. Maybe I will send you a package as well; with some pictures of the awesome, striking places I have lived and visited here in the U.S. For example, check out these pictures from my website of a trip I took in 2004 in the U.S. Northwest. I am definitely an amateur photographer, but these aren't too bad: http://www.romans618.org/hojotravels/showpic.php?PICNAME=images/standard/gla cierpics/glacierpics03.JPG http://www.romans618.org/hojotravels/tripday37.php http://www.romans618.org/hojotravels/showpic.php?PICNAME=images/standard/cra terpics/craterpics07.JPG http://www.romans618.org/hojotravels/tripday20.php http://www.romans618.org/hojotravels/showpic.php?PICNAME=images/standard/coo spics/coospics01.JPG http://www.romans618.org/hojotravels/tripday24.php Anyway, sorry it has taken so long. I will send one soon... :) ICS ROCKS! Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Question regarding Delphi 2007
Hello all. This might be a bit Off Topic. However, since Francois mentioned D2007, I hope you don't mind if I ask a more specific question regarding D2007. You mentioned that it is fast and reliable product. I was wondering if it is any better regarding resource usage than D2006? Specifically, regarding loading multiple instances / apps at a time for debugging dependant applications? Often, it is necessary for me to debug more than one Delphi app at a time, sometimes even 3 or more (server, client, broker, admin tools, etc). With older Delphis (D6, etc) this was NEVER a problem. But, D2006 is SO HORRIBLE regarding resource utilization, that I can't load more than 2 apps. On trying to load a third instance of D2006, it fails with numerous errors and just dies loading Delphi. Ugggh! And, annoyingly, I have a pretty beasty machine, more than I ever had with Delphi before for sure. I LOVE Delphi AND BORLAND, but D2006 is the most dissappointing and worst quality product that I have ever got from Borland. While it had some great new productivity features, it is full of so many bugs, including MAJOR memory leaks, that have yet to be fixed and that are so bad I sometimes have to reload Delphi 2006 several times a day or my machine comes to a grinding halt. If I am so bold as to try and load TWO instances of D2006, the problems appear to be more frequent. I am really shocked. I have never seen Borland put out a product this bad before, at least as regards quality. And QC is full of issues like that, but with no response or anticipated fix date. It has made me leery to try another product from them for a while. So, have you found that it can actually load more than 2 apps / instances at a time without dying? I would appreciate any feedback on this, because if it is better in this regard, it will be much easier for me to justify my company spending more money on yet another Delphi IDE. :) Thanks much... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Very strange problem with server and client software [thanks]...
Hello. Just wanted to say thanks for the great input over the weekend! I will consider the input. Francois, you are right, that particular part of the initial handshaking exchange probably is a flaw in the implementation, although it is a requirement in the protocol design. In the protocol design, it is not possible to prepare the CHANNEL before sending the packet, as modifying the channel parameters before the Client is ready will result in a garbled reception by the Client. The packet buffering to prepare the packet before re-syncing the channel works fine. That is the only place in the protocol design where that is a weakness. Every other packet exchange is basically non-modal and can occur in any random or unpredictable order. I guess I assumed that because I was not RECEIVING multi-threaded, or calling any message processing loops, or implementing any custom message pumps, that I wouldn't experience a sequencing issue of that nature. Obviously, I was wrong. :-) Thanks again. Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Very strange problem with server and client software...
Hello. I am experiencing some very strange symptoms with some programs that communicate using ICS. Summary: The programs communicate fine (stable and consistent) when they are run on DIFFERENT machines. However, when run on the SAME machine, the communication between the programs is very unpredictable (sporadic failures, varying in frequency). In contrast to this, there are CERTAIN machines, where they DO run fine (stable and consistent) on the same machine. Oddly the working machines have Delphi installed on them and the failing machines do NOT have Delphi installed on them (which shouldn't matter). Detail: (sorry so long) I have implemented a thin client architecture using a basic Server / Client pattern. Meaning, there are two programs; one functions as a Server (actually a Broker) using TWSocketServer and another Client program uses TWSocket. The Client program initiates the connection and generates TCP traffic of a varying nature, depending upon specific user activities. The sessions are persistent (the client stays connected for the life of the application). The session traffic uses my own packetized protocol, something I have done many times (including with ICS). The Server uses a single TWSocketServer on the Main Form for all communications, meaning no Server based threads for TCP handling. However, while processing inbound packets from the Client program, it does hand some chores off to a thread pool that does the mundane chores (DB lookups, etc). A thread may or may not send one or more packets to the client, depending upon the particular protocol chore they were handling (requesting data of different types, reporting on Client interface activities, etc). The Server's outbound packet calls via the TWSocketClients all funnel through one critical section, which should provide thread safety for outbound traffic. There are a few other items that may worth noting. There is an additional load balancing program (Router) that is not directly engaged in the session. The Client first connects to the Router, which provides Server connection information to the Client. The Client disconnects from the Router and re-connects to the directed Server. There is an Admin channel between the Router and Server (Broker) that provides intelligent load balancing info, which OBVIOUSLY uses a SEPARATE socket interface, distinct from Client / Server channel. Additionally, this all occurs on top of my own Secure Channel implementation, which uses bi-directional BlowFish encryptors and multiple private keys to seed various encryption and randomizing algorithms. Also, I am using a MySQL Database via the MicroOLAP MySQLDAC components, which MicroOLAP claims is thread safe and appears generally to be. Oddly, the Client and Server programs work fine when both are running on my 2 development machines. (As a result, I didn't catch this issue until later in development, like before a demo. :-( ). However, when I try to run both of them on any other machine (so far), they communicate very poorly. Basically, they experience symptoms of random packet drops, but I don't think that is actually the case. I can't really tell, because I can not reproduce the scenario on a development machine where I could trace and debug the issue. The session will just hang, usually during the handshaking stage (Channel setup), but not always. However, if the Server and Client are on DIFFERENT machines, they work wonderfully, no problems at all, no matter what MACHINES they are on. I have tried a BUNCH of things to attempt to isolate and / or alleviate the problem. I have changed the Threading property on the Server Socket, which seems to have no effect. I have also tried using local MySQL Server vs. Remote MySQL server, no effect either way. As well as a many other tests / solutions. All to no avail. I am stumped in a big way. I understand that this is a really complicated implementation and my issue could be caused by a NUMBER of things. However, most of my testing seems to isolate the issue to the communications layer, for some unknown reason. I have never seen a scenario where the programs run fine from different machines, but won't work on the same machine. I am really hoping someone might have seen something similar? Could it be a WinSock bug? Any help would be appreciated. Thanks much. Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Very strange problem with server and client software...
Thanks for the info guys... :) I assumed the issue must somehow be related to sequencing issues impacted by the faster, local speeds, although exactly how I wasn't sure. On further testing, I have discovered something of interest. After setting the TWSocketServer MultiThreaded property to true, it appears to possibly fix the instability problem, but seems to inject another problem. With MultiThreaded set to true, I generated some additional log info from the Server app. Oddly, I found that the packet handling sequence order is altered in a particular handshaking process, causing the channel security syncing to fail. Here is some pseudo logic for what happens when MultiThreaded is set to True. Again, for some reason, this DOESN'T happen on my development machine, just on some other machines, and only when Server and Client are on the same machine: - Discrete portion of session logic flow during channel sync: 1. [SERVER] Send Packet to Client with random Challenge parameters. 2. [SERVER] Resync Server side of Client channel using new security parameters. 3. [CLIENT] Upon receipt of Challenge parameters from Server, resync Client side of channel using new parameters. 4. [CLIENT] Send Challenge response to Server using new channel parameters. 5. [SERVER] Receive and validate Client Challenge response, asserting trusted or untrusted channel state on pass / fail. The problem is that #2 MUST occur before #5. However, with MultiThreaded set to true, it does not occur in the correct order on some machines when both programs are running on the same machine. I don't quite understand why that would be, as #1 and #2 are basically consecutive lines of code, where #5 is a process that will be handled by a different thread function after receiving the client response packet. It exhibits symptoms like ProcessMessages is being called after #1, which receives and process the Client response before #2 can occur, although I am not calling ProcessMessages. Pseudo logic for the code fragment for #1 and #2 would be: - Thread function 1 Line 1. Generate Random challenge parameters. Line 2. Send Client Packet with info. Line 3. Resync Client Channel parameters. ... - Thread Function 2 (upon receipt of Client Challenge Response) If ValidateClientParms then Session = trusted else AbortSession. Question: Why would setting MultiThreaded to True cause Thread Function 2 to fire after Thread Function Line 2 which handles the response, before Thread Function Line 3 can execute? Why would this only occur with MultiThreaded set to true and only on some machines and not others? I can work around this by preparing the Server Challenge packet first, re-initializing the channel, then bypassing the channel overhead and sending the prepared packet raw. However, that would be a real kludge and I am more concerned that similar sequencing issues might surface in other areas. Thanks much for your time... Hoby Previous Francois response: I don't think it is a winsock bug. The main difference between running programs on different machine and on the same machine is the speed. On the local machine, it is likely to be very fast. You have to pay attention to your DataAvailable event handler so that it doesn't care how data is split into packets. I suggest you use SocketSPY (see usermade page at ICS website) to spy on the communication between your programs, either when running on different computers or running on the same computer. You may modify SocketSpy slightly to log all traffic to a file foe later analysis and comparing running the program locally or remotely. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Very strange problem with server and client software [solution]...
Hello. Well, ultimately, here is my solution. This appears to work 99.9% of the time, maybe 100%. There was one failure, but not sure if it was a related problem... ;) Oddly, turning on the TWSocket MultiThreaded property tends to alleviate the majority of symptoms. As I understand, the only major difference when MultiThreaded is on is that the ClientSocket utilizes its own messaging interface? Right? As for the issue I mentioned in the last email I posted, I solved the sequencing issue this way. Previously, I had a global send packet routine, wrapped in a Critical Section for thread safety. I use a single Global send routine so that I can gather statistics in one place (total bytes sent, total packets sent, total session bytes, etc). Since the MultiThreaded message loop was picking up responses before I was prepared to receive them, I created an additional couple of global routines. One is a global post packet routine, which processes the packet, but uses PutDataInSendBuffer to prepare and que up the outbound traffic (framing, encryption, etc), instead of sending it immediately. Then, when I am prepared to handle any related responses, I call a new global Send Buffer procedure, that simply issues a 0 byte Send on the ClientSocket to begin sending the que'd data. I then changed the sequence order sensitive handshaking routines to use the PostPacket / SendBuffer routines instead. This approach has appeared to fix the issue sufficiently. I am perplexed as to the nature of the issue itself, but for now, this will suffice. The scenario of both programs (client and server) being on one system is never going to occur during a live production environment. Solving this issue is really only relevant for development, testing and demonstration purposes. None-the-less, I am concerned that the surfacing of this issue might reflect some deeper logic flaw in the system as a whole. Does this solution sound appropriate? It appears to work fine. And does the problem issue sound like it should occur at all? Or does it sound like I have done something really BAD? I still can't get over the fact that it works fine and never occurs on the proverbial my machine... :) Thanks... Hoby -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
