Re: [twsocket] Webserver only with local connections
What I tried to explain to you is the simplest method--if the client is in the same computer and if it uses 127.0.0.1 as the destination addess, then the source address from server's point of view must be 127.0.0.1 as well.. So if you use the listening IP as 127.0.0.1, then no other IP would be able to connect, which was your goal. For HTTP(S) authentication, you have 5 options: 1) HTTP/1.0 basic auth 2) HTTP/1.1 digest auth which is somewhat secure 3) NTLM, the securest yet the credentials-to-compare are out of your control, they are taken from Windows 4) GET param 5) POST data We support all except 4) implicitly in IQ Reverse Proxy: http://www.fastream.com/iqreverseproxy.php Best Regards, SZ On 12/4/07, Arno Garrels [EMAIL PROTECTED] wrote: Hoby Smith wrote: [Big snip] For example, you could use, TMyHttpConnection(Client).GetPeerAddr, to get the client's address and then determine if you want to disconnect it. You would have to provide this logic and any rules as you need. There are helper functions available in OverbyteIcsWinsock2.pas to get the list of interfaces including loopback and also a function to check whether an IP is in current subnet. -- Arno Garrels [Big snip] -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
No worries! Here's the update on this: I have *slightly* modified my application based on the suggestions and insight I received from this list. When I say slightly I mean a lot, but that sounds too ominous :) First, I switched to TWSocketThrdServer without a hitch (hurray! for Arno's hard work on it). This introduced a new bottleneck: the database calls needed to be synchronized, and so it was basically the same as running on a single thread but with the additional overhead of thread creation and synchronization. The end result was that it made my server slower. Which brings me to the second change I made: I re-factored all database requests into a separate thread and completely de-coupled the other two working threads from having to perform any database access or additional management logic; they just post messages to the new thread with the data they need stored in the database. This introduce a whole new level of complexity: that of inter-thread communications and how to cope with un-received posted messages if the thread needs to abort unexpectedly (since the other threads now send and forget and expect the database thread to do its thing). And this led me to the final change in my application: The new database thread became the overall manager of the application: it governs all other threads, instructs them when to start and stop, and appropriately deals with anybody's impromptu demise. So now I have 3 worker threads: 1. Queue Manager: performs all database access, inter-thread management, application initialization and recovery, and overall management of the entire queue. 2. Queue Receiver: accepts incoming client connections and stores their data into the queue, then post the necessary information to the Manager, making sure state is always recoverable in case of failure. 3. Queue Dispatcher: scans periodically the queue and sends the messages via SMTP, then posts a message to the Manager announcing their result (whether success or failure) so that it can update the database record and remove the message from the queue. It also receives notifications from the Manager whenever new messages arrive of higher priority, so that it can interrupt its current scan and address those. Overall, the new design is more elegant and flexible, and still very stable; but more importantly, it is now considerably faster than before (orders of magnitude), and none of the connection issues that I was encountering are manifesting anymore. For the sake of comparisson, it now can take about 30 to 40 seconds to send 1000 messages to the Queue Server. And that's with a backlog of 50. A backlog of 5 takes a few seconds more because (at most) a handful of connections need to be retried (10061 error). A backlog of 10 succeeds without retries and takes roughly the same time. This means that it was my application design which was impeding the performance of TWSocketServer, and not an inherent issue with TWSocket itself (DOH!). System resources are limited, of course, so in my opinion our empirical analysis on the usage of the backlog is still valid: a larger number seems to affect performance negatively without any overall gain in availability, especially under heavy stress. In conclusion, as Arno and Wilfried suggested from the beginning (and as Francois has always claimed), TWSocket is fast, efficient and fully capable of handling thousands of concurrent connections, provided there are sufficient resources for it, and that no _extensive_processing_ is competing with the socket communication. How's that for an endorsement :) Thanks to all of you who offered help and suggestions. Cheers! -dZ. --- Original Message --- From: Hoby Smith[mailto:[EMAIL PROTECTED] Sent: 12/5/2007 12:44:57 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog Hey DZ. Sorry, I didn't mean to drop out of this email thread. I have just been slammed for the last week and didn't have a chance to response to any of the further posts on this (they were buried in very long inbox). From what I see, Wilfried and Arno helped you out more than I would have anyway. Also, sorry I misunderstood your initial post about this. Story of my life... always coming in to the middle of a conversation confused and broke... ;) BTW, the pocket calculator comment was LOL... :) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS
Farhan, I'm posting a copy of this message to the list because I think other members can offer good help for you to migrate your application to ICS. The TWSocket component communicates with the WinSock API at the lowest levels, and therefore it may seem more complicated to deal with than other more abstracted components. However, this is precisely the reason it is so powerful, flexible and efficient. If I understand your question correctly, you want to send a stream of data of arbitrary length, or at least be able to send a large string without having to worry about buffer size or re-building the packets on the receiving end. You can do this quite easily by enabling LineMode and setting a very large number as the maximum line length (ideally, the maximum that your data can be). What this will do is make TWSocket receive all data packets and build the original string completely before triggering the OnDataAvailable event. Therefore, when the event is finally triggered, you are guaranteed to have the entire data. The only other thing you need to select an appropriate delimiter string or character that will let TWSocket know when end of the data has been received. Typically, for the higher-level Internet protocols (such as FTP, SMTP, POP3, etc), a combination of Carriage-Return and Line-Feed (#13#10) serves as the line delimiter. But for binary transfers, you may want to pick something that you can be sure to *not* exist in your data stream. It can be a single control character or a combination, as long as you can be sure it will not occur on your data. So, you set LineMode to true, LineLimit to a very high value (or the length of the data itself), and LineEnd as the end-of-data delimiter, then call SendLine() with the string you want to send: StrData := 'This is the data you want to send'; StrEnd:= #0#0; WSocket.LineMode := True; WSocket.LineLimit := 6553; // Length(StrData) + Length(StrEnd); WSocket.SendLine(StrData); Let me know if you need any more help with this. -dZ. --- Original Message --- From: Farhan Anwar[mailto:[EMAIL PROTECTED] Sent: 12/5/2007 1:23:23 PM To : [EMAIL PROTECTED] Cc : Subject : RE: ICS FARHAN ANWAR Yes Sir i have a question That will help me to convert The Project.If u can Answer then here is my question. In TSockets We can Send As Many data as we can In One line of Code.But In TWSocket We hae to Set the Buf Size,and at the receiving side we get te data in Small packets,or some thing like that.How can i assume that the data i got is the Continued Data sequence and what is the length of the actual data,or simply how can we send and receiove stream -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Winsock-Error 'Invalid Parameter' (10022) under Win98
Arno Garrels wrote: Try to set a breakpoint in WSocket.pas, procedure TCustomWSocket.Connect and single step thru the code. Then please post the line where the exception is fired. Or use MadExcept to generate a BugReport if you don't have Delphi installed. -- Arno Garrels [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html BTW: What happens if you remove FD_ACCEPT from FSelectEvent below? I'm sure this has been removed some months before but now it's back in my V6-version again, is it a regression? FSelectEvent := FD_READ or FD_WRITE or FD_CLOSE or FD_ACCEPT or FD_CONNECT; iStatus := WSocket_Synchronized_WSAASyncSelect(FHSocket, Handle, FMsg_WM_ASYNCSELECT, FSelectEvent); -- Arno Garrels [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Webserver only with local connections
Sorry, I missed that post. I never received it :( Thanks! - Original Message - From: Fastream Technologies [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Tuesday, December 04, 2007 2:21 PM Subject: Re: [twsocket] Webserver only with local connections I have already told you the best way. Make the listening IP 127.0.0.1 . On 12/4/07, George [EMAIL PROTECTED] wrote: Hello, I created a THttpServer that is used as a middleware between my delphi app and a flash application. I want to make sure that only me and flash will communicate each other so I want to restrict connections from outside internet or local IPs. What's the safest way to check it? Any piece of code is welcomed. Thanks George -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] AV in TWSocketThrdServer.PutDataInSendBuffer
Hello:
I was testing my application and and noticed in
the log that when I tried to destroy the
TWSocketThrdServer while it had opened connections, I
got a few (seemingly) random AVs which I traced to
the following lines in PutDataInSendBuffer method:
procedure TCustomWSocket.PutDataInSendBuffer(Data :
Pointer; Len : Integer);
var
oBuffer : TBuffer;
cWritten : Integer;
bMore : Boolean;
begin
if (Len = 0) or (Data = nil) then
exit;
{$IFDEF COMPILER2_UP}
EnterCriticalSection(GSendBufCritSect);
try
{$ENDIF}
{ THE FOLLOWING LINE }
if FBufList.Count = 0 then begin
oBuffer := TBuffer.Create(FBufSize);
FBufList.Add(oBuffer);
end
else
oBuffer := FBufList.Last;
Inc(FBufferedByteCount, Len);
bMore := TRUE;
while bMore do begin
cWritten := oBuffer.Write(Data, Len);
if cWritten = Len then
bMore := FALSE
else begin
Len := Len - cWritten;
Data := PChar(Data) + cWritten;
if Len 0 then
bMore := FALSE
else begin
oBuffer := TBuffer.Create(FBufSize);
FBufList.Add(oBuffer);
end;
end;
end;
bAllSent := FALSE;
{$IFDEF COMPILER2_UP}
finally
LeaveCriticalSection(GSendBufCritSect);
end;
{$ENDIF}
end;
The full error is [EAccessViolation] Access
violation at address 00469094 in module
'SmailQ_con.exe'. Read of address 0008
Its hard for me to reproduce exactly, but has
anybody any idea what could be causing this?
-dZ.
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hey DZ. Sorry, I didn't mean to drop out of this email thread. I have just been slammed for the last week and didn't have a chance to response to any of the further posts on this (they were buried in very long inbox). From what I see, Wilfried and Arno helped you out more than I would have anyway. Also, sorry I misunderstood your initial post about this. Story of my life... always coming in to the middle of a conversation confused and broke... ;) BTW, the pocket calculator comment was LOL... :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of DZ-Jay Sent: Thursday, November 29, 2007 4:52 AM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer and backlog Wait, I'm sorry, I perhaps did not explain correctly: It was taking 5 to 7 minutes for the server to *process* the client's request to completion, not the connection. My tests, although quick and dirty, are intended to check the behaviour of my application as a whole, not just the connection. For the sake of understanding, here's a brief explanation of my project: Its an e-mail queue server; it has a listening thread running TWSocketServer, which will receive requests from my queue client. The client communicates with a custom line-based protocol, and sends the message data, which will then be stored in the queue (filesystem) by the listening thread. A separate thread periodically scans the queue directories and dispatches the messages to the SMTP server. The client builds and encodes the entire message on the fly, line by line as the data is sent, to avoid having the entire thing on memory at once. But that's not really important to this discussion (I'm just proud of it :). A large message may take a few seconds to transmit. My tests all send the same message: a multi-part message with alternative text and html parts, and a small (4Kb) binary attachment, encoded in MIME. The whole thing was about 14Kb (I think, I'm not sure). I was sending 1000 of these. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) He, he, he. If it was processing 1000 *connections* in 5 minutes, I'd say a pocket calculator! 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? Not the client, but the server is performing some integrity checks, file IO, and eventually storing a record of the transaction in the database. The client does indeed build the message on the fly, even encoding all content as lines are sent to the server (I'm sorry, there I go again, but I think this is pretty nifty :), but it doesn't start doing that until after the connection is established and the message data is going to be sent. Plus both the server and the client were running on the same development machine, along with Delphi-hog-my-memory-2006, in debug mode, with no optimizations. Moreover, the client test app has a TMemo, displaying the progress, and in my rush to make a quick and dirty test, the test app does not free the client objects (all 1000 of them), until it finishes. So the slowness wasn't unexpected. The point of my previous message was to show the difference between two tests, when the only variable was the backlog value: a backlog of 5 took less than half the time to do the exact same work as a backlog of 500. The problem that I see is that the TWSocketServer seems to be taking too long (relatively speaking) to accept the connections. My client seems to be able to send lots of connection requests before a single one is established, thus abusing and exceeding the backlog queue. Of course, it could be my application that is preventing TWSocketServer from doing its work effectively, and if so, then perhaps I should consider using a multi-threaded server. I cringe at that thought, though, because I had so many problems getting TWSocketThrdServer to run properly (due to my own lack of experience with this sort of thing). Any recommendations would be greatly appreciated. dZ. On Nov 28, 2007, at 18:47, Hoby Smith wrote: H... If it is taking your system 5 to 7 MINUTES to process 1000 connect / disconnect cycles, then something is very wrong. I would have to rerun my tests, but I am thinking that I was doing 1K connect / disconnects in about 10 to 15 seconds when running both server and client on a single core P4. Perhaps a little faster using several client instances at the same time, although the performance maxed quickly on a single core CPU. I believe it was much faster on a 4 way Xeon machine I tested it on. I can get more specific stats for you, if you want them. But, whatever my specific results were, 5 to 7 MINUTES is just WAY off. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or
Re: [twsocket] Winsock-Error 'Invalid Parameter' (10022) under Win98
[EMAIL PROTECTED] wrote: Hello I'm using the ICS and Delphi 4 to test the mail-send demo on an system with Win98SE. When attempting to connect to the mailserver it throws the winsock error 'Invalid Parameter' (10022). Causing this the version of the winsock-dll? Installed: ws2_32.dll / 4.10. On an XP-system it works fine. Any ideas to solve that problem ? (Not changing the System ;-)) Try to set a breakpoint in WSocket.pas, procedure TCustomWSocket.Connect and single step thru the code. Then please post the line where the exception is fired. BTW: What happens if you remove FD_ACCEPT from FSelectEvent below? I'm sure this has been removed some months before but now it's back in my V6-version again, is it a regression? FSelectEvent := FD_READ or FD_WRITE or FD_CLOSE or FD_ACCEPT or FD_CONNECT; iStatus := WSocket_Synchronized_WSAASyncSelect(FHSocket, Handle, FMsg_WM_ASYNCSELECT, FSelectEvent); -- Arno Garrels [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Winsock-Error 'Invalid Parameter' (10022) under Win98
Hello I'm using the ICS and Delphi 4 to test the mail-send demo on an system with Win98SE. When attempting to connect to the mailserver it throws the winsock error 'Invalid Parameter' (10022). Causing this the version of the winsock-dll? Installed: ws2_32.dll / 4.10. On an XP-system it works fine. Any ideas to solve that problem ? (Not changing the System ;-)) Many Thanks. Hans -- GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS. Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] WSocketTS won't build in d2
Are there some conditional defines missing from this unit? Delphi 2 errors on defines for FreeAndNil, Longword and TagMsg. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
