[twsocket] SessionConnected/SessionClosed are confusing app's workflow
Hello all, I experienced a situation when TWSocket's handlers confuse application workflow. I have SessionConnected handler async-ly reporting an error and SessionClosed handler async-ly reconnecting/pausing/reporting an error - depends of several conditions. Everything worked fine for months, but today the server I was connecting to went down. Here's the log (Pos/SetState are my methods): 2011-10-12 10:37:39.3117FD10330 Socket handle created 3572 2011-10-12 10:37:39.311SocketChangeState wsClosed -> wsOpened 2011-10-12 10:37:39.311TWSocket will connect to 2011-10-12 10:37:39.311SocketChangeState wsOpened -> wsConnecting 2011-10-12 10:37:40.326SocketChangeState wsConnecting -> wsConnected 2011-10-12 10:37:40.326SocketSessionConnected, Connection refused (#10061) // OK. Error on connect. My code commands to report an error and reconnect 2011-10-12 10:37:40.326PostState: csConnecting -> csError 2011-10-12 10:37:40.3267FD10330 TCustomWSocket.Shutdown 1 3572 2011-10-12 10:37:40.326SocketChangeState wsConnected -> wsClosed // ! Oops. The socket was "kinda connected" and then closed 2011-10-12 10:37:40.326SocketSessionClosed // ! Looks like just an accident closing after successful connect, closing WITHOUT ANY ERROR. So my code commands to fall asleep 2011-10-12 10:37:40.326PostState: csConnecting -> csSleeping 2011-10-12 10:37:40.326SetState: csConnecting -> csError // OK. Here the state commanded in SocketSessionConnected is finally set 2011-10-12 10:37:40.326PostState: csError -> csReconnect // OK. Here we command to retry after an error 2011-10-12 10:37:40.326SetState: csError -> csSleeping // Oops! Command from SocketSessionClosed has arrived 2011-10-12 10:37:40.326SetState: csSleeping -> csReconnect // Damn, the previous reconnect command has arrived ... Frankly speaking, there was no serious trouble except for strange high CPU load which I still can't track. But I think the very principle of calling the methods here enables possible troubles. I see two major danger sources here: 1) Despite of error on connect, socket state changes to wsConnected anyway. 2) Close is called without any error code so it looks like an accidental line break. What have I do to handle the descibed situation properly? * I can't use only OnSessionConnected because then I won't catch disconnects after successful connect * I can't use only OnSessionClosed because then I won't realise that an error occured on connect * I can't use both of them because they're messing up. So for now I have to invent some flag to tell the OnSessionClosed that OnSessionConnected has already handled the situation. Maybe you have more clever alternative? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] DLL implementation advice (or example)...
>I have an existing DLL that was written to use TClientSocket via a data >module. It has a kind of message pump but it works in reverse... it has a >timer thread that uses postmessage to trigger the main app to call a >ReadData function which checks the socket for received data, then processes it. It's a really overcomplicated one! I'd recommend you to isolate main app objects/variables from DLL ones as strongly as possible. For example: * Main app calls DLL's Connect() with a pointer to callback method. * DLL's Connect() saves this pointer, allocates buffer for data and connects to the host specified * In OnDataAvailable DLL copies data to buffer and calls the callback with parameters (PBufStart: PByte; PDataLen: Integer). * Callback copies data to application's buffer thread-safely (you'll have to use additional thread for socket messages otherwise you'll get funny side effect when DLL's Socket.MessageLoop processes main app's messages) and signals main app thread to process the data received. Callback should return number of bytes it has read so that DLL could move a pointer to data start in the buffer. In general, only simple and cross-language types should be used when interacting with DLL. This means NO CLASSES in parameters/results! -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] DLL implementation advice (or example)...
Adam, you should read my recent message called "ICS in DLL. My report about results" here in mail list. I faced with tons of troubles because of DLL nature so finally I returned to my own blocking socket classes (just a simple WinSock wrapper - fortunately I needed only basic functions in DLL). For client library it works flawlessly. For server and complicated client apps I continue using ICS. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: Mustangpeak Virtual Shell Tools
I think it's not the proper place for this subject but I see no problem here: take the sources and modify them as you wish, it's OpenSource! I'd also remove all TNT stuff because it became useless since D2009 but maybe backward compatibility must be kept. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] Nice forum engine based on the mailing list
Guys, I know most of you are fond of newsgroups and all that stuff, but no one tells you to stop using them. The resource I've found provides an additional forum-like interface for the maillist, with no efforts for the admins excepting first-time registration. You may continue using what you like while other people like me who are incompatible with newsgroubs could use what they like. P.S. It also saves from wrong topics and replying from other mail account what happens to me quite often. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [OT] Nice forum engine based on the mailing list
I just found a nice thing at http://www.nabble.com/. It provides forum-like interface based on the mailing list. I find it much more useful than mail archives especially if you want to read complete conversation. Would be awesome if ICS list is accessible via this site. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS in DLL. My report about results
>Why don't you use initialization/finalization sections? Do you really think that these sections are executed not from DLLMain? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] ICS in DLL. My report about results
After 1.5 days of struggling with DLL and async model I decided to share some experience. What I needed is DLL with several functions interacting via sockets with a remote app. They must send request messages, receive responces, process them and return as out parameters. Function call must be blocking to avoid bugs in the host app. So at first I simply tried MessageLoop inside the function call. But - it's running in the main thread's (i.e. thread in host app which called the function) context - so it processed all the windows messages to host app like Application.ProcessMessages does. Interesting side effect but not usable in my case. Second thing is that I wanted all sockets to be created only on demand so I couldn't use its MessageLoop method as I couldn't tell which socket currently exists. So I came up to additional thread with its own message loop. But how could I achieve synchronization between calling functions and thread methods? Events? They increased complicateness much. So I had a nice idea: SendMessage. It blocks the calling thread while processing is done in the context of the window owner. Awesome! I've done all the stuff and everything seemed good and working when I faced with DLLMain. In my previous version I used some init stuff with no troubles so I thought some additional method calls would be OK. Unfortunately DLLMain appeared to be a very tricky thing: you can't call methods from other DLLs from there (incl GetProcAddress) as they could have not been loaded yet, and no sync methods too. But I used them widely to ensure resource cleanup on DLL unload. This resulted in deadlock and overall app hang when unloading DLL. I revised the situation but found only one way: provide special Start/Finish methods for init/cleanup. Not a good variant in my case but at least something. But the strangest thing is that deadlocks kept happening even when I explicitly call Finish before FreeLibrary (btw Finish executes all its commands without errors)! So I finally got tired of all this stuff and returned to my own simple blocking sockets - for DLL and in my case this seems much more applicable. Anyway I decided to share my experience - maybe it'll make a little help to someone (I really loved that method with SendMessage; things also must be working with explicit Start/Finish procedures but they don't and I have no wish to investigate why). Here's demo project I created for testing. It uses OverbyteIcsTcpSrv.exe launched on localhost as a server: http://uploadbox.com/files/7466a54787/ P.S. I examined ICS DLL demo but it closes the connection every time - it's not an option for me. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Suggestion: set FLastError on session connect/disconnect
>I think you should use only one thread for all socket operation within your application and one thread for each lengthy processing/computation you have to do. The rule is to avoid lengthy processing within the thread doing communication. Yes, that's exactly I intended to do except I don't need long processing so just a single thread inside DLL for socket message loop >In my opinion, you should revise you design to better fit the programming model used for ICS. Otherwise you'll end up with a complex application difficult to maintain a prone to errors. Well, I can't imagine something else without complicating the structure. I need several functions to sync-ly communicate thru sockets. So WaitFor-s and again WaitFor-s... >See above. Pausing/resuming the socket shouldn't be used, specially if you have your own worker thread. Just have the socket event handler blocked waiting for some synchronization object. You mean blocking the entire message loop? I'll implement it myself without using socket method as I don't know what set of sockets would exist. >Yes, you have to detach the socket from the worker thread before destroying the socket. Got it! >As I said before, you can avoid thread Attach/detach with careful design. Having to use attached/detach is the sign of a too complex design. Another ways to reach comatibility between sync and async is are ugly (Synchronize) or complicated (SendMessage or smth else). I'll try with attach/detach -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Suggestion: set FLastError on session connect/disconnect
>You get the best isolation from >the host application by creating a worker thread and run most of your code >in the context of this thread. Okay, I think I got the structure which seems correct. As I need several sockets for several purposes, I can't use socket's message loop (I don't want to create all of them on DLL load). And I export several functions each of them uses its own socket. TSocketThread GetSocket(channel: TChannel) * return appropriate socket (create and init it if necessary, attach to the thread with ThreadAttach - I can't create them inside Execute because it's more complicated with my design). Execute() cycle while not terminated * WaitFor activate event (to avoid socket events outside the DLL functions — or maybe I just may pause all the sockets?) * do message loop TCommSocket InitConnection() * connect if closed, WaitFor connection event (set in SessionConnected/Closed/Timeout/... handlers) * send some init stuff Communicate() * send request * WaitFor reply received event (set in DataAvail/SessionClosed/Timeout/... handlers) DLLDoCommunicate * GetSocket * set thread's activate event (or resume socket?) * InitConnection * Communicate * process the received data * unset thread's activate event (or pause socket?) Questions are: 1) How to avoid socket events outside the DLL functions (I guess pausing/resuming the sockets is better cause I'll only have one active socket at a time — the one I need) 2) Do I have to do socket.ThreadDetach on app closing or it's OK? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Creatively annoying a host
>I could leave the connection open without sending a response? You may delay it for, say, 5 sec. But then I guess the vulnerability to DDOS increases (thousands of connections simultaneously). I'm not sure how much of them is possible. >Or send a vast amount of rubbish data in response? Maybe — if you have unlimited outgoing traffic. But no chances that scanner software would catch all of the rubbish sent. It could abort the connection after some amount of data received and leave your server sending something to nowhere. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Suggestion: set FLastError on session connect/disconnect
2Arno: >No problem to use async in a DLL, have a look at OverbyteIcsDll1 demo, >it shows a multi-thread capable implementation. If you don't need MT >have a look at the console demos, those with "con" in name, the same >princple works in a DLL as well. Thanks! I digged into OverbyteIcsDll1 demo and got a question. I see that in DLL demo the thread is responsible for message processing. But what if I don't need a thread? I tryed with while .. do MessagePump but noticed that it processes even the messages of the host app! So no sync here. I suspect console demo approach won't work for me too (am I right?) 2Francois: >If you need to export synchronous functions out of your DLL, I would suggest >using a thread >withing the DLL, having his own message ump and working purely >asynchronoulsy as ICS does so well. Then the exported function will talk to >this thread using traditional multithread synchonization mechanisms to looks >blocking from the outside. A Windows event is a perfect item for that kind >of job: the exported function will start the asynchonous operation by the >worker thread and wait on an event which will be set by the worker thread >once the work is done. Quite easy to implement without any change in TWSocket. Yeah, I guess it's the only way. I have to create separate thread or a socket would process host app events while waiting. I'll consider DoneFlag or even Events to easily implement waiting. >If you really want to do something like you have show, do NOT loop on >the state property value but use the events and your own application level >state variable. TWSocket.State has never been intended for waiting on a >given state. Hmm you mean State isn't reliable? But why? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Suggestion: set FLastError on session connect/disconnect
Hmm, things appeared not so simple as they seemed. Assigning LastError on both session connect and disconnect immediately rewrites connection error code with 0 caused by successful close. Currently I have no nice idea but maybe someone has? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Suggestion: set FLastError on session connect/disconnect
I now trying to implement sync socket actions (I'll need to make DLL so async model doesn't fit). procedure TSockClient.Request; begin if State <> wsConnected then begin Connect; while not (State in [wsConnected, wsClosed]) do MessagePump; if State <> wsConnected then begin //.. Exit; end; end; ...send request... end; If I try to connect to not opened addr-port, the error is transferred to OnSessionConnected. But I'd like to have result code of the last operation in the socket's field without adding an OnSessionConnected handler. LastError is what I want but it's not changed on session connect/disconnect. If that hasn't been done by intention, I guess that should be implemented. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Possible CliId duplication
>Let's do some computation: the ID wrap around after 2.147.483.647 >connections. If you have 100 connections/disconnections per second, then it >will take 248 days to wrap around. Well, "if FClientNum >= $7F then" currently it wraps around after 8,388,607 what is very little and might be achieved in 23 hours (less than 1 day!). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Possible CliId duplication
>It is only guaranteed that _connected clients never share the same CliId. I couldn't see _where_ it is guaranteed... It's quite possible to have 1st client with ID 1 connected for a long, long time while others connect and disconnect very frequently (i.e. large proxy and a client downloading huge file thru it). They could fill the counter and it would reset to ID 1 which already exists. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Possible CliId duplication
In OverbyteIcsWSocketS unit, TCustomWSocketServer.TriggerSessionAvailable method, Client.FCliId is determined as incremental counter. But as this counter is resetted after $7F, it is possible that some client IDs are duplicated. Of course it would be a very rare case but it could lead to serious troubles if an application uses CliId fields for some identification. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Bomb
That Bomb trick causes some troubles like Code insight failure or something
else - I don't remember exactly, but I experienced one. Actually it could be
replaced by {$MESSAGE FATAL 'Wrong defines'} undestandable by new compilers,
and old compilers would stop on it anyway.
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout socket request
Arno, thanks for the tip, I'll give it a try. P.S. I noticed right now that there are useless things in TCustomWSocket.WMASyncSelect : TTTCount constant and FSelectMessage field. They probably should be removed. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Timeout socket request
Hello all, I have a small request for timeout socket. By Arno's advice writing to this list. I recently faced with necessity of tracking ticks inside a socket (actually I want to pause it for some time and then resume). As I use BUILTIN_TIMEOUT, I tried to attach to TimeoutTimer but it's unaccessible to descendant classes. So my suggestion is to somehow make timer itself or its OnTimer event accessible from outside. I guess the simplest way is to add published property OnTimer and launch it in TimeoutHandleTimer (or maybe TriggerTimer method should be added to fully follow ICS event handler style). I've already implemented this and may share the modifications. -- Regards, Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTML encoding to char
>You may also use my function which I wrote for a XML: Ahh, sorry, you need decoding... here's algorythm: 1) Search for & 2) Search for ; after & 3) Get text between & and ; as S 4) idx := AnsiIndexStr(S, XmlEntities); if idx = -1 then ... // Error or ignore 5) replacement char is XmlCtrlChars[idx+1]; -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTML encoding to char
>Delphi HTTPApp unit, function HTMLDecode
Wow! I had no idea of this unit, Thanks a lot! HTTP date parsing, Cookies (!),
entuity en/decoding... THttpCli might take some useful code from this unit :)
But as I see, HTMLEncode touches only special chars. If that's what you need,
fine. You may also use my function which I wrote for a XML:
const
XmlCtrlChars = '<>&"''';
XmlEntities: array[1..Length(XmlCtrlChars)] of TXmlString = ('lt', 'gt',
'amp', 'quot', 'apos');
// Replaces special chars by entity-codes
function TextToXML(const aText: TXmlString): TXmlString;
var CtrlCharIdx, PrevIdx, CurrIdx: Integer;
begin
PrevIdx := 1; Result := '';
CurrIdx := 1;
while CurrIdx <= Length(aText) do
begin
CtrlCharIdx := Pos(aText[CurrIdx], XmlCtrlChars);
if CtrlCharIdx <> 0 then
begin
Result := Result + Copy(aText, PrevIdx, CurrIdx - PrevIdx) +
'&'+XmlEntities[CtrlCharIdx]+';';
PrevIdx := CurrIdx + 1;
end;
Inc(CurrIdx);
end;
if Result = ''
then Result := aText
else Result := Result + Copy(aText, PrevIdx, CurrIdx - PrevIdx);
end;
If you need full replacement of all entities, I may share that code too but
it's much bigger (because of all entities declaration)
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] OverbyteIcsHttpProt suggestions
Hello Francois, thanks for answering.
2) $IF defines
for my units, I replaced the include file with COMPILER_##_UP defines to the
file with following text:
// compiler version constants for checking with $IF directive (only for
Delphi7+)
const
RAD_XE = 22;
RAD_2010 = 21;
RAD_2009 = 20;
RAD_2007 = 19;
RAD_2006 = 18;
Delphi_9 = 17;
Delphi_8 = 16;
Delphi_7 = 15;
so my version checks look like {$IF CompilerVersion >= RAD_2009} ... {$IFEND}
And compilers lower than D7 won't compile conditional defines anyway.
Concerning BCB, it has no separate meaning (BCB6 is too old, and later it was
included into BDS/RAD).
I remember this subject to be discussed some time ago, and your answer was NO,
but as long as $IF defines are already used, why don't use them everywhere.
4) THttpBigInt
>Strictly not needed, but make code easier to update.
Well, I don't think that in the near time the value of 17,179,869,183 GIGABYTES
would be not enough :) Besides, don't other components like FTP and Socket
itself need similar types too?
7) Last-modified and Date header fields
>I don't see exactly what you mean
I mean:
...
property LastModified : TDateTimeread FLastModified;
property Date : TDateTimeread FDate;
...
procedure THttpCli.GetHeaderLineNext;
...
else if Field = 'date' then
FDate := RFC1123_StrToDate(Trim(Data))
...
else if Field = 'last-modified' then
FLastModified := RFC1123_StrToDate(Trim(Data))
...
end
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] OverbyteIcsHttpProt suggestions
ICS guys, I humbly ask you to pay attention on my previous message. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Cookie Manager
Daniele, great! Built-in coookie manager would be a great improvement! Looked quickly at your unit. You have many code in StringToCookie that formats date, but OverbyteIcsHttpProt.pas already has functions RFC1123_Date and RFC1123_StrToDate -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] OverbyteIcsHttpProt suggestions
I'm now working with HttpCli and after examining OverbyteIcsHttpProt unit I
have several suggestions
1) Excess ancient defines
While only D7+ is supported currently in ICS7, there are some legacy defines:
{$IFNDEF COMPILER7_UP}
'Sorry, we do not want to support ancient compilers any longer'
{$ENDIF}
{$IFDEF DELPHI6_UP} << useless
{$WARN SYMBOL_PLATFORM OFF}
{$WARN SYMBOL_LIBRARYOFF}
{$WARN SYMBOL_DEPRECATED OFF}
{$ENDIF}
{$IFDEF BCB3_UP} << {$IFDEF BCB} ?
{$ObjExportAll On}
{$ENDIF}
There's even some code for DELPHI1!
2) $IF defines
While COMPILER##_UP defines are used for determining compiler version, there
are some checks with $IF like {$IF CompilerVersion < 21}. Maybe it's better to
change compiler detection to IF-s? IF-s are supported in D7, the only thing
this version requires is defining a const CompilerVersion = 15.
3) HTTPCliWindowProc
function HTTPCliWindowProc is never used, maybe it should be removed?
4) THttpBigInt
THttpBigInt type - is it really needed?
5) Header value names
Such header value names as 'Content-Length: ' are repeated several times
throughout the unit, I think they should be declared as constants.
6) Current http doc properties clearing
There's a block of code which clears http doc properties, it is repeated three
times: once in THttpCli.InternalClear and twice in
THttpCli.LocationSessionClosed.
{ Clear header from previous operation }
FRcvdHeader.Clear;
FHeaderLineCount := 0;
FBodyLineCount:= 0;
FContentLength:= -1;
FContentType := ''; { 25/09/1999 }
FTransferEncoding := ''; { 28/12/2003 }
{$IFDEF UseContentCoding}
FContentEncoding := '';
{$ENDIF}
I'd extract this into separate method (what would ease addition of new http doc
properties)
7) Last-modified and Date header fields
I slightly modified RFC1123_Date and RFC1123_StrToDate and added
auto-conversion of Last-modified and Date header fields into HttpCli's
TDateTime fields. If you're interested, I can send my modifications to ICS team.
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] 64-bit Delphi preview
Angus >Only if your ICS project needs to access more than 2 gigs of memory (or >is it 4), or is a DLL called by other 64-bit applications. Well, native apps get more efficiency anyway. >ICS is currently being tested for 64-bit computability, which >unfortunately means replacing 32-bit assembler. I guess asm blocks could be adopted to x64 too. Emb-ro advices to reject asm at all though. I examined the ICS units some time ago and found only several places that *may* require changing. But maybe WinSock wrappers would require it as Windows' types lika UINT, LRESULT etc are used quite rarely, most coders use general integer types instead. Tobias >When working with Windows messages I often use a pattern like... Well, they paid a special attention of "message crackers" in the preview video. They are likely to be changed. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [OT] 64-bit Delphi preview
Finally, it's here! http://www.embarcadero.com/products/delphi/64-bit Maybe it's time to check all our projects for 64-bit compatibility? :) -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] I think I found an important bug in wsocket.pas
IMHO such operations as Abort should be as much quiet as possible. Nobody cares the results, they just must work. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [OT] Twitter tells third-party devs to stop making Twitter client apps
Just FYI Twitter tells third-party devs to stop making Twitter client apps "In a statement issued today by Twitter on its official developer mailing list, the company informed third-party developers that they should no longer attempt to build conventional Twitter client applications. In a move to increase the "consistency" of the user experience, Twitter wants more control over how its service is presented to users in all contexts." http://arstechnica.com/software/news/2011/03/twitter-tells-third-party-devs-to-stop-making-twitter-client-apps.ars -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Request: add a couple of debug log messages
Nice! Thanks Arno :) -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Request: add a couple of debug log messages
Would be nice if there would be debug log messages when creating (and maybe
accepting) sockets (including pointer and socket handle like in "... will be
connected to " message). It will help to track main stages of socket life.
I added these lines
{$IFNDEF NO_DEBUG_LOG}
if CheckLogOptions(loWsockInfo) then { V5.21 } { replaces $IFDEF
DEBUG_OUTPUT }
DebugLog(loWsockInfo, {$IFNDEF CLR}_IntToHex(INT_PTR(Self),
SizeOf(Pointer) * 2) + ' ' +{$ENDIF}
'Socket created ' + _IntToStr(FHSocket));
{$ENDIF}
to TCustomWSocket.Connect after "if FHSocket = INVALID_SOCKET then begin ...
end;" and these
{$IFNDEF NO_DEBUG_LOG}
if CheckLogOptions(loWsockInfo) then { V5.21 } { replaces $IFDEF
DEBUG_OUTPUT }
DebugLog(loWsockInfo, {$IFNDEF CLR}_IntToHex(INT_PTR(Self),
SizeOf(Pointer) * 2) + ' ' +{$ENDIF}
'Socket accepted ' + _IntToStr(FASocket));
{$ENDIF}
to TCustomWSocket.Accept after "Result := FASocket;".
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] AQTime reporting leak withDispatchMessage() usingTHttpCli
Albert, do you really need multithreadness with ICS? It's asynchronous so you just need one additional thread that will launch and control all sockets. And regarding the leaks, consider using FastMM4 in full debug mode to get more info (i.e. type of variable that has been leaked). DispatchMessage calls all message handlers so it could be any of them that is to blame for leak. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] 3proxy
3proxy developer answered (translation is mine) "What vesion was used? 0.7 has quite many fixes for HTTP/1.1 to work but old vesions are transferring 1.1 as 1.1 as well. Let them log request/replies or smth like that" Arno maybe you'll wish to contact 3APA3A personally? I've little idea of HTTP proxies so I'm not the one to link you two. There's 3proxy board at http://3proxy.ru/board/ (no auth required) and developer's email "3APA3A at security.nnov.ru". -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TTwitter component
>There's already a method through httpcli, but PIN approach should always be >encouraged. I don't know what PIN is but couldn't it be implemented without external apps/activeX? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TTwitter component
TWebbrowser? ShellExec?? Chromium?? Why don't you use ICS HTTPCli for authorizing? You may even display an original page with powerful ThtmlViewer component. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] udp packet loss
>var > lBuffer:array[0..1500] of AnsiChar; I'd advice you not to allocate static buffer inside a method because it is placed inside the stack every time method is called. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SVN update notifications
I think another maillist would be fine bacause you won't have to maintain an actual list of people interested in notifications. RSS would be good too. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: 3Proxy malware?
Arno, I'll post a message to the 3proxy's forum and give a link to this thread so that 3APA3A could be aware. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: 3Proxy malware?
>Is there some switch to turn that on? >I just tried briefly yesterday and it returned HTTP/1.0 >responses to my HTTP/1.1 request. Don't know, it should simply work! Are you sure to use latest version? I now tried to test with Opera with option "Use HTTP 1.1 for proxies" enabled but it used to send GET requests with "HTTP 1.0". I don't know how to force 1.1 mode. BTW, regarding "malwareness": developer refers to this link to read http://vil.nai.com/vil/content/v_100768.htm -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] OT: 3Proxy malware?
Arno, "18/02/2009 3[APA3A]tiny proxy 0.6 New features since 0.5 are marked with !. Features: 1. General + HTTP/1.1 Proxy with keep-alive client and server support, transparent proxy support." http://3proxy.ru/0.6.1/Release.notes.txt and this software is unlikely a trojan. I used it for long time, speaked with developer. And it also opensource. Maybe AV's detect its net communication parts. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Proxy traversal error handling
>however with socks "#20xxx is not a winsock error" >is displayed everywhere with socks errors What's the problem? ICS uses custom WSocketErrorDesc anyway so just add a definition of socks errors there. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Built-in timeout questions
Thanks Arno, I followed your advices and successfully solved both issues. Regarding timer I'll try ThreadTimer as I don't need precise resolution here -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Built-in timeout questions
2Arno >Method DnsLookup doesn't require a custom timeout since the underlaying >winsock function uses system's DNS lookup timeout. Same for Connect >with or without DNS name. >And if method Connect is called with a DNS name it performs a blocking >DNS lookup while a Built-In timeout won't trigger at all. Yes, this is blocking DNS lookup which I want to avoid. And I dislike relying on OS timeouts, maybe it'll be 5 minutes long? >You can do what ever you like i.e. (from memory): Thanks! I'll examine the snipped you provided! >Listening sockets do not connect so use TimeoutIdle in OnClientCreate >event Ah, the trick is not using connect timeout! I've got it. >TIcsThreadTimer is a lazy, low resolution timer, so in general no >TTimer replacement, use TIcsTimer if you need a thread-safe TTimer >replacement. TIcsTimer doesn't create a hidden window but uses >TWSocket's window to receive WM_TIMER messages, that saves >one window handle. Well, I want this timer to execute periodical and not time-precise tasks so resolution of GMinIcsTimerResolution : Longword = 100; is far than enough. And regarding TIcsTimer, would it work if there's no any TWSocket object created yet? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Built-in timeout questions
Regarding accepted sockets: that's how I solved the problem for now: procedure TServer.TriggerClientConnect(Client: TWSocketClient; Error: Word); begin inherited; if Error <> 0 then Exit; with TSrvClient(Client) do begin TimeoutSampling := 1000; TimeoutConnect := InactiveTimeout; TimeoutIdle := InactiveTimeout; OnTimeout := ClientTimeout; TimeoutStartSampling; Counter.SetConnected; // <-- init Counter manually end; end; This good method might be called in TimeoutStartSampling after Counter creation when state is wsConnected, and after socket accepting too. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Two things in ICS (probably bug/potential bug)
Arno wrote: >The .NET code in V7 is unfinished work, it never compiled. Francois wrote: >Unfinished work, sure. But it worked at some point. And from trhat point >changes have been made with no effort to keep it running because no one >seems interested anymore in a .NET version. Ah, now things are clear. Very pity - lots of efforts in vain! Delphi.Net seem to be unuseful thing at all though idea is great. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Built-in timeout questions
Hello all, recently I've started migrating from my own timeout-featured TWSocket descendant towards the ICS built-in timeout. First of all, thanks to Arno for implementing this feature! Usage is quite simple and most of the questions could be answered by reading the code but some things are still lurking in the dark. Here are my questions: 1) In my descendant class I implement true asynchronous Connect with asynchronous DNS lookup before connecting. Of course, I wish to use built-in timeout mechanism for tracking timeouts. But descendant classes haven't access to necessary fields, namely FTimeoutConnectStartTick. And without it I have no means of controlling the timeout. There are two ways as I see it: make FTimeoutConnectStartTick protected or even public - like Counter field is - or add a parameter to TimeoutStartSampling with type TTimeoutReason. Inside this method when parameter is torConnect, FTimeoutConnectStartTick would be assigned. Or, async DNS lookup before connect could be implemented in TWSocket what is the best decision IMHO :) 2) I have listening socket and wish to have its clients disconnected by inactivity timeout. Alas, the sockets created by Accept are never ininialized with timeout code. Of course, I may execute their TimeoutStartSampling manually but then another issue is revealing: TimeoutStartSampling creates Counter but doesn't init it, leaving its fields empty. So in case when client is connecting but sends or receives nothing, it is timeouted immediately! So I guess there should be FCounter.LastSendTick := _GetTickCount after CreateCounter. Regarding accepted sockets I guess some additions should be made too (for example, copying timeout fields from Server to newly created socket) but it's not so critical. 3) I have a TTimer in my thread which owns sockets. May I use TIcsThreadTimer instead and would it be more effective or no difference? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: client not removed from server list on disconnect
Tobias, oh, thanks a lot! Today is the day when I use to forget to call ancestors' methods :) -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] TWSocketServer: client not removed from server list on disconnect
I faced strange issue when connected clients seem not to be removed from
server's list on disconnect.
I wrote simple demo project to test this, it'll be later. In short words,
client objects seem to never be deleted from FClientList. I've set breakpoint
on the line "FClientList.Remove(AComponent)" in
TCustomWSocketServer.Notification - it's never reached. So client is destroyed,
but server still thinks he has it, and when destroying the server or calling
DisconnectAll the exception "Invalid pointer operation" is raised.
Here's the demo:
== start ==
unit Servers;
interface
uses
Windows, Messages, SysUtils, Classes, StrUtils, DateUtils,
OverbyteIcsWSocket, OverbyteIcsWinsock, OverbyteIcsLogger,
OverbyteIcsWSocketS, OverbyteIcsWndControl;
procedure ServersInit;
procedure ServersFree;
implementation
type
TSrvClient = class(TWSocketClient)
protected
FList: TList;
public
constructor Create(AOwner: TComponent); override;
destructor Destroy; override;
end;
TServer = class(TCustomWSocketServer)
protected
procedure TriggerClientConnect(Client: TWSocketClient; Error: Word);
override;
procedure TriggerClientDisconnect(Client: TWSocketClient; Error: Word);
override;
public
constructor Create(AOwner: TComponent); override;
end;
var Server: TServer;
procedure ServersInit;
begin
Server := TServer.Create(nil);{}
Server.ClientClass := TSrvClient;
Server.Listen;
end;
procedure ServersFree;
begin
FreeAndNil(Server);
end;
constructor TSrvClient.Create(AOwner: TComponent);
begin
inherited;
CreateCounter;
ComponentOptions := [wsoNoReceiveLoop];
FList := TList.Create;
end;
destructor TSrvClient.Destroy;
begin
FreeAndNil(FList); // *** set breakpoint here to ensure client is destroyed
***
end;
constructor TServer.Create(AOwner: TComponent);
begin
inherited;
Proto := 'tcp';
Addr:= '0.0.0.0'; // Use any interface
Port:= '1';
ClientClass := TSrvClient; // Use our component
Banner := '';
BannerTooBusy := '';
end;
procedure TServer.TriggerClientConnect(Client: TWSocketClient; Error: Word);
begin
inherited;
Client.SendLine('Connect OK, closing...');
Client.CloseDelayed;
end;
procedure TServer.TriggerClientDisconnect(Client: TWSocketClient; Error: Word);
begin
inherited;
end;
end.
== end ==
How to test.
Save this as unit Servers, start new VCL app, use the unit, drop 2 buttons and
create handlers:
procedure TForm1.Button1Click(Sender: TObject);
begin
ServersInit;
end;
procedure TForm1.Button2Click(Sender: TObject);
begin
ServersFree;
end;
Run the app, push the 1st button. Launch cmd.exe and type "telnet localhost
1". You'll receive a message and connection will be closed. But breakpoint
on the line "FClientList.Remove(AComponent)" in
TCustomWSocketServer.Notification isn't reached. Now push the 2nd button.
Exception.
I believe in TCustomWSocketServer.WMClientClosed RemoveComponent should be
called before Client.Free. I tried it and everything's OK.
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Two things in ICS (probably bug/potential bug)
>It's not for .NET however for Delphi 7+ and BCB 2006+, what is the >problem? Do you mean that ICS currently couldn't be compiled under .NET? >FHSocket actually is of type TSocket = u_int = Integer defined in >Winsock.pas, WPARAM and LPARAM are of type Integer defined in >Windows.pas, there is no cast required. Do you use Delphi? Aah, I remembered. I changed Winsock.pas so that TSocket is Cardinal which really is what it should be (U_int is unsigned undoubtedly, and declaring socket as signed is a bug leading to another range check errors in some cases). I hope this bug would sometimes be fixed so it's better to cast it explicitly. >Currently both WPARAM and LPARAM are 32-bit and will be 64-bit >in x64, no problem IMO. Ok, I didn't know what WPARAM is on x64, tried searching in header files but haven't found the declaration. So if they are identical, no problem here. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Two things in ICS (probably bug/potential bug)
I just downloaded recent version from SVN and found a strange define in
OverbyteIcsDefs.inc
{$IFDEF CLR}
{$DEFINE BOMB}
{$ELSE}
{$IFNDEF COMPILER7_UP}
{$DEFINE BOMB}
{$ENDIF}
{$ENDIF}
It seems that it shouldn't really be CLR but old versions of BCB or something
else.
And the second thing, I faced with Range check errors inside ICS code. After
some investigations I've found the root of evil: _PostMessage in
TCustomWSocket.Send takes wParam as Integer while FHSocket it is called with
has unsigned type Cardinal. Casting FHSocket to Integer resolved the issue.
There are several other places where socket handle is posted so I think they
should be fixed too. And consider using WPARAM/LPARAM types for casting instead
of general integer ones for higher compatibility.
Moreover, after I've learned some thing regarding x64 I accepted the habit of
sending/posting pointers/handles ONLY in LParam which is what it intended for.
So it'd be nice to see it in ICS.
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli, bandWidthLimit ?
Angus, Arno, thanks for info about threads in ICS! I'll probably take a look at built-in timeouts and use them instead of my own ones which I implemented by TIcsTimer. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
Maybe you'd try to extract USB interaction into separate thread and assign to it a higher priority? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli, bandWidthLimit ?
2Angus >The old bandwith control code is unused, indeed not compiled, so removing >it does not break existing applications. But there is no point in >rewriting it which is what you were suggesting. I don't think so. OverbyteIcsDefs.inc has BUILTIN_TIMEOUT & BUILTIN_THROTTLE defines commented by default so "old code" is still in use. Moreover, as these features require an additional thread there certainly could be some people who won't like an idea of turning their apps to multi-threaded ones. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
>Is it possible that the FTP transactions can, for whatever network reasons, hog the CPU resources so that my application cannot get access to the USB data? Is it possible to completely turn off all FTP exchanges and try only with USB? You may also check CPU usage during FTP sessions. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli, bandWidthLimit ?
2Bruno:
>Could you guide me to do the modification please ? and where ?
>it would be really nice from you.
Well, I'm not an expert in ICS and have little knowledge of HttpCli but here's
dummy vision of the issue.
Search inside OverbyteIcsFtpCli.pas for "BUILTIN_THROTTLE" and copy-paste all
the stuff dealing with properties BandwidthLimit & BandwidthSampling to
corresponding methods of FtpCli based in OverbyteIcsHttpProt.pas. Then find
TCustomFtpCli.DataSocketGetInit and copypaste {$IFDEF
BUILTIN_THROTTLE}...{$ENDIF} block into THttpCli.DoBeforeConnect (you'll also
need to add httpoBandwidthControl to THttpCliOption type - it'll be replacement
of ftpBandwidthControl option). And as long as I understand, you're done.
2Angus:
>Anton was not referring to the HTTP client, but the FTP client. He was
>asking for some code to be simplified, but the code he posted is actually
>obsolete and no longer used by the FTP client.
>It has been replaced by new common throttling code in TWSocket which is
>enabled by the BUILTIN_THROTTLE definition. This was only added last
>year, so we tend to keep old code for a while until new code is
>thoroughly tested.
Well, my report of repeating code pieces has little connection with the topic,
moreover, I guess it's only Arno and/or Francois who might be interested in my
report. But as we are talking about bandwidth stuff I decided to mention it
inside this topic.
As long as obsoleteness is concerned, I have no info on future plans of this
feature but the ICS "NOT BREAK EXISTING CODE" commandment hints that
UseBandwidthControl would probably remain.
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT: Why you insist in use SSL if the client and server applications are coded by you?!! Why use a standard that only exist because of the need to connect many different implementations of clients to many different implementations of servers? It's easy to implement encryption but you'll have to add key exchange also and exclude possibility of key sniffing what is quite harder. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli, bandWidthLimit ?
Arno, in FtpCli unit there's quite large piece of code
{$IFDEF UseBandwidthControl}
FBandwidthCount := 0; // Reset byte counter
if ftpBandwidthControl in FOptions then begin
if not Assigned(FBandwidthTimer) then
FBandwidthTimer := TIcsTimer.Create(Self);
FBandwidthTimer.Enabled := FALSE;
FBandwidthTimer.Interval := FBandwidthSampling;
FBandwidthTimer.OnTimer := BandwidthTimerTimer;
FBandwidthTimer.Enabled := TRUE;
// Number of bytes we allow during a sampling period
FBandwidthMaxCount := Int64(FBandwidthLimit) * FBandwidthSampling
div 1000;
FBandwidthPaused := FALSE;
end;
{$ENDIF}
repeating 4 times in
TCustomFtpCli.DataSocketGetSessionConnected
TCustomFtpCli.DataSocketPutSessionConnected
TCustomFtpCli.DataSocketGetSessionAvailable
TCustomFtpCli.DataSocketPutSessionAvailable
Shouldn't it be extracted into separate procedure?
Bruno, using BUILTIN_THROTTLE it's quite easy to add bandwidth control, just
add Socket.BandwidthLimit and Socket.BandwidthSampling assignment after socket
creation and that's all.
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] x64 adaptation
I recently reviewed all my projects to be ready to x64 (replaced things like Integer(Pointer) ) and also checked code of some components I use. The main idea is not to cast pointers/handles to integer/cardinal as they may exceed 4 bytes on x64 systems. Emb-ro introduced two new types: Native(U)Int that will always have the same size as Handle/Pointer. So I believe the changes could be done right now. Here's what I found in ICS. OverbyteIcsUtils.pas 1) if Integer(FHandle) >= 0 then // NativeInt() but not necessary as it's for old compilers 2) TIcsIntegerList.Get***, all Integer() casts 3) Cardinal(Pointer(Result)^) := Len; 4) if Cardinal(FHandle) = INVALID_HANDLE_VALUE then // (twice) to NativeUInt OverbyteIcsWndControl.pas 1) KillTimer(FIcsWndControl.Handle, Cardinal(Integer(IntPtr(FHandleGC; 2)if SetTimer(FIcsWndControl.Handle, Cardinal(Integer(IntPtr(FHandleGC))), FInterval, nil) = 0 then begin 3) KillTimer(FIcsWndControl.Handle, Cardinal(Self)); 4)if SetTimer(FIcsWndControl.Handle, Cardinal(Self), FInterval, nil) = 0 then begin There are other some Integer() stuff but inside IFDEF CLR sections and I have no idea on what things in x64 .Net are (and even if they differ from x32 at all). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS-based SNTP (Simple network time protocol) client
Francois wrote: >Actually, there is an ICS version for Linux :-) Thanks! I downloaded it and examined. So, it's thread-based actually - not event-driven. You will still have a lot of work to port ICS provided that async model would stay for *nix :) >Any Vista or Windows 7 has IPv6 installed. You don't need special hardware. Well, I meant time-servers who support IPv6. 2All: my SNTP client is available for downoad at the Overbyte's Usermade page (demo included). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS-based SNTP (Simple network time protocol) client
Angus wrote: >So the SNTP component is already testing with ICS for IPv6? >And with Delphi for Mac and Linux? I'm afraid I hadn't got your point. Especially concerning "Delphi for Mac and Linux". There is still neither Delphi nor ICS for Mac/Linux (and porting ICS may be a greatly complicated task). IP6 isn't tested as wel as multicast because I haven't this technologies and servers nearby. But I'm not leaving my creation and will add features as soon as possible. Arno wrote: >Don't you use SVN locally for version control? >I can't live without it, it's so easy with TortoiseSVN, and you >do not have to run a SVN server in your LAN. A shared, file based >repository is enough (two mouse clicks to create a new repository). Well, for now a simple batch file which archives a project folder is still enough for me. I'm quite old-school at this, I know :) -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS-based SNTP (Simple network time protocol) client
Francois wrote: >The problem is not that I don't want to expand ICS. The problem is to >support ICS. There are a lot of interesting components that could be added >to ICS. By moving it into "usermade" page, I'm not commited to support it. >If included in ICS distribution, it has to be supported and kept up-to-date >with current ICS and Delphi. A huge work actually. Well, it's so simple that I guessed it wouldn't require any support or modifications. >Could you also write the text to be published on the usermade page ? Have a >look at the other text to get inspired I've sent you all the required files, hope I did everything correctly. Arno wrote: >I wonder if a new "usermade" SVN repository wasn't a better place? >Most of the downloads from current ICS "usermade" page are obsolete. Well, if these usermades are obsolete SVN would change nothing. And I personally am still not familiar with SVN, especially in committing a changes. Maybe others prefer old good archive down/uploading also :). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] ICS-based SNTP (Simple network time protocol) client
Angus wrote: >I did SNTP client and server 10 years ago, they are improved versions of >code from the usermade page: Ahhh what a trouble! Why I hadn't know that! Anyway I like my implementation more :) Francois wrote: >Do you want an entry on the "user made" web page at ICS website ? Well, if you don't wish to expand ICS, it will work too. I'll polish the source and create simple demo and send it to you. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Useless lines in POP3 and Finger client code
There are useless lines in OverbyteIcsFingCli and OverbyteIcsPop3Prot. Abort procedures act like this procedure TFingerCli.Abort; begin FWSocket.CancelDnsLookup; // < this line is doubled FWSocket.Abort; end; while CancelDnsLookup is included into TWSocket.Abort. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] ICS-based SNTP (Simple network time protocol) client
Hello all, I just completed a TSNTPCli class which performs time-sync with NTP servers. I guess it might be included into standard ICS toolkit. What do you think, is someone interested? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] IndexOf is slow when there are thousands ofclientsconnected
>One OT question: Would it be faster if we use the cat() method of AnsiString >instead of the + operand? It would be faster for you to check it yourself. I have no idea with BCB but on Delphi + operator is counted to be faster than Append(). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] IndexOf is slow when there are thousands of clientsconnected
Arno wrote: >I don't think so, why should it? Well, the connection to HTTP client is closed when response is sent (usually; there are some options in the request header to keep it but these cases are rarer). I'm just supposing, not asserting. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] IndexOf is slow when there are thousands of clients connected
Fastream Technologies wrote: >clientString := IntToStr(ULong(Client)) Won't it be much faster to use TList with custom Compare method which would compare Client pointers as integers? String comparison is a waste. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] IndexOf is slow when there are thousands of clients connected
I dared to start the discussion in a separate thread. >I have seen a bottleneck in ICS with tens of thousands of clients: When a >client is disconnected the TWSocketServer calls FClientList->IndexOf() for >finding the index to remove the object! IndexOf is simply exhaustive search >and it is a huge CPU hog when there are 30k clients! In the one old internet component library implemented a hashed list to store sockets. Hash is computed as Integer(SocketHandle) div HashBase and in that position of the array the first socket object is stored with pointer to the next socket object having the same hash (simple one-d\irectional list). But I'm unsure whether this way is much better. >In most cases IndexOf() is not used frequently if not just once >when the client disconnects from the server. Doesn't it used in HTTP server every time response is sent? It happens pretty often. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] FYI: asynchronous sockets on Linux
The thing that ICS would sometimes become multi-platform is great! I myself have quite little knowledge of what things are on *nix but I found an article that seems like a good start. It describes several approaches to doing async on *nix. I think it would be easier to use one of these and keep ICS's async ideology. Here it is: http://www.kegel.com/c10k.html -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening toseveralIP:Port?
>Probably breaking changes could be iontroduced the day ICS stops supporting >non unicode compiler. >Probably ICS-V8 could be that one, maybe to be released at the same time as >Delphi crossplatfom, abandonning support for most of old Delphi version. I'm >thinking about letting ICS-V7 live for D7-D2010 and have ICS-V8 for >Delphi-XE and the next one. >Just ideas right now. Oh, maybe that will be a good occasion for some cleanup and slight code redesign. And maybe these changes will include more clear FTPcli error reporting (the thing I wrote about several months ago). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPcli: source path question
Hello SZ! >You must parse the HTML for this. We use a Delphi HTML parser which I >downloaded from sourceforge for this but sometimes it raises an exception. >Search for that and if you cannot find it I will do my best to search it for >you in our projects... Actually I'm trying to extend Angus' Magenta Copy so HTML parsing is already implemented. Zvone, right! I'll take more intent look at the headers. Thanks to all for answers! -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPcli: source path question
Francois wrote: >In HTTP world, there is no real directory concept. There are only documents. >It happens that some webservers, if configured so could display a directory >content if the default document is missing. That directory content is a HTML >page built automatically by the webserver. Yes, I've realized it already >This is not always the case. >I would not rely on that behaviour. Zvone wrote: >So you cannot really know how folders are structured on the server is >just by looking at the URL. Sad :( That's what I was afraid of... Well, then I have a question: maybe you have some ideas of how to organize recursive download: for example, if user started to download www.example.com/path/index.html, we should also accept www.example.com/path/logo.jpg and so on, but not www.example.com/index.php. If user started www.example.com/path/foo, we should accept www.example.com/path/foo/index.php but NOT www.example.com/path/bar.jpg. Applications like Wget do support this behavior but the question is how they do it. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HTTPcli: source path question
Currently I'm starting some research on HTTP downloads with ICS THttpCli. I want to add recursive download functionality but faced with impossibility to distinguish file of directory. Then I noticed that requests to folder without trailing slash (GET /somepath/foo/bar) are redirected to locations with slash (/somepath/foo/bar/) so it's easy to tell it's a directory. I'm far not close to HTTP specs and don't know whether it's obligatory behavior and could I rely on it. Moreover, though THttpCli implements redirect internally (fortunately!!!) I don't know whether I could rely on FPath (of FLocation?) fields. I looked in RFC but found nothing on this issue. And another thing. Shouldn't header fields Modified-since and Date be parsed as well as other ones? RFC1123_StrToDate is already present in the unit, just 2 date formats are left to implement. And the last (finally!). RFC1123_Date seem to double standard Delphi FormatDateTime routine. You may use it with english TFormatSettings record specified (get it by GetLocaleFormatSettings(LOCALE_INVARIANT) ) to fix month and day names. This could be also used in data parsing. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Request for better Delphi version detection
In addition to previous message: Jon's suggestion with {$IF CompilerVersion >=
##} looks pretty, and if the only thing preventing from using it is a new
compiler versions, it could be easily solved by this code:
{$IF CompilerVersion > LastKnownAndTestedCompilerVersion}
Error ( Sorry, I don't know this compiler version )
{$ENDIF}
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Request for better Delphi version detection
As for me, I counted current compilers.inc code not-optimal too so I redesigned
it. Haven't tested it with .Net or Linux (these targets use quite different
approach) but Windows versions seem working OK.
Here's my version:
// Compiler defines not specific to a particlular platform.
// BDS 2010 (BDS 7.0)
{$ifdef VER210}
{$define COMPILER_13}
{$define COMPILER_13_UP}
{$endif}
// BDS 2009 (BDS 6.0)
{$ifdef VER200}
{$define COMPILER_12}
{$define COMPILER_12_UP}
{$endif}
// BDS 2007 (BDS 5.0)
{$ifdef VER190}
{$define COMPILER_11}
{$define COMPILER_11_UP}
{$endif}
// BDS 2006 (BDS 4.0)
{$ifdef VER180}
{$define COMPILER_10}
{$define COMPILER_10_UP}
{$endif}
// DELPHI 9
{$ifdef VER170}
{$define COMPILER_9}
{$define COMPILER_9_UP}
{$endif}
// DELPHI 8
{$ifdef VER160}
{$define COMPILER_8}
{$define COMPILER_8_UP}
{$endif}
// DELPHI 7
{$ifdef VER150}
{$define COMPILER_7}
{$define COMPILER_7_UP}
{$endif}
// DELPHI/CPPB 6
{$ifdef VER140}
{$define COMPILER_6}
{$define COMPILER_6_UP}
{$endif}
// DELPHI/CPPB 5
{$ifdef VER130}
{$define COMPILER_5}
{$define COMPILER_5_UP}
{$endif}
// CPPB 4
{$ifdef VER125}
{$define COMPILER_4}
{$define COMPILER_4_UP}
{$endif}
// DELPHI 4
{$ifdef VER120}
{$define COMPILER_4}
{$define COMPILER_4_UP}
{$endif}
// CPPB 3
{$ifdef VER110}
{$define COMPILER_3}
{$define COMPILER_3_UP}
{$endif}
// DELPHI 3
{$ifdef VER100}
{$define COMPILER_3}
{$define COMPILER_3_UP}
{$endif}
// CPPB 1
{$ifdef VER93}
{$define COMPILER_2} // C++ Builder v1 compiler is really v2
{$define COMPILER_2_UP}
{$endif}
// DELPHI 2
{$ifdef VER90}
{$define COMPILER_2}
{$define COMPILER_2_UP}
{$endif}
// DELPHI 1
{$ifdef VER80}
{$define COMPILER_1}
{$define COMPILER_1_UP}
{$endif}
{$else (not Windows)}
// Linux is the target
{$define QT_CLX}
// KYLIX 1/2
{$ifdef VER140}
{$define COMPILER_6}
{$define COMPILER_6_UP}
{$endif}
// KYLIX 3
{$ifdef VER150}
{$define COMPILER_7}
{$define COMPILER_7_UP}
{$endif}
{$endif Win32}
//
{$ifdef COMPILER_13_UP}
{$define COMPILER_12_UP}
{$endif}
{$ifdef COMPILER_12_UP}
{$define COMPILER_11_UP}
{$endif}
{$ifdef COMPILER_11_UP}
{$define COMPILER_10_UP}
{$endif}
{$ifdef COMPILER_10_UP}
{$define COMPILER_9_UP}
{$endif}
{$ifdef COMPILER_9_UP}
{$define COMPILER_8_UP}
{$endif}
{$ifdef COMPILER_8_UP}
{$define COMPILER_7_UP}
{$endif}
{$ifdef COMPILER_7_UP}
{$define COMPILER_6_UP}
{$endif}
{$ifdef COMPILER_6_UP}
{$define COMPILER_5_UP}
{$endif}
{$ifdef COMPILER_5_UP}
{$define COMPILER_4_UP}
{$endif}
{$ifdef COMPILER_4_UP}
{$define COMPILER_3_UP}
{$endif}
{$ifdef COMPILER_3_UP}
{$define COMPILER_2_UP}
{$endif}
{$ifdef COMPILER_2_UP}
{$define COMPILER_1_UP}
{$endif}
So when a new compiler is released all we need to add is just 4+3=7 lines of
copy-pasted code. Maybe this way would be more optimal than current one?
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] New SVN URLs. Please switch your work copies!
>I suggest you read the change log with a SVN client, I've read it just now, and all it said was about XE support :) I'll seek changes with WinMerge. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] New SVN URLs. Please switch your work copies!
>Note that the old icsv7 branch doesn't get any more >updates, updates go to the trunk from now. What's the difference between old and new v7 ? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TIcsLogger.DoDebugLog optimization
Arno, >I do not think that people use multiple log destinations at the >same time. Currently the string is allocated and concatenated >only when actually required. But it's possible. My modification is more compact also. >This is non-optimal as well. >You always concatenate and allocate the string even though it's >not used. That sure leads to smaller source code but only performs >faster _if you actually write to multiple log destinations. Where? I concatenate it _only_ if loAddStamp is set and it's the same as execute procedure with AddTimeStamp + Msg. Maybe you mean that additional concatenation with #13#10? I think it would be better to move this to WriteToLogFile because newline is added before call to this method anyway. Moreover I think additional check should be added to determine whether newline is already present in Msg - in that case it shouldn't attach another one. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] TIcsLogger.DoDebugLog optimization
TIcsLogger.DoDebugLog uses non-optimal algorithm dealing with timestamps: it is
being added to Msg three times.
I suggest change the code as following:
procedure TIcsLogger.DoDebugLog(
Sender : TObject;
LogOption : TLogOption;
Msg : String); // remove const to avoid declaring a local variable
begin
{$IFNDEF NO_LOGGER_MT}
Lock;
try
{$ENDIF}
{$IFDEF VCL}
if csDestroying in Componentstate then { V1.02 }
Exit;
{$ENDIF}
if loAddStamp in FLogOptions then
Msg := AddTimeStamp + FTimeStampSeparator + Msg; // construct new Msg
if loDestEvent in FLogOptions then
if Assigned(FOnIcsLogEvent) then
FOnIcsLogEvent(Sender, LogOption, Msg);
if loDestOutDebug in FLogOptions then
OutputDebugString(TOutputDebugStringType(Msg));
if loDestFile in FLogOptions then
WriteToLogFile(Msg + #13#10); {V6.03}
{$IFNDEF NO_LOGGER_MT}
finally
Unlock;
end;
{$ENDIF}
end;
--
Anton
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] ICS Message handlers
Eric, talking about your shared timer: do you call OnTimer of all attached objects directly or post WM_TIMER to them? I had though over the same timer class and that was my idea to broadcast timer signal. I think it'll eliminate the issue of blocking during loop since PostMessage is asynchronous. But I'm not sure, maybe it'll overload message queues. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Somebody should block "andy at ietgroup.com" to stop his auto-answerer flooding the list
Subj -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
