Re: [twsocket] Timeout in sync operations
Anton Sviridov wrote: >> Connect-timeouts are not the only possible ones. > Yes, I know, but connect ones are the most general and used by > everyone, because w/o connection there's nothing to do anyway. > >> If you need different timeout values for different states it has to >> be implemented using a timer. We need to keep the message pump >> as fast as possible. > But what about convenience? And every programmer implements these > timeouts anyway. Or he hopes on ICS and doesnt do this receiving > endless connect waiting... > >> Actually it _is called on StateChange(httpReady). > Yes, but it is called with ErrCode = 0 and StatusCode = 0. Also I > have strange things here: > > log of my program, trying to download http://test1.ru/1.php, which is > my script with endless loop to simulate timeout: [..] This has been discussed many times before. The sync methods are not finished until they return, OnRequestDone actually fires before the sync method returns! So either use async methods or post a custom message from OnRequestDone handler and call the next method from the custom message handler. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
> Connect-timeouts are not the only possible ones. Yes, I know, but connect ones are the most general and used by everyone, because w/o connection there's nothing to do anyway. > If you need different timeout values for different states it has to be > implemented using a timer. We need to keep the message pump > as fast as possible. But what about convenience? And every programmer implements these timeouts anyway. Or he hopes on ICS and doesnt do this receiving endless connect waiting... >Actually it _is called on StateChange(httpReady). Yes, but it is called with ErrCode = 0 and StatusCode = 0. Also I have strange things here: log of my program, trying to download http://test1.ru/1.php, which is my script with endless loop to simulate timeout: URL http://test1.ru/1.php : retrieving header... (attempt 1/3) // FHttpCli.Head called tick... 1 // this from my timer, which does nothing but counting tick... 2 tick... 3 tick... 4 tick... 5 Error: HEAD Failed (404, Request aborted on timeout)! // exception from Head, everything's OK URL http://test1.ru/1.php : retrieving header... (attempt 2/3) // FHttpCli.Head called 2nd time Done.// HttpRequestDone with ErrCode = 0! Done.// HttpRequestDone with ErrCode = 0! Header received, StatusCode = 0 // Head successfully passed! with my timer everything's OK: URL http://test1.ru/1.php : retrieving header... (attempt 1/3) tick... 1 tick... 2 tick... 3 tick... 4 tick... 5 Failed, error #3 Error: HEAD Failed (404, Connection aborted on request)! URL http://test1.ru/1.php : retrieving header... (attempt 2/3) tick... 1 tick... 2 tick... 3 tick... 4 tick... 5 Failed, error #3 Error: HEAD Failed (404, Connection aborted on request)! URL http://test1.ru/1.php : retrieving header... (attempt 3/3) tick... 2 tick... 3 tick... 4 tick... 5 Failed, error #3 Error: HEAD Failed (404, Connection aborted on request)! *** finish *** * BTW, I have noticed that Last-Modified filed in header isn't parsed! IMHO it's too important field to leave parsing of it. Maybe Magenta's code could be added for this purpose? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Anton Sviridov wrote: >> Any alternates >> (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)? It's not a >> constant error AFAIK. IMO 408 was more confusing. > > Yeah, that's why transport level stuff should be done in transport > (socket) classes :) We would have OnConnect(Request)Timeout or > something like and no mess with statuses... Connect-timeouts are not the only possible ones. If you need different timeout values for different states it has to be implemented using a timer. We need to keep the message pump as fast as possible. > >> OK, what about this one? > > That's pretty better, although OnRequestDone is never called and > hence FRequestDoneError isn't used. Maybe make it readable property? Actually it _is called on StateChange(httpReady). GET Failed ! StatusCode = 404 ReasonPhrase = Request aborted on timeout RequestDone Error = 8. Status = 404 -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
>Any alternates (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)? >It's not a constant error AFAIK. IMO 408 was more confusing. Yeah, that's why transport level stuff should be done in transport (socket) classes :) We would have OnConnect(Request)Timeout or something like and no mess with statuses... >OK, what about this one? That's pretty better, although OnRequestDone is never called and hence FRequestDoneError isn't used. Maybe make it readable property? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Anton Sviridov wrote:
> Arno Garrels wrote
>> Please guys, take a look at this fix for V7 and test it, anything
>> wrong with it?
> It works, but I think Abort isn'tt a very good way to tell that there
> was timeout...
Abort on timeout is commonly used in ICS.
> and 404 even stranger. 404 means that file not exist
> (constant error),
Any alternates (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes)?
It's not a constant error AFAIK. IMO 408 was more confusing.
404 Not Found
The requested resource could not be found but may be available again
in the future. Subsequent requests by the client are permissible.
408 Request Timeout
The server timed out waiting for the request.
"Connection aborted on request" means that user or
> control proc decided to break the waiting, and timeout means that
> there's no answer from server and we should wait and retry later.
OK, what about this one?
{code}
procedure THttpCli.DoRequestSync(Rq : THttpRequest);
var
DummyHandle : {$IFDEF CLR}Borland.Vcl.Windows.THandle;
{$ELSE} THandle;{$ENDIF}
TimeOutMsec : UINT;
bFlag : Boolean;
begin
DoRequestAsync(Rq);
if not Assigned(FCtrlSocket.Counter) then
FCtrlSocket.CreateCounter;
FCtrlSocket.Counter.SetConnected; // Reset counter
DummyHandle := INVALID_HANDLE_VALUE;
TimeOutMsec := 5000;//FTimeOut * 1000; // hardcoded 5 sec for testing only
while FState <> httpReady do
begin
if MsgWaitForMultipleObjects(0, DummyHandle, FALSE, 1000,
QS_ALLINPUT) = WAIT_OBJECT_0 then
FCtrlSocket.MessagePump;
if (FState <> httpReady) and (
{$IFNDEF NOFORMS} Application.Terminated or {$ENDIF} FTerminated or
(IcsCalcTickDiff(FCtrlSocket.Counter.LastAliveTick,
GetTickCount) >= TimeOutMsec)) then
begin
bFlag := (FState = httpDnsLookup);
StateChange(httpAborting);
if bFlag then
try
FCtrlSocket.CancelDnsLookup;
except
{ Ignore any exception }
end;
FStatusCode := 404;
if {$IFNDEF NOFORMS} Application.Terminated or {$ENDIF} FTerminated
then
begin
FReasonPhrase := 'Request aborted';
FRequestDoneError := httperrAborted;
end
else begin
FReasonPhrase := 'Request aborted on timeout';
FRequestDoneError := 8; //new const will be added
httperrCustomTimeOut = 8
end;
if bFlag then
SocketSessionClosed(Self, 0)
else
FCtrlSocket.Close;
StateChange(httpReady);
break;
end;
end;
{* Jul 12, 2004
WARNING: The component now doesn't consider 401 status
as a fatal error (no exception is triggered). This required a
change in the application code if it was using the exception that
is no more triggered for status 401 and 407.
*}
{* if FStatusCode > 401 thenDec 14, 2004 *}
if (FStatusCode >= 400) and (FStatusCode <> 401) and (FStatusCode <> 407)
then
raise EHttpException.Create(FReasonPhrase, FStatusCode);
end;
{code}
--
Arno Garrels
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Arno Garrels wrote >TWSocketCounter has been added to TWSocket in V7 with properties >ConnectTick, ConnectDT, LastAliveTick, LastRecvTick and LastSendTick. Yes, I'm using it in my FTP class, but Start/StopTimer seem easier in my case SZ wrote: >I think Arno and Francois is right about not making a TTimer per Thttpcli as >default. It would overwhelm our reverse proxy with 10k connections each with >two sockets. However for simple needs, Anton is right about asking for a >solution which I think sould only be switched on by a define. The timer could be created only by special call, like it's done for Counters... also there are no such necessary in timers in server components, they are content with a single one to trace inactive connections. And I doubt that anybody will have so much client sockets that he'll run out of Windows resources. Arno Garrels wrote >Please guys, take a look at this fix for V7 and test it, anything wrong >with it? It works, but I think Abort isn'tt a very good way to tell that there was timeout... and 404 even stranger. 404 means that file not exist (constant error), "Connection aborted on request" means that user or control proc decided to break the waiting, and timeout means that there's no answer from server and we should wait and retry later. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Arno Garrels wrote: > Please guys, take a look at this fix for V7 and test it, anything > wrong with it? Just noticed that it still lacks a timeout status. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
> > But most components do have a timeout for sync methods, for > > instance TSyncSmtpCli and TFtpClient. > > Yes they have, but do they work properly when a send or receive > lasts longer then the timeout value? I fixed the timeout in TSyncSmtpCli eight years ago, one of the first changes I made to ICS. Sync TFtpClient certainly works as well, I use it all the time, ditto POP3. I also use sync THttpCli and did not notice a timeout missing, but I have my own abort timers anyway. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Angus Robertson - Magenta Systems Ltd wrote:
>>> I'm writing simple console downloader and decided to use sync
>>> operations in order to simplify structure. But I've found that
>>> sync requests (HttpCli) do not use any timeout, so they may last
>>> forever in case of error.
>>
>> That's IMHO a bug.
>
> Indeed, poor implementation in THttpCli with no timeout.
>
> But most components do have a timeout for sync methods, for instance
> TSyncSmtpCli and TFtpClient.
Yes they have, but do they work properly when a send or receive lasts
longer then the timeout value?
Please guys, take a look at this fix for V7 and test it, anything wrong
with it?
{code}
procedure THttpCli.DoRequestSync(Rq : THttpRequest);
var
DummyHandle : {$IFDEF CLR}Borland.Vcl.Windows.THandle;
{$ELSE} THandle;{$ENDIF}
TimeOutMsec : UINT;
begin
DoRequestAsync(Rq);
if not Assigned(FCtrlSocket.Counter) then
FCtrlSocket.CreateCounter;
FCtrlSocket.Counter.SetConnected; // Reset counter
DummyHandle := INVALID_HANDLE_VALUE;
TimeOutMsec := 5000;//FTimeOut * 1000; // hardcoded 5 sec for testing only
while FState <> httpReady do
begin
if MsgWaitForMultipleObjects(0, DummyHandle, FALSE, 1000,
QS_ALLINPUT) = WAIT_OBJECT_0 then
FCtrlSocket.MessagePump;
if (FState <> httpReady) and (
{$IFNDEF NOFORMS} Application.Terminated or {$ENDIF} FTerminated or
(IcsCalcTickDiff(FCtrlSocket.Counter.LastAliveTick,
GetTickCount) >= TimeOutMsec)) then
begin
Abort;
break;
end;
end;
{* Jul 12, 2004
WARNING: The component now doesn't consider 401 status
as a fatal error (no exception is triggered). This required a
change in the application code if it was using the exception that
is no more triggered for status 401 and 407.
*}
{* if FStatusCode > 401 thenDec 14, 2004 *}
if (FStatusCode >= 400) and (FStatusCode <> 401) and (FStatusCode <> 407)
then
raise EHttpException.Create(FReasonPhrase, FStatusCode);
end;
{code}
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
> > I'm writing simple console downloader and decided to use sync > > operations in order to simplify structure. But I've found that > > sync requests (HttpCli) do not use any timeout, so they may last > > forever in case of error. > > That's IMHO a bug. Indeed, poor implementation in THttpCli with no timeout. But most components do have a timeout for sync methods, for instance TSyncSmtpCli and TFtpClient. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Anton Sviridov wrote: > Hello! > I'm writing simple console downloader and decided to use sync > operations in order to simplify structure. But I've found that sync > requests (HttpCli) do not use any timeout, so they may last forever > in case of error. That's IMHO a bug. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
I think Arno and Francois is right about not making a TTimer per Thttpcli as default. It would overwhelm our reverse proxy with 10k connections each with two sockets. However for simple needs, Anton is right about asking for a solution which I think sould only be switched on by a define. Just my two cents, SZ On Wed, Sep 16, 2009 at 8:01 PM, Arno Garrels wrote: > Anton Sviridov wrote: > >> If you build a console mode application, you have to create one > >> yourself. See various console mode ICS d?mos. Simple... > > Yes, I created TIcsTimer with HttpCli.CtrlSocket as Owner and > > implemented Start/StopTimer methods. Seems working, although I don't > > see yet how can I control the main function flow from OnTimer > > procedure (i.e., how to tell CtrlSocket that is's timeout). > > TWSocketCounter has been added to TWSocket in V7 with properties > ConnectTick, ConnectDT, LastAliveTick, LastRecvTick and LastSendTick. > That makes it very easy to implement a custom timeout. > Have a look at current THttpServer for an example. > > > > >> This is underway. but designing a general purpose timeout is not so > >> simple because you potentially have thousands of components (server > >> side). You can't create a TTimer for each one or you'll run out > >> Windows resources very quickly (bad scalability). > > > No problem, you could implement only one timer in the very root of > > socket (as you did with socket's window handle, AFAIR). > > If you have a better idea than TIcsThreadTimer, demo in the MiscDemos > folder, please let us know. > > -- > Arno Garrels > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
Anton Sviridov wrote: >> If you build a console mode application, you have to create one >> yourself. See various console mode ICS d?mos. Simple... > Yes, I created TIcsTimer with HttpCli.CtrlSocket as Owner and > implemented Start/StopTimer methods. Seems working, although I don't > see yet how can I control the main function flow from OnTimer > procedure (i.e., how to tell CtrlSocket that is's timeout). TWSocketCounter has been added to TWSocket in V7 with properties ConnectTick, ConnectDT, LastAliveTick, LastRecvTick and LastSendTick. That makes it very easy to implement a custom timeout. Have a look at current THttpServer for an example. > >> This is underway. but designing a general purpose timeout is not so >> simple because you potentially have thousands of components (server >> side). You can't create a TTimer for each one or you'll run out >> Windows resources very quickly (bad scalability). > No problem, you could implement only one timer in the very root of > socket (as you did with socket's window handle, AFAIR). If you have a better idea than TIcsThreadTimer, demo in the MiscDemos folder, please let us know. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
>If you build a console mode application, you have to create one yourself. >See various console mode ICS d?mos. Simple... Yes, I created TIcsTimer with HttpCli.CtrlSocket as Owner and implemented Start/StopTimer methods. Seems working, although I don't see yet how can I control the main function flow from OnTimer procedure (i.e., how to tell CtrlSocket that is's timeout). >This is underway. but designing a general purpose timeout is not so simple >because you potentially have thousands of components (server side). You >can't create a TTimer for each one or you'll run out Windows resources very >quickly (bad scalability). No problem, you could implement only one timer in the very root of socket (as you did with socket's window handle, AFAIR). >Currently you should implement a timeout yourself. Using a single TTimer and >carefully crafted code will do the job Yeah, and every developer should do the same, so we are inventing the bicycle again. Maybe then some general TTimeoutOperationsSeeker? Anyway, at least for connection stage timeouts should be implemented, imho. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
> And second question, more global: Isn't it worth to add good and native > timeout functionality to TWSocket so that thousands of developers > hadn't invent a bicycle again and again? I know about TIcsTimer > and so on, but it is not 'out of the box'. This is underway. but designing a general purpose timeout is not so simple because you potentially have thousands of components (server side). You can't create a TTimer for each one or you'll run out Windows resources very quickly (bad scalability). Currently you should implement a timeout yourself. Using a single TTimer and carefully crafted code will do the job. Basically you need a list of all your timeout \"jobs\" and have the timer event scan the list a do whatever appropriate is to evaluation and handle the timeout. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
>> how can I add timeouts without messageloops and threads and so on? You need a messageloop to make ICS component work anyway. Every GUI application has a messageloop. If you build a console mode application, you have to create one yourself. See various console mode ICS démos. Simple... -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be-- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Timeout in sync operations
> how can I add timeouts without messageloops and threads and so on? I see that there is still ControlSocket.MesagePump in SyncRequest, so I suppose creating timer will work, won't it? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Timeout in sync operations
Hello! I'm writing simple console downloader and decided to use sync operations in order to simplify structure. But I've found that sync requests (HttpCli) do not use any timeout, so they may last forever in case of error. My question is: how can I add timeouts without messageloops and threads and so on? And second question, more global: Isn't it worth to add good and native timeout functionality to TWSocket so that thousands of developers hadn't invent a bicycle again and again? I know about TIcsTimer and so on, but it is not 'out of the box'. -- Best regards, Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
