Re: [twsocket] HttpCli StatusCode = 0 causes problems
I am using the latest version but I don't want to try a BETA version. I am using a workaround and simply returning from the RequestDone event if the LocationChange event was called and the status code is 0. I hope this is sufficient. This should definitely be fixed ASAP. It can cause significant problems and crashes in applications when a link is redirected. Beta are fix proposal waiting to be tested in real worl application. You can't ask to fix something asap if you don't want to test yourself for the beta. Or do you prefer that I skip beta stage and put that code directly in production ? I guess not. -- Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
Hello, I am having a problem using THttpCli and redirected links. It appears similar to this issue I am replying to, probably the same issue. It seems OnRequestDone is being called twice when I try to get a link that is redirected. Has this been addressed/fixed? Thank you! -- Albert Wiersch AI Internet Solutions [EMAIL PROTECTED] http://www.htmlvalidator.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Maurizio Lotauro Sent: Wednesday, July 27, 2005 12:18 PM To: ICS support mailing Subject: Re: [twsocket] HttpCli StatusCode = 0 causes problems Scrive Merijn Terheggen [EMAIL PROTECTED]: I'm trying to write a spider that collects web pages. In the LocationChange event, I check the properties of the new location. For instance: Ok, nothig strange that can interact with the message queue. Francois, I reproduced the problem using HttpTst demo with the url http://help.lycos.com This happen because an OnRequestDone is triggered after the StartRelocation, that call InternalClear. Maybe the problem is that the OnRequestDone should not be called at all, or that it should be called after the StartRelocation? I tried to trace the flow but it is not so simple. Bye, Maurizio. This mail has been sent using Alpikom webmail system http://www.alpikom.it -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
The best is that you try the latest version, both the latest released version and the beta version Maurizio has done (both available from ICS page at my website). -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be - Original Message - From: Albert Wiersch [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Thursday, September 22, 2005 12:47 PM Subject: Re: [twsocket] HttpCli StatusCode = 0 causes problems Hello, I am having a problem using THttpCli and redirected links. It appears similar to this issue I am replying to, probably the same issue. It seems OnRequestDone is being called twice when I try to get a link that is redirected. Has this been addressed/fixed? Thank you! -- Albert Wiersch AI Internet Solutions [EMAIL PROTECTED] http://www.htmlvalidator.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Maurizio Lotauro Sent: Wednesday, July 27, 2005 12:18 PM To: ICS support mailing Subject: Re: [twsocket] HttpCli StatusCode = 0 causes problems Scrive Merijn Terheggen [EMAIL PROTECTED]: I'm trying to write a spider that collects web pages. In the LocationChange event, I check the properties of the new location. For instance: Ok, nothig strange that can interact with the message queue. Francois, I reproduced the problem using HttpTst demo with the url http://help.lycos.com This happen because an OnRequestDone is triggered after the StartRelocation, that call InternalClear. Maybe the problem is that the OnRequestDone should not be called at all, or that it should be called after the StartRelocation? I tried to trace the flow but it is not so simple. Bye, Maurizio. This mail has been sent using Alpikom webmail system http://www.alpikom.it -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
On 22-Sep-05 14:12:39 Francois Piette wrote: The best is that you try the latest version, both the latest released version and the beta version Maurizio has done (both available from ICS page at my website). I doubt that the beta solve this problem because the changes are in a different part of the source. I'll try. But which beta are you speaking? Actually we have two, one fix an authentication problem and the other add the handling of content coding. Should I prepare a version that merge the two? Bye, Maurizio. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
I'm trying to write a spider that collects web pages. In the LocationChange event, I check the properties of the new location. For instance: - Is the new location a URL that is present on a black-list of banned URL's or domains? - Is there a robots.txt file for the new location? - Other properties that would lead to an abort of the request. - Empty the received data buffer so that the buffer only contains the contents of the new location (I also use the OnDocData event to add received data to my buffer).. In the RequestDone event, I process the collected data. A flag is set for the client to be deleted when the clean up routine comes by. I use the RequestDone event because this way I'm sure now other events can be triggered and it is save to delete the client. Thanks, --Merijn P.S. Would it be more efficient to do things another way? On 27 Jul 2005 3:48:57 +0100, Maurizio Lotauro [EMAIL PROTECTED] wrote: On 26-Jul-05 09:31:34 Merijn Terheggen wrote: Yes, I'm using the original client. Also the latest version. Don't you get the same results when you add a LocationChange event handler and set breakpoints in both the LocationChange and the RequestDone event handlers? What are you doing in these events? Bye, Maurizio. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
Scrive Merijn Terheggen [EMAIL PROTECTED]: I'm trying to write a spider that collects web pages. In the LocationChange event, I check the properties of the new location. For instance: Ok, nothig strange that can interact with the message queue. Francois, I reproduced the problem using HttpTst demo with the url http://help.lycos.com This happen because an OnRequestDone is triggered after the StartRelocation, that call InternalClear. Maybe the problem is that the OnRequestDone should not be called at all, or that it should be called after the StartRelocation? I tried to trace the flow but it is not so simple. Bye, Maurizio. This mail has been sent using Alpikom webmail system http://www.alpikom.it -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
Yes, I'm using the original client. Also the latest version. Don't you get the same results when you add a LocationChange event handler and set breakpoints in both the LocationChange and the RequestDone event handlers? Thanks, --Merijn On 26 Jul 2005 1:44:16 +0100, Maurizio Lotauro [EMAIL PROTECTED] wrote: On 23-Jul-05 17:04:48 Merijn Terheggen wrote: I noticed some behavior of HttpCli that caused me a LOT of headaches before I discovered what was going on and made a work-around. [...] Are you using the original HttpCli or an inherited version made by you? Bye, Maurizio. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
On 23-Jul-05 17:04:48 Merijn Terheggen wrote: I noticed some behavior of HttpCli that caused me a LOT of headaches before I discovered what was going on and made a work-around. [...] Are you using the original HttpCli or an inherited version made by you? Bye, Maurizio. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
Using HttpTst sample program provided with ICS, I get only on OnRequestDone with StatusCode = 200. Not using proxy cmd GET / HTTP/1.1 cmd Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* cmd Connection: Keep-Alive cmd Accept-Language: en, fr cmd User-Agent: Mozilla/3.0 (compatible) cmd Host: help.lycos.com cmd text/html = document.htm Location = http://help.lycos.com/network_1_help.asp URL = http://help.lycos.com Document = document.htm hdrHTTP/1.1 302 Object moved hdrServer: Microsoft-IIS/5.0 hdrDate: Sun, 24 Jul 2005 08:57:30 GMT hdrP3P: CP=CAO DSP COR CURa ADMa DEVa CONo PSAa OUR IND DEM PRE PUR TAIa NAV UNI hdrConnection: close hdrLocation: network_1_help.asp hdrContent-Length: 121 hdrContent-Type: text/html hdrCache-control: private Location changed to http://help.lycos.com/network_1_help.asp; cmd GET /network_1_help.asp HTTP/1.1 cmd Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */* cmd Connection: Keep-Alive cmd Accept-Language: en, fr cmd User-Agent: Mozilla/3.0 (compatible) cmd Host: help.lycos.com cmd Cookie: lubid=01508BD39CB1060842E357FA0006835F; expires=Mon, 18-Jan-2038 05:00:00 GMT; domain=.lycos.com; path=/ Cookie: ASPSESSIONIDQSCRSSCC=BCEDCMIAIEKNELPFODIDOLIA; path=/ text/html = network_1_help.asp Location = http://help.lycos.com/network_1_help.asp URL = http://help.lycos.com Document = network_1_help.asp RequestDone, no error. Status =200 StatusCode = 200 hdrHTTP/1.1 200 OK hdrServer: Microsoft-IIS/5.0 hdrDate: Sun, 24 Jul 2005 08:57:29 GMT hdrP3P: CP=CAO DSP COR CURa ADMa DEVa CONo PSAa OUR IND DEM PRE PUR TAIa NAV UNI hdrConnection: close hdrContent-Length: 19340 hdrContent-Type: text/html hdrSet-Cookie: lubid=01508BD39CB1060842E357FA0006835F; expires=Mon, 18-Jan-2038 05:00:00 GMT; domain=.lycos.com; path=/ hdrSet-Cookie: ASPSESSIONIDQSCRSSCC=BCEDCMIAIEKNELPFODIDOLIA; path=/ hdrCache-control: private -- [EMAIL PROTECTED] http://www.overbyte.be - Original Message - From: Merijn Terheggen [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Saturday, July 23, 2005 9:12 PM Subject: Re: [twsocket] HttpCli StatusCode = 0 causes problems Here you go: 'http://help.lycos.com' redirects into 'http://help.lycos.com/network_1_help.asp'. 'http://help.lycos.com/network_1_help.asp' redirects into 'http://help.lycos.com/network_1_help.asp', which is into itself. After the 2 LocationChange events, there will be 2 RequestDone events. The first one with a StatusCode = 0 and the second one with a StatusCode = 200. --Merijn On 7/23/05, Francois PIETTE [EMAIL PROTECTED] wrote: Why is there a RequestDone event while the StatusCode is still zero anyway? I would be interested in finding a reproductible case where it occurs. -- [EMAIL PROTECTED] http://www.overbyte.be - Original Message - From: Merijn Terheggen [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Saturday, July 23, 2005 6:04 PM Subject: [twsocket] HttpCli StatusCode = 0 causes problems I noticed some behavior of HttpCli that caused me a LOT of headaches before I discovered what was going on and made a work-around. This mail is to ask you: - If the behavior that I noticed indeed happens the way I describe - If the behavior is consiously designed this way or needs to be changed On the RequestDone event, the StatusCode can be 0. I only notice this happen rarely in certain cases of a redirect (FollowRelocation set to 'true'). If I've seen things correct, there can be ANOTHER RequestDone event following in this case. Why is this a problem? The RequestDone event is typically used to process the HttpCli data. After processing, the client potentially could get deleted since it has no function anymore (as happens in my code at least). However, another RequestDone event following the first one when the client has been deleted already causes access violations etc. The only way to be sure the RequestDone event is the 'final' event seems to be a check on the StatusCode. In case the StatusCode = 0, I let the event handler return because another RequestDone event will follow. Why is there a RequestDone event while the StatusCode is still zero anyway? Thanks, --Merijn -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http
Re: [twsocket] HttpCli StatusCode = 0 causes problems
Here you go: 'http://help.lycos.com' redirects into 'http://help.lycos.com/network_1_help.asp'. 'http://help.lycos.com/network_1_help.asp' redirects into 'http://help.lycos.com/network_1_help.asp', which is into itself. After the 2 LocationChange events, there will be 2 RequestDone events. The first one with a StatusCode = 0 and the second one with a StatusCode = 200. --Merijn On 7/23/05, Francois PIETTE [EMAIL PROTECTED] wrote: Why is there a RequestDone event while the StatusCode is still zero anyway? I would be interested in finding a reproductible case where it occurs. -- [EMAIL PROTECTED] http://www.overbyte.be - Original Message - From: Merijn Terheggen [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Saturday, July 23, 2005 6:04 PM Subject: [twsocket] HttpCli StatusCode = 0 causes problems I noticed some behavior of HttpCli that caused me a LOT of headaches before I discovered what was going on and made a work-around. This mail is to ask you: - If the behavior that I noticed indeed happens the way I describe - If the behavior is consiously designed this way or needs to be changed On the RequestDone event, the StatusCode can be 0. I only notice this happen rarely in certain cases of a redirect (FollowRelocation set to 'true'). If I've seen things correct, there can be ANOTHER RequestDone event following in this case. Why is this a problem? The RequestDone event is typically used to process the HttpCli data. After processing, the client potentially could get deleted since it has no function anymore (as happens in my code at least). However, another RequestDone event following the first one when the client has been deleted already causes access violations etc. The only way to be sure the RequestDone event is the 'final' event seems to be a check on the StatusCode. In case the StatusCode = 0, I let the event handler return because another RequestDone event will follow. Why is there a RequestDone event while the StatusCode is still zero anyway? Thanks, --Merijn -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
I don't know if this applies to your situation or not but I have seen a StatusCode of 0 in log files from our customers. Most often, we find that Norton Internet Security, or Norton Firewall is blocking the responses and giving us the StatusCode of 0. I have them check Norton and they usually find that Norton is blocking our program. Clearing that blocking gets rid of the StatusCode of 0. Steve Endicott At 09:04 AM 7/23/2005, you wrote: I noticed some behavior of HttpCli that caused me a LOT of headaches before I discovered what was going on and made a work-around. This mail is to ask you: - If the behavior that I noticed indeed happens the way I describe - If the behavior is consiously designed this way or needs to be changed On the RequestDone event, the StatusCode can be 0. I only notice this happen rarely in certain cases of a redirect (FollowRelocation set to 'true'). If I've seen things correct, there can be ANOTHER RequestDone event following in this case. Why is this a problem? The RequestDone event is typically used to process the HttpCli data. After processing, the client potentially could get deleted since it has no function anymore (as happens in my code at least). However, another RequestDone event following the first one when the client has been deleted already causes access violations etc. The only way to be sure the RequestDone event is the 'final' event seems to be a check on the StatusCode. In case the StatusCode = 0, I let the event handler return because another RequestDone event will follow. Why is there a RequestDone event while the StatusCode is still zero anyway? Thanks, --Merijn -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
