Re: [twsocket] [THTTPCLI] What am i doing wrong? ;-(
Wiadomość Oryginalna Od: Francois PIETTE [EMAIL PROTECTED] Do: ICS support mailing twsocket@elists.org Data: Sun, 18 Dec 2005 12:22:54 +0100 Temat: Re: [twsocket] [THTTPCLI] What am i doing wrong? ;-( I've changed .CloseAsync to .Abort and nothing changed :-( Still got access violation. Access violation only occures if Timer2 is Enabled. Somewhere in this code (Even without calling abort): for i:=HTTPCliList.Count - 1 downto 0 do begin todestroy:=THTTPCLI(HTTPCLILIST[i]); if todestroy.Tag = -1 then begin HTTPCliList.Delete(i); todestroy.Destroy; httpcli_count:=httpcli_count-1; end; end; end; With the debugger, you'll be able to pin-point exactly where the AV occur. Try changing the logic as I said: for i:=HTTPCliList.Count - 1 downto 0 do begin todestroy:= HTTPCLILIST[i] as THTTPCLI; if todestroy.Tag = -1 then todestroy.Abort; end; Then in OnRequestDone event, handle as if it where success but of course don't process data. There is something wrong with HTTPList... ;-( Maybe TObjectList cant handle so many async requests? I changed the code to something like this: for i:=Form1.ComponentCount -1 downto 0 do begin If form1.Components[i] is THttpCli then begin if (abs((noww-THTTPCLI(form1.Components[i]).Tag)) 60) and (THTTPCLI(form1.Components[i]).Tag -2)and (THTTPCLI(form1.Components[i]).Tag -1) then begin THTTPCLI(form1.Components[i]).Abort; end; if THTTPCLI(form1.Components[i]).Tag = -1 then begin utworzono:=utworzono-1; THTTPCLI(form1.Components[i]).Destroy; end; end; end; ... but its not a good code. Works fine but it shouldnt be done like that... :/ Any ideas how to make it on TObjectList so it wouldnt crash? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HTTPCli.GetAsync problem.
Ok. I made simple application based on Http.GetAsync function. Heres what i got: Components: -TList_1 - List of busy components -TList_2 - List of free components -HTTPCli - 30 more created dynamicly -TStringList - a queue of hosts to gather data from -ListView - table to store gathered data -Timer_1 - to make threads workin' -Timer_2 - to check all over again -ToolBar - tools to save the list, load list etc etc. -StatusBar - to show how many components are in use -ProgressBar - shows the progress of current job -CoolTrayIcon - to be able to minimalize app to tray Here's how it works: At startup 30 HTTP components are being created and added to TList_2. U can add new hosts to gather data, and they appear in TStringList. When timer ticks it checks if there is something in TStringList (queue) and if there are any components in TList_2 (free). If there are, it moves HTTPCli component from TList_2 (free) to TList_1 (busy) and uses it to GetAsync required data. Data isn't to large so I'm gathering it OnDocData converting buffer with strpas() function. Then some function searches the ListView for requested record and changes the value to current. And when its done, component is moved back to TList_2 (free components). After all records are done, Timer_2 starts and counts 1 minute to start another cycle (to keep the data in ListView up-to-date). But... But there seems to be a problem... don't know exacltly what is it, but after few hours of working (i guess minimalized to tray) after Application.restore, toolbar, and sometimes statusbar dissapear! I checked it with WinDowse and those objects are just being destroyed or something - they are not there even hidden... Oh... and after closing application it often crashes and sometimes crashes without closing with access violation. Any ideas what could possibly be wrong? My idea is that when 2 HttpCli components are trying to access ListView, something bad happens... but not sure if thats the problem... Please HELP! :) Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] What am i doing wrong? [HttpCli]
Got problem... Finally wrote some semi-multithreaded application using HTTPCli and GetAsync, but it doesn't seem to work and i dunno why ;-( Generally on OnDocData i got some procedure that changes something on ListView (changes are based on what the HTTPCli Buffer is)... but it doesn't work ;-( It does work on single request, but not on multiple GetAsync... what am i doing wrong? How do i fix it? For example when i got only Memo1.Lines.Add(strpas(buffer)) everything works fine ...but when ill add some procedure: ChangeListViewValues(DataFromUrl(THttpCli(sender).URL),strpas(buffer),#1,#1, #1,datetostr(now),#1,#1); (procedure finds the substring[0] that contains DataFromUrl(THttpCli(sender).URL) and changes the values of next substrings... (if its not #1)) Help? Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Little question... TList of HTTPCli
Hello ;-) It's me again... Just one little question... When i have a TList of busy HTTPCli'ents how do i know which one should i remove on OnRequestDone event? Thank you, Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: Re[2]: [twsocket] Little question... TList of HTTPCli
- Original Message - From: Wilfried Mestdagh [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Tuesday, May 03, 2005 2:08 PM Subject: Re[2]: [twsocket] Little question... TList of HTTPCli Hello Ann, Yes. What you also can do is: i := List.IndexOf(Sender); Then i is the index in the List where Sender is, or -1 if not found and you can thus use it to add the right item to another list and/or remove it etc... Thank you guys! I love ya! :-) Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Redirection? [HTTPCli]
- Original Message - From: Dan [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Saturday, April 30, 2005 12:29 AM Subject: Re: [twsocket] Redirection? [HTTPCli] If i would like to make some exploit, i would use socks, not htttp. Haha, funniest mailing list post of the year. Whats so funny? :-) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Redirection? [HTTPCli]
- Original Message - From: Francois Piette [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Friday, April 29, 2005 8:54 AM Subject: Re: [twsocket] Redirection? [HTTPCli] Ok... i got follow redirection Enabled but it doesn't do what i thought it should do. I got a site... site with loading animation... But HTTPCli does not show the site after loading but only the loading state... :-( What can i do with that? I'm not sure I understand what you mean... If you load a page with an animation, you get that page and that's all. If the page contains a HTML (not HTTP) relocation, it is _not_ executed by the HTTP component which doesn't know anything about HTML. Francois... here's example of what i ment: http://nazwa.pl/whois.php?s=zbd=google.com So you're saying that in similar case, there is no way to get the content of final site? :-( -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Redirection? [HTTPCli]
Ok... i got follow redirection Enabled but it doesn't do what i thought it should do. I got a site... site with loading animation... But HTTPCli does not show the site after loading but only the loading state... :-( What can i do with that? Thank you! Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: Re[4]: [twsocket] Multithreaded http...
I don't really know what to do with TMemoryStream... i need TStringList ... the site isn't that big, so it should be any problem... But i still don't know how do i convert the recived stream to TStringList... ;-( - Original Message - From: Francois Piette [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Monday, April 18, 2005 9:23 AM Subject: Re: Re[4]: [twsocket] Multithreaded http... in On requestDone: THttpCli(Sender).RcvdStream But how do i convert it to StringList ? Convert the received stream to a string list ? Arno told you: AStream.Seek(0, sofromBeginning); StringList.LoadFromStream(AStream); But if you want to convert to a string list for analysis, it is probably not the most efficient way to do that. You should probably use a TMemoryStream and direction use the data in memory : var P : PChar; // Will point to HTML data (or whatever data you received) N : Integer; // Will contains the number of bytes begin P := PChar((THttpCli(Sender).RcvdStream as TMemoryStream).Memory); N := THttpCli(Sender).RcvdStream.Size; // Now you can use P and N to analyse the document received. Very fast. // Don't forget to free the stream when processing is done or you'll have a memory leak. -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: Re[4]: [twsocket] Multithreaded http...
So this ruin everything... semi-multithreading won't make its role... - Original Message - From: Dan [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Wednesday, April 20, 2005 5:45 PM Subject: Re: Re[4]: [twsocket] Multithreaded http... Before you start your request (before the get) you need to assign the receive stream: HTTPCli.RcvdStream:=TMemoryStream.Create; Then, in the OnRequestDone, as has been explained, you have the rcvdstream to read from: var StringList: TStrings; ... StringList:=TStringList.Create; THTTPCli(Sender).RcvdStream.Seek(0,soBeginning); StringList.LoadFromStream(THTTPCli(Sender).RcvdStream); //Use StringList... Stringlist.Free; You are getting the access violation because you didnt allocate the stream before starting the request. Dan. - Original Message - From: Ann [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Wednesday, April 20, 2005 3:05 PM Subject: Re: Re[4]: [twsocket] Multithreaded http... I don't really know what to do with TMemoryStream... i need TStringList ... the site isn't that big, so it should be any problem... But i still don't know how do i convert the recived stream to TStringList... ;-( - Original Message - From: Francois Piette [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Monday, April 18, 2005 9:23 AM Subject: Re: Re[4]: [twsocket] Multithreaded http... in On requestDone: THttpCli(Sender).RcvdStream But how do i convert it to StringList ? Convert the received stream to a string list ? Arno told you: AStream.Seek(0, sofromBeginning); StringList.LoadFromStream(AStream); But if you want to convert to a string list for analysis, it is probably not the most efficient way to do that. You should probably use a TMemoryStream and direction use the data in memory : var P : PChar; // Will point to HTML data (or whatever data you received) N : Integer; // Will contains the number of bytes begin P := PChar((THttpCli(Sender).RcvdStream as TMemoryStream).Memory); N := THttpCli(Sender).RcvdStream.Size; // Now you can use P and N to analyse the document received. Very fast. // Don't forget to free the stream when processing is done or you'll have a memory leak. -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Multithreaded http...
:-( I cant handle it ;- I dont really understand the HTTPAsy example ;-( Could anybody help me? For example i got a an edit, where im deciding how may threads (simaltaneous async http components) are going to be used... for example ... 50..., got some memo... every line of memo is some http address... and other memo as output... now i woulike it to check every server at memo (like it had 50 threads) and put the output to output memo... but if the connection times out, it add the server again to check to the end of the list ... Could anybody help me please... Im really new in delphi and i really need it... Desperaited Ann... - Original Message - From: Francois Piette [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Thursday, April 14, 2005 9:58 AM Subject: Re: [twsocket] Multithreaded http... because there are alot of records to check, i wanted it to be threaded... Well, this is defenitely not a case where multithreading is needed because ICS HTTP component is asynchronous. HTTP component doesn't block ! It automatically handle your requests in the background using events. If you need to handle 100 simultaneous requests, then simply use 100 HTTP components and async methods (GetAsync and the likes) and events (OnRequestDone and the likes). btw: Not all Windows versions are able to handle 100 simultaneous connections ! This is not a limitation of ICS, it is a built-in limitation in some Windows versions. You need a Windows Server edition to be able to handle hundreds of simultaneous connections. -- [EMAIL PROTECTED] http://www.overbyte.be - Original Message - From: Flash Thunder [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Wednesday, April 13, 2005 11:09 PM Subject: Re: [twsocket] Multithreaded http... - Original Message - From: Francois PIETTE [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Wednesday, April 13, 2005 10:13 PM Subject: Re: [twsocket] Multithreaded http... I'm kinda new in Delphi but i really really need to write some application... My application would have some TStringList where it would contain urls... then i wouldlike to process this list in some threads... for example 100 threads... the thread would get the content of url, run some procedure/s and give the results/s to some other TStringList... Well, you don't need multithreading with ICS. ICS is asynchronous and non blocking. Except is very special situations, using multithreading will actually _reduce_ performance, even on a multiprocessor computer. If you tell us what you really want to do, we can tell you if multithreading is needed or not. I need to constantly gather data form a server about diffrent items ... that generally what i need... alot of items... the site isnt too user friendly so i got to copy/cut some things and put them into for example listview... and when it changes, the data in listview is being updated... for example: http://server.com/showiteminfo.php?itemid=12342352 this gives me some html with some table in whitch there is some information about product ... for example: name: canon 3200f resolution: 2048dpi price: 333$ and i need to check alot of items continously so i would be the first one that knows that the price has changed or something... because there are alot of records to check, i wanted it to be threaded... thank your for your help... Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Multithreaded http...
- Original Message - From: Francois PIETTE [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Sunday, April 17, 2005 3:47 PM Subject: Re: [twsocket] Multithreaded http... You need: - A TStringList with all URL to get, call it UrlToGetList Understood - A TSringList with all URL that are being processed, call it UrlBeingGetList Understood - A TObjectList with 50 HTTP Component, call it FreeHttpCliList Understood - A TObjectList with all Http components that are busy getting URL, call it BusyHttpCliList Understood - When you start the process, you have a loop that will iterate thru the FreeHttpCliList. For each one, extract an url from UrlToGetList, move tha url to the UrlBeingGetList and start a HttpCli.GetAsync, move the Http component to the BusyHttpCliList. When the UrlToGetList is empty or FreeHttpCliList is empty, break the loop. Use the Tag property of the Http component to store the time you started to get the URL. - Use a TTimer which will iterate thru the BusyHttpCliList, checking the tag against the current time to find out which one has timed out. If timed out, call his Abort method. What will happen when someone will change the system time? What does (TDataTime-TDataTime) return? miliseconds? - From the OnRequestDone on all Http component, print the result of the get, move the component to the FreeHttpCliList and postmessage a custome message to your form. The corresponding message handler will start the process again. If the url failed to be got, move it again to the UrlToGetList, otherwise, remove it from the list where it is. Don't undertand... what message to form? How? What for? And I dont really know how to get the result... I have OnRequestDone... but how do i get the request result? Thank you very much for your patience. Ann -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: Re[4]: [twsocket] Multithreaded http...
But how do i convert it to StringList ? - Original Message - From: Wilfried Mestdagh [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Sunday, April 17, 2005 10:38 PM Subject: Re[4]: [twsocket] Multithreaded http... Hello Ann, in On requestDone: THttpCli(Sender).RcvdStream --- Rgds, Wilfried http://www.mestdagh.biz Sunday, April 17, 2005, 22:10, Ann wrote: - Original Message - From: Marco van de Voort [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Sunday, April 17, 2005 9:05 PM Subject: Re: Re[2]: [twsocket] Multithreaded http... you are ritght because working with float is more CPU. But GetTickCount roll over every 49 day... So without extra code the timeout can fail if it is (by Murphy's law) just on the particular moment.. One can still use gettickcount then. One only has to combine it with your technique to keep track of the roll over. (to get the most significant bits so to say) So set a timer every 49 days (or the maximal resolution of the timer if 49days )instead of each and every second The easiest way is to just set the SecondsBetween or something like that... it wont matter if its less or more... for example we are starting thread at 48day,23h,59m,30s... and tick... 0... difference is still too high, thread will get terminated... it doesnt really matter... few more timeouts every 48 days... nothing to cry about... Anyway i still dont know how can i get da whole recived document OnRequestDone... and can i put in there some procedures and they still will be async? And is there any way to get the least possible data? I dont need anything but pure text of the page... -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
