[twsocket] HTTPClient and comet / chunked transfer and post data with active connection
Hi, I am working on a project where the server side delivers HTTP-data as comet / chunked transfers e.g. the connection is never closed and as new events come along it sends new data over the same connection. There is no way to change the server side and use sockets instead (server side isn't mine). I have figured out that I can receive their data on the OnDocData-events. However I am also supposed to post some data over the same connection. When I try to do this of course the component say it's busy. Is it possible to do the post while the component is connected in some way? Regards, Fredrik. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPClient and comet / chunked transfer and post datawith active connection
Fredrik Larsson wrote: Hi, I am working on a project where the server side delivers HTTP-data as comet / chunked transfers e.g. the connection is never closed and as new events come along it sends new data over the same connection. There is no way to change the server side and use sockets instead (server side isn't mine). I have figured out that I can receive their data on the OnDocData-events. However I am also supposed to post some data over the same connection. When I try to do this of course the component say it's busy. Is it possible to do the post while the component is connected in some way? As I understand Comet from a brief look at http://en.wikipedia.org/wiki/Comet_%28programming%29 the problem is the persistent or long-lasting HTTP connection between the server and the client. As long as a THttpCli-request isn't done you cannot issue another request. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HttpClient download and Progress Bar
Hi, I'm trying to have a progress bar working during a file download by HttpClient.Get (and HttpClient.RcvdStream). I've seen the OnDocData that seems to receive the length of data downloaded. Unfortunaly, the data length isn't really the size of the piece of file downloaded because the packed received include the HTTP protocol and the data. Is a way to know : - the size of the entire file (before starting the download) - the size of all data (I mean the size of the file piece received and not the size of the entire packet received). Thanks JP -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient download and Progress Bar
Jean-Paul Passama wrote: Hi, I'm trying to have a progress bar working during a file download by HttpClient.Get (and HttpClient.RcvdStream). I've seen the OnDocData that seems to receive the length of data downloaded. Unfortunaly, the data length isn't really the size of the piece of file downloaded because the packed received include the HTTP protocol and the data. Is a way to know : - the size of the entire file (before starting the download) Quoted from http://www.rfc-editor.org/rfc/rfc2068.txt: An HTTP/1.1 server may also establish persistent connections with HTTP/1.0 clients upon receipt of a Keep-Alive connection token. However, a persistent connection with an HTTP/1.0 client cannot make use of the chunked transfer-coding, and therefore MUST use a Content-Length for marking the ending boundary of each message. So it should be possible to force the server to send a content-length header. In event OnDocBegin property ContentLength should return that value. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient download and Progress Bar
Le 09/01/2011 15:49, Arno Garrels a écrit : Jean-Paul Passama wrote: Hi, I'm trying to have a progress bar working during a file download by HttpClient.Get (and HttpClient.RcvdStream). I've seen the OnDocData that seems to receive the length of data downloaded. Unfortunaly, the data length isn't really the size of the piece of file downloaded because the packed received include the HTTP protocol and the data. Is a way to know : - the size of the entire file (before starting the download) Quoted from http://www.rfc-editor.org/rfc/rfc2068.txt: An HTTP/1.1 server may also establish persistent connections with HTTP/1.0 clients upon receipt of a Keep-Alive connection token. However, a persistent connection with an HTTP/1.0 client cannot make use of the chunked transfer-coding, and therefore MUST use a Content-Length for marking the ending boundary of each message. So it should be possible to force the server to send a content-length header. In event OnDocBegin property ContentLength should return that value. In the Event OnDocBegin , no such property (ContentLength) : procedure TMyForm.HttpCli1DocBegin(Sender: TObject); And THttpClient doesn't have this property. So where I could retrieve this property from ? Thank you JP -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient download and Progress Bar
Le 09/01/2011 16:08, Jean-Paul Passama a écrit : Le 09/01/2011 15:49, Arno Garrels a écrit : Jean-Paul Passama wrote: Hi, I'm trying to have a progress bar working during a file download by HttpClient.Get (and HttpClient.RcvdStream). I've seen the OnDocData that seems to receive the length of data downloaded. Unfortunaly, the data length isn't really the size of the piece of file downloaded because the packed received include the HTTP protocol and the data. Is a way to know : - the size of the entire file (before starting the download) Quoted from http://www.rfc-editor.org/rfc/rfc2068.txt: An HTTP/1.1 server may also establish persistent connections with HTTP/1.0 clients upon receipt of a Keep-Alive connection token. However, a persistent connection with an HTTP/1.0 client cannot make use of the chunked transfer-coding, and therefore MUST use a Content-Length for marking the ending boundary of each message. So it should be possible to force the server to send a content-length header. In event OnDocBegin property ContentLength should return that value. In the Event OnDocBegin , no such property (ContentLength) : procedure TMyForm.HttpCli1DocBegin(Sender: TObject); And THttpClient doesn't have this property. So where I could retrieve this property from ? Thank you JP Sorry for that message, I tried and it works : procedure TClubFTP.HttpCli1DocBegin(Sender: TObject); var x : integer; begin x := (Sender as THttpCli).ContentLength; if (x -1) and (SizeFile = 0) then begin SizeFile := x; end; end; The ProgressBar works with RcvdStream.Position procedure TClubFTP.HttpCli1DocData(Sender: TObject; Buffer: Pointer; Len: Integer); var x : longint; begin x := HttpCli1.RcvdStream.Position * 100 div SizeFile; if x 0 then PB.Position := x; end; Thank you JP -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HttpClient Add a Header Variable
How does one add a Header variable to ICS HttpClient ? thank you. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient Add a Header Variable
Hello Wayne, There is an event called OnBeforeHeader... which gets the request header as a TStringlist where you can change. Regards, SZ On Tue, Mar 10, 2009 at 1:11 PM, wayne forrest wayneforr...@gmail.comwrote: How does one add a Header variable to ICS HttpClient ? thank you. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Httpclient
On Jul 21, 2007, at 11:58, Joakim Krassman wrote: Is there anyone that do have a code snippet that actually tells me the exactly weight of all the content that the httpclient downloaded? There is no way for any client to know the entire weight of all content without requesting each piece individually. Like Arno said in his response, you can call HEAD for each item and check the Content-Length header returned (if supported) without having to actually download everything. You _do_ need to download the main document first in order to parse it. This is how the HTTP protocol works; the fact that Web browsers make it look seemlessly as a single request and operation is mostly behind-the-scenes magic. If you pay close attention, particularly on a slow connection, you'll see the Progress Bar in your browser stretch and contract constantly as it updates the total size when it finds images and other media while parsing the document (that's why usually the text is rendered before the images start loading). The HttpCli component in ICS does not provide this feature automatically, but it shouldn't be too hard to implement. For parsing the document and extracting all urls for image, css, js, and other external resources, you can use any one of the many open source HTML parsers out there (I even believe the JEDI project has a couple). Implementing your own parser shouldn't be that hard either: recall that you are not interested in understanding all layout information; you are only looking for things that look like img src=... and the like. So, to recap, here's a simple algorithm for doing what you requested: 1. GET document; 2. TOTAL_BYTES = Content-Length; 3. ResourceList := extract_resources(document); 4. For i := 0 To (ResourceList.Count - 1) 4.a. HEAD resource[i]; 4.b. TOTAL_BYTES = TOTAL_BYTES + Content-Length; 5. Next; dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Httpclient
Hi all, I hope that I managed to post this question to the right newsgroup, if not - be nice to me ;) Question, I am looking at the http client and I want that component to get the total size of a webpage with all graphics, flash etc. As I see it - it just looks at the header and sending this back as the result? Right or wrong? Is there anyone that do have a code snippet that actually tells me the exactly weight of all the content that the httpclient downloaded? Best regards, Joakim -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Httpclient
Joakim Krassman wrote: Hi all, I hope that I managed to post this question to the right newsgroup, if not - be nice to me ;) Question, I am looking at the http client and I want that component to get the total size of a webpage with all graphics, flash etc. Since the THttpCli is not a webbrowser you would have to follow all local links on a webpage (download them and parse them to find local links). If the server supports the Content-Length header you do not need to download all graphics, flashs etc. to know their size. As I see it - it just looks at the header and sending this back as the result? Right or wrong? Is there anyone that do have a code snippet that actually tells me the exactly weight of all the content that the httpclient downloaded? I do not know. Be creative it's not too difficult. -- Arno Garrels [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient inside HttpServer
Inside a THttpServer.OnGetDocument event, I have been using several THttpCli objects sucessfully for quite a while using Get. As my data returned is growing every month, i would like to start using GetAsync. The problem i have is, my data is returned in THttpCli.OnDocDone i am unable to access THttpConnection from there to pass the response stream back to the original requesting client connection. Is there a way round this? Derived a class from THttpCli and add any required data or methods you need. You can for example add a filed for the THttpConnection. -- Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient inside HttpServer
Hello, You can put it in the Tag property if you cast it to an integer. Later in OnDocDone or whatever you restore it back by typecasting to THttpConnection. Dont forget to make it null when the connection is gone. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Saturday, July 29, 2006, 21:11, JohnW wrote: Inside a THttpServer.OnGetDocument event, I have been using several THttpCli objects sucessfully for quite a while using Get. As my data returned is growing every month, i would like to start using GetAsync. The problem i have is, my data is returned in THttpCli.OnDocDone i am unable to access THttpConnection from there to pass the response stream back to the original requesting client connection. Is there a way round this? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HttpClient inside HttpServer
Inside a THttpServer.OnGetDocument event, I have been using several THttpCli objects sucessfully for quite a while using Get. As my data returned is growing every month, i would like to start using GetAsync. The problem i have is, my data is returned in THttpCli.OnDocDone i am unable to access THttpConnection from there to pass the response stream back to the original requesting client connection. Is there a way round this? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpClient, loadsave png file
How to load some image and save in to file or directly assign to image object on form? HttpCli1.URL := 'http://www.myserver.com/images/test.png'; HttpCli1.RcvdStream := TMemoryStream.Create; HttpCli1.Get; MyPngComponent.LoadFromStream(HttpCli1.RcvdStream); HttpCli1.RcvdStream.Free; Of course you must add error checking ! -- Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPClient
I think the timer should be reset for each data sent/got. Otherwise, for lenghtly downloads, it would simply break the it!! Regards, SZ - Original Message - From: Francois Piette [EMAIL PROTECTED] To: ICS support mailing twsocket@elists.org Sent: Wednesday, February 15, 2006 9:39 AM Subject: Re: [twsocket] HTTPClient I am having a problem with HttpClient that sits for nearly hours on a url while trying to download it. If I am not mistaken there is no timeout property to specify. How would I go about this? Use a TTimer you start along with your request and disable from OnRequestDone. Should the timer event trigger, you call Abort. -- Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] HTTPClient
Hi ICS, I am having a problem with HttpClient that sits for nearly hours on a url while trying to download it. If I am not mistaken there is no timeout property to specify. How would I go about this? Thanks for helping! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPClient
I am having a problem with HttpClient that sits for nearly hours on a url while trying to download it. If I am not mistaken there is no timeout property to specify. How would I go about this? Use a TTimer you start along with your request and disable from OnRequestDone. Should the timer event trigger, you call Abort. -- Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] httpClient, total bytes received / sent
OnDocDate event: - TotalReceived := TotalReceived + Len; - Yes, but be carefull 'cause if you do UI stuff in OnDocData event the transfell could slow down awfully! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] httpClient, total bytes received / sent
Yes, but be carefull 'cause if you do UI stuff in OnDocData event the transfell could slow down awfully! Is there any other way around besides in OnDocData event? Because performance is of utter most importance to me. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] httpClient, total bytes received / sent
If you just inrease an integer in the OnDocData event this shouldn't be a performance issue. Marius van Tubbergh wrote: Yes, but be carefull 'cause if you do UI stuff in OnDocData event the transfell could slow down awfully! Is there any other way around besides in OnDocData event? Because performance is of utter most importance to me. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] httpClient, total bytes received / sent
Hello Marius, As Tibor noticed, just increment a integer and with a Timer you can update the UI every few seconds. This way you will not have performacne degrease. It is not the calculation that take time, it is graphical user interface is very slow. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPClient
Hello Robert, If the server does not give you the length, then it is impossible to know. ps: please dont reply on a thread if you start a new topic. Many people will not see your question if they use an email reader that sort on topic and when they dont follow the topic. I only saw it by accident :( --- Rgds, Wilfried http://www.mestdagh.biz Thursday, March 31, 2005, 18:21, GSE Sas wrote: How is possible to know if the full zipped file is downloaded? I've not info about the file size in the web site. Tks Robert -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
