Re: [twsocket] Adding gzip to HttpCli
> >I think I said that RcvdCount that a choice has to be made and Ithat I have > >no defitive answer. The idea is to break as less as possible existing code. > >RcvdCount is used for progress bar updating and should be compressed byte > >count. It is also used to allocate storage (or similar use) for data and > >should be decompressed data. Maybe for simplicity we should let RcvdCount be > >the compressed byte count ? Really a question, the debate is open ! > > I doubt that RcvdCount could be used to allocate storage. The body > data will be put into RcvdStream that is a stream, and normally a > stream is able to allocate the storage itself. RcvdStream is not the only way to get data from the component. Some (many ?) applications sue OnDocData event to get data on the fly. > It could be useful if the size of the body is known in advance so the > whole storage is allocated in one step, but eventually you have this > information only with the content length and not from RcvdCount. > And the content length is not always specified, not to mention the > case when it is wrong. If specified but wrong, the component will hang, unless the server close the connexion before the length specified is reached. > After all this considerations my conclusions are: > - RcvdCount contains the count of bytes received from the server > - ContentLength contains what specified in the header Agreed. -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TPop3Cli is halted while receiving a message over 4Mb
>> (taking into account that >> they disclose themselves in such a negligible, infrequent manner)? > >Do you mean that it doesn't always occur with the same 4MB message ? Yes. Today the program fetches the same message (it remains on a server) successfully (several times for testing). Now I can't investigate the matter, alas. Best wishes, Stanislav Korotky, Russia, Moscow, CET +02:00 >-- >[EMAIL PROTECTED] >http://www.overbyte.be > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TPop3Cli is halted while receiving a message over 4Mb
Stanislav Korotky wrote: >>> (taking into account that >>> they disclose themselves in such a negligible, infrequent manner)? >> >> Do you mean that it doesn't always occur with the same 4MB message ? > > Yes. Today the program fetches the same message (it remains on a server) > successfully (several times for testing). Then it is most likely due to a bad line to the server/internet. You should implement a custom timeout that triggers when the server didn't send a packet/line within let's say 60 seconds. Or try a sniffer to check what's going on. Arno Garrels > Now I can't investigate the matter, alas. > > Best wishes, > Stanislav Korotky, > Russia, Moscow, CET +02:00 > >> -- >> [EMAIL PROTECTED] >> http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
I'm trying to write a spider that collects web pages. In the LocationChange event, I check the properties of the new location. For instance: - Is the new location a URL that is present on a black-list of banned URL's or domains? - Is there a robots.txt file for the new location? - Other properties that would lead to an abort of the request. - Empty the received data buffer so that the buffer only contains the contents of the new location (I also use the OnDocData event to add received data to my buffer).. In the RequestDone event, I process the collected data. A flag is set for the client to be deleted when the clean up routine comes by. I use the RequestDone event because this way I'm sure now other events can be triggered and it is save to delete the client. Thanks, --Merijn P.S. Would it be more efficient to do things another way? On 27 Jul 2005 3:48:57 +0100, Maurizio Lotauro <[EMAIL PROTECTED]> wrote: > On 26-Jul-05 09:31:34 Merijn Terheggen wrote: > > >Yes, I'm using the original client. Also the latest version. > > >Don't you get the same results when you add a LocationChange event > >handler and set breakpoints in both the LocationChange and the > >RequestDone event handlers? > > What are you doing in these events? > > > Bye, Maurizio. > > -- > To unsubscribe or change your settings for TWSocket mailing list > please goto http://www.elists.org/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Monitor sockets components
Hello Angus, Testing it. Seems good ! --- Rgds, Wilfried http://www.mestdagh.biz Monday, July 25, 2005, 21:36, Angus Robertson - Magenta Systems Ltd wrote: > I've just written two new components for monitoring ethernet packets: > 1 - Raw sockets (W2K and later) using ICS, does not any other software > installed, but may not capture send packets on W2K and XP, only W2K3, > and ignores non-IP traffic. > 2 - Winpcap device driver, needs to be installed (two DLLs and a driver), > but captures all packets including non-IP, from http://www.winpcap.org/. > Note the Delphi Winpcap pcap.pas and packet32.pas were originally > written by Lars Peter Christiansen, but have several bug fixes and many > new features. In theory Winpcap will run on Windows 9x, but it's not > been tested. It has only been tested with WinPcap 3.0 and 3.1 beta 4. > Both components have properties to filter local traffic, and to exclude > data for better performance. > There are two test applications: > 1 - A simple packet analyser, showing ethernet packets sent and received > 2 - A traffic analyser, that totals IP traffic for each different IP > address, by protocol and service. > Note you don't need to install WinPcap to runs these applications, they > will work with raw sockets alone. > The source and executables are available from: > http://www.magsys.co.uk/download/software/monsock1.zip > Any comments appreciated. > Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HttpCli StatusCode = 0 causes problems
Scrive Merijn Terheggen <[EMAIL PROTECTED]>: > I'm trying to write a spider that collects web pages. > > In the LocationChange event, I check the properties of the new > location. For instance: Ok, nothig "strange" that can interact with the message queue. Francois, I reproduced the "problem" using HttpTst demo with the url http://help.lycos.com This happen because an OnRequestDone is triggered after the StartRelocation, that call InternalClear. Maybe the problem is that the OnRequestDone should not be called at all, or that it should be called after the StartRelocation? I tried to trace the flow but it is not so simple. Bye, Maurizio. This mail has been sent using Alpikom webmail system http://www.alpikom.it -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Adding gzip to HttpCli
Scrive Francois Piette <[EMAIL PROTECTED]>: [...] > > I doubt that RcvdCount could be used to allocate storage. The body > > data will be put into RcvdStream that is a stream, and normally a > > stream is able to allocate the storage itself. > > RcvdStream is not the only way to get data from the component. Some (many ?) > applications sue > OnDocData event to get data on the fly. In that case the decompression should be made completly by the application, or should we add some support for this? [...] > > After all this considerations my conclusions are: > > - RcvdCount contains the count of bytes received from the server > > - ContentLength contains what specified in the header > > Agreed. This is how it is implemented. Bye, Maurizio. This mail has been sent using Alpikom webmail system http://www.alpikom.it -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Adding gzip to HttpCli
>> > I doubt that RcvdCount could be used to allocate storage. The body >> > data will be put into RcvdStream that is a stream, and normally a >> > stream is able to allocate the storage itself. >> >> RcvdStream is not the only way to get data from the component. Some (many >> ?) >> applications sue >> OnDocData event to get data on the fly. > > In that case the decompression should be made completly by the > application, or > should we add some support for this? If decompression is done on the fly (streaming, we already talked about this), then nothing special is required. -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Adding gzip to HttpCli
Scrive Francois PIETTE <[EMAIL PROTECTED]>: > >> RcvdStream is not the only way to get data from the component. Some (many > > >> ?) > >> applications sue > >> OnDocData event to get data on the fly. > > > > In that case the decompression should be made completly by the > > application, or > > should we add some support for this? > > If decompression is done on the fly (streaming, we already talked about > this), then nothing special is required. The decompression will be made only if RcvdStream is assigned. Bye, Maurizio. This mail has been sent using Alpikom webmail system http://www.alpikom.it -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] [OT] OverByte Home
Do you know Google Earth (http://earth.google.com/) ? If you have installed it, you can open the attached document and find where OverByte Home is. Really ! I'm located in a high resolution area. You can really see my house, my garden,... To really see my house, you have a picture on the author's page at my website. -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Adding gzip to HttpCli
>> >> RcvdStream is not the only way to get data from the component. Some >> >> (many >> >> >> ?) >> >> applications sue >> >> OnDocData event to get data on the fly. >> > >> > In that case the decompression should be made completly by the >> > application, or >> > should we add some support for this? >> >> If decompression is done on the fly (streaming, we already talked about >> this), then nothing special is required. > > The decompression will be made only if RcvdStream is assigned. That is not needed. The decompression can be made always. If RcvdStream is assigned, then decompressed data is written there. Anytime is is made available from OnDocData event. -- [EMAIL PROTECTED] http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] [OT] OverByte Home
Well, something went wrong with the attached document. I've uploaded it at: http://www.overbyte.be.arch/OverByteHome.kmz (This is a kind of hyperlink for Google Earth). -- [EMAIL PROTECTED] http://www.overbyte.be - Original Message - From: "Francois PIETTE" <[EMAIL PROTECTED]> To: Sent: Wednesday, July 27, 2005 7:58 PM Subject: [twsocket] [OT] OverByte Home > Do you know Google Earth (http://earth.google.com/) ? > If you have installed it, you can open the attached document and find > where > OverByte Home is. Really ! > I'm located in a high resolution area. You can really see my house, my > garden,... To really see my house, you have a picture on the author's page > at my website. > > -- > [EMAIL PROTECTED] > http://www.overbyte.be > > -- > To unsubscribe or change your settings for TWSocket mailing list > please goto http://www.elists.org/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Monitor sockets components
On 25-Jul-05 20:36:00 Angus Robertson - Magenta Systems Ltd wrote: [...] >There are two test applications: [...] >The source and executables are available from: >http://www.magsys.co.uk/download/software/monsock1.zip I downloaded it today, but I see only one executable. Bye, Maurizio. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Adding gzip to HttpCli
On 27-Jul-05 18:59:30 Francois PIETTE wrote: [...] >>> If decompression is done on the fly (streaming, we already talked about >>> this), then nothing special is required. >> >> The decompression will be made only if RcvdStream is assigned. >That is not needed. The decompression can be made always. If RcvdStream is >assigned, then decompressed data is written there. Anytime is is made >available from OnDocData event. What I mean is that actually the decompression works only on RcvdStream. Do you want that the data passed to OnDocData is decompressed? This could eventually be done only if the class that make the decompression supports "on the fly", or better if all classes neded to decompress the body suport it. If we want to accept both scenarios (on the fly and not, but I don't know if on the fly is always appliable) then we will have an incostincency: is it possible that OnDocData will receive compressed data. I forget to mention one thing. Xavier in his code set FLastResponse with the whole content of the decompressed stream. Is it correct and/or needed? Bye, Maurizio. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
