Re: urllib2 slow for multiple requests

2009-05-15 Thread Tomas Svarovsky
On May 14, 6:33 pm, "Richard Brodie" wrote: > "Tomas Svarovsky" wrote in message > > news:747b0d4f-f9fd-4fa6-bb6d-0a4365f32...@b1g2000vbc.googlegroups.com... > > > This is a good point, but then it would manifest regardless of the > > language used A

Re: urllib2 slow for multiple requests

2009-05-14 Thread Tomas Svarovsky
On May 14, 11:57 am, "Richard Brodie" wrote: > "cgoldberg" wrote in message > > news:9ae58862-1cb2-4981-ae6a-0428c7684...@z5g2000vba.googlegroups.com... > > > you aren't doing a read(), so technically you are just connecting to > > the web server and sending the request but never reading the cont

Re: urllib2 slow for multiple requests

2009-05-14 Thread Tomas Svarovsky
One more thing, since I am stuck with 2.4 (and if this is really 2.4 issue), is there some substitution for urllib2? On May 14, 11:00 am, Tomas Svarovsky wrote: > On May 13, 4:55 pm, cgoldberg wrote: > > > > Bascally it just grabs a page xy > > > times and tells me how

Re: urllib2 slow for multiple requests

2009-05-14 Thread Tomas Svarovsky
On May 13, 4:55 pm, cgoldberg wrote: > > Bascally it just grabs a page xy > > times and tells me how long it took. > > you aren't doing a read(), so technically you are just connecting to > the web server and sending the request but never reading the content > back from the socket.  So your timing

urllib2 slow for multiple requests

2009-05-13 Thread Tomas Svarovsky
Hello everybody, really new to python, so bear with me. I am trying to do some very basic scraping tool. Bascally it just grabs a page xy times and tells me how long it took. When I do this once, it is blazingly fast, but when I increase the number of repetitions, it is slowing down considerably (1