On Thu, 04 Dec 2008 10:48:54 -0800, Lawrence Wickline wrote: > Thanks for the help I think I got it. > > As far as lines go I believe it will be processing hundreds of thousands > of lines if not a million or more lines per run. I haven't gotten to do > a full run but it has been running acceptably fast on my test files. > > I ended up putting it into a main function and adding: > > if __name__ == "__main__": > main() >
In most cases, in processing involving networking, the bottleneck is the network speed itself. To speed things up by optimizing your own code might not make your download significantly faster (getting 60 seconds faster is great for scripts that usually runs for 70 seconds, but is a waste of development time for scripts that usually run for 1 hour) Usually a multi-threading downloader might be a better chance to improvement, especially for 1) downloading from different site, 2) the remote sites have speed limit, 3) you have faster download link than the server can gives _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
