On 5 January 2013 21:33, vigna <[email protected]> wrote: >> But why would you want a web crawler to have 10-20K simultaneously >> opened connections in the first place? > > (I thought I answered this, but it's not on the archive. Boh.) > > Having a few thousands connection open is the only way to retrieve data > respecting politeness (e.g., not banging the same site too often).
Huh? There are surely other ways to achieve that goal. > I have another question: Please start a new thread for a new question. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
