Hi, They take records or (FetchItems) from the QueueFeeder. Queues are based on domain, host or ip and a URL exists only once, so nothing collides. The redirect will be followed in the next fetch cycle.
Markus -----Original message----- > From:Vangelis karv <[email protected]> > Sent: Friday 14th February 2014 12:39 > To: [email protected] > Subject: Threads > > Hello people! > > Lets say we choose 20 threads to fetch. How do they cooperate? I mean, who > tells them what pages each one of them will fetch? > Is it possible some of them to collide or fetch the same page without them > knowing? > I read the code and found that if the redirect is to the same page, it will > not follow that redirect. Any advice would be very helpful! > > Vangelis
