Hi,How do I go about recrawling websites? Essentially I want to repeat the following tasks repeatedly:
[one off task] inject the database with a url list 1. create a segment with the initial list 2. fetch the segment 3. update the database 4. create a new segment with the outlinks from [2] 5. fetch the segement created in [4]. I basically want to repeat steps 2 through 5. How would I do this? Thanks for the help Suhail
