Re: how to deal with large/slow sites

[Doug Cutting]

EM wrote:

Doug Cutting wrote:

> (2) Will the dropped urls be picked up again in subsequent cycles of fetchlist/segment/fetch/updatedb?

They will be retried in the next cycle, up to db.fetch.retry.max.

After the next bin/nutch generate... or are you using 'cycle' for something else?

updatedb + generate

Doug