Hi everyone,

Anyone knows if there is an option to fetch a single or a group of wanted
urls using the fetcher, but (!!) not fetching previous links extracted from
urls which have already been fetched (meaning depth 1+x).

for e.g.

if i did a inject-->generate-->fetch-->update loop once
now the crawldb is updated with depth 1 links from the root url list.

i now want to inject a new url list for fetching, but want to exclude the
current links from fetching.

Eyal Edri

Reply via email to