Tim Gautier wrote:
I ran a fetch on a fetch list of around 3 million urls and it has failed on a single reduce task. Is there any way to recover the data that's been pulled down already? It's my understanding that the pages have all been pulled down to disk at this point and since it takes 3 days to pull them down, I'd really like to avoid doing it again.
Did you use DFS, or did you run this on a single machine? -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
