Thanks, this helps.
D.
Andrzej Bialecki wrote:
Andy Liu wrote:
This is built into Nutch. Instead of injecting http:// url's, use
file:// , and Nutch will use protocol-file to fetch the files locally.
Andy
Also, there is a tool I created to skip importing these URLs into
database first... Take a look at
http://issues.apache.org/jira/browse/NUTCH-68
So, you can do the following:
cd target
find . -exec printf "file:/`pwd`/{}\n" \; > url.lst
and convert this url.lst to a new segment containing a fetchlist.
