My problem is that when I go to run bin/nutch/analyze db 2 it runs out of space. Is it normal that it'll eat up 200+ gigs for a 7.5 gigs worth of segments?
I assume i need to run analyze frequently...?
I also don't really understand what this does. The way I think things work is like this:
generate db segment -topN <x> * Generates a fetchlist of unfetched or out of date URLs.
fetch <segment> * Builds a content segment of the URLs in that fetchlist
updatedb db
* Tells the web db which url's have been fetched so they don't get added to a new fetchlist (unless they're outdated)
analyze db 2
* Performs DB analysis. Two passes. I think this is the actual ranking mechanism - is that right?
merge index segments/* * Merges indexed segments into a master index.
Thanks, Gus
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
