Hmm... perhaps I'm not clear then on the purpose of these two tools. Is this new tool not simply a SegmentMergeTool that does a smarter merge, albeit at a cost?
No, this is a segment updater tool. The anchors and scores are copied from the database into a segment when its fetchlist is created. Database updates after the fetchlist generation are never currently reflected in a segment. This tool would permit one to update all segments with the latest anchor and score data.
On a related note, would a single-site crawler have any need of this new tool? My sense is that inlink-based ranking has little value in a site-restricted crawl, so you'd just want to use the straight SegmentMergeTool.
Anchor text from incoming links is valuable in single-site crawlers. Consider for example a help page that is frequently linked to with the text "help" but may not contain the word "help" many times. This should probably be the first hit when searching for "help" on the site.
Doug
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/
_______________________________________________
Nutch-developers mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
