when segments are generated, each pages' anchors and page scores are stored within the segment. any subsequent updates to the webdb are not propagated to the segments.
a tool that updates segments with page scores and anchors from the webdb does not yet exist. so generating a new segment with all the pages in the webdb and refetching them all was a quick and dirty way of ensuring that the pages had the most up to date anchor and score data. --- Christophe Noel <[EMAIL PROTECTED]> wrote: > In Nutch 0.5, I see following comment at > CrawlTool.java line 103 : > > // Re-fetch everything to get the complete set of > incoming anchor texts > // associated with each page. We should fix this, > so that we can update > // the previously fetched segments with the > anchors that are now in the > // database, but until that algorithm is written, > we re-fetch. > > What do you mean ? > > Thanks ! > > Chris. > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT > Products from real users. > Discover which products truly live up to the hype. > Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Nutch-developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/nutch-developers > __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
