Re: [Nutch-dev] fast database update

[Pierre]

nutch should come up with something like this open source engine that automatically installs. http://www.egothor.org/download/ takes alot of headaches away. ----- Original Message ----- From: Brice Tsakam To: [EMAIL PROTECTED] Sent: Wednesday, March 10, 2004 11:41 AM Subject: [Nutch-dev] fast database update Hello, I am new to the nutch project. Actually, I have been looking for an open source search engine project to propose the introduction of an optimization method for the database update.   The idea is simple: - Let a large number of anonymous computers monitor of small regions of the world wide web. These computer reports web pages changes to a centralized indexing engine.   - The centralized engine maintains the duration since the last updates for each area of the web. This allows to prioritize the monitoring needs per web area.   - Volunteer anonymous computers can load the next area to monitor (e.g. at the same time as they get web search results).   The result is that the indexes are kept up to date with minimal efforts for the indexing engine and for the anonymous computers. And the reaction of the search engine to web pages update is extremly quick.   If the anonymous computers monitor web sites that are in their location, we also obtain that the (long distance) crawling bandwith is minimized.   Is anybody working on something similar ? Are there any effort to distribute the search engine components ?   Bye, Brice Protect your PC - Click here for McAfee.com VirusScan Online ------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers