You may need to be more specific, there is a wiki page that give some
ideas about hardware requirements.
Nutch 0.8 is very scalable.
With some tricks it is no problem to run a several hundred million
page search engine - that is proved.
In general it very much depends on the servers, but you can say more
cheap servers are better than less expansive.
However more expansive than your severs is the bandwidth you need.
Stefan
Am 05.01.2006 um 22:22 schrieb Dan Segel:
Do you know if Nutch can handle something this large??
Dan
----- Original Message ----- From: "Byron Miller"
<[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Thursday, January 05, 2006 4:20 PM
Subject: Re: please disregard last post....
Yes and no. It's my personal belief that when you
read they run the search engine on 40 servers that is
40 query servers. I doubt they have a 2billion page
index/db/analysis/processing and spidering on just 40
servers but prove me wrong :)
-byron
--- Dan Segel <[EMAIL PROTECTED]> wrote:
correct post:
I am looking to be able to have an engine the size
of gigablast.com, I understand they run EVERYTHING
on just 40 servers! Can this software do that, and
what are the server specifications needed.
Dan
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
