My mistake, you're right. The last processing clusters we built were
using Xeon quad cores, not i7s. The i7s were search servers which
didn't need ecc memory. AFAICT, wikipedia is correct and the i7s don't
yet support ECC.
So my suggestion would be to stick with Xeon procs or something that
supports ECC for the processing clusters. I would never build a
processing cluster that doesn't have ECC memory. We spent a few weeks
when we first started trying to tracking down weird corruption checksum
bugs ultimately related to using non-ECC memory on a cluster.
Dennis
Doğacan Güney wrote:
Hi Dennis,
On Fri, Jul 24, 2009 at 16:46, Dennis Kubesku...@apache.org wrote:
fredericoagent wrote:
If I want to setup nutch with lets say 400 million urls in the database.
Is it better to have a 4-5 super fast and loaded servers or have 12-15
smaller , cheaper servers.
More smaller servers. Make sure they are energy efficient though and have a
decent amount of Ram. If a server goes down, you aren't affected as much.
By superfast I mean cpu is latest quad core or latest six core processor
with 6 Gigs Ram and 1. or 1.5 TB HD.
By cheap I mean something like a Xeon quad core 2.26 cpu with 3 Gig Ram
and
500 Sata HD.
or if anyone can suggest a better spec ideal
Our first servers were 1Ghz (Yes really) running hadoop 0.04 way back when.
Our first production clusters were core2, 4G ECC, 1 750G hard drive. These
days been building i7 8-core, 12G ECC, 4T raid-5 machines with up to 8
disks, 2U for around 2200.00 each. If you are looking for a good server
builder check out swt.com. They are supermicro resellers and build solid
machines.
It suggests here:
http://en.wikipedia.org/wiki/Core_i7#Drawbacks
that core i7's do not support ECC rams. Have you ran into any issues or is WP
wrong here?
Suggestions. Don't skimp on the hard drive, do at least 750G or more. Price
difference is negligible. Do at least 2G Ram, 4G is better, 8G is better
than that. You can get up to 12G on regular motherboards these days. After
that it gets much more expensive. Ao more recent processors, such as core2
or i7. They are more power efficient per processing unit. If you want a
really fast machine, do multiple disks in a raid-5 format.
Dennis
