Hey Roger,
SSH is only needed to start and stop daemons - it's not really needed
for running Hadoop itself. Currently, we do this through custom site
mechanisms, and not through SSH.
Brian
On Nov 4, 2008, at 10:36 AM, Zhang, Roger wrote:
Brian,
You seem to have a pretty large cluster. How do you think about the
overall performance?
Is your implementation on Open-SSH or SSH2?
I'm new to this and trying to setup a 20 node cluster. But our Linux
boxes enforced F-secure SSH2 already, which I found HDFS 0.18 does
not support right now.
Anyone has any idea of a workaround?
Thanks and best Rgds.
Roger Zhang
-----Original Message-----
From: Brian Bockelman [mailto:[EMAIL PROTECTED]
Sent: 2008年11月4日 21:36
To: [email protected]
Subject: Re: Hadoop hardware specs
Hey Arjit,
We use all internal SATA drives in our cluster, which is about 110TB
today; if we grow it to our planned 350TB, it will be a healthy mix of
worker nodes w/ SATA, large internal chases (12 - 48TB), SCSI attached
vaults, and fibre channel vaults.
Brian
On Nov 4, 2008, at 4:16 AM, Arijit Mukherjee wrote:
Hi All
We're thinking of setting up a Hadoop cluster which will be used to
create a prototype system for analyzing telecom data. The wiki page
on
machine scaling (http://wiki.apache.org/hadoop/MachineScaling) gives
an
overview of the node specs and from the Hadoop primer I found the
following specs -
* 5 x dual core CPUs
* RAM - 4-8GB; ECC preferred, though more expensive
* 2 x 250GB SATA drives (on each of the 5 nodes)
* 1-5 TB external storage
I'm curious to find out what sort of specs do people use normally. Is
the external storage essential or will the individual disks on each
node
be sufficient? Why would you need an external storage in a hadoop
cluster? How can I find out what other projects on hadoop are using?
Cheers
Arijit
Dr. Arijit Mukherjee
Principal Member of Technical Staff, Level-II
Connectiva Systems (I) Pvt. Ltd.
J-2, Block GP, Sector V, Salt Lake
Kolkata 700 091, India
Phone: +91 (0)33 23577531/32 x 107
http://www.connectivasystems.com
