Sorry for the previous post. I haven't finished. Please skip it.
Hi all,
I've made some experiments on Hadoop on Amazon EC2.
I would like to share the result and any feedback would be appreciated.
Environment:
-Xen VM (Amazon EC2 instance ami-ee53b687)
-1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of lo
Hi all,
I've made some experiments on Hadoop on Amazon EC2.
I would like to share the result and any feedback would be appreciated.
Environment:
-Xen VM (Amazon EC2 instance ami-ee53b687)
-1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network
bandwidth (small instance)
-Hadoo
What does ganglia show for load and network?
You should also be able to see gc stats (count and time). Might help
as well.
fyi,
running
> hadoop-ec2 proxy
will both setup a socks tunnel and list available urls you can cut/
paste into your browser. one of the urls is for the ganglia interfa
On Wed, 9 Apr 2008, Chris K Wensel wrote:
make sure all nodes are running in the same 'availability zone',
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1347
check!
and that you are using the new xen kernels.
http://developer.amazonwebservices.com/connect/entry.jspa?ext
On Thu, 10 Apr 2008, Ted Dziuba wrote:
I have seen EC2 be slower than a comparable system in development, but
not by the factors that you're experiencing. One thing about EC2 that
has concerned me - you are not guaranteed that your "/mnt" disk is an
uncontested spindle. Early on, this was the
I have seen EC2 be slower than a comparable system in development, but
not by the factors that you're experiencing. One thing about EC2 that
has concerned me - you are not guaranteed that your "/mnt" disk is an
uncontested spindle. Early on, this was the case, but Amazon made no
promises.
A
On Thu, 10 Apr 2008, Ted Dunning wrote:
Are you trying to read from mySQL?
No, we're outputting to MySQL. I've also verified that the MySQL server is
hardly seeing any load, isn't waiting on slow queries, etc.
If so, it isn't very surprising that you could get lower performance
with more re
Are you trying to read from mySQL?
If so, it isn't very surprising that you could get lower performance with
more readers.
On 4/9/08 7:07 PM, "Nate Carlson" <[EMAIL PROTECTED]> wrote:
> Hey all,
>
> We've got a job that we're running in both a development environment, and
> out on EC2. I've
a few things..
make sure all nodes are running in the same 'availability zone',
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1347
and that you are using the new xen kernels.
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1353&categoryID=101
http://de
ginal Message-
From: Nate Carlson [mailto:[EMAIL PROTECTED]
Sent: Wednesday, April 09, 2008 7:07 PM
To: core-user@hadoop.apache.org
Subject: Hadoop performance on EC2?
Hey all,
We've got a job that we're running in both a development environment, and
out on EC2. I've been rather
Hey all,
We've got a job that we're running in both a development environment, and
out on EC2. I've been rather displeased with the performance on EC2, and
was curious if the results that we've been seeing are similar to other
people's, or if I've got something misconfigured. ;) In both
en
11 matches
Mail list logo
