Thank you, Bryan, that is very important and clear some cloudiness in my mind.
Sincerely, Mark On Fri, Dec 30, 2011 at 10:54 AM, Bryan Beaudreault < [email protected]> wrote: > We have also seen this in our testing, though we focused mainly on MR more > than HBase. > > Keep in mind that EC2 Compute Units are defined as follows: > > The amount of CPU that is allocated to a particular instance is expressed > > in terms of these EC2 Compute Units. We use several benchmarks and tests > to > > manage the consistency and predictability of the performance of an EC2 > > Compute Unit. One EC2 Compute Unit provides the equivalent CPU capacity > of > > a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. > > > This does not even account for CPU contention that Amandeep mentioned, > which we have noticed at times as well. Also, c1.mediums have a I/O > Performance rating of "Moderate." I think this mainly refers to ethernet > speed, but it could refer to disk speed as well. > > If your local workstation is a reasonably modern system, it is very > possible for you to see much better performance locally. The difference > between 2.5 1.0 GHz 2007 processors (2.5 compute units) and a modern i5, > i7, or equivalent is huge not just in speed and number of cores, but > architecture, cache, etc. In terms of HBase write speed, if you are > running on an SSD this could cause a substantial gap as well. > > On Fri, Dec 30, 2011 at 12:38 AM, Amandeep Khurana <[email protected]> > wrote: > > > Is your client program running on the same node? Given that c1.mediums > are > > on shared hosts, your neighbor might be overloading his VM, causing yours > > to starve. > > > > On Fri, Dec 30, 2011 at 9:50 AM, Mark Kerzner <[email protected]> > > wrote: > > > > > Hi, > > > > > > I am running a small program to load about 1 million rows into HBase. > It > > > takes 200 seconds on my dev machine, and 800 seconds on a c1.medium EC2 > > > machine. Both are running the same version of Ubuntu and the same > version > > > of HBase. Everything is local on one machine in both cases. > > > > > > What could the difference between the two environments be? I did notice > > > that my local machine has higher CPU loads: > > > > > > hbase 64% > > > java (my app) 38% > > > hdfs 20% > > > > > > whereas the EC2 machine > > > hbase 47% > > > java (my app) 23% > > > hdfs 14% > > > > > > > > > Sincerely, > > > Mark > > > > > >
