Yes, I think so, but.... My worry is that AWS does suffer outages - their AZs (== DCs) do get knocked out (at least the ones in N. Virginia), so I figured is one can spread data over multiple AZs/DCs, then the chances of the whole cluster being wiped out when 1 AZ goes down will be lower. Spreading things over N racks is good for guarding against things like rack-specific equipment failures, so this wouldn't help if the whole AZ/DC goes down.
Otis -- Search Analytics - http://sematext.com/search-analytics/index.html HBASE Performance Monitoring - http://sematext.com/spm/index.html On Mon, May 6, 2013 at 3:18 PM, Mike Spreitzer <[email protected]> wrote: > If there were a way to ask for an HBase cluster to be spread across > multiple racks within the same AZ, would this be valuable? > > Thanks, > Mike > > > > From: Amandeep Khurana <[email protected]> > To: "[email protected]" <[email protected]>, > Date: 05/06/2013 02:09 PM > Subject: Re: HBase cluster over multiple EC2 Availability Zones? > > > > Based on some testing I did on running MR on clusters across AZs, you take > a 20-30% hit in performance purely due to latencies in replication. Of > course, the numbers would vary based on the workload and my testing was > definitely not comprehensive enough to cover all scenarios. This does not > take into account the extra $$ you pay for going cross AZ, which as you > can > imagine would add up pretty quickly when looking at a large scale > deployment. > > > On Mon, May 6, 2013 at 11:00 AM, Otis Gospodnetic < > [email protected]> wrote: > >> Re traffic and free vs. not free - Amandeep was right - according to >> http://aws.amazon.com/ec2/pricing/#DataTransfer it's NOT free. The >> AWS person I recently spoke to about this told me it was free, but I >> guess it's not! >> >> Otis >> -- >> Search Analytics - http://sematext.com/search-analytics/index.html >> HBASE Performance Monitoring - http://sematext.com/spm/index.html >> >> >> >> >> >> On Mon, May 6, 2013 at 1:58 PM, Otis Gospodnetic >> <[email protected]> wrote: >> > Hi, >> > >> > Actually, cross-AZ traffic is actually free. It's cross-Region >> > traffic that's not free. I realize the cross-AZ (translates to >> > cross-DC) traffic will be slower, but I wonder if there are any HBase >> > (or HDFS)-specific reasons why one should not attempt to do this? >> > >> > Thanks, >> > Otis >> > -- >> > Search Analytics - http://sematext.com/search-analytics/index.html >> > HBASE Performance Monitoring - http://sematext.com/spm/index.html >> > >> > >> > >> > >> > >> > On Mon, May 6, 2013 at 1:41 PM, Amandeep Khurana <[email protected]> >> wrote: >> >> I've not come across anyone spanning clusters cross AZ. You pay for >> cross >> >> AZ traffic and the link is slower than within a single AZ. >> >> >> >> Amandeep >> >> >> >> >> >> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic < >> >> [email protected]> wrote: >> >> >> >>> Hi, >> >>> >> >>> Do people spread HBase clusters over multiple EC2 Availability > Zones, >> >>> or is that a big no-no? Cross-AZ communication is supposedly going >> >>> over fast fiber optic cables. >> >>> >> >>> Thanks, >> >>> Otis >> >>> -- >> >>> Search Analytics - http://sematext.com/search-analytics/index.html >> >>> HBASE Performance Monitoring - http://sematext.com/spm/index.html >> >>> >> >
