I've had good experience running a large hadoop cluster on EC2 instances. After almost 1 year we haven't had any significant down time, just lost a small # of data nodes. I don't think EMR is an ideal solution if your cluster will be running 24/7.
But for running a large cluster, I don't see how you it's more cost efficient to run in the cloud than to own the hardware and we're trying to move off the cloud onto our own hardware. Can I ask why you're looking to move to the cloud? On Fri, Aug 9, 2013 at 10:42 AM, Nitin Pawar <[email protected]>wrote: > check altiscale as well > > > On Fri, Aug 9, 2013 at 3:05 AM, Dhaval Shah > <[email protected]>wrote: > >> Thanks for the list Marcos. I will go through the slides/links. I think >> that's helpful >> >> Regards, >> Dhaval >> >> ------------------------------ >> *From:* Marcos Luis Ortiz Valmaseda <[email protected]> >> *To:* Dhaval Shah <[email protected]> >> *Cc:* [email protected] >> *Sent:* Thursday, 8 August 2013 4:50 PM >> *Subject:* Re: Hosting Hadoop >> >> Well, all depends, because many companies use Cloud Computing >> platforms like Amazon EMR. Vmware, Rackscpace Cloud for Hadoop >> hosting: >> http://aws.amazon.com/elasticmapreduce >> http://www.vmware.com/company/news/releases/vmw-mapr-hadoop-062013.html >> http://bitrefinery.com/services/hadoop-hosting >> http://www.joyent.com/products/compute-service/features/hadoop >> >> There a lot of companies using HBase hosted in Cloud. The last >> HBaseCon was full of great use-cases: >> HBase at Pinterest: >> http://www.hbasecon.com/sessions/apache-hbase-operations-at-pinterest/ >> >> HBase at Groupon >> http://www.hbasecon.com/sessions/apache-hbase-at-groupon/ >> >> A great talk by Benoit for Networking design for HBase: >> >> http://www.hbasecon.com/sessions/scalable-network-designs-for-apache-hbase/ >> >> Using Coprocessors to Index Columns in an Elasticsearch Cluster >> http://www.hbasecon.com/sessions/using-coprocessors-to-index-columns/ >> >> 2013/8/8, Dhaval Shah <[email protected]>: >> > We are exploring the possibility of hosting Hadoop outside of our data >> > centers. I am aware that Hadoop in general isn't exactly designed to >> run on >> > virtual hardware. So a few questions: >> > 1. Are there any providers out there who would host Hadoop on dedicated >> > physical hardware? >> > 2. Has anyone had success hosting Hadoop on virtualized hardware where >> 100% >> > uptime and performance/stability are very important (we use HBase as a >> real >> > time database and it needs to be up all the time)? >> > >> > Thanks, >> > Dhaval >> >> >> -- >> Marcos Ortiz Valmaseda >> Product Manager at PDVSA >> http://about.me/marcosortiz >> >> >> > > > -- > Nitin Pawar >
