Our company is also using bigtop to build up our private hadoop distribution. A nice thing about bigtop is that you can patch the sourse code, add or modify features you'd like to have and then leverage bigtop's packaging and testing framework to produce production ready packages for cluster admin to easily upgrade the cluster either to fix bugs or introduce new features. We're currently upgrading our production hadoop to bigtop 1.0, which consist some of our private patches already. The distribution looks promising since we've been backed by bigtop's CI and testing framework.
2015-06-02 10:27 GMT+08:00 Demai Ni <[email protected]>: > Andrew, > > great to hear that you are also using BigTop. I will surely try it out, to > replace my (a little bit) old CDH cluster. :-) > > cheers > > Demai > > On Mon, Jun 1, 2015 at 5:29 PM, Andrew Purtell <[email protected]> > wrote: > > > Bigtop, in a nutshell, is a non-commercial multi-stakeholder Apache > > project that produces a build framework that takes as input source from > > Hadoop and related big data projects and produces as output OS native > > packages for installation and management - certainly, a distribution of > the > > Hadoop ecosystem - coupled with a suite of integration tests for ensuring > > the distribution components are working well together, coupled with a > suite > > of Puppet scripts for post-deploy configuration management. It's a rather > > large nutshell. (Smile) Bigtop distribution packages are supported by > > Cask's Coopr (coopr.io) and I think to some extent by Ambari (haven't > > tried it). > > > > I've personally used Bigtop for years to produce several custom Hadoop > > distributions. For this purpose it is a great tool. > > > > Please mail [email protected] if you would like to know more, we'd > > love to talk with you. > > > > > > On Jun 2, 2015, at 7:16 AM, Demai Ni <[email protected]> wrote: > > > > Chris and Roman, > > > > many thanks for the quick response. I will take a look at bigtop. > > Actually, I heard about it, but thought it is a installation framework, > > instead of a hadoop distribution. Now I am looking at the BigTop 0.7.0 > > hadoop instruction, which probably will work fine for my needs. > Appreciate > > the pointer. > > > > Roman, I will ping you off list for ODP. I was hoping ODP will be the one > > for me. Well, in reality, it is owned by a few companies, at least not by > > ONE company. :-) It is fine with me, as long as ODP is open to be used > by > > others. I am just having trouble to find document/installation info of > the > > ODP. maybe I should google harder? :-) > > > > Demai > > > > > > On Mon, Jun 1, 2015 at 1:46 PM, Roman Shaposhnik <[email protected]> wrote: > > > >> On Mon, Jun 1, 2015 at 1:37 PM, Demai Ni <[email protected]> wrote: > >> > My question is besides the commercial distributions: CDH(Cloudera) , > >> HDP > >> > (Horton work), and others like Mapr, IBM... Is there a distribution > >> that is > >> > NOT owned by a company? I am looking for something simple for cluster > >> > configuration/installation for multiple components: hdfs, yarn, > >> zookeeper, > >> > hive, hbase, maybe Spark. Surely, for a well-experience person(not > me), > >> > he/she can build the distribution from Apache releases. Well, I am > more > >> > interested on building application on top of it, and hopefully to find > >> one > >> > packed them together. > >> > >> Apache Bigtop (CCed) aims at delivering a 100% open and > >> community-driven distribution of big data management technologies > >> around Apache Hadoop. Same as, for example, what Debian is trying > >> to do for Linux. > >> > >> > BTW, I don't need the latest releases like other commercial > distribution > >> > offered. I am also looking into the ODP(the open data platform), but > >> that > >> > project is kind of quiet after the initial Feb announcement. > >> > >> Feel free to ping me off list if you want more details on ODP. > >> > >> Thanks, > >> Roman. > >> > > > > >
