Yeah, I think HDFS is pretty important, esp. if we want to use SolrCloud + HBase, both of which need a good DFS working under the hood. So it sounds like the most important packages I've heard so far are:
HDFS yarn Spark HBase+Phoenix Kafka Solr are the ones which seem to be most appealing. On Fri, Dec 12, 2014 at 9:54 PM, Andrew Purtell <[email protected]> wrote: > > Well if HDFS is dropped, and all packages that depend on it, the > usefulness of Bigtop for me drops to zero. > > On Fri, Dec 12, 2014 at 6:24 PM, jay vyas <[email protected]> > wrote: > >> youre right - this thread is just a discussion ... nothing at all has >> been replaced, nor even proposed to be replaced. >> I think the purpose of this thread is to discuss the **entire** bigtop >> stack, from a theoretical perspective... so there are no sacred cows :)... >> >> >> On Fri, Dec 12, 2014 at 7:59 PM, Andrew Purtell <[email protected]> >> wrote: >> >>> > thanks for the input maybe yarn and HDFS should continue to stick >>> around >>> >>> Huh? When/where was HDFS not sticking around? To be replaced with what? >>> >>> >>> On Thu, Dec 11, 2014 at 6:57 PM, jay vyas <[email protected]> >>> wrote: >>>> >>>> great feedback guys. my thoughts: >>>> >>>> @andrew, yeah thats some very good points you make. thanks for the >>>> input maybe yarn and HDFS should continue to stick around. >>>> >>>> @RJ, i think python, and the mgmt tooling can be complimentary, but >>>> more ** on top ** of bigtop, i.e. in a vendor product based on bigtop : or >>>> a community repackaging of bigtop --- managing them as part of bigtop >>>> itself -- might be extra features are a little out of scope of bigtop ; >>>> which is more around deployment and packaging / testing for the core of a >>>> big data infrastructure, as opposed to an e2e solution. but *at the least* >>>> i think it makes sense to keep in mind the ambaris and also the >>>> ipythons/tableus of the world when producing bigtop releases, b/c if those >>>> represent use cases we can have good tests for (i.e. REST Apis and PySpark >>>> tests and so on)... >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> -- >>> Best regards, >>> >>> - Andy >>> >>> Problems worthy of attack prove their worth by hitting back. - Piet Hein >>> (via Tom White) >>> >> >> >> >> -- >> jay vyas >> > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > -- jay vyas
