On Tue, Dec 23, 2014 at 08:55AM, Andrew Purtell wrote: > > So, in this sense commercials aren't releasing Apache software, but > rather its derivatives. I don't see how Bigtop would compete in this field > neither culturally nor resource wise. > > This. From personal experience, once you start patching Apache releases you > kick off a snowball of curation that gets larger with every upstream commit > on each project. The only way to stop the ball rolling is to periodically > nuke it with a rebase on a Apache release, reset that patch delta on as > many components as possible to near 0. In practice Bigtop can't do this > because we don't have the bandwidth. But in principle it is also good in my > opinion: If the Apache releases cannot stand up a stable and cohesive stack > then the ecosystem has issues, and Bigtop can provide a corrective > influence as patches, JIRAs, email.
Same here. Although, when we were doing a Hadoop distribution at WANdisco it was 100% Apache, e.g. Bigtop stack with a couple of our own components on top of it. Otherwise, you risk a divergence or a huge effort to simply keep this business of yours afloat. Cos > On Tue, Dec 23, 2014 at 12:36 AM, Konstantin Boudnik <[email protected]> wrote: > > > > Sorry for being a bit late to this discussion, but I will try to make a > > very > > sort summary of what I've read: > > > > 1. there's the intention to focus on the vertical value-add, > > rather than just a platform [Andrew, et all] > > 2. Focus more on in-memory technologies (which seemed to be our trend ever > > since > > we added Spark and now Ignite (incubating)). [Jay, Evans, RJ] > > 3. while many data processing components aren't HDFS centric anymore, the > > storage layer still seems to be important for anything related to > > Hadoop. > > Since, I don't think HDFS can be dropped tomorrow. > > > > And I don't see these three interfering with each other. To me they are > > quite > > complemektary. > > > > As for lower appeal of Bigtop stack to commercials as Evans alluded. > > There's > > that. And the main reason is that Bigtop releases are always based on > > official > > Apache release of upstream components. Wheres non of the Hadoop vendors can > > say the same. Anything that Cloudera or HortonWorks put out there is > > Hadoop 2.x > > + N-patches, where N could be anywhere between 1 and 2000, but is never 0. > > > > So, in this sense commercials aren't releasing Apache software, but rather > > its > > derivatives. I don't see how Bigtop would compete in this field neither > > culturally nor resource wise. > > > > Cos > > > > On Sat, Dec 06, 2014 at 06:23PM, jay vyas wrote: > > > hi bigtop ! > > > > > > I thought id start a thread a few vaguely related thoughts i have around > > > next couple iterations of bigtop. > > > > > > 1) Hive: How will bigtop to evolve to support it, now that it is much > > > more than a mapreduce query wrapper? > > > > > > 2) I wonder wether we should confirm cassandra interoperability of spark > > in > > > bigtop distros, > > > > > > 3) Also, as per , https://issues.apache.org/jira/browse/BIGTOP-1561 --- > > > What about presto ? Who is interested in supporting it - packaging it > > > -testing it etc..? (con) I don't know if its really ready to be in > > bigtop, > > > but (pro) i think if there is someone really dedicated to testing its > > > interop w/ the bigtop stack, that could be great news for us. > > > > > > *** Now three concrete questions lead to a more interesting question *** > > > > > > 4) in general, i think bigtop can move in one of 3 directions. > > > > > > EXPAND ? : Expanding to include new components, with just basic > > interop, > > > and let folks evolve their own stacks on top of bigtop on their own. > > > > > > CONTRACT+FOCUS ? Contracting to focus on a lean set of core > > components, > > > with super high quality. > > > > > > STAY THE COURSE ? Staying the same ~ a packaging platform for just > > > hadoop's direct ecosystem. > > > > > > I am intrigued by the idea of A and B both have clear benefits and > > > costs... would like to see the opinions of folks --- do we lean in one > > > direction or another? What is the criteria for adding a new feature, > > > package, stack to bigtop? > > > > > > ... Or maybe im just overthinking it and should be spending this time > > > testing spark for 0.9 release.... > > > > > > Either way, looking forward to some feedback on these thoughts from the > > > bigtop community ! > > > > > > jay vyas > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White)
