> So, in this sense commercials aren't releasing Apache software, but rather its derivatives. I don't see how Bigtop would compete in this field neither culturally nor resource wise.
This. From personal experience, once you start patching Apache releases you kick off a snowball of curation that gets larger with every upstream commit on each project. The only way to stop the ball rolling is to periodically nuke it with a rebase on a Apache release, reset that patch delta on as many components as possible to near 0. In practice Bigtop can't do this because we don't have the bandwidth. But in principle it is also good in my opinion: If the Apache releases cannot stand up a stable and cohesive stack then the ecosystem has issues, and Bigtop can provide a corrective influence as patches, JIRAs, email. On Tue, Dec 23, 2014 at 12:36 AM, Konstantin Boudnik <[email protected]> wrote: > > Sorry for being a bit late to this discussion, but I will try to make a > very > sort summary of what I've read: > > 1. there's the intention to focus on the vertical value-add, > rather than just a platform [Andrew, et all] > 2. Focus more on in-memory technologies (which seemed to be our trend ever > since > we added Spark and now Ignite (incubating)). [Jay, Evans, RJ] > 3. while many data processing components aren't HDFS centric anymore, the > storage layer still seems to be important for anything related to > Hadoop. > Since, I don't think HDFS can be dropped tomorrow. > > And I don't see these three interfering with each other. To me they are > quite > complemektary. > > As for lower appeal of Bigtop stack to commercials as Evans alluded. > There's > that. And the main reason is that Bigtop releases are always based on > official > Apache release of upstream components. Wheres non of the Hadoop vendors can > say the same. Anything that Cloudera or HortonWorks put out there is > Hadoop 2.x > + N-patches, where N could be anywhere between 1 and 2000, but is never 0. > > So, in this sense commercials aren't releasing Apache software, but rather > its > derivatives. I don't see how Bigtop would compete in this field neither > culturally nor resource wise. > > Cos > > On Sat, Dec 06, 2014 at 06:23PM, jay vyas wrote: > > hi bigtop ! > > > > I thought id start a thread a few vaguely related thoughts i have around > > next couple iterations of bigtop. > > > > 1) Hive: How will bigtop to evolve to support it, now that it is much > > more than a mapreduce query wrapper? > > > > 2) I wonder wether we should confirm cassandra interoperability of spark > in > > bigtop distros, > > > > 3) Also, as per , https://issues.apache.org/jira/browse/BIGTOP-1561 --- > > What about presto ? Who is interested in supporting it - packaging it > > -testing it etc..? (con) I don't know if its really ready to be in > bigtop, > > but (pro) i think if there is someone really dedicated to testing its > > interop w/ the bigtop stack, that could be great news for us. > > > > *** Now three concrete questions lead to a more interesting question *** > > > > 4) in general, i think bigtop can move in one of 3 directions. > > > > EXPAND ? : Expanding to include new components, with just basic > interop, > > and let folks evolve their own stacks on top of bigtop on their own. > > > > CONTRACT+FOCUS ? Contracting to focus on a lean set of core > components, > > with super high quality. > > > > STAY THE COURSE ? Staying the same ~ a packaging platform for just > > hadoop's direct ecosystem. > > > > I am intrigued by the idea of A and B both have clear benefits and > > costs... would like to see the opinions of folks --- do we lean in one > > direction or another? What is the criteria for adding a new feature, > > package, stack to bigtop? > > > > ... Or maybe im just overthinking it and should be spending this time > > testing spark for 0.9 release.... > > > > Either way, looking forward to some feedback on these thoughts from the > > bigtop community ! > > > > jay vyas > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
