I wanted to single out the issue of overlapping components that Cos raised for discussion without derailing the Crail conversation, so I'm starting a new thread.
Cos said: > - from a quick glance at the project, looks like it fits the general bucket > of data fabric platforms. Something like Apache Ignite comes to mind, while > Aluxio, as a simple caching solution, only has partial target > functionality. While there's no limitation on having multiple components > with overlapping functionality in a given Bigtop stack (after all, we have > HDFS and QFS), it's an aspect worthy of some consideration. I don't think having overlapping components is bad or should be avoided. Apache has tons of TLPs that overlap: Drill vs Hive vs Phoenix, Spark vs Flink vs Apex vs Storm vs Samza, Bookkeeper vs Zookeeper. Rather, the "Apache way" seems to be to provide even ground for all projects and let "the market" decide whether projects succeed or fail. >From that standpoint, I think BigTop should not be opinionated about which components are included as long as someone is willing to maintain them.
