Evans, Thanks for starting this discussion. Hopefully, It would be valuable to integrate the real-time event streaming and query processing stack e.g., Apache Druid, Pinot, Pulsar and etc.
And 'k8s operator for Bigtop' looks promising for me! Thanks, Youngwoo On Wed, Nov 3, 2021 at 1:34 AM Evans Ye <[email protected]> wrote: > Hi folks, > > With Bigtop 3.0 been released, I think it's time to discuss what's new as > our next steps. Of course the open source ver. of unified compatible Hadoop > Distro. is still our core product going forward. But the surrounding value > added features might be something that can take us further beyond where we > were at. Now, let me post some ideas to start the brainstorming. > > 1. Deployment on K8S: Ambari or Bigtop Puppet as K8S operators. > 2. MLOps integrations: MLFlow, Submarine. > 3. Data Lake integrations: Hudi, Iceberg, Delta. > > And for some software engineering stuffs, I think we can do a clean up on > out-dated features such as: > 1. vagrant provisioner > 2. docker sandbox > 3. bigtop-ci > 4. bigtop-data-generators > 5. bigtop-bigpetstore > > Any thoughts? Would love to hear all of you. >
