Hi Matt Andruff, Agree. Hi Youngwoo, Could you elaborate what component is needed for the real-time event streaming? Kafka + Flink in current stack are the solution for it. Pulsar can be an addition. Regarding query processing, Could you share more insight on the difference of Pinot V.S. Presto? Does Pinot suitable for having Looker plugged in front of it for analytical purposes?
Hi Kengo/Masatake, Do you have any feature that is needed for your company to move business forward? BTW, we have some discussion in the past for this topic you can take as a reference[1]. [1] https://docs.google.com/document/d/1F2Gxu8GARQDZXgqHn12LKkQ5wCV_AF4b_tVmjYB6YfA/edit# Youngwoo Kim (김영우) <[email protected]> 於 2021年11月3日 週三 下午4:02寫道： > Evans, > Thanks for starting this discussion. > > Hopefully, It would be valuable to integrate the real-time event streaming > and query processing stack e.g., Apache Druid, Pinot, Pulsar and etc. > > And 'k8s operator for Bigtop' looks promising for me! > > Thanks, > Youngwoo > > On Wed, Nov 3, 2021 at 1:34 AM Evans Ye <[email protected]> wrote: > > > Hi folks, > > > > With Bigtop 3.0 been released, I think it's time to discuss what's new as > > our next steps. Of course the open source ver. of unified compatible > Hadoop > > Distro. is still our core product going forward. But the surrounding > value > > added features might be something that can take us further beyond where > we > > were at. Now, let me post some ideas to start the brainstorming. > > > > 1. Deployment on K8S: Ambari or Bigtop Puppet as K8S operators. > > 2. MLOps integrations: MLFlow, Submarine. > > 3. Data Lake integrations: Hudi, Iceberg, Delta. > > > > And for some software engineering stuffs, I think we can do a clean up on > > out-dated features such as: > > 1. vagrant provisioner > > 2. docker sandbox > > 3. bigtop-ci > > 4. bigtop-data-generators > > 5. bigtop-bigpetstore > > > > Any thoughts? Would love to hear all of you. > > >
