- No need for our own k8s. Let’s just test with nfs volume provisioned and note what version of nfs, networking, helm, and cluster were using to test releases .
- I think since we are using helm charts , we can add anything and curate and test. Cost of packaging with helm and docker is very low. - If you go to the repo there is a 20 line snippet we can start with on the readme that sets up Kafka spark NiFi zk minio and presto. > On Sep 15, 2019, at 10:10 PM, Youngwoo Kim (김영우) <yw...@apache.org> wrote: > > Good job Jay & Sid. Nice work! very informative and also thanks Jay > for sharing the stuffs. > > I have a couple of questions and suggestions: > - Do we need our own k8s distribution? (pros vs. cons) > - k8s is complex and also people love (easy)UI :-) How can we reduce the > gap? > - How can we test the distribution? (integrity and performance?) > - Rest of the elements?, Such as logging, metrics & security. > - How do we manage base container images? official images or github? > > As a distribution for big data and analytics, it would be nice to have such > components(already... there are nice projects in ASF :-)): > - Jupyter Notebook and jupyterhub > - More choices on stream data processing: Flink, Pulsar > - Task scheduling: Airflow > - Interactive analytics and dashboard: Superset > - Functions(serverless): OpenWhisk or KNative > - Purpose-built data store for big data: Redis, Cassandra and Druid. > - and more... > > WDYT? > > Thanks, > Youngwoo > > On Sun, Sep 15, 2019 at 2:07 AM jay vyas <jayunit100.apa...@gmail.com> > wrote: > >> Hi everyone ! >> >> The contents for this that were finally presented are here... >> https://github.com/jayunit100/apachecon-2019-bigtop3x >> >> and the video is here ! https://youtu.be/LUCE63qPMyM >> >> Let us know what you think >> >> >> On Thu, Sep 12, 2019 at 4:35 PM jay vyas <jayunit100.apa...@gmail.com> >> wrote: >> >>> hi folks, am in the conference lobby area now. have 2 laptops out so >> easy >>> to find :) . . >>> >>> >>> On Thu, Sep 12, 2019 at 2:41 PM Konstantin Boudnik <c...@apache.org> >> wrote: >>> >>>> Well, here's an interesting piece of news: >>>> >>>> >>>> >> https://www.theregister.co.uk/2019/09/10/google_cloud_lights_spark_for_kubernetes/ >>>> >>>> Mind you, these nice fellas [/sarc] promised on two different occasions >>>> that >>>> they will be contributing back to Apache Bigtop - the foundation of the >>>> Dataproc - and yet they never did. But I guess this is fine ... not the >>>> first >>>> company taking this kind of advantage :) >>>> >>>> -- >>>> Cos >>>> >>>> >>>>> On Wed, Sep 04, 2019 at 05:54AM, Jay Vyas wrote: >>>>> Hi folks. Couple of updateys in the next gen brainstorming we’ve >> been >>>> doing... >>>>> >>>>> Sid Mani from minio is going to copresent with me at our bigtop talk >> at >>>>> apachecon next week, around the internals of minio integrations etc. >>>>> >>>>> Spoke with cos and sounds like we definetly have a quorum for hacking >>>> on this next gen implementation. >>>>> >>>>> Whose in on hacking on it, and who will be in Vegas on the 12th? >>>>> >>>>> I’ll help w the k8s stuff so don’t be intimidated if that’s your weak >>>> spot. >>>> >>> >>> >>> -- >>> jay vyas >>> >> >> >> -- >> jay vyas >>
