Good job Jay & Sid. Nice work! very informative and also thanks Jay for sharing the stuffs.
I have a couple of questions and suggestions: - Do we need our own k8s distribution? (pros vs. cons) - k8s is complex and also people love (easy)UI :-) How can we reduce the gap? - How can we test the distribution? (integrity and performance?) - Rest of the elements?, Such as logging, metrics & security. - How do we manage base container images? official images or github? As a distribution for big data and analytics, it would be nice to have such components(already... there are nice projects in ASF :-)): - Jupyter Notebook and jupyterhub - More choices on stream data processing: Flink, Pulsar - Task scheduling: Airflow - Interactive analytics and dashboard: Superset - Functions(serverless): OpenWhisk or KNative - Purpose-built data store for big data: Redis, Cassandra and Druid. - and more... WDYT? Thanks, Youngwoo On Sun, Sep 15, 2019 at 2:07 AM jay vyas <jayunit100.apa...@gmail.com> wrote: > Hi everyone ! > > The contents for this that were finally presented are here... > https://github.com/jayunit100/apachecon-2019-bigtop3x > > and the video is here ! https://youtu.be/LUCE63qPMyM > > Let us know what you think > > > On Thu, Sep 12, 2019 at 4:35 PM jay vyas <jayunit100.apa...@gmail.com> > wrote: > > > hi folks, am in the conference lobby area now. have 2 laptops out so > easy > > to find :) . . > > > > > > On Thu, Sep 12, 2019 at 2:41 PM Konstantin Boudnik <c...@apache.org> > wrote: > > > >> Well, here's an interesting piece of news: > >> > >> > >> > https://www.theregister.co.uk/2019/09/10/google_cloud_lights_spark_for_kubernetes/ > >> > >> Mind you, these nice fellas [/sarc] promised on two different occasions > >> that > >> they will be contributing back to Apache Bigtop - the foundation of the > >> Dataproc - and yet they never did. But I guess this is fine ... not the > >> first > >> company taking this kind of advantage :) > >> > >> -- > >> Cos > >> > >> > >> On Wed, Sep 04, 2019 at 05:54AM, Jay Vyas wrote: > >> > Hi folks. Couple of updateys in the next gen brainstorming we’ve > been > >> doing... > >> > > >> > Sid Mani from minio is going to copresent with me at our bigtop talk > at > >> > apachecon next week, around the internals of minio integrations etc. > >> > > >> > Spoke with cos and sounds like we definetly have a quorum for hacking > >> on this next gen implementation. > >> > > >> > Whose in on hacking on it, and who will be in Vegas on the 12th? > >> > > >> > I’ll help w the k8s stuff so don’t be intimidated if that’s your weak > >> spot. > >> > > > > > > -- > > jay vyas > > > > > -- > jay vyas >
