Hi, Let's make a step forward... I created an umbrella Jira to manage the development issues https://issues.apache.org/jira/browse/BIGTOP-3225 and also created the branch 'cnb' https://gitbox.apache.org/repos/asf?p=bigtop.git;a=shortlog;h=refs/heads/cnb
@Jay, Please push your work into 'cnb' branch and then, let's find out what we can improve. Thanks, Youngwoo On Fri, Sep 20, 2019 at 8:39 PM Jay Vyas <jayunit100.apa...@gmail.com> wrote: > Btw ... kind or minikube is even easier than kubespray for small local > testing . > > > On Sep 20, 2019, at 12:57 AM, Evans Ye <evan...@apache.org> wrote: > > > > +1 > > > > Youngwoo Kim (김영우) <yw...@apache.org> 於 2019年9月20日 週五 上午11:12寫道： > > > >> I did work through the snippet@Jay's repo on my end in using > Kubespray[1] > >> and vagrant VM. There are minor nits but overall it works well as > >> expected... > >> In my opinion, it's breaking changes for us because the tooling that > we've > >> used is not integrated(or replaced) yet. so I think it would be good to > >> start with a development branch based on Jay's work... and also new > branch > >> is harmless on current development. > >> > >> I'm proposing a new branch for **incubating** cloud-native distribution > >> which is based on Jay's ground work! > >> > >> What do you think? > >> > >> Thanks, > >> Youngwoo > >> > >> 1. https://github.com/kubernetes-sigs/kubespray > >> > >> On Mon, Sep 16, 2019 at 8:02 PM Jay Vyas <jayunit100.apa...@gmail.com> > >> wrote: > >> > >>> - No need for our own k8s. Let’s just test with nfs volume provisioned > >>> and note what version of nfs, networking, helm, and cluster were using > >> to > >>> test releases . > >>> > >>> - I think since we are using helm charts , we can add anything and > curate > >>> and test. Cost of packaging with helm and docker is very low. > >>> > >>> - If you go to the repo there is a 20 line snippet we can start with on > >>> the readme that sets up Kafka spark NiFi zk minio and presto. > >>> > >>>> On Sep 15, 2019, at 10:10 PM, Youngwoo Kim (김영우) <yw...@apache.org> > >>> wrote: > >>>> > >>>> Good job Jay & Sid. Nice work! very informative and also thanks Jay > >>>> for sharing the stuffs. > >>>> > >>>> I have a couple of questions and suggestions: > >>>> - Do we need our own k8s distribution? (pros vs. cons) > >>>> - k8s is complex and also people love (easy)UI :-) How can we reduce > >> the > >>>> gap? > >>>> - How can we test the distribution? (integrity and performance?) > >>>> - Rest of the elements?, Such as logging, metrics & security. > >>>> - How do we manage base container images? official images or github? > >>>> > >>>> As a distribution for big data and analytics, it would be nice to have > >>> such > >>>> components(already... there are nice projects in ASF :-)): > >>>> - Jupyter Notebook and jupyterhub > >>>> - More choices on stream data processing: Flink, Pulsar > >>>> - Task scheduling: Airflow > >>>> - Interactive analytics and dashboard: Superset > >>>> - Functions(serverless): OpenWhisk or KNative > >>>> - Purpose-built data store for big data: Redis, Cassandra and Druid. > >>>> - and more... > >>>> > >>>> WDYT? > >>>> > >>>> Thanks, > >>>> Youngwoo > >>>> > >>>> On Sun, Sep 15, 2019 at 2:07 AM jay vyas <jayunit100.apa...@gmail.com > > > >>>> wrote: > >>>> > >>>>> Hi everyone ! > >>>>> > >>>>> The contents for this that were finally presented are here... > >>>>> https://github.com/jayunit100/apachecon-2019-bigtop3x > >>>>> > >>>>> and the video is here ! https://youtu.be/LUCE63qPMyM > >>>>> > >>>>> Let us know what you think > >>>>> > >>>>> > >>>>> On Thu, Sep 12, 2019 at 4:35 PM jay vyas < > jayunit100.apa...@gmail.com > >>> > >>>>> wrote: > >>>>> > >>>>>> hi folks, am in the conference lobby area now. have 2 laptops out > so > >>>>> easy > >>>>>> to find :) . . > >>>>>> > >>>>>> > >>>>>> On Thu, Sep 12, 2019 at 2:41 PM Konstantin Boudnik <c...@apache.org> > >>>>> wrote: > >>>>>> > >>>>>>> Well, here's an interesting piece of news: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>> > >> > https://www.theregister.co.uk/2019/09/10/google_cloud_lights_spark_for_kubernetes/ > >>>>>>> > >>>>>>> Mind you, these nice fellas [/sarc] promised on two different > >>> occasions > >>>>>>> that > >>>>>>> they will be contributing back to Apache Bigtop - the foundation of > >>> the > >>>>>>> Dataproc - and yet they never did. But I guess this is fine ... > not > >>> the > >>>>>>> first > >>>>>>> company taking this kind of advantage :) > >>>>>>> > >>>>>>> -- > >>>>>>> Cos > >>>>>>> > >>>>>>> > >>>>>>>> On Wed, Sep 04, 2019 at 05:54AM, Jay Vyas wrote: > >>>>>>>> Hi folks. Couple of updateys in the next gen brainstorming we’ve > >>>>> been > >>>>>>> doing... > >>>>>>>> > >>>>>>>> Sid Mani from minio is going to copresent with me at our bigtop > >> talk > >>>>> at > >>>>>>>> apachecon next week, around the internals of minio integrations > >> etc. > >>>>>>>> > >>>>>>>> Spoke with cos and sounds like we definetly have a quorum for > >> hacking > >>>>>>> on this next gen implementation. > >>>>>>>> > >>>>>>>> Whose in on hacking on it, and who will be in Vegas on the 12th? > >>>>>>>> > >>>>>>>> I’ll help w the k8s stuff so don’t be intimidated if that’s your > >> weak > >>>>>>> spot. > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> jay vyas > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> jay vyas > >>>>> > >>> > >> >
