just revisiting this thread... re presubmit strategy: i don't think this would be easy to set up... and i'm not sure what benefit it will give us.
re inadvertent errors: since we're checking out the same hash from the PR for both builds, and they'll run simultaneously, i don't think it'll be an issue. re overloading the workers: nah. the regular PRB takes ~4hr, and the k8s PRB takes ~30m and runs in parallel. i'll set this up right now and keep an eye on the queue/build results today. shane On Thu, Aug 20, 2020 at 2:28 PM Holden Karau <hol...@pigscanfly.ca> wrote: > Sounds good, thanks for the heads up. I hope you get some time to relax :) > > On Thu, Aug 20, 2020 at 2:26 PM shane knapp ☠ <skn...@berkeley.edu> wrote: > >> fyi, i won't be making this change until the 1st week of september. i'll >> be out, off the grid all next week! :) >> >> i will send an announcement out tomorrow on how to contact my team here @ >> uc berkeley if jenkins goes down. >> >> shane >> >> On Thu, Aug 20, 2020 at 4:40 AM Prashant Sharma <scrapco...@gmail.com> >> wrote: >> >>> Another option is, if we could have something like "presubmit" PR build. >>> In other words, running the entire 4 H + K8s integration on each commit >>> pushed is too much at the same time and there are chances that one thing >>> can inadvertently affect other components(as you just said). >>> >>> A presubmit(which includes K8s integration tests) build will be run, >>> once the PR receives LGTM from "Approved reviewers". This is one criteria >>> that comes to my mind, others may have better suggestions. >>> >>> On Thu, Aug 20, 2020 at 12:25 AM shane knapp ☠ <skn...@berkeley.edu> >>> wrote: >>> >>>> we'll be gated by the number of ubuntu workers w/minikube and docker, >>>> but it shouldn't be too bad as the full integration test takes ~45m, vs 4+ >>>> hrs for the regular PRB. >>>> >>>> i can enable this in about 1m of time if the consensus is for us to >>>> want this. >>>> >>>> On Wed, Aug 19, 2020 at 11:37 AM Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>> >>>>> Sounds good. In the meantime would folks committing things in core run >>>>> the K8s PRB or run it locally? A second change this morning was committed >>>>> that broke the K8s PR tests. >>>>> >>>>> On Tue, Aug 18, 2020 at 9:53 PM Prashant Sharma <scrapco...@gmail.com> >>>>> wrote: >>>>> >>>>>> +1, we should enable. >>>>>> >>>>>> On Wed, Aug 19, 2020 at 9:18 AM Holden Karau <hol...@pigscanfly.ca> >>>>>> wrote: >>>>>> >>>>>>> Hi Dev Folks, >>>>>>> >>>>>>> I was wondering how people feel about enabling the K8s PRB >>>>>>> automatically for all core changes? Sometimes I forget that a change >>>>>>> might >>>>>>> impact one of the K8s integration tests since a bunch of them look at >>>>>>> log >>>>>>> messages. Would folks be OK with turning on the K8s integration PRB for >>>>>>> all >>>>>>> core changes as well as K8s changes? >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> Holden :) >>>>>>> >>>>>>> -- >>>>>>> Twitter: https://twitter.com/holdenkarau >>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Twitter: https://twitter.com/holdenkarau >>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>> >>>> >>>> >>>> -- >>>> Shane Knapp >>>> Computer Guy / Voice of Reason >>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>> https://rise.cs.berkeley.edu >>>> >>> >> >> -- >> Shane Knapp >> Computer Guy / Voice of Reason >> UC Berkeley EECS Research / RISELab Staff Technical Lead >> https://rise.cs.berkeley.edu >> > > > -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
