ok, i'm gonna have to reboot all the workers tomorrow and wipe the m2 caches. it looks like zombie builds were lingering post-jenkins-wedging and corrupting the repos.
fixed on -05. On Mon, Jul 6, 2020 at 2:17 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > Just encountered the same and it's worker-05 again. (You can find [error] > in the console to see what's the problem. I guess jetty artifacts in the > worker might be messed up.) > > > https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125127/consoleFull > > > On Tue, Jul 7, 2020 at 5:35 AM Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Could this be a flaky or persistent issue? It failed with Scala gendoc >> but it didn't fail with the part the PR modified. It ran from worker-05. >> >> >> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125121/consoleFull >> >> On Tue, Jul 7, 2020 at 2:10 AM shane knapp ☠ <skn...@berkeley.edu> wrote: >> >>> i killed and retriggered the PRB jobs on 04, and wiped that workers' m2 >>> cache. >>> >>> On Mon, Jul 6, 2020 at 9:24 AM shane knapp ☠ <skn...@berkeley.edu> >>> wrote: >>> >>>> once the jobs running on that worker are finished, yes. >>>> >>>> On Sun, Jul 5, 2020 at 7:41 PM Hyukjin Kwon <gurwls...@gmail.com> >>>> wrote: >>>> >>>>> Shane, can we remove .m2 in worker machine 4? >>>>> >>>>> 2020년 7월 3일 (금) 오전 8:18, Jungtaek Lim <kabhwan.opensou...@gmail.com>님이 >>>>> 작성: >>>>> >>>>>> Looks like Jenkins service itself becomes unstable. It took >>>>>> considerable time to just open the test report for a specific build, and >>>>>> Jenkins doesn't pick the request on rebuild (retest this, please) in >>>>>> Github >>>>>> comment. >>>>>> >>>>>> On Thu, Jul 2, 2020 at 2:12 PM Hyukjin Kwon <gurwls...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Ah, okay. Actually there already is - >>>>>>> https://issues.apache.org/jira/browse/SPARK-31693. I am reopening. >>>>>>> >>>>>>> 2020년 7월 2일 (목) 오후 2:06, Holden Karau <hol...@pigscanfly.ca>님이 작성: >>>>>>> >>>>>>>> We don't I didn't file one originally, but Shane reminded me to in >>>>>>>> the future. >>>>>>>> >>>>>>>> On Wed, Jul 1, 2020 at 9:44 PM Hyukjin Kwon <gurwls...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Nope, do we have an existing ticket? I think we can reopen if >>>>>>>>> there is. >>>>>>>>> >>>>>>>>> 2020년 7월 2일 (목) 오후 1:43, Holden Karau <hol...@pigscanfly.ca>님이 작성: >>>>>>>>> >>>>>>>>>> Huh interesting that it’s the same worker. Have you filed a >>>>>>>>>> ticket to Shane? >>>>>>>>>> >>>>>>>>>> On Wed, Jul 1, 2020 at 8:50 PM Hyukjin Kwon <gurwls...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hm .. seems this is happening again in amp-jenkins-worker-04 ;(. >>>>>>>>>>> >>>>>>>>>>> 2020년 6월 25일 (목) 오전 3:15, shane knapp ☠ <skn...@berkeley.edu>님이 >>>>>>>>>>> 작성: >>>>>>>>>>> >>>>>>>>>>>> done: >>>>>>>>>>>> -bash-4.1$ cd .m2 >>>>>>>>>>>> -bash-4.1$ ls >>>>>>>>>>>> repository >>>>>>>>>>>> -bash-4.1$ time rm -rf * >>>>>>>>>>>> >>>>>>>>>>>> real 17m4.607s >>>>>>>>>>>> user 0m0.950s >>>>>>>>>>>> sys 0m18.816s >>>>>>>>>>>> -bash-4.1$ >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jun 24, 2020 at 10:50 AM shane knapp ☠ < >>>>>>>>>>>> skn...@berkeley.edu> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> ok, i've taken that worker offline and once the job running on >>>>>>>>>>>>> it finishes, i'll wipe the cache. >>>>>>>>>>>>> >>>>>>>>>>>>> in the future, please file a JIRA and assign it to me so i >>>>>>>>>>>>> don't have to track my work through emails to the dev@ list. >>>>>>>>>>>>> ;) >>>>>>>>>>>>> >>>>>>>>>>>>> thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> shane >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Jun 24, 2020 at 10:48 AM Holden Karau < >>>>>>>>>>>>> hol...@pigscanfly.ca> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> The most recent one I noticed was >>>>>>>>>>>>>> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124437/console >>>>>>>>>>>>>> which >>>>>>>>>>>>>> was run on amp-jenkins-worker-04. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Jun 24, 2020 at 10:44 AM shane knapp ☠ < >>>>>>>>>>>>>> skn...@berkeley.edu> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> for those weird failures, it's super helpful to provide >>>>>>>>>>>>>>> which workers are showing these issues. :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> i'd rather not wipe all of the m2 caches on all of the >>>>>>>>>>>>>>> workers, as we'll then potentially get blacklisted again if we >>>>>>>>>>>>>>> download too >>>>>>>>>>>>>>> many packages from apache.org. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Jun 23, 2020 at 5:58 PM Holden Karau < >>>>>>>>>>>>>>> hol...@pigscanfly.ca> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Folks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I've been see some weird failures on Jenkins and it looks >>>>>>>>>>>>>>>> like it might be from the m2 cache. Would it be OK to clean it >>>>>>>>>>>>>>>> out? Or is >>>>>>>>>>>>>>>> it important? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Holden >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Twitter: https://twitter.com/holdenkarau >>>>>>>>>>>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>>>>>>>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>>>>>>>>>>>>> YouTube Live Streams: >>>>>>>>>>>>>>>> https://www.youtube.com/user/holdenkarau >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Shane Knapp >>>>>>>>>>>>>>> Computer Guy / Voice of Reason >>>>>>>>>>>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>>>>>>>>>>>>> https://rise.cs.berkeley.edu >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Twitter: https://twitter.com/holdenkarau >>>>>>>>>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>>>>>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>>>>>>>>>>> YouTube Live Streams: >>>>>>>>>>>>>> https://www.youtube.com/user/holdenkarau >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Shane Knapp >>>>>>>>>>>>> Computer Guy / Voice of Reason >>>>>>>>>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>>>>>>>>>>> https://rise.cs.berkeley.edu >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Shane Knapp >>>>>>>>>>>> Computer Guy / Voice of Reason >>>>>>>>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>>>>>>>>>> https://rise.cs.berkeley.edu >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> Twitter: https://twitter.com/holdenkarau >>>>>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>>>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Twitter: https://twitter.com/holdenkarau >>>>>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Shane Knapp >>>> Computer Guy / Voice of Reason >>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>> https://rise.cs.berkeley.edu >>>> >>> >>> >>> -- >>> Shane Knapp >>> Computer Guy / Voice of Reason >>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>> https://rise.cs.berkeley.edu >>> >> -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
