Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-30 Thread Shane Knapp
alright, they switched over from generator power back to the grid about an hour ago... all the workers are back up and building! shane On Tue, Sep 24, 2019 at 3:23 PM Takeshi Yamamuro wrote: > > Shane, thanks for the hard work! > > Bests, > Takeshi > > On Wed, Sep 25, 2019 at 6:07 AM Jungtaek

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Takeshi Yamamuro
Shane, thanks for the hard work! Bests, Takeshi On Wed, Sep 25, 2019 at 6:07 AM Jungtaek Lim wrote: > Awesome, thanks for the quick update! > > On Wed, Sep 25, 2019 at 6:04 AM Shane Knapp wrote: > >> no worries. since we deprecated the packaging builds i put that >> worker back in to the

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Jungtaek Lim
Awesome, thanks for the quick update! On Wed, Sep 25, 2019 at 6:04 AM Shane Knapp wrote: > no worries. since we deprecated the packaging builds i put that > worker back in to the rotation... there was a slight env issue but > that's fixed and we should be g2g w/python. > > On Tue, Sep 24,

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Shane Knapp
no worries. since we deprecated the packaging builds i put that worker back in to the rotation... there was a slight env issue but that's fixed and we should be g2g w/python. On Tue, Sep 24, 2019 at 1:50 PM Jungtaek Lim wrote: > > Hi Shane, > > Thanks for the update, and take care of build

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Jungtaek Lim
Hi Shane, Thanks for the update, and take care of build system! Looks like some of build just got failed without test failures, looks like env. issue. https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111308/ + ./dev/run-tests-jenkins Python versions prior to 2.7 are not

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Shane Knapp
quick update from our colo admin: they are going to keep the colo on generator power until monday morning and not switch back and forth. this is great as we'll have solid uptime until monday morning when we go back to grid power. if this changes i will be certain to update everyone here.

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Shane Knapp
aand that was quick! everything is back up and building On Tue, Sep 24, 2019 at 9:39 AM Shane Knapp wrote: > > power switchover is happening now. more updates to come once machines > come back up. > > On Mon, Sep 23, 2019 at 3:16 PM Shane Knapp wrote: > > > > the main transformer for our

Re: [build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-24 Thread Shane Knapp
power switchover is happening now. more updates to come once machines come back up. On Mon, Sep 23, 2019 at 3:16 PM Shane Knapp wrote: > > the main transformer for our colo is experiencing major issues, and > campus be performing emergency work on it starting tomorrow morning > (tuesday sept

[build system] our colo is having power issues again. there will be a few 'events' this week

2019-09-23 Thread Shane Knapp
the main transformer for our colo is experiencing major issues, and campus be performing emergency work on it starting tomorrow morning (tuesday sept 24, 9am PDT). it's pretty dire. :( there's a lot going on, but please expect some sporadic jenkins downtime until monday. here's the abbreviated