Re: [discuss] ending support for Java 6?

that's kinda what we're doing right now, java 7 is the default/standard on our jenkins. or, i vote we buy a butler's outfit for thomas and have a second jenkins instance... ;) On Sat, May 2, 2015 at 1:09 PM, Mridul Muralidharan mri...@gmail.com wrote: We could build on minimum jdk we support

Re: [discuss] ending support for Java 6?

. -source and -target is insufficient to ensure api usage is conformant with the minimum jdk version we are supporting. Regards, Mridul [1] Not jdk7 as you mentioned On Sat, May 2, 2015 at 8:53 PM, shane knapp skn...@berkeley.edu wrote: that's kinda what we're doing right now, java

[build infra] quick downtime again tomorrow morning for DOCKER

yes, docker. that wonderful little wrapper for linux containers will be installed and ready for play on all of the jenkins workers tomorrow morning. the downtime will be super quick: i just need to kill the jenkins slaves' ssh connections and relaunch to add the jenkins user to the docker

Re: [build system] quick jenkins restart thursday morning (5-6-15) 7am PDT

things are currently rebooting. On Thu, May 7, 2015 at 7:18 AM, shane knapp skn...@berkeley.edu wrote: this is happening now. On Wed, May 6, 2015 at 5:44 PM, shane knapp skn...@berkeley.edu wrote: we've had a spate of issues since the power outage, and now the github pull request builder

Re: [build system] quick jenkins restart thursday morning (5-6-15) 7am PDT

and we're back up and building. thanks for your patience! On Thu, May 7, 2015 at 7:48 AM, shane knapp skn...@berkeley.edu wrote: things are currently rebooting. On Thu, May 7, 2015 at 7:18 AM, shane knapp skn...@berkeley.edu wrote: this is happening now. On Wed, May 6, 2015 at 5:44 PM

Re: Pull request builder errors (taking Jenkins worker 3 offline)

On Wed, May 6, 2015 at 10:12 AM, shane knapp skn...@berkeley.edu wrote: ok, i looked deeper and this is only happening on -03, and not linked specifically to the pull request builder: 3 NewSparkPullRequestBuilder 13 Spark-Master-SBT 4 Spark-1.4-SBT 49

Re: Pull request builder errors (taking Jenkins worker 3 offline)

and reinstall the node. sorry for the inconvenience! On Tue, May 5, 2015 at 3:08 PM, shane knapp skn...@berkeley.edu wrote: alright, this is happening again w/this worker and i will be taking it offline for further investigation. i'm OOO for the rest of the day, but will check in again later

[build system] quick jenkins restart thursday morning (5-6-15) 7am PDT

we've had a spate of issues since the power outage, and now the github pull request builder is randomly deciding who can and can't trigger builds[1]. i think it's time for a quick restart of the master and workers, which i'll do early tomorrow morning. the outage should be very brief, and i'll

Re: Pull request builder errors (taking Jenkins worker 3 offline)

alright, this is happening again w/this worker and i will be taking it offline for further investigation. i'm OOO for the rest of the day, but will check in again later this evening. On Tue, May 5, 2015 at 9:33 AM, shane knapp skn...@berkeley.edu wrote: ok, i reset the maven cache on amp

Re: [build infra] quick downtime again tomorrow morning for DOCKER

: will docker allow new capabilities for the Spark build? (Where can I read more?) Punya On Fri, May 8, 2015 at 10:00 AM shane knapp skn...@berkeley.edu wrote: this is happening now. On Thu, May 7, 2015 at 3:40 PM, shane knapp skn...@berkeley.edu wrote: yes, docker. that wonderful little wrapper

Re: [build infra] quick downtime again tomorrow morning for DOCKER

this is happening now. On Thu, May 7, 2015 at 3:40 PM, shane knapp skn...@berkeley.edu wrote: yes, docker. that wonderful little wrapper for linux containers will be installed and ready for play on all of the jenkins workers tomorrow morning. the downtime will be super quick: i just need

Re: [build infra] quick downtime again tomorrow morning for DOCKER

...and this is done. thanks for your patience! On Fri, May 8, 2015 at 7:00 AM, shane knapp skn...@berkeley.edu wrote: this is happening now. On Thu, May 7, 2015 at 3:40 PM, shane knapp skn...@berkeley.edu wrote: yes, docker. that wonderful little wrapper for linux containers

Re: Pull request builder errors (taking Jenkins worker 3 offline)

hmm, still happening. looking deeper. On Tue, May 5, 2015 at 8:54 AM, shane knapp skn...@berkeley.edu wrote: taking a look now. On Tue, May 5, 2015 at 3:23 AM, Patrick Wendell pwend...@gmail.com wrote: For unknown reasons, pull requests on Jenkins worker 3 have been failing

Re: Pull request builder errors (taking Jenkins worker 3 offline)

ok, i reset the maven cache on amp-jenkins-worker-03 and some stuff is currently building and not failing... i'll keep a close eye on this for now. On Tue, May 5, 2015 at 9:15 AM, shane knapp skn...@berkeley.edu wrote: hmm, still happening. looking deeper. On Tue, May 5, 2015 at 8:54 AM

Re: Pull request builder errors (taking Jenkins worker 3 offline)

taking a look now. On Tue, May 5, 2015 at 3:23 AM, Patrick Wendell pwend...@gmail.com wrote: For unknown reasons, pull requests on Jenkins worker 3 have been failing with an exception[1]. After trying to fix this by clearing the ivy and maven caches on the node, I've given up and simply

[build system] QA infrastructure wiki updated w/latest package installs/versions

so i spent a good part of the morning parsing out all of the packages and versions of things that we have installed on our jenkins workers: https://cwiki.apache.org/confluence/display/SPARK/Spark+QA+Infrastructure if you're looking to set up something to mimic our build system, this should be a

Re: [discuss] ending support for Java 6?

, Patrick Wendell pwend...@gmail.com wrote: If there is broad consensus here to drop Java 1.6 in Spark 1.5, should we do an ANNOUNCE to user and dev? On Mon, May 4, 2015 at 7:24 PM, shane knapp skn...@berkeley.edu wrote: sgtm On Mon, May 4, 2015 at 11:23 AM, Patrick Wendell pwend...@gmail.com

Re: [build system] brief downtime tomorrow morning (5-12-15, 7am PDT)

this is already done On Tue, May 12, 2015 at 1:14 PM, shane knapp skn...@berkeley.edu wrote: i will need to restart jenkins to finish a plugin install and resolve https://issues.apache.org/jira/browse/SPARK-7561 this will be very brief, and i'll retrigger any errant jobs i kill. please let

[build system] brief downtime tomorrow morning (5-12-15, 7am PDT)

i will need to restart jenkins to finish a plugin install and resolve https://issues.apache.org/jira/browse/SPARK-7561 this will be very brief, and i'll retrigger any errant jobs i kill. please let me know if there are any comments/questions/concerns. thanks! shane

[build system] scheduled datacenter downtime, sunday may 17th

our datacenter is rejiggering our network (read: fully re-engineering large portions from the ground up) and has downtime scheduled from 9am-3pm PDT, this sunday may17th. this means our jenkins instance will not be available to the outside world, and i will be putting jenkins in to quiet mode the

Re: [build system] scheduled datacenter downtime, sunday may 17th

our sysadmins fixed the auth issue about an hour ago... /etc/shadow's perms got borked somehow and that was breaking logins for local (non-ldap) accounts. we're all green. On Sun, May 17, 2015 at 2:46 PM, shane knapp skn...@berkeley.edu wrote: ...and we're back up. it looks like things

Re: [build system] scheduled datacenter downtime, sunday may 17th

jenkins is being a little recalcitrant and i'm looking at logs to see why it won't start.

Re: [build system] scheduled datacenter downtime, sunday may 17th

ok, i think it's time to reboot the jenkins master. On Sun, May 17, 2015 at 1:44 PM, shane knapp skn...@berkeley.edu wrote: jenkins is being a little recalcitrant and i'm looking at logs to see why it won't start.

Re: [build system] scheduled datacenter downtime, sunday may 17th

machine rebooted, but auth is completely broken (web and CLI on the server). i'm trying to fix this now. On Sun, May 17, 2015 at 1:51 PM, shane knapp skn...@berkeley.edu wrote: ok, i think it's time to reboot the jenkins master. On Sun, May 17, 2015 at 1:44 PM, shane knapp skn

Re: [build system] scheduled datacenter downtime, sunday may 17th

auth is fixed, and jenkins is out of quiet mode and now building. sorry for the delay! On Sun, May 17, 2015 at 2:06 PM, shane knapp skn...@berkeley.edu wrote: machine rebooted, but auth is completely broken (web and CLI on the server). i'm trying to fix this now. On Sun, May 17, 2015 at 1

Re: [build system] scheduled datacenter downtime, sunday may 17th

...and we've lost network connectivity again. things are still very flaky. more updates as they come. On Sun, May 17, 2015 at 2:32 PM, shane knapp skn...@berkeley.edu wrote: actually, LDAP auth is fixed, but if you have a local account that i've created for you, it's not letting you log

Re: [build system] scheduled datacenter downtime, sunday may 17th

actually, LDAP auth is fixed, but if you have a local account that i've created for you, it's not letting you log in to jenkins' UI. looking at this now. On Sun, May 17, 2015 at 2:13 PM, shane knapp skn...@berkeley.edu wrote: auth is fixed, and jenkins is out of quiet mode and now building

Re: extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python more)

knapp skn...@berkeley.edu wrote: ok, we're looking good. i'll keep an eye on this for the rest of the day, and if you happen to notice any infrastructure failures before i do (i updated a LOT), please let me know immediately! :) On Thu, Apr 9, 2015 at 8:38 AM, shane knapp skn

Re: extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python more)

reminder! this is happening thurday morning. On Fri, Apr 3, 2015 at 9:59 AM, shane knapp skn...@berkeley.edu wrote: welcome to python2.7+, java 8 and more! :) i'll be doing a major upgrade to our build system next thursday morning. here's a quick list of what's going on: * installation

Re: Unit test logs in Jenkins?

i agree with all of this. but can we please break up the tests and make them shorter? :) On Thu, Apr 2, 2015 at 8:54 AM, Nicholas Chammas nicholas.cham...@gmail.com wrote: This is secondary to Marcelo’s question, but I wanted to comment on this: Its main limitation is more cultural than

Re: extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python more)

ok, we're looking good. i'll keep an eye on this for the rest of the day, and if you happen to notice any infrastructure failures before i do (i updated a LOT), please let me know immediately! :) On Thu, Apr 9, 2015 at 8:38 AM, shane knapp skn...@berkeley.edu wrote: things are looking pretty

Re: extended jenkins downtime, thursday april 9th 7am-noon PDT (moving to anaconda python more)

and this is now happening. On Tue, Apr 7, 2015 at 4:38 PM, shane knapp skn...@berkeley.edu wrote: reminder! this is happening thurday morning. On Fri, Apr 3, 2015 at 9:59 AM, shane knapp skn...@berkeley.edu wrote: welcome to python2.7+, java 8 and more! :) i'll be doing a major upgrade

Re: Ivy support in Spark vs. sbt

interesting... i definitely haven't seen it happen that often in our build system, and when it has happened, i wasn't able to determine the cause. On Thu, Jun 4, 2015 at 10:16 AM, Marcelo Vanzin van...@cloudera.com wrote: On Thu, Jun 4, 2015 at 10:04 AM, shane knapp skn...@berkeley.edu wrote

Re: Ivy support in Spark vs. sbt

this has occasionally happened on our jenkins as well (twice since last august), and deleting the cache fixes it right up. On Thu, Jun 4, 2015 at 4:29 AM, Sean Owen so...@cloudera.com wrote: I've definitely seen the dependency path must be relative problem, and fixed it by deleting the ivy

[build system] jenkins downtime tomorrow morning ~730am PDT

i'm going to be performing system, jenkins, and plugin updates tomorrow morning beginning at 730am PDT. 0700: pause build queue 0800: kill off any errant jobs (retrigger when everything comes back up) 0800-0900: system and plugin updates 0900-1000: final debugging, roll back versions of

Re: [build system] jenkins downtime tomorrow morning ~730am PDT

well, i started early and am pretty much done. sadly, i had to roll back most of the plugin updates (which doesn't surprise me), but the system and jenkins core updates went swimmingly. anyways, we're up and building again! now, back to my coffee... :) On Wed, May 27, 2015 at 2:11 PM, shane

Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

+1, and i know i've been guilty of this in the past. :) On Wed, Jun 10, 2015 at 10:20 PM, Joseph Bradley jos...@databricks.com wrote: +1 On Sat, Jun 6, 2015 at 9:01 AM, Patrick Wendell pwend...@gmail.com wrote: Hey All, Just a request here - it would be great if people could create

Re: Jenkins having issues?

hey all... so this has been happening intermittently and i'm not sure what's causing it. sometimes directories under the target/tmp/ dir get created w/o the owner write bit set, so that they look like this: dr-xr-xr-x. 2 jenkins jenkins 4096 Aug 9 01:28

Re: update on git timeouts for jenkins builds

newp. still happening, and i'm still looking in to it: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38880/console On Wed, Jul 29, 2015 at 12:20 PM, shane knapp skn...@berkeley.edu wrote: ok, i think i found the problem and solution to the git timeouts: https

Re: update on git timeouts for jenkins builds

a difference. On Tue, Jul 28, 2015 at 11:51 AM, shane knapp skn...@berkeley.edu wrote: hey all, i'm just back in from my wedding weekend (woot!) and am working on figuring out what's happening w/the git timeouts for pull request builds. TL;DR: if your build fails due to a timeout, please

update on git timeouts for jenkins builds

hey all, i'm just back in from my wedding weekend (woot!) and am working on figuring out what's happening w/the git timeouts for pull request builds. TL;DR: if your build fails due to a timeout, please retrigger your builds. i know this isn't the BEST solution, but until we get some stuff

Re: update on git timeouts for jenkins builds

btw, the directory perm issue was only happening on amp-jenkins-worker-04 and -05. both of the broken dirs were clobbered, so we won't be seeing any more of these again. On Tue, Jul 28, 2015 at 12:28 PM, shane knapp skn...@berkeley.edu wrote: ++joshrosen ok, i found out some of what's going

Re: update on git timeouts for jenkins builds

^dr-x; echo; echo; done as for what exactly is messing up the perms, i'm not entirely sure. josh, you have any ideas? shane On Tue, Jul 28, 2015 at 11:51 AM, shane knapp skn...@berkeley.edu wrote: hey all, i'm just back in from my wedding weekend (woot!) and am working on figuring out what's

Re: update on git timeouts for jenkins builds

, shane knapp skn...@berkeley.edu wrote: btw, the directory perm issue was only happening on amp-jenkins-worker-04 and -05. both of the broken dirs were clobbered, so we won't be seeing any more of these again. On Tue, Jul 28, 2015 at 12:28 PM, shane knapp skn...@berkeley.edu wrote

shane will be OOO 8-5-15 through 8-18-15

so i done gone and got myself hitched, and will be disappearing in to the rainy island of kol chang in thailand for the next ~2 weeks. :) this means i will be completely out of contact, and have to leave jenkins in the gentle hands of jon kuroda (a sysadmin here at the lab) and matt massie (my

Re: jenkins downtime 7/13/15, 7am PDT

this is happening now. On Sun, Jul 12, 2015 at 8:49 PM, shane knapp skn...@berkeley.edu wrote: reminder: this is happening tomorrow morning! On Thu, Jul 9, 2015 at 1:07 PM, shane knapp skn...@berkeley.edu wrote: i'll be taking jenkins down for system and jenkins app updates. this should

Re: jenkins downtime 7/13/15, 7am PDT

. On Mon, Jul 13, 2015 at 7:09 AM, shane knapp skn...@berkeley.edu wrote: this is happening now. On Sun, Jul 12, 2015 at 8:49 PM, shane knapp skn...@berkeley.edu wrote: reminder: this is happening tomorrow morning! On Thu, Jul 9, 2015 at 1:07 PM, shane knapp skn...@berkeley.edu wrote: i'll

Re: jenkins downtime 7/13/15, 7am PDT

reminder: this is happening tomorrow morning! On Thu, Jul 9, 2015 at 1:07 PM, shane knapp skn...@berkeley.edu wrote: i'll be taking jenkins down for system and jenkins app updates. this should be pretty quick and i'm expecting to have everything back up and building by 9am. i will send

[build system] emergency restart to temporarily patch a massive java security hole

hey everyone! i'm about to shut down jenkins to deploy a temporary fix for a massive security hole i found out about late friday: http://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/ read the whole thing.

Re: [build system] emergency restart to temporarily patch a massive java security hole

ok, we're good to go. https://amplab.cs.berkeley.edu/jenkins/cli/ returns a 404, as it should. thanks for your patience... shane On Sun, Nov 8, 2015 at 2:53 PM, shane knapp <skn...@berkeley.edu> wrote: > hey everyone! > > i'm about to shut down jenkins to deploy a temporary fi

[build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

i will admit that it does seem like a bad idea to poke jenkins on friday the 13th, but there's a release that fixes a lot of security issues: https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2015-11-11 i'll set jenkins to stop kicking off any new builds around 5am PST, and

Re: [build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

this is still ongoing. the update is running 'chown -R jenkins' on the jenkins root directory, which is a hair under 3T. this might take a while... :\ shane On Fri, Nov 13, 2015 at 6:36 AM, shane knapp <skn...@berkeley.edu> wrote: > this is happening now. > > On Thu, Nov 12, 2

Re: Seems jenkins is down (or very slow)?

were you hitting any particular URL when you noticed this, or was it generally slow? On Thu, Nov 12, 2015 at 6:21 PM, Yin Huai wrote: > Hi Guys, > > Seems Jenkins is down or very slow? Does anyone else experience it or just > me? > > Thanks, > > Yin

Re: [build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

phew. this is finally done... jenkins is up and building. On Fri, Nov 13, 2015 at 7:16 AM, shane knapp <skn...@berkeley.edu> wrote: > this is still ongoing. the update is running 'chown -R jenkins' on > the jenkins root directory, which is a hair under 3T. > > this

[build system] shane OOO until monday, nov 16

i'll be at the USENIX LISA conference in DC, so josh and jon will be keeping an eye on jenkins and making sure it doesn't misbehave. since attending every session of every day will drive one insane, i will be sporadically checking in and making sure things are humming along... but for

[BUILD SYSTEM] quick jenkins downtime, november 5th 7am

i'd like to take jenkins down briefly thursday morning to install some plugin updates. this will hopefully be short (~1hr), but could easily become longer as the jenkins plugin ecosystem is fragile and updates like this are known to cause things to explode. the only reason why i'm contemplating

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

well, i forgot to put this on my calendar and didn't get around to getting it done this morning. :) anyways, i'll be shooting for tomorrow (friday) morning instead. shane On Mon, Nov 2, 2015 at 9:55 AM, shane knapp <skn...@berkeley.edu> wrote: > i'd like to take jenkins down briefly

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

this is happening now. On Thu, Nov 5, 2015 at 11:08 AM, shane knapp <skn...@berkeley.edu> wrote: > well, i forgot to put this on my calendar and didn't get around to > getting it done this morning. :) > > anyways, i'll be shooting for tomorrow (friday) morning instead. > &g

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

and we're back! On Fri, Nov 6, 2015 at 7:39 AM, shane knapp <skn...@berkeley.edu> wrote: > this is happening now. > > On Thu, Nov 5, 2015 at 11:08 AM, shane knapp <skn...@berkeley.edu> wrote: >> well, i forgot to put this on my calendar and didn't get around to >

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

a pox on the github pull request builder... the update wiped out the github auth creds. :\ On Fri, Nov 6, 2015 at 12:30 PM, shane knapp <skn...@berkeley.edu> wrote: > looking in to this now. > > On Fri, Nov 6, 2015 at 12:28 PM, Michael Armbrust > <mich...@databricks.com>

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

alright, i'm downgrading our ghprb plugin back to the last known working version. this will require a jenkins restart, which i will do immediately. sorry about this! :( On Fri, Nov 6, 2015 at 12:35 PM, shane knapp <skn...@berkeley.edu> wrote: > a pox on the github pull reques

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

hro...@databricks.com> wrote: > Are you sure that the credentials are missing? Also: did you enable GitHub > commit status updating by accident / configuration loss? That might explain > the errors here, since our keys don't have permissions to use that API. > > On Fri, Nov 6, 2015 at

Re: [BUILD SYSTEM] quick jenkins downtime, november 5th 7am

for our staging instance just showed up? :) On Fri, Nov 6, 2015 at 1:13 PM, shane knapp <skn...@berkeley.edu> wrote: > gonna have to kick jenkins again, folks. sorry! > > On Fri, Nov 6, 2015 at 1:11 PM, shane knapp <skn...@berkeley.edu> wrote: >> i (stupidly) updated the

Re: BUILD SYSTEM: amp-jenkins-worker-05 offline

;> >> This is what I'm looking at: >> >> >> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/ >> >> >> >> On Mon, Oct 19, 2015 at 12:58 PM, shane knapp <skn...@berkeley.edu> wrote: >>> >>> all we did

Re: BUILD SYSTEM: amp-jenkins-worker-05 offline

ery Spark build is failing right now. Could it be > related to your changes? > > - Patrick > > On Mon, Oct 19, 2015 at 11:13 AM, shane knapp <skn...@berkeley.edu> wrote: >> >> worker 05 is back up now... looks like the machine OOMed and needed >> to be kicked. &

Re: BUILD SYSTEM: amp-jenkins-worker-05 offline

worker 05 is back up now... looks like the machine OOMed and needed to be kicked. On Mon, Oct 19, 2015 at 9:39 AM, shane knapp <skn...@berkeley.edu> wrote: > i'll have to head down to the colo and see what's up with it... it > seems to be wedged (pings ok, can't ssh in) and i'll upd

Re: BUILD SYSTEM: amp-jenkins-worker-05 offline

things are green, nice catch on the job config, josh. On Mon, Oct 19, 2015 at 1:57 PM, shane knapp <skn...@berkeley.edu> wrote: > ++joshrosen > > some of those 1.4 builds were incorrectly configured and launching on > a reserved executor... josh fixed them and we're lo

BUILD SYSTEM: builds are OOMing the jenkins workers, investigating. also need to reboot amp-jenkins-worker-06

starting this saturday (oct 17) we started getting alerts on the jenkins workers that various processes were dying (specifically ssh). since then, we've had half of our workers OOM due to java processes and have had now to reboot two of them (-05 and -06). if we look at the current machine

Re: BUILD SYSTEM: builds are OOMing the jenkins workers, investigating. also need to reboot amp-jenkins-worker-06

amp-jenkins-worker-06 is back up. my next bets are on -07 and -08... :\ https://amplab.cs.berkeley.edu/jenkins/computer/ On Tue, Oct 20, 2015 at 3:39 PM, shane knapp <skn...@berkeley.edu> wrote: > here's the related stack trace from dmesg... UID 500 is jenkins. > > Out of memor

Re: BUILD SYSTEM: builds are OOMing the jenkins workers, investigating. also need to reboot amp-jenkins-worker-06

ok, based on the timing, i *think* this might be the culprit: https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=spark-test/3814/console On Tue, Oct 20, 2015 at 3:35 PM, shane knapp <skn...@berkeley.edu> wrote: > -06 just kinda

Re: BUILD SYSTEM: builds are OOMing the jenkins workers, investigating. also need to reboot amp-jenkins-worker-06

. On Tue, Oct 20, 2015 at 3:46 PM, shane knapp <skn...@berkeley.edu> wrote: > amp-jenkins-worker-06 is back up. > > my next bets are on -07 and -08... :\ > > https://amplab.cs.berkeley.edu/jenkins/computer/ > > On Tue, Oct 20, 2015 at 3:39 PM, shane knapp <skn...@

jenkins downtime 7/13/15, 7am PDT

i'll be taking jenkins down for system and jenkins app updates. this should be pretty quick and i'm expecting to have everything back up and building by 9am. i will send a reminder email this weekend, and again when i start the maintenance. if there's any reason for me to delay this, please let

Re: [build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

this is happening now. On Thu, Nov 12, 2015 at 12:14 PM, shane knapp <skn...@berkeley.edu> wrote: > i will admit that it does seem like a bad idea to poke jenkins on > friday the 13th, but there's a release that fixes a lot of security > issues: > > https://wiki.jenkins-ci

Re: Maven issues with 1.5-RC

we build on jenkins w/3.1.1, but also have 3.0.4. On Wed, Aug 26, 2015 at 8:18 AM, Sean Owen so...@cloudera.com wrote: It sounds like you're doing the right things. I believe the Jenkins test machines also have 3.0.4, but successfully build by using build/mvn --force. Not sure what to make of

[build system] java package updates on the amplab jenkins workers

i've installed the latest java 7 and 8 packages on all of the jenkins workers! i haven't updated the /usr/java/latest and /usr/java/default symlinks to point to the new java 7 package, as i'd like to wait for downtime when no builds are running. switching java versions mid-build might be fun,

JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)

good morning, denizens of the aether! your hard working build system (and some associated infrastructure) has been in need of some updates and housecleaning for quite a while now. we will be splitting the maintenance over two mornings to minimize impact. here's the plan: 7am-9am wednesday,

Re: JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)

> 630am-10am thursday, 9-24-15: > * jenknins update to 1.629 (we're a few months behind in versions, and > some big bugs have been fixed) > * jenkins master and worker system package updates > * all systems get a reboot (lots of hanging java processes have been > building up over the months) > *

Re: JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)

this is happening now. On Tue, Sep 22, 2015 at 10:07 AM, shane knapp <skn...@berkeley.edu> wrote: > ok, here's the updated downtime schedule for this week: > > wednesday, sept 23rd: > > firewall maintenance cancelled, as jon took care of the update > saturday mornin

Re: JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)

...and we're finished and now building! On Thu, Sep 24, 2015 at 7:19 AM, shane knapp <skn...@berkeley.edu> wrote: > this is happening now. > > On Tue, Sep 22, 2015 at 10:07 AM, shane knapp <skn...@berkeley.edu> wrote: >> ok, here's the updated downtime schedule for

Re: JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)

a copy of our post-mortem once the dust settles. it's been, shall we say, a pretty crazy few days. http://news.berkeley.edu/2015/09/19/campus-network-outage/ :) On Mon, Sep 21, 2015 at 10:11 AM, shane knapp <skn...@berkeley.edu> wrote: > quick update: we actually did some of the maintenan

BUILD SYSTEM: fire and power event at UC berkeley's IST colo, jenkins offline

TL; DR: jenkins is currently down and will probably not be brought back up until monday morning. a machine caught fire in the colo this evening, and this tripped the halon, and now IST is overheating... it looks like it may have been one of our servers that popped and caused the event, and

Re: BUILD SYSTEM: fire and power event at UC berkeley's IST colo, jenkins offline

we're up and building! time for breakfast... :) https://amplab.cs.berkeley.edu/jenkins/ On Sat, Sep 19, 2015 at 7:35 AM, shane knapp <skn...@berkeley.edu> wrote: > it was definitely one of our servers... we have no ETA on when > jenkins will be back online. we will need to insp

Re: AMP JENKINS - unplanned outage at 1845, ongoing

we're up and building! time for breakfast... :) https://amplab.cs.berkeley.edu/jenkins/ On Fri, Sep 18, 2015 at 9:30 PM, jon kuroda wrote: > Starting tonight at about 6:45PM, the AMP Jenkins instance, which is > hosted at the main UC Berkeley Campus Datacenter, went

Re: JENKINS: downtime next week, wed and thurs mornings (9-23 and 9-24)

with what we'll be covering on wednesday once we get our current situation more under control. :) On Wed, Sep 16, 2015 at 12:15 PM, shane knapp <skn...@berkeley.edu> wrote: >> 630am-10am thursday, 9-24-15: >> * jenknins update to 1.629 (we're a few months behind in versions, an

[build system] jenkins downtime, thursday 12/10/15 7am PDT

there's Yet Another Jenkins Security Advisory[tm], and a big release to patch it all coming out next wednesday. to that end i will be performing a jenkins update, as well as performing the work to resolve the following jira issue: https://issues.apache.org/jira/browse/SPARK-11255 i will put

Re: [build system] jenkins downtime, thursday 12/10/15 7am PDT

reminder! this is happening tomorrow morning. On Wed, Dec 2, 2015 at 7:20 PM, shane knapp <skn...@berkeley.edu> wrote: > there's Yet Another Jenkins Security Advisory[tm], and a big release > to patch it all coming out next wednesday. > > to that end i will be performin

Re: [build system] jenkins downtime, thursday 12/10/15 7am PDT

here's the security advisory for the update: https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2015-12-09 On Wed, Dec 9, 2015 at 9:55 AM, shane knapp <skn...@berkeley.edu> wrote: > reminder! this is happening tomorrow morning. > > On Wed, Dec 2, 2015 at 7:20

Re: [build system] jenkins downtime, thursday 12/10/15 7am PDT

this is happening now. On Wed, Dec 9, 2015 at 11:56 AM, shane knapp <skn...@berkeley.edu> wrote: > here's the security advisory for the update: > https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2015-12-09 > > On Wed, Dec 9, 2015 at 9:55 AM, shane knapp &l

Re: [build system] jenkins downtime, thursday 12/10/15 7am PDT

jenkins is done, but we'll also be updating the firewall. this shouldn't take very long and i'll let everyone know when we're done. On Thu, Dec 10, 2015 at 6:35 AM, shane knapp <skn...@berkeley.edu> wrote: > this is happening now. > > On Wed, Dec 9, 2015 at 11:56 AM, s

Re: [build system] jenkins downtime, thursday 12/10/15 7am PDT

and we're done! this was a quick one. :) On Thu, Dec 10, 2015 at 6:54 AM, shane knapp <skn...@berkeley.edu> wrote: > jenkins is done, but we'll also be updating the firewall. this > shouldn't take very long and i'll let everyone know when we're done. > > On Thu, Dec 10, 2015

Re: [build system] brief downtime right now

something is up w/apache. looking. On Mon, Dec 14, 2015 at 11:37 AM, shane knapp <skn...@berkeley.edu> wrote: > after killing and restarting jenkins, things seem to be VERY slow. > i'm gonna kick jenkins again and see if that helps. > > > > On Mon, Dec 14, 2015 at 11

Re: [build system] brief downtime right now

...and we're back. we were getting reverse proxy timeouts, which seem to have been caused by jenkins churning and doing a lot of IO. i'll dig in to the logs and see if i can find out what happened. weird. shane On Mon, Dec 14, 2015 at 11:51 AM, shane knapp <skn...@berkeley.edu>

Re: Maven build against Hadoop 2.4 times out

++joshrosen This Is Known[tm], and we have a bug open against it: https://issues.apache.org/jira/browse/SPARK-11823 On Mon, Dec 14, 2015 at 7:42 AM, Ted Yu wrote: > Attached was the tail of test suite output from local run. > I got test failure. > > FYI > > On Sun, Dec 13,

Re: [build system] brief downtime right now

ok, we're back up and building. On Mon, Dec 14, 2015 at 10:31 AM, shane knapp <skn...@berkeley.edu> wrote: > last week i forgot to downgrade R to 3.1.1, and since there's not much > activity right now, i'm going to take jenkins down and finish up the > ticket. > > https://i

[build system] brief downtime right now

last week i forgot to downgrade R to 3.1.1, and since there's not much activity right now, i'm going to take jenkins down and finish up the ticket. https://issues.apache.org/jira/browse/SPARK-11255 we should be back up and running within 30 minutes. thanks! shane

Re: [build system] brief downtime right now

b/Spark-Master-SBT/4260/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=spark-test/console. > Is it related to the upgrade work of R? > > Thanks, > > Yin > > On Mon, Dec 14, 2015 at 11:55 AM, shane knapp <skn...@berkeley.edu> wrote: >> >> ...and we're back. we w

Re: [build system] jenkins downtime, thursday 12/10/15 7am PDT

enkins has been in the status of "Jenkins is going to shut > down" for at least 4 hours (from ~23:30 Dec 9 to 3:45 Dec 10, PDT). Not sure > whether this is part of the schedule or related? > > Cheng > > On Thu, Dec 10, 2015 at 3:56 AM, shane knapp <skn...@berkeley.

Re: Write access to wiki

> Shane may be able to fill you in on how the Jenkins build is set up. > mark: yes. yes i can. :) currently, we have a set of bash scripts and binary packages on our jenkins master that can turn a bare centos install in to a jenkins worker. i've also been porting over these bash tools in to

Re: Write access to wiki

> Ok, sounds good. I think it would be great, if you could add installing the > 'docker-engine' package and starting the 'docker' service in there too. I > was planning to update the playbook if there were one in the apache/spark > repo but I didn't see one, hence my question. > we currently have

[build system] jenkins process wedged, need to do restart

of course, on my first day back from vacation, i notice that the jenkins process got wedged immediately upon my visiting the page. one quick jenkins/httpd restart later and we're back up and building. sorry for any inconvenience! shane

<    1   2   3   4   5   6   7   8   >