Damn, centos6 slave still behaves - I think I will dive into this tomorrow to find out why the stuff like this
http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-Datafu/label=centos6/513/console is happening. I have already rebooted it - perhaps the timing was wrong and all these jobs failed because the slave was down? It is an unlikely explanation, but I can't come up with anything better atm ;( Cos On Mon, Jul 14, 2014 at 10:50PM, Konstantin Boudnik wrote: > Ok, Jenkins is at 1.572 now - let's see how the build proceeds from here. > > Cos > > On Mon, Jul 14, 2014 at 03:53PM, Konstantin Boudnik wrote: > > Damn, I can't: only my home computer's keys are added to the CI setup ;( > > Ok, I > > will do the upgrade in a couple of hours. > > > > On Mon, Jul 14, 2014 at 03:30PM, Konstantin Boudnik wrote: > > > Good one Chandrasekhar! Ok, Jenkins is completely stuck right now, so let > > > me > > > start the upgrade right away and restart the server. We'll see how it goes > > > after. > > > > > > Cos > > > > > > On Mon, Jul 14, 2014 at 02:13PM, Chandrasekhar Gopal wrote: > > > > My 2 cents on Julien's comments about the OOM JIRAs. We had the same > > > > issue when we installed version 1.560. > > > > > > > > <> Copying artifacts from other build jobs would hang > > > > <> Slaves would disconnect at random. > > > > > > > > *However, as pointed out > > > > in https://issues.jenkins-ci.org/browse/JENKINS-22734 > > > > <https://issues.jenkins-ci.org/browse/JENKINS-22734>, this problem was > > > > specifically introduced in version 1.560.* > > > > *It does not occur in previous versions and is fixed in versino 1.563.* > > > > > > > > So given that we currently use v1.455, I doubt that this JIRA is the one > > > > creating the problem that we currently are experiencing. > > > > > > > > Chandrasekhar Gopal > > > > Pivotal Hadoop -- Build, Release and Deployments > > > > [email protected] > > > > > > > > > > > > On Mon, Jul 14, 2014 at 1:38 PM, Konstantin Boudnik <[email protected]> > > > > wrote: > > > > > > > > > Julien, > > > > > > > > > > if you feel like helping to resolve this problem then I think I can > > > > > give > > > > > you > > > > > the access to the system. Please reach out to me privately, so I can > > > > > get > > > > > your > > > > > keys setup, etc. > > > > > > > > > > Thanks, > > > > > Cos > > > > > > > > > > On Mon, Jul 14, 2014 at 12:04PM, Julien Eid wrote: > > > > > > This looks to be this issue > > > > > > https://issues.jenkins-ci.org/browse/JENKINS-22734 or > > > > > > https://issues.jenkins-ci.org/browse/JENKINS-22734 but I don't have > > > > > access > > > > > > to build slaves to actually get down and dirty with logs. Either > > > > > > way, our > > > > > > Jenkins setup is pretty old, latest is 1.572 and ours is 1.455. I'm > > > > > > going > > > > > > to talk to a few people here and see about updating our Jenkins > > > > > > setup and > > > > > > any other work we need to do infra side to get this issue fixed. > > > > > > I'll > > > > > > report back with anything I find so we can get a gameplan going. > > > > > > > > > > > > Julien > > > > > > > > > > > > > > > > > > On Fri, Jul 11, 2014 at 6:08 PM, Konstantin Boudnik > > > > > > <[email protected]> > > > > > wrote: > > > > > > > > > > > > > Thanks for checking it out, Mark. I've tried to untangle it last > > > > > night, but > > > > > > > without much success. The most weird part is that cleaning up > > > > > > > build > > > > > queue > > > > > > > and > > > > > > > restarting to the trunk-all job grinds to the same halt every > > > > > > > time ;( > > > > > > > > > > > > > > Cos > > > > > > > > > > > > > > On Fri, Jul 11, 2014 at 10:29AM, Mark Grover wrote: > > > > > > > > Maybe, it's archiving the artifacts on the Jenkins master? > > > > > > > > (sorry, > > > > > don't > > > > > > > > understand much about how the Jenkins artifact archiver works, > > > > > > > > I will > > > > > > > poke > > > > > > > > more). > > > > > > > > > > > > > > > > I don't have access to it, can someone with access to Jenkins > > > > > > > > master > > > > > > > check > > > > > > > > if life is good on it? > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 11, 2014 at 10:25 AM, Mark Grover <[email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > Yeah, I noticed that too. Looking into it, ssh'ing into the > > > > > > > > > slave, > > > > > disk > > > > > > > > > space looks fine. I am trying to understand where the Jenkins > > > > > archiver > > > > > > > is > > > > > > > > > archiving to, maybe that thing ran out of space... > > > > > > > > > Mark > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 11, 2014 at 10:24 AM, Julien Eid < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> http://bigtop01.cloudera.org:8080/job/Bigtop-trunk-Hadoop/ > > > > > > > > >> > > > > > > > > >> Fedora 18 box is blocking jobs from running for the past two > > > > > > > > >> days. > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > http://bigtop01.cloudera.org:8080/job/Bigtop-trunk-Sqoop/label=fedora18/624/console > > > > > > > > >> Stuck on archiving Sqoop artifacts. > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
