The jenkins process actually ran out of the PermGen - fixed now. Thanks On Wed, Jul 16, 2014 at 04:17PM, pengwenwu wrote: > is possible network overloading cause failure as > https://issues.jenkins-ci.org/browse/JENKINS-14332 ? > > > > > Regards, > Wenwu,Peng > > > > > > > > At 2014-07-16 02:49:26, "Konstantin Boudnik" <[email protected]> wrote: > >Damn, centos6 slave still behaves - I think I will dive into this tomorrow to > >find out why the stuff like this > > > >http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-Datafu/label=centos6/513/console > > > >is happening. I have already rebooted it - perhaps the timing was wrong and > >all these jobs failed because the slave was down? It is an unlikely > >explanation, but I can't come up with anything better atm ;( > > > >Cos > > > >On Mon, Jul 14, 2014 at 10:50PM, Konstantin Boudnik wrote: > >> Ok, Jenkins is at 1.572 now - let's see how the build proceeds from here. > >> > >> Cos > >> > >> On Mon, Jul 14, 2014 at 03:53PM, Konstantin Boudnik wrote: > >> > Damn, I can't: only my home computer's keys are added to the CI setup ;( > >> > Ok, I > >> > will do the upgrade in a couple of hours. > >> > > >> > On Mon, Jul 14, 2014 at 03:30PM, Konstantin Boudnik wrote: > >> > > Good one Chandrasekhar! Ok, Jenkins is completely stuck right now, so > >> > > let me > >> > > start the upgrade right away and restart the server. We'll see how it > >> > > goes > >> > > after. > >> > > > >> > > Cos > >> > > > >> > > On Mon, Jul 14, 2014 at 02:13PM, Chandrasekhar Gopal wrote: > >> > > > My 2 cents on Julien's comments about the OOM JIRAs. We had the > >> > > > same > >> > > > issue when we installed version 1.560. > >> > > > > >> > > > <> Copying artifacts from other build jobs would hang > >> > > > <> Slaves would disconnect at random. > >> > > > > >> > > > *However, as pointed out > >> > > > in https://issues.jenkins-ci.org/browse/JENKINS-22734 > >> > > > <https://issues.jenkins-ci.org/browse/JENKINS-22734>, this problem > >> > > > was > >> > > > specifically introduced in version 1.560.* > >> > > > *It does not occur in previous versions and is fixed in versino > >> > > > 1.563.* > >> > > > > >> > > > So given that we currently use v1.455, I doubt that this JIRA is the > >> > > > one > >> > > > creating the problem that we currently are experiencing. > >> > > > > >> > > > Chandrasekhar Gopal > >> > > > Pivotal Hadoop -- Build, Release and Deployments > >> > > > [email protected] > >> > > > > >> > > > > >> > > > On Mon, Jul 14, 2014 at 1:38 PM, Konstantin Boudnik > >> > > > <[email protected]> wrote: > >> > > > > >> > > > > Julien, > >> > > > > > >> > > > > if you feel like helping to resolve this problem then I think I > >> > > > > can give > >> > > > > you > >> > > > > the access to the system. Please reach out to me privately, so I > >> > > > > can get > >> > > > > your > >> > > > > keys setup, etc. > >> > > > > > >> > > > > Thanks, > >> > > > > Cos > >> > > > > > >> > > > > On Mon, Jul 14, 2014 at 12:04PM, Julien Eid wrote: > >> > > > > > This looks to be this issue > >> > > > > > https://issues.jenkins-ci.org/browse/JENKINS-22734 or > >> > > > > > https://issues.jenkins-ci.org/browse/JENKINS-22734 but I don't > >> > > > > > have > >> > > > > access > >> > > > > > to build slaves to actually get down and dirty with logs. Either > >> > > > > > way, our > >> > > > > > Jenkins setup is pretty old, latest is 1.572 and ours is 1.455. > >> > > > > > I'm going > >> > > > > > to talk to a few people here and see about updating our Jenkins > >> > > > > > setup and > >> > > > > > any other work we need to do infra side to get this issue fixed. > >> > > > > > I'll > >> > > > > > report back with anything I find so we can get a gameplan going. > >> > > > > > > >> > > > > > Julien > >> > > > > > > >> > > > > > > >> > > > > > On Fri, Jul 11, 2014 at 6:08 PM, Konstantin Boudnik > >> > > > > > <[email protected]> > >> > > > > wrote: > >> > > > > > > >> > > > > > > Thanks for checking it out, Mark. I've tried to untangle it > >> > > > > > > last > >> > > > > night, but > >> > > > > > > without much success. The most weird part is that cleaning up > >> > > > > > > build > >> > > > > queue > >> > > > > > > and > >> > > > > > > restarting to the trunk-all job grinds to the same halt every > >> > > > > > > time ;( > >> > > > > > > > >> > > > > > > Cos > >> > > > > > > > >> > > > > > > On Fri, Jul 11, 2014 at 10:29AM, Mark Grover wrote: > >> > > > > > > > Maybe, it's archiving the artifacts on the Jenkins master? > >> > > > > > > > (sorry, > >> > > > > don't > >> > > > > > > > understand much about how the Jenkins artifact archiver > >> > > > > > > > works, I will > >> > > > > > > poke > >> > > > > > > > more). > >> > > > > > > > > >> > > > > > > > I don't have access to it, can someone with access to > >> > > > > > > > Jenkins master > >> > > > > > > check > >> > > > > > > > if life is good on it? > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > On Fri, Jul 11, 2014 at 10:25 AM, Mark Grover > >> > > > > > > > <[email protected]> > >> > > > > wrote: > >> > > > > > > > > >> > > > > > > > > Yeah, I noticed that too. Looking into it, ssh'ing into > >> > > > > > > > > the slave, > >> > > > > disk > >> > > > > > > > > space looks fine. I am trying to understand where the > >> > > > > > > > > Jenkins > >> > > > > archiver > >> > > > > > > is > >> > > > > > > > > archiving to, maybe that thing ran out of space... > >> > > > > > > > > Mark > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > On Fri, Jul 11, 2014 at 10:24 AM, Julien Eid < > >> > > > > [email protected]> > >> > > > > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > >> http://bigtop01.cloudera.org:8080/job/Bigtop-trunk-Hadoop/ > >> > > > > > > > >> > >> > > > > > > > >> Fedora 18 box is blocking jobs from running for the past > >> > > > > > > > >> two days. > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > > > > http://bigtop01.cloudera.org:8080/job/Bigtop-trunk-Sqoop/label=fedora18/624/console > >> > > > > > > > >> Stuck on archiving Sqoop artifacts. > >> > > > > > > > >> > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > > >
signature.asc
Description: Digital signature
