On Thu 05 Jun 2014 05:10:48 PM CEST, Eyal Edri wrote: > i think this happens when gerrit server is too overloaded, and can't return > an answer for more than 10min (default value). > latest git plugin allows us to set a different value than 10min. > > for 1st action, i'd change those failing jobs to use maybe 20min instead, and > see if it's fixed. > can you provide list of links for the failing jobs you saw? > > for longer run, i'd think on how to reduce/manage load on gerrit server, > maybe using zuul. [1] > david - any thoughts? > > [1] http://ci.openstack.org/zuul/ > > ----- Original Message ----- >> From: "Michal Skrivanek" <[email protected]> >> To: [email protected] >> Sent: Thursday, June 5, 2014 5:36:59 PM >> Subject: Jenkins issue (findbugs) >> >> >>> ----- Forwarded Message ----- >>> From: "Frantisek Kobzik" <[email protected]> >>> To: [email protected] >>> Sent: Thursday, June 5, 2014 11:23:12 AM >>> Subject: Jenkins issue (findbugs) >>> >>> Hi guys, >>> >>> I have an issue with jenkins build and I was advised to contact infra. >>> The console output is below (Similar issue also happens with other kinds of >>> jobs). >>> >>> I'm seeing this issue quite often (~50 % of pushes), is there anything you >>> can do about it? >>> >>> Thanks, >>> Franta. >>> >>> >>> 9:47:36 ERROR: Timeout after 10 minutes >>> 09:47:36 FATAL: Failed to fetch from git://gerrit.ovirt.org/ovirt-engine >>> 09:47:37 hudson.plugins.git.GitException: Failed to fetch from >>> git://gerrit.ovirt.org/ovirt-engine >>> 09:47:37 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:623) >>> 09:47:37 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:855) >>> 09:47:37 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:880) >>> 09:47:37 at >>> org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:118) >>> 09:47:37 at >>> hudson.model.AbstractProject.checkout(AbstractProject.java:1320) >>> 09:47:37 at >>> hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609) >>> 09:47:37 at >>> jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88) >>> 09:47:37 at >>> hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:518) >>> 09:47:37 at hudson.model.Run.execute(Run.java:1700) >>> 09:47:37 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) >>> 09:47:37 at >>> hudson.model.ResourceController.execute(ResourceController.java:88) >>> 09:47:37 at hudson.model.Executor.run(Executor.java:231) >>> 09:47:37 Caused by: hudson.plugins.git.GitException: Command "git fetch >>> --tags --progress git://gerrit.ovirt.org/ovirt-engine >>> refs/changes/14/28314/3 --prune" returned status code 143: >>> 09:47:37 stdout: >>> 9:47:37 stderr: remote: Counting objects: 136 >>> remote: Counting objects: 375 >>> remote: Counting objects: 582 >>> . >>> etc. >>> . >>> 09:47:37 at >>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1276) >>> 09:47:37 at >>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1146) >>> 09:47:37 at >>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:87) >>> 09:47:37 at >>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:254) >>> 09:47:37 at >>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) >>> 09:47:37 at >>> org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) >>> 09:47:37 at hudson.remoting.UserRequest.perform(UserRequest.java:118) >>> 09:47:37 at hudson.remoting.UserRequest.perform(UserRequest.java:48) >>> 09:47:37 at hudson.remoting.Request$2.run(Request.java:328) >>> 09:47:37 at >>> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) >>> 09:47:37 at java.util.concurrent.FutureTask.run(FutureTask.java:262) >>> 09:47:37 at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> 09:47:37 at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> 09:47:37 at java.lang.Thread.run(Thread.java:744) >>> . >> >> _______________________________________________ >> Infra mailing list >> [email protected] >> http://lists.ovirt.org/mailman/listinfo/infra >>
Maybe we can start investigating on create mirrors for gerrit to pull from the slaves (also downstream if able) at least for read operations, that will leverage a lot of load from the jenkins slaves (if not all of it) -- David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: [email protected] Web: www.redhat.com RHT Global #: 82-62605
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Infra mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/infra
