Are you trying to clone a huge repo with JGit on the git server? If so, you might try cloning the same repo with command line git on the server.
I don't recognize "Finding sources" or "Getting sizes" as common messages from command line git as a server. Mark Waite On Thursday, June 16, 2016 at 10:57:59 AM UTC-6, Jérémie Faucher-Goulet wrote: > > Hello all, > I've got a nagging problem for which I'd welcome some thoughtful expert > input... > > Once in a while, our builds will fail with the output pasted below in the > console. This is a build triggered by another job, which itself is > triggered remotely by Atlassian Fisheye when a commit is pushed to the > repository. This is part of our CI system. > > Building remotely on grs-jslave > <http://grs-jenkins03.trilliant.local/computer/grs-jslave> (linux openwrt > cmockery2) in workspace > /var/grs-jenkins03/workspace/smv6-6lr-core_Develop_Linux > > git rev-parse --is-inside-work-tree # timeout=10 > Fetching changes from the remote Git repository > > git config remote.origin.url > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core # > timeout=10 > Cleaning workspace > > git rev-parse --verify HEAD # timeout=10 > Resetting working tree > > git reset --hard # timeout=10 > > git clean -fdx # timeout=10 > Fetching upstream changes from > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > > git --version # timeout=10 > using .gitcredentials to set credentials > > git config --local credential.username grfbuild # timeout=10 > > git config --local credential.helper store > --file=/tmp/git9025647881849389319.credentials # timeout=10 > > git -c core.askpass=true fetch --tags --progress > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > +refs/heads/*:refs/remotes/origin/* > > git config --local --remove-section credential # timeout=10 > ERROR: Error fetching remote repo 'origin' > hudson.plugins.git.GitException > <http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException>: > Failed to fetch from > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.plugins.git.GitSCM.fetchFrom&entity=method> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.plugins.git.GitSCM.retrieveChanges&entity=method> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.plugins.git.GitSCM.checkout&entity=method> > at hudson.scm.SCM.checkout(SCM.java:485) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.scm.SCM.checkout&entity=method> > at hudson.model.AbstractProject.checkout(AbstractProject.java:1275) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractProject.checkout&entity=method> > at > hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610) > > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout&entity=method> > at > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) > <http://stacktrace.jenkins-ci.org/search/?query=jenkins.scm.SCMCheckoutStrategy.checkout&entity=method> > at > hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractBuild$AbstractBuildExecution.run&entity=method> > at hudson.model.Run.execute(Run.java:1741) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Run.execute&entity=method> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.FreeStyleBuild.run&entity=method> > at > hudson.model.ResourceController.execute(ResourceController.java:98) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.ResourceController.execute&entity=method> > at hudson.model.Executor.run(Executor.java:408) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Executor.run&entity=method> > Caused by: hudson.plugins.git.GitException > <http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException>: > Command "git -c core.askpass=true fetch --tags --progress > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > +refs/heads/*:refs/remotes/origin/*" returned status code 1: > stdout: Fetching submodule contrib/exegin-nan > > stderr: remote: Counting objects: 1 > remote: Counting objects: 57, done > remote: Finding sources: 4% (1/25) > remote: Finding sources: 8% (2/25) > remote: Finding sources: 12% (3/25) > remote: Finding sources: 16% (4/25) > remote: Finding sources: 20% (5/25) > remote: Finding sources: 24% (6/25) > remote: Finding sources: 28% (7/25) > remote: Finding sources: 32% (8/25) > remote: Finding sources: 36% (9/25) > remote: Finding sources: 40% (10/25) > remote: Finding sources: 44% (11/25) > remote: Finding sources: 48% (12/25) > remote: Finding sources: 52% (13/25) > remote: Finding sources: 56% (14/25) > remote: Finding sources: 60% (15/25) > remote: Finding sources: 64% (16/25) > remote: Finding sources: 68% (17/25) > remote: Finding sources: 72% (18/25) > remote: Finding sources: 76% (19/25) > remote: Finding sources: 80% (20/25) > remote: Finding sources: 84% (21/25) > remote: Finding sources: 88% (22/25) > remote: Finding sources: 92% (23/25) > remote: Finding sources: 96% (24/25) > remote: Finding sources: 100% (25/25) > remote: Finding sources: 100% (25/25) > remote: Getting sizes: 2% (1/42) > remote: Getting sizes: 4% (2/42) > remote: Getting sizes: 7% (3/42) > remote: Getting sizes: 9% (4/42) > remote: Getting sizes: 11% (5/42) > remote: Getting sizes: 14% (6/42) > remote: Getting sizes: 16% (7/42) > remote: Getting sizes: 19% (8/42) > remote: Getting sizes: 21% (9/42) > remote: Getting sizes: 23% (10/42) > remote: Getting sizes: 26% (11/42) > remote: Getting sizes: 28% (12/42) > remote: Getting sizes: 30% (13/42) > remote: Getting sizes: 33% (14/42) > remote: Getting sizes: 35% (15/42) > remote: Getting sizes: 38% (16/42) > remote: Getting sizes: 40% (17/42) > remote: Getting sizes: 42% (18/42) > remote: Getting sizes: 45% (19/42) > remote: Getting sizes: 47% (20/42) > remote: Getting sizes: 50% (21/42) > remote: Getting sizes: 52% (22/42) > remote: Getting sizes: 54% (23/42) > remote: Getting sizes: 57% (24/42) > remote: Getting sizes: 59% (25/42) > remote: Getting sizes: 61% (26/42) > remote: Getting sizes: 64% (27/42) > remote: Getting sizes: 66% (28/42) > remote: Getting sizes: 69% (29/42) > remote: Getting sizes: 71% (30/42) > remote: Getting sizes: 73% (31/42) > remote: Getting sizes: 76% (32/42) > remote: Getting sizes: 78% (33/42) > remote: Getting sizes: 80% (34/42) > remote: Getting sizes: 83% (35/42) > remote: Getting sizes: 85% (36/42) > remote: Getting sizes: 88% (37/42) > remote: Getting sizes: 90% (38/42) > remote: Getting sizes: 92% (39/42) > remote: Getting sizes: 95% (40/42) > remote: Getting sizes: 97% (41/42) > remote: Getting sizes: 100% (42/42) > remote: Getting sizes: 100% (42/42) > remote: Total 25 (delta 12), reused 25 (delta 12) > From https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > 029fa64..dee5ad4 develop -> origin/develop > * [new branch] feature/fb-new-exegin-3_6_0 -> > origin/feature/fb-new-exegin-3_6_0 > fatal: remote error: Invalid username or password. > > at > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1710) > at > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1454) > at > org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:63) > at > org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:314) > at > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152) > at > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145) > at hudson.remoting.UserRequest.perform(UserRequest.java:121) > at hudson.remoting.UserRequest.perform(UserRequest.java:49) > at hudson.remoting.Request$2.run(Request.java:326) > at > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > at ......remote call to grs-jslave(Native Method) > at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:221) > at hudson.remoting.Channel.call(Channel.java:778) > at > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145) > at sun.reflect.GeneratedMethodAccessor570.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at > org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131) > at com.sun.proxy.$Proxy70.execute(Unknown Source) > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:764) > ... 11 more > ERROR: null > > > Other tidbits or observations: > > - It seems to fail when there hasn't been any builds for a while, as > if something was cached. I've never seen two failures in rapid succession. > When we get this failure, usually immediately relaunching the same build > will work the second time around. > - I'm confused by the fact that it seems to try fetching git > submodules when it fails. We haven't enabled any advanced submodule > behavior and a successful build doesn't show any submodule fetches being > made. > - This occurs on both Windows and Linux (We run a similar build on a > Linux machine configured as a Jenkins slave). > - From the windows error, I've read somewhere that it might have > something to do with "wincreds" being configured globally while we use > "store" locally. But how would that apply to Linux? Furthermore I've > confirmed that wincred isn't configured on the windows machine. > > On a windows build, we get a slightly different output: > > ERROR: Error fetching remote repo 'origin' > hudson.plugins.git.GitException > <http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException>: > Failed to fetch from > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:766) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.plugins.git.GitSCM.fetchFrom&entity=method> > at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1022) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.plugins.git.GitSCM.retrieveChanges&entity=method> > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1053) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.plugins.git.GitSCM.checkout&entity=method> > at > org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:129) > <http://stacktrace.jenkins-ci.org/search/?query=org.jenkinsci.plugins.multiplescms.MultiSCM.checkout&entity=method> > at hudson.scm.SCM.checkout(SCM.java:485) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.scm.SCM.checkout&entity=method> > at hudson.model.AbstractProject.checkout(AbstractProject.java:1275) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractProject.checkout&entity=method> > at > hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610) > > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout&entity=method> > at > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) > <http://stacktrace.jenkins-ci.org/search/?query=jenkins.scm.SCMCheckoutStrategy.checkout&entity=method> > at > hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.AbstractBuild$AbstractBuildExecution.run&entity=method> > at hudson.model.Run.execute(Run.java:1741) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Run.execute&entity=method> > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.FreeStyleBuild.run&entity=method> > at > hudson.model.ResourceController.execute(ResourceController.java:98) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.ResourceController.execute&entity=method> > at hudson.model.Executor.run(Executor.java:408) > <http://stacktrace.jenkins-ci.org/search/?query=hudson.model.Executor.run&entity=method> > Caused by: hudson.plugins.git.GitException > <http://stacktrace.jenkins-ci.org/search?query=hudson.plugins.git.GitException>: > Command "C:\Program Files\Git\bin\git.exe -c core.askpass=true fetch --tags > --progress > https://grs-softsvnrepo1.trilliant.local:8081/scm/git/smv6-6lr-core > +refs/heads/*:refs/remotes/origin/*" returned status code 1: > stdout: > stderr: > This application has requested the Runtime to terminate it in an unusual way. > Please contact the application's support team for more information. > Assertion failed! > > Program: C:\Program > Files\Git\mingw64\libexec\git-core\git-credential-store.exe > File: exec_cmd.c, Line 23 > > Expression: argv0_path > remote: Counting objects: 1 > remote: Counting objects: 593, done > remote: Finding sources: 1% (4/322) > remote: Finding sources: 2% (7/322) > remote: Finding sources: 3% (10/322) > > > Normally developers would have a look when the build fails and they > receive the notification email. However, since the build often fails for > Git fetch reasons, nobody looks at the failures anymore and the real issues > get overlooked for a while, defeating the very purpose of having a CI > system in place. I'm really hoping to figure this one out... > > Has anyone encountered such behavior before? Any hints on what is > happening? Anything I should do to investigate this further? > > Thank you, > > Jérémie > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/cf2036b0-3b6c-450d-89da-d691f441dbdb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
