On Wed, Jun 12, 2019 at 9:52 AM Giles <[email protected]> wrote: > Hi Mark. > > Correct again: > > - Git plugin 3.10.0, now downgraded to 3.9.1 >
Git plugin 3.10.0 is a good choice. Upgrade to it. > - Git client plugin 3.0.0-rc, now downgraded to 2.7.6 > > Git client plugin 3.0.0-rc is a bad choice. Don't upgrade (yet) to any git client plugin newer than 2.7.x > Deploys do now appear to all be working (this time I waited until we'd > done a fair bit of testing before reporting ...). > > I upgraded the above plugins to 4.0.0-rc and 3.0.0-rc back in February > because there were security advisories: I don't usually upgrade major > versions immediately (especially not RC), instead waiting for them to have > time to stabilize. > > I think that you may have misread the security advisory. No security advisory has been issued that would require installation of a release candidate. Those two releases (git plugin 4.0.0-rc and git client plugin 3.0.0-rc) have been removed from the update center. They have serious compatibility problems which have been resolved in later pre-releases of the plugins. > These downgrades means that I'm now getting a fair bit of messaging from > Jenkins about things I should upgrade or change: > > * Dependency errors:* > > Some plugins could not be loaded due to unsatisfied dependencies. Fix > these issues and restart Jenkins to restore the functionality provided by > these plugins. > Git Parameter Plug-In version 0.9.10 Jenkins Git plugin version > 3.9.1 is older than required. To fix, install version 3.9.3 or later. > GitHub Branch Source Plugin version 2.5.3 Jenkins Git plugin version > 3.9.1 is older than required. To fix, install version 3.9.3 or later. > > Warnings have been published for the following currently installed > components. Git plugin 3.9.1 > <http://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin> CSRF > vulnerability > <https://jenkins.io/security/advisory/2019-01-28/#SECURITY-1095> > > Given that everything is working, I take it I should just ignore these for > now? Or possibly remove "Git Parameter Plug-In" and "GitHub Branch Source > Plugin" as we apparently don't use them at all ... > > Once again, thank you very much for your help. > > > On Tuesday, 11 June 2019 19:02:56 UTC-4, Mark Waite wrote: >> >> >> >> On Tue, Jun 11, 2019 at 4:53 PM Giles <[email protected]> wrote: >> >>> Unfortunately, I appear to have gone from the frying pan to the fire. I >>> ran one deploy that hadn't worked previously and it did work, and I >>> immediately responded with an "it's working" message. And that single >>> deploy still works. But multiple others (all based on the same code, with >>> only varying parameters) don't, all failing with this: >>> >>> Started by user Giles >>> java.lang.NoSuchMethodError: >>> org.eclipse.jgit.lib.Repository.getRef(Ljava/lang/String;)Lorg/eclipse/jgit/lib/Ref; >>> at >>> jenkins.plugins.git.GitSCMFileSystem$1.invoke(GitSCMFileSystem.java:117) >>> >>> That may indicate that something has installed a newer version of JGit >> than JGit 4.5 which is bundled with git client plugin 2.7.x. JGit 4.5 and >> prior versions provided the getRef(String) API. Newer JGit versions (like >> 4.9, 5.1, 5.2, and 5.3) have deprecated or stopped delivering the >> getRef(String) API. >> >> Can you confirm that the git client plugin version you're running is >> 2.7.x and not one of the pre-release 3.0 versions? If it is not, then you >> likely need to install the most recent 2.7 version. >> >> Can you confirm that the git plugin version you're running is 3.9.x and >> not one of the pre-release 4.0 versions? I expect that is the case since >> pre-release versions of git plugin 4.0.0 generally do not refer to >> getRef(String). >> >> That message is unrelated to the version of command line git installed on >> the master. That message is related to an internal mismatch between the >> JGit API that is available in the running Jenkins process and the JGit API >> that the git plugin requires. The git plugin is expecting to find JGit 4.5 >> in the Jenkins process. It does not seem to be there in this case. >> >> This at least is a known error: it appears to happen when there's a newer >>> version of git, or a misalignment of the git version and the git plugin. I >>> had initially installed the very new version of Git for Windows 2.22.0. I >>> downgraded that to 2.17.1.2, and eventually went to 2.21.0 - all resulting >>> in the same failure. Any thoughts? It certainly doesn't seem to be the >>> Git version, and the one working pipeline muddies the water further. >>> >>> On Tuesday, 11 June 2019 17:12:11 UTC-4, Mark Waite wrote: >>>> >>>> I'm glad that helped. In the past, we've seen cases where a new >>>> version of ssh surprises the credential integration that is used by the git >>>> plugin. I don't think your case is related to that, but I did see that Git >>>> for Windows 2.22.0 now includes OpenSSH 8.0. I need to add that version >>>> into my test environment. I'm not aware of any breakage with OpenSSH 8.0, >>>> but we've been surprised in the past. >>>> >>>> On Tue, Jun 11, 2019 at 3:09 PM Giles <[email protected]> wrote: >>>> >>>>> Nailed it in one. >>>>> >>>>> I was sure it was a Jenkins issue ... but I'd upgraded Cygwin at the >>>>> same time, and that probably included their version of git. Which yes, is >>>>> my default git. I changed "Manage Jenkins" -> "Global Tool Configuration" >>>>> -> git to the just-installed "Git for Windows", and ... fixed. I may have >>>>> to make some changes in the Jenkinsfiles themselves, but it seems clear >>>>> now >>>>> that the latest Cygwin git broke something, and that Git for Windows fixed >>>>> it ... >>>>> >>>>> This was crisis-level breakage for my department - I can't thank you >>>>> enough. >>>>> >>>>> On Tuesday, 11 June 2019 16:36:38 UTC-4, Mark Waite wrote: >>>>>> >>>>>> Are you quite sure that your configuration did not change in about >>>>>> the same time as that upgrade? >>>>>> >>>>>> The log file indicates that you're running command line git as >>>>>> provided by Cygwin. Is that intentional? Has it always been that way? >>>>>> >>>>>> I don't test the Jenkins git plugin with Cygwin. I test with Git for >>>>>> Windows. I'm glad cygwin git works, but don't intend to apply effort to >>>>>> make it work. >>>>>> >>>>>> I don't see anything suspicious or troubling in the list of updates. >>>>>> As far as I can tell, none of them should have changed that behavior of >>>>>> the >>>>>> git plugin. >>>>>> >>>>>> On Tue, Jun 11, 2019 at 1:54 PM Giles <[email protected]> wrote: >>>>>> >>>>>>> I'm running Jenkins 2.164.3 on a Windows server. It's been running >>>>>>> well for several months. Every Friday evening I do all the Jenkins >>>>>>> plugin >>>>>>> updates. After this past Friday's updates (details below), all our jobs >>>>>>> are broken - apparently because the GitSCM checkout is broken. Our >>>>>>> Jenkinsfiles are all stored in a git repo, so Jenkins attempts to pull a >>>>>>> new version before running the job. This now fails. I created a test >>>>>>> Jenkinsfile (editing directly in Jenkins rather than in the git repo) >>>>>>> with >>>>>>> this code in it: >>>>>>> >>>>>>> def scmVars = checkout( >>>>>>> [ >>>>>>> $class: 'GitSCM', >>>>>>> branches: [[name: '*/' + branch ]], >>>>>>> doGenerateSubmoduleConfigurations: false, >>>>>>> extensions: [], >>>>>>> submoduleCfg: [], >>>>>>> userRemoteConfigs: [ >>>>>>> [ >>>>>>> credentialsId: jenkinsCred, >>>>>>> url: jenkinsRepo, >>>>>>> ] >>>>>>> ] >>>>>>> ] >>>>>>> ); >>>>>>> >>>>>>> (Assume the "jenkinsCred" and "jenkinsRepo" values are correct.) >>>>>>> The failure looks like this: >>>>>>> >>>>>>> [Pipeline] {[Pipeline] checkoutusing credential >>>>>>> 29d83033-ebf6-4811-9c45-b0aadec775c2 >>>>>>> > C:\cygwin64\bin\git.exe rev-parse --is-inside-work-tree # timeout=10 >>>>>>> Fetching changes from the remote Git repository >>>>>>> > C:\cygwin64\bin\git.exe config remote.origin.url >>>>>>> [email protected]:*/*.git # timeout=10 >>>>>>> Fetching upstream changes from [email protected]:*/*.git >>>>>>> > C:\cygwin64\bin\git.exe --version # timeout=10 >>>>>>> using GIT_SSH to set credentials Read-only access to the "Jenkins" >>>>>>> repository at github.com/*. >>>>>>> > C:\cygwin64\bin\git.exe fetch --tags --force --progress >>>>>>> [email protected]:*/*.git +refs/heads/*:refs/remotes/origin/* # timeout=10 >>>>>>> ERROR: Error fetching remote repo 'origin' >>>>>>> hudson.plugins.git.GitException: Failed to fetch from >>>>>>> [email protected]:*/*.git >>>>>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:894) >>>>>>> at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) >>>>>>> at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) >>>>>>> at >>>>>>> org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) >>>>>>> at >>>>>>> org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) >>>>>>> at >>>>>>> org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) >>>>>>> at >>>>>>> org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) >>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown >>>>>>> Source) >>>>>>> at java.util.concurrent.FutureTask.run(Unknown Source) >>>>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown >>>>>>> Source) >>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >>>>>>> Source) >>>>>>> at java.lang.Thread.run(Unknown Source) >>>>>>> Caused by: hudson.plugins.git.GitException: Command >>>>>>> "C:\cygwin64\bin\git.exe fetch --tags --force --progress >>>>>>> [email protected]:*/*.git +refs/heads/*:refs/remotes/origin/*" returned >>>>>>> status code 128: >>>>>>> stdout: >>>>>>> stderr: error: cannot run >>>>>>> D:\Jenkins\workspace\DEBUG_07_git_Checkout@tmp\jenkins-gitclient-ssh1436722960383118000.bat: >>>>>>> No such file or directory >>>>>>> fatal: unable to fork >>>>>>> >>>>>>> at >>>>>>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2318) >>>>>>> at >>>>>>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1905) >>>>>>> at >>>>>>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:81) >>>>>>> at >>>>>>> org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:488) >>>>>>> at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:892) >>>>>>> ... 11 more[Pipeline] }[Pipeline] // node[Pipeline] End of >>>>>>> PipelineERROR: Error fetching remote repo 'origin' >>>>>>> Finished: FAILURE >>>>>>> >>>>>>> This is essentially identical to the failure we're seeing for all jobs. >>>>>>> I assume it's normal behaviour that GitSCM generates and >>>>>>> tries to run a random-named bat file? It's unclear to me if the bat >>>>>>> file isn't created or is unreadable, although Jenkins is running >>>>>>> as SYSTEM and has generally been able to read and write everything. >>>>>>> >>>>>>> Does anyone have any thoughts on what may have gone wrong? >>>>>>> >>>>>>> ----- >>>>>>> >>>>>>> >>>>>>> The updates made Friday night: >>>>>>> >>>>>>> >>> Artifactory 3.2.2 -> 3.2.4 [ security warning >>>>>>> ] >>>>>>> >>> Pipeline: API 2.34 -> >>>>>>> 2.35 >>>>>>> >>> Pipeline: Basic Steps 2.16 -> >>>>>>> 2.18 >>>>>>> >>> Pipeline: Declarative 1.3.8 -> >>>>>>> 1.3.9 >>>>>>> >>> Pipeline: Declarative Extension Points API 1.3.8 -> >>>>>>> 1.3.9 >>>>>>> >>> Pipeline: Groovy 2.69 -> >>>>>>> 2.70 >>>>>>> >>> Pipeline: Model API 1.3.8 -> >>>>>>> 1.3.9 >>>>>>> >>> Pipeline: Nodes and Processes 2.30 -> >>>>>>> 2.31 >>>>>>> >>> Pipeline: SCM Step 2.7 -> >>>>>>> 2.9 >>>>>>> >>> Pipeline: Stage Tags Metadata 1.3.8 -> >>>>>>> 1.3.9 >>>>>>> >>> Pipeline: Step API 2.19 -> >>>>>>> 2.20 >>>>>>> >>> Slack Notification 2.23 -> 2.24 >>>>>>> >>>>>>> I've since rolled back all of the "Pipeline: ..." changes. The >>>>>>> problem didn't change. >>>>>>> >>>>>>> About 40 Cygwin updates (I didn't enumerate those.) >>>>>>> >>>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/jenkinsci-users/18b9de6c-9c98-455e-b149-9f43b2e4011e%40googlegroups.com >>> <https://groups.google.com/d/msgid/jenkinsci-users/18b9de6c-9c98-455e-b149-9f43b2e4011e%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> -- >> Thanks! >> Mark Waite >> > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-users/2769d445-024a-4ffa-a593-b37958466581%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-users/2769d445-024a-4ffa-a593-b37958466581%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Thanks! Mark Waite -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtH%2BVCzfbKvHF6f-yRtJ%2BRhCNrWdYZAK6HqpK_xttLZVvg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
