Obviously, going from 1.509.4 to 1.554.3 is a pretty big jump that included lots and lots of changes. However, the fact that the singular act of downgrading that library got us back to our prior build times is a big smoking gun to me.
> I wonder if something changed upstream... >From the upstream release notes: > > build217, 2013-06-03: > > - Support for SSH agent based authentication. > > build216, 2013-03-04: > > - Support of unencrypted entries in the known_hosts file. > - Improved timeout handling. > BTW you are using /dev/./urandom as an entropy source for the JVM? Nope. Should we? -- Dean From: Stephen Connolly <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Tuesday, July 29, 2014 at 2:16 PM To: "[email protected]" <[email protected]> Subject: Re: SSH slave performance degradation > * KK's changes to window sizes should have *increased* performance > * My connection bug fixes were surgical IIRC > * Nicolas's merge of upstream seems to include an EOL change, so hard to see > what changed there with the Github diff tool: > https://github.com/jenkinsci/trilead-ssh2/compare/trilead-ssh2-build214-jenkin > s-3...trilead-ssh2-build217-jenkins-5 > > I wonder if something changed upstream... > > BTW you are using /dev/./urandom as an entropy source for the JVM? > > > On 29 July 2014 19:51, Dean Yu <[email protected]> wrote: >> Hi folks, >> We just upgraded our cluster from 1.509.4 to 1.554.3, and discovered a >> significant increase in our build times. Builds that typically took ~50 to >> complete started taking ~90 minutes to finish, sometimes spiking to 2 hours. >> While researching, we found this JIRA[1] which reported that downgrading the >> trilead-ssh2 jar solved the performance issues. >> While this ticket talks specifically artifact downloads, we see that our >> builds as a whole were slower. >> The trilead-ssh2 dependency version was updated by [2], so it was >> introduced into 1.536, show would only have made it to LTS with 1.554.1 in >> April. >> Looking at the trilead-ssh2 repo[3], it looks like there were a small set >> of changes: >> - changes by ndeloof to merge a newer upstream (build214 to build217) >> - changes by stephenc to fix connection bugs >> - changes by kohsuke to support package window sizes >> >> Anyone have thoughts on the likely culprit? Given the severity of the >> performance hit we took, I'm surprised that more people haven't reported >> this. >> >> -- Dean >> >> [1] https://issues.jenkins-ci.org/browse/JENKINS-20550 >> [2] >> https://github.com/jenkinsci/jenkins/commit/bb265c5e95b0fe39128720b9039142369 >> 62db41b >> [3] https://github.com/jenkinsci/trilead-ssh2/commits/master >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
