IIUC http://bugs.java.com/view_bug.do?bug_id=4705093 assigned special meaning to "/dev/urandom" so to avoid that special meaning you need to add the /./
On 30 July 2014 17:24, Stephen Connolly <[email protected]> wrote: > yep > > > On 30 July 2014 16:07, Dean Yu <[email protected]> wrote: > >> > Oh and yes that crazy path is the only way to get it to work... it's a >> bug/feature of the JVM >> >> I'm taking that by the lack of mention of JVM version that this is an >> outstanding issue? >> >> From: Stephen Connolly <[email protected]> >> Reply-To: "[email protected]" < >> [email protected]> >> Date: Wednesday, July 30, 2014 at 7:51 AM >> >> To: "[email protected]" <[email protected]> >> Subject: Re: SSH slave performance degradation >> >> on all linux machines you can just add >> `-Djava.security.egd=file:/dev/./urandom` to the JVM startup command. >> >> This is more critical on the Jenkins master than the slaves as the slaves >> typically only have one connection back to the master where as the master >> has multiple slaves. >> >> If you have multiple slaves sharing the same machine then you would >> probably need it for the slaves also. >> >> Windows machines do not have this issue as far as I am aware. >> >> Oh and yes that crazy path is the only way to get it to work... it's a >> bug/feature of the JVM >> >> >> On 30 July 2014 15:23, Mike Chmielewski <[email protected]> wrote: >> >>> What's the most straightforward way to add this to my installation, >>> add to container args for master, and the node configuration for slaves? Is >>> this just needed on master or just needed on slaves? >>> >>> >>> ----- Original message ----- >>> From: Stephen Connolly <[email protected]> >>> To: "[email protected]" <[email protected]> >>> Subject: Re: SSH slave performance degradation >>> Date: Wed, 30 Jul 2014 15:04:30 +0100 >>> >>> On an AWS m3.large I could not even get to 10 SSH slaves connected >>> without switching to /dev/./urandom >>> >>> >>> On 30 July 2014 14:48, Dean Yu <[email protected]> wrote: >>> >>> This is great info, but how big of a pool of ssh slaves does this become >>> a problem at? We have 12. (And again, the problem goes away by downgrading >>> the library.) >>> >>> -- Dean >>> >>> *From: *Stephen Connolly <[email protected]> >>> *Reply-To: *"[email protected]" < >>> [email protected]> >>> *Date: *Tuesday, July 29, 2014 at 11:42 PM >>> >>> *To: *"[email protected]" <[email protected]> >>> *Subject: *Re: SSH slave performance degradation >>> >>> >>> In my scalability testing I have found you cannot scale out ssh slaves >>> with /dev/random as the entropy source. You need to use /dev/./urandom (JVM >>> bug requires that name btw) >>> >>> The master on windows is a different story though >>> >>> On Wednesday, 30 July 2014, Mark Waite <[email protected]> >>> wrote: >>> >>> I thought that a common default on Linux was to block if /dev/random was >>> to block if the pool of random data was emptied. Refer to >>> http://en.wikipedia.org/?title=/dev/random for a description. >>> >>> I thought that /dev/urandom did not block if the pool of random data was >>> emptied. That same article describes the differences between the two. >>> >>> I've seen cases with some versions of Java and some Linux variants where >>> Java performance suffered badly when I had emptied the pool of random data. >>> I think that is why Stephen recommends using /dev/urandom so that your >>> program won't block while waiting for random data. >>> >>> Mark Waite >>> >>> >>> On Tue, Jul 29, 2014 at 10:44 PM, Dean Yu <[email protected]> wrote: >>> >>> Obviously, going from 1.509.4 to 1.554.3 is a pretty big jump that >>> included lots and lots of changes. However, the fact that the singular act >>> of downgrading that library got us back to our prior build times is a big >>> smoking gun to me. >>> >>> > I wonder if something changed upstream... >>> >>> From the upstream release notes: >>> >>> >>> >>> build217, 2013-06-03: >>> >>> - Support for SSH agent based authentication. >>> >>> build216, 2013-03-04: >>> >>> - Support of unencrypted entries in the known_hosts file. >>> - Improved timeout handling. >>> >>> > BTW you are using /dev/./urandom as an entropy source for the JVM? >>> >>> Nope. Should we? >>> >>> -- Dean >>> >>> >>> *From: *Stephen Connolly <[email protected]> >>> *Reply-To: *"[email protected]" < >>> [email protected]> >>> *Date: *Tuesday, July 29, 2014 at 2:16 PM >>> *To: *"[email protected]" <[email protected]> >>> *Subject: *Re: SSH slave performance degradation >>> >>> >>> * KK's changes to window sizes should have *increased* performance >>> * My connection bug fixes were surgical IIRC >>> * Nicolas's merge of upstream seems to include an EOL change, so hard to >>> see what changed there with the Github diff tool: >>> https://github.com/jenkinsci/trilead-ssh2/compare/trilead-ssh2-build214-jenkins-3...trilead-ssh2-build217-jenkins-5 >>> >>> I wonder if something changed upstream... >>> >>> BTW you are using /dev/./urandom as an entropy source for the JVM? >>> >>> >>> On 29 July 2014 19:51, Dean Yu <[email protected]> wrote: >>> >>> Hi folks, >>> We just upgraded our cluster from 1.509.4 to 1.554.3, and discovered a >>> significant increase in our build times. Builds that typically took ~50 to >>> complete started taking ~90 minutes to finish, sometimes spiking to 2 >>> hours. While researching, we found this JIRA[1] which reported that >>> downgrading the trilead-ssh2 jar solved the performance issues. >>> While this ticket talks specifically artifact downloads, we see that >>> our builds as a whole were slower. >>> The trilead-ssh2 dependency version was updated by [2], so it was >>> introduced into 1.536, show would only have made it to LTS with 1.554.1 in >>> April. >>> Looking at the trilead-ssh2 repo[3], it looks like there were a small >>> set of changes: >>> - changes by ndeloof to merge a newer upstream (build214 to build217) >>> - changes by stephenc to fix connection bugs >>> - changes by kohsuke to support package window sizes >>> >>> Anyone have thoughts on the likely culprit? Given the severity of the >>> performance hit we took, I'm surprised that more people haven't reported >>> this. >>> >>> -- Dean >>> >>> [1] https://issues.jenkins-ci.org/browse/JENKINS-20550 >>> [2] >>> https://github.com/jenkinsci/jenkins/commit/bb265c5e95b0fe39128720b903914236962db41b >>> [3] https://github.com/jenkinsci/trilead-ssh2/commits/master >>> >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> -- >>> Thanks! >>> Mark Waite >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> -- >>> Sent from my phone >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> -- >>> Mike Chmielewski >>> [email protected] >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Jenkins Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
