Ok so it sounds like exhuming Stephens scalability stuff (not sure if it did startup time, but it doesn't sound like it would be hard to ad) would be a great place to start. Like you said, turning the dials and seeing what happens is super useful. Even on VMs (vs bare metal) would be informative as its increasingly common to run Jenkins not on bare hardware. On Wed, 14 Oct 2015 at 2:42 AM, Jesse Glick <[email protected]> wrote:
> On Tue, Oct 13, 2015 at 7:29 AM, Michael Neale <[email protected]> > wrote: > > Looking at the variation of times people see, I am questioning the > utility > > of a generic test suite. Things vary so much there may be too many > variables > > at play to make something like this useful right now. > > Well a generic test suite is not going to predict any given > installation’s performance, of course. But it can serve a controlled > baseline by which you can measure the effects of changes. And many > widely applicable bugs, like the ones Google engineers found, can be > reproduced this way. When Stephen and I were poring over results from > sample tests using his scalability framework, which did really generic > stuff—run lots of builds from lots of jobs, each build producing gobs > of output—it was immediately clear what was broken. You set n=10 and > all is well. You set n=500 and things start to look worse. You set > n=10000 and the system basically hangs, and you look at a thread dump, > and oh yes a thousand threads are waiting on this one lock for no good > reason. So you fix that problem and rerun the test and you find the > next problem. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Jenkins Developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/jenkinsci-dev/1F9DHyMOutw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1kkepm0P%2B9QzLzZq9a5dmjzos3jYu6r4W6r4wbvXPX2A%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAKVMTi5sSUG%3DY9rNcQ4MoZLdV1%3DWq9uQoYt6rGTrB8LJOL1poQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
