#2480: Thundering herd of pollers overwhelms ssh server, causing nearly silent
GitPoller failure
----------------------+-----------------------
Reporter: dank | Owner:
Type: undecided | Status: new
Priority: minor | Milestone: undecided
Version: 0.8.7p1 | Keywords:
----------------------+-----------------------
buildbots with more than ten masters using GitPoller against the same git
server via ssh seem to fail to notice changes. The only indication of
error is messages like
ssh_exchange_identification: Connection closed by remote host
in twisted.log on the master.
This happens because the ssh daemon drops incoming connections when they
exceed the MaxStartups or MaxSessions limits from sshd.conf. Both default
to 10. Users can raise those limits, but it'd be better to avoid the
thundering herd in the first place.
A workaround is for the user to fuzz the pollIntervals, e.g.
GitPoller(repourl, branches=branchnames, pollinterval=300 +
random.uniform(-10, 10))
A fix might be to build the fuzzing into GitPoller or PollingChangeSource,
so users didn't have to worry about it.
Perhaps add a pollIntervalFuzz parameter that defaults to 10.
--
Ticket URL: <http://trac.buildbot.net/ticket/2480>
Buildbot <http://buildbot.net/>
Buildbot: build/test automation
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_mar
_______________________________________________
Buildbot-commits mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/buildbot-commits
