Issue #3362 has been updated by Mark Plaksin.
Can we have an option to disable this change? We have adapted to how splay currently works and our client checkins are almost perfectly spread out. We have 400+ clients and have 15-20 checkins each minute. Last time splay changed we spent a lot of time re-adjusting to spread out the load. This change might be awesome and work perfectly for us (and everybody else) but it might have unanticipated consequences so a switch to leave things as they were before the change would be great. ---------------------------------------- Bug #3362: splay drift occurs when passenger/mongrel get too much load. http://projects.reductivelabs.com/issues/3362 Author: Dan Bode Status: Investigating Priority: Normal Assigned to: Category: plumbing Target version: Statler Affected version: 0.25.4 Keywords: passenger load splay mongrel connection timeouts Branch: http://github.com/MarkusQ/puppet/tree/ticket/0.25.x/3362 not sure if this counts as a bug... I could not concretely prove the assumptions below. I did some investigation and this is my best guess as to the cause. Splay was drifting for hundreds of machines so that over time, most were checking in at the same time, while at other times none were checking in. here is my theory as to why. splay only runs the first time after puppet starts. Assumption: runinterval starts counting only after the client finishes its last run? Here is the chain of events that I think causes this: 1. passenger or mongrel is under heavy load. 2. processes get used up, they start queuing hosts. 3. Once a machine falls into the queue, it gets stuck with the group of machines that cause the queue to fill up, since it will now use runinterval and check in at the same time as the other machines that were running at that same time. 4. Over time, splay drifts so that most machines are checking in at the same time. Basically, once performance starts getting bad, the splaying falls apart so that it gets much worse. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://reductivelabs.com/redmine/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
