Thanks for the lengthy explanation. I suppose that, for one thing, I didn't understand the algorithm that was used in puppetcommander. The one thing that will complicate this for me is my horribly poor ruby skills, coupled with the fact that the MCo python bindings appear to be abandoned (though I found what appears to be a not-yet-complete replacement for them at https://github.com/rafaduran/python-mcollective).
Understood about the 3 variables, and I can more or less pick 2 of 3. At the moment I'm running in daemon mode with splay = false, so I'm not terribly worried about concurrency (as a matter of fact, my test environment has 25 nodes and 21 of them appear to be running within the same minute); my end goal is to have all my nodes running every 30 minutes and be as evenly dispersed as practical within that timeframe. For the time being, as I have some more pressing issues, I'm going to give up on the "evenly dispersed" part, and opt to use runall with a concurrency of 10, and revisit this when my infrastructure grows to the point that it becomes a problem (or when I finally pull the ticket to fix this). I just did a "mco puppet runall 10" in my 25-node test environment, and the mco command completed in 57 seconds. The puppet master looked perfectly happy with that. Thanks for all of the advice. -Jason On 12/31/2013 11:58 AM, R.I.Pienaar wrote: > > ----- Original Message ----- >> From: "Jose Luis Ledesma" <[email protected]> >> To: [email protected] >> Sent: Tuesday, December 31, 2013 4:51:04 PM >> Subject: Re: [Puppet Users] MCollective/Puppet - one-time run with other >> options >> >> The problem about using runall/concurrency is that you cannot know when it >> will finish, so setting it on the crontab could be a source of a problems if >> it has not yet finished when the next run is schedulee > You could just use a lock file and not run then. Or wait or whatever. > > There are 3 variables: > > - how many nodes are running > - how often they run > - how long to sleep between them > > you cannot optimise all 3 these variables, the old puppetcommander would skip > nodes that it cannot run in the time so it would always finish in the > allocated > time. This had the draw back that if you miss-specify your max run time or > just > can't be bothered to monitor it over time as you added nodes you can end up > with > nodes that just never run. > > the new runall algorithm will run all nodes as quick as it can, but cannot > guarantee > it will finish within your half an hour. This is the better algo since all > your nodes > will run and your master will not be DOS'd. But the important thing is that > all nodes > do get a run. > > So now you have to just decide what is important for you. If you would like > to know > when your concurrency and run times are such that it can't finish in 30 > minutes given > your master resources just make your cronjob write a lock file and notify > when the > next one finds a lock - you should then look why, maybe your manifests have > gotten > so slow that it cant possibly finish anymore. > > Or just run the thing in a loop forever let it make best efforts without > killing your > master - in this case a simple daemon will work. > > >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/cff5bbd6-c856-44c7-836c-9e98355eb2c7%40googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/52C30F4F.7010300%40jasonantman.com. For more options, visit https://groups.google.com/groups/opt_out.
