We have been using jenkins for close to everything lately...we have a 7 node (1 master 6 slaves) cluster that essentially replaced all of our cron servers, so now we have 1 console that manages all the automated tasks between our colocation facility and office...this is about 300 jobs
We have another standalone jenkins install that does puppet syntax checking, and then does push based puppet runs on about 300 servers (we dont run puppet in daemon mode) we have another jenkins server that is for our dev group to do all their automated builds, pushes to QA, some automated tasks and things like that then we have another 7 or so jenkins servers for various other automated tasks that we wanted separated for various reasons...all together we probably have close to 1000 jobs setup..they range across the board..running automated SQL reports and emailing the results, deploying code to production, backing up load balancer configs, restarting services....etc jenkins has to be one of the best tools to come out in recent years, and you cant beat the price..we are basically using it for anything we want to run more than once...on a schedule or manually Mike On Tue, Jan 15, 2013 at 5:34 AM, Saj Goonatilleke <[email protected]> wrote: > Hi Adam, > > On Mon Jan 14, 2013 at 19:41:48 -0500, Adam Moskowitz wrote: > > Is anyone using continuous build or continuous integration tools -- > > stuff like Jenkins or Bamboo or CruiseControl [1] -- for sysadmin tasks? > > If so, can you say what kinds of tasks you're doing with these tools? > > I'm one of a band of merry sysadmins who use BuildBot to provision > servers. > > Over time, we started provisioning so many of the darned things by hand > that, one day, we decided to invest in (almost) completely robotising > our provisioning process. We have had Kickstart (and CFEngine, later > Puppet) from Day One, but Kickstart couldn't power our machines on by > itself. Neither could it click through the monotonous forms on our > asset management and billing systems, or help us satisfy our > documentation requirements. All up, dull auxiliary tasks were consuming > hours of operator time a pop! > > Fast forward. Our entire provisioning process was automated end-to-end. > New problem: it only takes a single misbehaving mirror or a subtle > problem in a change to a firewall configuration to trip up a robot [1]. > When performing or supervising this work by hand, people don't tend to > dwell on little hiccups: they get fixed and we move on. For the first > time, we understood the significant risk our volatile environment was > placing on our automation goals. > > The problem was largely solved by replicating a subset of our live > infrastructure (DNS nameservers and resolvers; DHCP+TFTP servers; proxy > servers; Puppetmasters; backup servers; availability, performance and > intrusion monitoring systems; etc.) to a mostly isolated test > environment. BuildBot pushes make-believe server orders with varying > desired specifications through this test environment 24 hours a day > looking for failure points that we are not already monitoring. > Minute-by-minute changes from our configuration management system get > applied here, too, so we can proactively address changes that have > introduced subtle unwanted side effects on the provisioning process. > > Don't just chuck your code into the CI wringer; chuck in your whole > distributed operating environment, too. :) > > 1: http://what-if.xkcd.com/5/ > _______________________________________________ > Discuss mailing list > [email protected] > https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss > This list provided by the League of Professional System Administrators > http://lopsa.org/ >
_______________________________________________ Discuss mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss This list provided by the League of Professional System Administrators http://lopsa.org/
