On 2014-11-13 at 00:42 +0000, Edward Ned Harvey (lopser) wrote:
> If I could smoke a crazy pipe and make a wish for what I wanted to do, it
> would be this - I would build a system, perhaps on AWS, Vmware, whatever.
> Get it configured and working. And then tell some tool to basically snapshot
> that machine's configuration, including list of packages installed, and their
> configurations, and all the other stuff that defines the machine state... In
> most of my environments, I have some ability to actually snapshot the machine
> storage, and then spin up clones of the machine. But I have to keep a
> documented procedure of how the original config was created, so it's not
> magic "special sauce." But snapshotting & cloning the storage is undesirable
> because it is not portable. I'd like to build a VM on my local vmware or
> virtualbox or whatever, and then essentially clone it to AWS or vice versa...
> Make some change on a development machine, test it, and then after it's
> validated, replicate that change to the production environment by sending
> essent
ia
> lly the snapshot differential of the configuration.
>
> For some reason, this is what I thought puppet/chef/etc did. Am I wrong? Is
> this a pipe dream?
[disclaimer: my employer offers cloud services in this space, but not as
a fit for what you're asking for right now, so I'm not going to further
mention us]
If you want to build manually, then snapshot: Docker
Pros: You can build manually
Cons: You built manually, and have no idea if you can recreate when it
comes time to apply some system updates.
You still need to set up something to run Docker in the VM.
If you want to script the build, but then end up with something which
can be used in vmware or virtualbox or AWS or whatever: Packer
Pros: You can script the setup with whatever you want; typically the
top-level JSON sequence of commands will invoke something like
Chef.
You get out binaries for different platforms.
Cons: Still no ongoing updates.
Note that both Docker and Packer are written in Golang, which is a quite
excellent systems programming language.
https://www.packer.io/
We use Packer to build our base images, with Chef pre-installed. Those
are targetted at various backends. To use AWS as an example, we upload
AMIs and record the id numbers, so that a deployment run can ensure
boxes are built from the correct initial OS/framework.
You don't want to download/build stuff from the Internet in Prod, so
using PXE and Kickstart to do more than apply configuration and fetch
your own pre-built packages is a mistake. Packer just continues this,
to where you can make arbitrary system changes to get a system image
which is usable as-is for everything except final role tuning (and if
you want, you can do that in Packer too).
We _then_ use Chef as part of the process for ongoing maintenance of the
core system, but using some in-house customisations. Chef is what
updates the images, but isn't what decides if new images, or changed
counts, are necessary.
For us, we use Chef because enough of the programmers already had Ruby
experience that it was the sane choice; back when we were choosing and I
was the main Ops person, it was far more important to pick something
which we could persuade the people without any Ops background to support
than to pick something perfect for those with an prior Ops experience.
I shrugged and learnt Chef.
Prior job it was Puppet, prior to that it was in-house and I learnt
Python there. Prior to that, I moved us from entirely ad-hoc "do the
usual things to secure a box" to having a baseline install floppy (!!)
with stuff to run immediately after the system disk, which would get us
to the baseline with secured state and remote SSH access for NOC staff,
and then customizations could be run.
The most important step is rigour. It comes even before deploy
automation. Keeping logbooks, and being stringent, lets you figure out
what is done to set something up and lets you repeat, and sanity-check,
and spot commonality, and then start to automate. Using automated
deploy systems follows on naturally from logbooks written in the style
of shell commands to run. The logbooks which you've said you're already
doing get you in the mindset of rigour, and after that the deploy
automation is just the same thing sysadmins have always done, with
automating repetitive tasks.
It's just that we've moved from a world where machines are set up rarely
and repetitive tasks are scripts you write for users on those machines,
to a world where "machines" should be spun up in at most minutes, are
spun up frequently, and the end-users normally don't have accounts on
those machines. So the focus of the sysadmin automation moves to
earlier in the life of the machine. It's the same job, just applied in
a different world.
-Phil
_______________________________________________
Tech mailing list
Tech@lists.lopsa.org
https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
