Happy anniversary to you all !!
On 25/06/14 18:53, Ashley Penney wrote:
The 1st anniversary of the module team!
Hello from the module team here at Puppet Labs! I’m starting this
email with a lie because I’m not sure exactly when our first
anniversary really is, but I started on the 1st of July and the team
had only just gotten started, so that’s as good a date as any.
For those readers who are unaware, the module team at Puppet Labs
exists primarily to implement the supported modules initiative. For
anyone that missed the announcement last year, the goal of supported
modules is to help you more easily discover amazing modules and offer
support for those modules to Puppet Enterprise customers. Over the
last year we’ve been laying the foundations to make this sustainable
(and making it up as we go along). In order to support modules across
the diverse set of platforms PE runs on, we’ve had to experiment with
and learn how to test modules in a sustainable, scalable way, and over
the last year we’ve been trying to accomplish that.
Members of the team
Before we talk about what we’ve been doing over the last year, I
thought it would be nice to briefly talk about who is in the team, our
backgrounds, and where you can get hold of us. I’ll list everyone in
the order they joined the team to make life easy for me.
Hunter Haugen
Hunter was the very first member of the team and many of you know him
as “hunner” on IRC. Previously a member of the Professional Services
team, Hunter spent his time traveling and visiting customers all over
the world. His background, like mine, is mostly UNIX systems
administration. He’s responsible for the huge refactoring of the
apache module a while back, and is all over the popular puppetlabs
modules we hope you’re all using.
Ashley Penney
This one is me. I’m “ashp” on IRC and hopefully I know many of you.
I’ve been a Puppet user since the start of 2008, when I spent most of
my time harassing Luke on IRC over “bugs” I found that turned out to
be fundamental design decisions. I’ve been in operations for ~12
years, and this is the only job I’ve ever had where nobody will wake
me up at 0300 to let me know everything has crashed and the world is
on fire. It’s pretty awesome.
Chris Hoge
Anyone here who has used the openstack modules can thank Chris for
putting in a ton of work to make them awesome. Just before I took
this job, I tried to use the puppetlabs openstack modules and after a
week I threw up my hands and gave up as nothing worked. Now they
actually work and are awesome. Progress! Chris primarily focuses on
openstack, but he sometimes has to wrestle modules that are
dependencies into shape (like mongodb!). You can find him as
“hogepodge” on IRC.
Travis Fields
Travis joined to help the module team build out and build up awesome
modules specifically for Windows. The rest of us are Linux users, so
we often just threw up our hands and said “I can’t fix that!” when
modules had issues on Windows. Since joining the team, he’s taken
over the reboot and registry modules, fixed vcsrepo to work on
windows, taken on the new acl module, as well as fixed a number of
issues throughout our tooling to make sure Windows is a true first
class platform for modules instead of something we hide under the bed
from. Travis goes by “cyberious” on IRC.
Morgan Haskel
Morgan previously worked with Onyxpoint (a long time Puppet community
member!) on Puppet modules. Battle-scarred from forcing complex
modules into behaving properly, she joined Puppet Labs to help us
write amazing supported modules. She’s brought some adult supervision
to the team and ensures we’re on a regular cadence for module
releases. You can ask her questions about Hadoop (she’ll love it, I
promise) on IRC as “_morgan”.
AJ Johnson
The almost-newest member of the team is our boss; he's in charge of
ensuring we’re all pointing in the right direction and focused on
actually building things the community benefits from. He escaped from
IBM to come wrangle the team into a semblance of order and make sure
we’re on track to deliver supported modules!
Colleen Murphy
The actual-newest member of the team comes to us for the summer as an
intern from PSU (that’s the portland one, not the Pennsylvania one).
She’s a Linux sysadmin, Puppet user, and developer, and she is
already helping us tackle a project we’ve been putting off for months.
You can find her on IRC as “crinkle”. If you’re igalic or blkperl
then I preemptively ban you from asking her for PR merges! :)
Other People
This is already longer than an Oscar acceptance speech, so I want to
wrap up by just saying that we have a bunch of other fantastic people
that help us keep this show on the road. Lauren Rother helps ensure
modules have documentation that makes sense, Heidi Pio shouts at us
when we don’t close JIRA tickets, Justin Stoller makes the CI
environment work, John Duarte shakes his head at our attempts at risk
assessments and testing plans, Ryan Coleman helps us figure out what
we’re even meant to be building in the first place, and I’m sure I’ve
forgotten someone who is going to be so mad at me.
What we’ve been doing
Onwards to something more of you might care about: what is it we DO
around here? Over the last year we’ve been focused on two main
things: refactoring, rewriting, and testing existing modules, and
helping shape the tools, documentation, and environment to make
testing hundreds of modules feasible. The time has flown by and I
think the entire team would like to have released more modules than we
did, but we’ve managed to get a lot of the groundwork done.
First four months
In the first four months Hunter and I ran rampant over the modules
trying to deal with some of the backlog of PRs, feature requests, and
rewrites that were needed. We rewrote the mysql and rabbitmq modules,
refactored a bunch of smaller helper modules (passenger, etc), and
merged or closed hundreds (and hundreds) of PRs that had been
outstanding (for years in some cases). We tried to aggressively manage
the backlog of bug reports and just generally get the modules into
some better shape.
Alongside this work, we started talking over standards and what
constituted a well-written module in our opinion. As Puppet users,
obviously we all disagreed, but over time we’ve managed to wrestle
consensus down to what we consider to be reasonably state of the art
in modules.
Next four months
The story of the next four months was acceptance testing. We started
out with a bunch of tests we had written in rspec-system during the
initial refactoring efforts of the first four months, but we switched
to beaker-rspec (https://github.com/puppetlabs/beaker-rspec) in order
to handle our acceptance tests. This allows us to run `rspec
spec/acceptance` and get virtual machines automatically spun up,
manifests applied, the results checked through various shell commands,
and then have the virtual machines killed off. This allowed us to
test the true behaviors of modules for the first time rather than
testing the contents of the puppet catalog.
In these four months we added thousands of tests across the modules
that were to be supported. These tests exposed issues on many
platforms and over the months we worked to try and deal with those issues.
Alongside this work, we continued trying to handle merging PRs in, as
well as the odd bit of refactoring to types and providers as we went.
We wrote the “Beginners Guide to Modules
<http://docs.puppetlabs.com/guides/module_guides/bgtm.html>” as well
as continued to work on making sure Beaker works well for community
members. We started rebuilding the vagrant boxes and uploading those
so everyone had recent operating systems to test against. These
eventually transitioned off tohttps://vagrantcloud.com/puppetlabs/.
Last four months
Our most recent four months have also involved testing. As the number
of acceptance tests grew and grew, it became harder and harder to get
our full set of tests to pass. We continued work on managing the
modules in general, including an almost full rewrite of the SOAP based
f5 module. We also wrote the “Advanced Guide to Modules” which is
still in the editing stage, but attempts to walk you through building
a module from the ground up (with types and providers, facts,
functions, unit tests, acceptance tests, all of it).
Faced with the increasing amount of time we’re spending focused on
testing we’ve put together a plan to fix things. I won’t give you all
the tedious details but we’re going to change how we currently
acceptance test, and it’ll affect anyone who contributes to our
modules. Once we had our hammer of beaker, everything looked like a
nail and we started doing unit level tests in the acceptance
framework. Things like “checking the rpm is really installed” for
postgres rather than just checking “postgres is running”. We’re going
to rework these acceptance tests to have a much smaller number of end
to end tests. We think this will snowball over the next four months
and really help us increase our speed when it comes to development work.
What we’re doing right now
The good news is we’re writing modules! I’m writing a drastically
improved REST based f5 module that will replace the SOAP module in
time. Morgan has written a tomcat module that handles multiple
installs of tomcat on a single server, compiled from source if needed,
and we’re preparing to release that next week. Travis is working on a
powershell module. Hunter is working on recovering from being sick,
as well as improvements to the haproxy module.
Alongside modules, we’re (I say 'we', I mean Colleen is) also working
on a project to allow us to centrally manage files that are common
across modules. Things like the .travis.yml or Gemfile. To do that
we need to account for local module differences, however, and Colleen
has written us a framework that allows us to manage these files
properly. It’ll mean we can update .travis.yml in a single repository
and then push it out automatically to all our modules. This has been
something that’s made certain mass changes needed for testing really
difficult.
I’m also working on a fixtures module which is temporarily living
athttps://github.com/apenney/pe_fixtures. This will eventually
contain a `facter -p` run for every single PE platform we have, and
exists to be used with rspec-puppet. Right now, if you want to make
sure your module fails gracefully on unsupported platforms you have to
go mock up the facts for all those platforms by hand. When we finish
this work, you’ll be able to easily iterate over all platforms and
check how your module performs on them with rspec-puppet.
Things we’d like to do
Sadly, I can’t give away all our secrets here! We’re continuing to
work on making it easier for community members to author modules. Our
first year was spent focused on figuring out the answers to questions
we know many of you have, and now we want to spend some time to make
sure we fully document and expose those answers so everyone can
benefit. We’ve got some other stuff in the pipeline to make it easier
to find high quality modules, as well as to write high quality
modules. We hope you bear with us as we continue to get all the
pieces in place to allow us to scale to hundreds of support modules,
modules hopefully for everything you need to do under the sun. I
spent from 2008-2013 writing private modules for a variety of
companies that I couldn’t release and every new job meant rewriting
the same things again. I came to Puppet Labs (hell, I begged them
over and over to create this team) so that we can write these modules
once and have everyone contribute to making them amazing, so that you
never have to write the same module twice in your career. We won’t
stop until you’re never faced with the horror of writing a module for
some boring piece of open source software, freeing you up to do things
that you actually care about. :)
--
Ashley Penney
[email protected] <mailto:[email protected]>
Module Engineer
*Join us at PuppetConf 2014**, September 23-24 in San Francisco -
http://puppetconf.com <http://puppetconf.com/>*
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAC9eg%2B%3DJh9J7yxN5O8nJDeCPb%3Do0NRcf7ZW%3D5rWozD6e7izH_g%40mail.gmail.com
<https://groups.google.com/d/msgid/puppet-users/CAC9eg%2B%3DJh9J7yxN5O8nJDeCPb%3Do0NRcf7ZW%3D5rWozD6e7izH_g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Johan De Wit
Open Source Consultant
Red Hat Certified Engineer (805008667232363)
Puppet Certified Professional 2013/2014 (PCP0000006)
_________________________________________________________
Open-Future Phone +32 (0)2/255 70 70
Zavelstraat 72 Fax +32 (0)2/255 70 71
3071 KORTENBERG Mobile +32 (0)474/42 40 73
BELGIUM http://www.open-future.be
_________________________________________________________
Next Events:
Linux Training | https://www.open-future.be/linux-training-8-till-12th-september
Puppet Introduction Course |
https://www.open-future.be/puppet-introduction-course-15th-september
Puppet Fundamentals Training | https://www.open-future.be/puppet-fundamentals-training-16-till-18th-september
Zabbix Certified Specialist | https://www.open-future.be/zabbix-certified-specialisttraining-22-till-24th-september
Zabbix Certified Professional |
https://www.open-future.be/zabbix-certified-professional-training-25-till-26th-september
Subscribe to our newsletter | http://eepurl.com/BUG8H
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/53ABC8DF.6040904%40open-future.be.
For more options, visit https://groups.google.com/d/optout.
