(bcc: infra@)
TL;DR: the goal of this VM is to make it so that everybody on the Whimsy
PMC can deploy both changes and new applications. This is a progress
report.
---
For starters, here is the new website on a new VM:
https://whimsy-test.apache.org/
The new VM is based on Ubuntu 14.04 (the current Whimsy is based on
Ubuntu 12.04), and is more completely managed by Puppet. The puppet
definition is contained in the following two files:
Content:
https://github.com/apache/whimsy
VM Definition:
https://github.com/apache/infrastructure-puppet/blob/deployment/data/nodes/whimsy-vm2.apache.org.yaml
https://github.com/apache/infrastructure-puppet/blob/deployment/modules/whimsy_server/manifests/init.pp
Instructions:
https://github.com/rubys/puppet-kitchen#readme
https://github.com/apache/infrastructure-puppet/blob/deployment/modules/vhosts_whimsy/README.md
Workflow:
https://cwiki.apache.org/confluence/display/INFRA/Git+workflow+for+infrastructure-puppet+repo
---
Applications consist of static data, CGI scripts, Rack applications, and
a single Puma application (the board agenda, which has some unique
response time requirements), and cron jobs. Current status:
(*) static content is working. Changes pushed to GitHub master will be
automatically deployed every 30 minutes. Note that this includes the
contents of scripts and applications too.
(*) CGI scripts are working, but dependencies (gems, svn checkouts) have
not been installed. A simple CGI:
https://github.com/apache/whimsy/blob/master/www/test.cgi
https://whimsy-test.apache.org/test.cgi
Many CGI scripts will require user authentication. This is done by
adding a single line to the deployment data identifying the location of
the script:
https://github.com/apache/infrastructure-puppet/blob/deployment/data/nodes/whimsy-vm2.apache.org.yaml#L65
Note that the LDAP module does not currently handle boolean conditions
(example: members or officers). The way to handle this is to do
authentication in two passes. The first pass will be done by the Apache
web server, and verify that the user is a part of the most inclusive
group (typically: committers). The CGI scripts that need to do more
will need to
(*) Rack applications are working and run under Passenger under Apache
httpd. Again, dependencies are not yet ready. In addition to simply
checking the application, one line per passenger application needs to be
added to the deployment data:
https://github.com/apache/infrastructure-puppet/blob/deployment/data/nodes/whimsy-vm2.apache.org.yaml#L60
A simple rack application (two empty directories, and a one line file):
https://github.com/apache/whimsy/tree/master/www/racktest
https://whimsy-test.apache.org/racktest
Authentication requirements will also need to be two phase, like with
CGI above; but more common conditions can be handled at the "Rack" level
instead of at the application level making use of Rack middleware such as:
https://github.com/apache/whimsy/blob/master/lib/whimsy/asf/rack.rb#L56
(*) I've not started work on deploying the Puma application, but have
verified that there is a Puppet module that handle do the necessary
deployment.
(*) I've not started work on cron jobs. The definitions for these are
likely to be placed in the infrastructure-puppet repository, with the
code that are executed by the cron jobs placed in the whimsy repository.
---
The biggest todos are:
1) get a handle on how to test LDAP changes locally. LDAP is installed
on whimsy-vm2, but not on local puppet-kitchen-vagrant repositories.
2) define a strategy for maintaining checkout data (mostly SVN) which is
needed by multiple applications. For production, that will mean
deployment on things like credentials for the whimsysvn user. For local
testing that likely will mean mounting local directories by the VM.
3) define a strategy for maintaining dependencies. Whimsy (as a site)
hosts multiple independent applications, each of which may have
different requirements. I'd like to make it so that not every
application has to be upgraded at the same time when a new version of a
dependency comes out. This likely will involve the use of bundler
(https://forge.puppetlabs.com/ploperations/bundler).
4) deploy puma (https://forge.puppetlabs.com/deversus/puma)
5) set up cron jobs.
---
A big shout out to to the infrastructure team for helping me get this
far. It has been a team effort with help from (at a minimum) Chris,
Daniel, David, Geoff, and Tony. With the investment in Puppet
automation, replacing a dependency, the OS, or even the VM entirely
should be relatively painless.
- Sam Ruby
