jenkins-bot has submitted this change and it was merged. Change subject: Introduce the notion of 'roles' ......................................................................
Introduce the notion of 'roles' This change refactors the Puppet code around the notion of 'roles' (a common Puppet pattern, also in use in operations/puppet). Roles are meant to facilitate the use of Mediawiki-Vagrant for sharing development environments amongst developers of some particular component or extension. Role classes are one level more abstract than module classes. They parametrize the provisioning of several related software components that together enable the machine to fulfill some particular function. Role classes do not take parameters themselves, but rather exist as a place for aggregating all parameters required for provisioning a virtual machine of a certain kind. Roles are enabled by including them in the node definition in site.pp. Declaring a role is not tantamount to enabling it -- it merely makes it available. Thus additional roles could be incorporated into this repository while keeping the basic setup quite generic and lightweight. My hope is that it would be rather straightforward for individual teams that use Vagrant internally to declare their development setup as role. Arthur Richards from the mobile team told me some moons ago that many of the bugs they ended up deploying were undetected by dint of differences in the development environment of various mobile developers. I think Vagrant + Puppet roles provide a possible solution. As part of this change, I moved various high-level configuration options from class parameter defaults to roles, to facilitate centralized management of key configuration options. Change-Id: I88ca30607c0ff91f4fcafa49a01765eb1013bb31 --- A puppet/manifests/base.pp D puppet/manifests/extras.pp A puppet/manifests/roles.pp M puppet/manifests/site.pp M puppet/modules/mediawiki/files/phpsh.sh M puppet/modules/mediawiki/manifests/init.pp M puppet/modules/mediawiki/manifests/phpsh.pp M puppet/modules/mediawiki/manifests/user.pp M puppet/modules/mediawiki/templates/mediawiki-apache-site.erb M puppet/modules/mediawiki/templates/rc.php.erb M puppet/modules/misc/manifests/init.pp M puppet/modules/mysql/manifests/init.pp M puppet/modules/mysql/templates/my.cnf.erb 13 files changed, 199 insertions(+), 132 deletions(-) Approvals: Ori.livneh: Looks good to me, approved jenkins-bot: Verified diff --git a/puppet/manifests/base.pp b/puppet/manifests/base.pp new file mode 100644 index 0000000..044ad41 --- /dev/null +++ b/puppet/manifests/base.pp @@ -0,0 +1,53 @@ +# == Base Puppet manifest +# +# This manifest declares resource defaults for the Mediawiki-Vagrant +# Puppet site. All Puppet modules bundled with this project have an +# implicit dependency on this manifest and the declarations it contains. +# Modify this file with care. +# +# For more information about resource defaults in Puppet, see +# <http://docs.puppetlabs.com/puppet/2.7/reference/lang_defaults.html>. +# + +# Declares a default search path for executables, allowing the path to +# be omitted from individual resources. Also configures Puppet to log +# the command's output if it was unsuccessful. +Exec { + path => [ '/bin', '/usr/bin', '/usr/sbin/', '/usr/local/bin' ], + logoutput => on_failure, +} + +# Ensure that apt-get update has ran in the last 24 hours before +# installing any packages. +Package { + require => Exec['update package index'], +} + +# Declare default uid / gid and permissions for file resources, and +# tells Puppet not to back up configuration files by default. +File { + backup => false, + owner => 'root', + group => 'root', + mode => '0644', +} + +# Emit a notice indicating whether or not the host's VirtualBox version +# was detected. See ../../Vagrantfile for the version detection code. +if ( $::virtualbox_version ) { + notice("Detected VirtualBox version ${::virtualbox_version}") +} else { + warning('Could not determine VirtualBox version.') +} + +# Run 'apt-get update' if the package index has not been updated in the +# last 24 hours. +exec { 'update package index': + command => 'apt-get update', + unless => 'bash -c \'(( $(date +%s) - $(stat -c %Y /var/lib/apt/periodic/update-success-stamp) < 86400 ))\'' +} + +# Configures a 'puppet' user group, required by Puppet. +group { 'puppet': + ensure => present, +} diff --git a/puppet/manifests/extras.pp b/puppet/manifests/extras.pp deleted file mode 100644 index 0a7f884..0000000 --- a/puppet/manifests/extras.pp +++ /dev/null @@ -1,13 +0,0 @@ -# == Puppet manifest for optional modules -# -# Uncomment lines below to enable Puppet modules that ship with Vagrant -# but are not enabled by default. -# -# Once uncommented, run 'vagrant up' and 'vagrant provision'. -# - -# Selenium browser tests for MediaWiki -# class { 'browsertests': } - -# User metrics API -# class { 'user_metrics': } diff --git a/puppet/manifests/roles.pp b/puppet/manifests/roles.pp new file mode 100644 index 0000000..c7945fb --- /dev/null +++ b/puppet/manifests/roles.pp @@ -0,0 +1,71 @@ +# == Roles for Mediawiki-Vagrant +# +# A 'role' represents a set of software configurations required for +# giving this machine some special function. If you'd like to use the +# Vagrant-Mediawiki codebase to describe a development environment that +# you could then share with other developers, you should do so by adding +# a role below and submitting it as a patch to the Mediawiki-Vagrant +# project. +# +# To enable a particular role on your instance, include it in the +# mediawiki-vagrant node definition in 'site.pp'. +# +# + + +# == Class: role::generic +# Configures common tools and shell enhancements. +class role::generic { + class { '::misc': } + class { '::git': } +} + +# == Class: role::mediawiki +# Provisions a MediaWiki instance powered by PHP, MySQL, and memcached. +class role::mediawiki { + $wiki_name = 'devwiki' + $server_url = 'http://127.0.0.1:8080' + $dir = '/vagrant/mediawiki' + + # Database access + $db_name = 'wiki' + $db_user = 'root' + $db_pass = 'vagrant' + + # Initial admin account + $admin_user = 'admin' + $admin_pass = 'vagrant' + + class { '::memcached': } + + class { '::mysql': + default_db_name => $db_name, + root_password => $db_pass, + } + + class { '::mediawiki': + wiki_name => $wiki_name, + admin_user => $admin_user, + admin_pass => $admin_pass, + db_name => $db_name, + db_pass => $db_pass, + db_user => $db_user, + dir => $dir, + server_url => $server_url, + } +} + +# == Class: role::browsertests +# Configures this machine to run the Wikimedia Foundation's set of +# Selenium browser tests for MediaWiki instances. +class role::browsertests { + class { '::browsertests': } +} + +# == Class: role::umapi +# Configures this machine to run the User Metrics API (UMAPI), a web +# interface for obtaining aggregate measurements of user activity on +# MediaWiki sites. +class role::umapi { + class { '::user_metrics': } +} diff --git a/puppet/manifests/site.pp b/puppet/manifests/site.pp index 1407086..9eb694a 100644 --- a/puppet/manifests/site.pp +++ b/puppet/manifests/site.pp @@ -1,57 +1,26 @@ -# == Puppet site manifest +# == Mediawiki-Vagrant Puppet Manifest # -# This manifest is the main entrypoint for the Puppet provisioner. It -# sets some default values for Puppet resource types and specifies which -# Puppet modules should be enabled. +# This manifest is the main entrypoint for Puppet, the configuration +# management tool that sets up this machine to run MediaWiki. The +# manifest specifies which classes of services should be enabled on this +# virtual machine. # +# By default, the Mediawiki-Vagrant virtual machine is configured to run +# a plain MediaWiki instance, with some small enhancements designed to +# make it easy to hack on MediaWiki code. However, MediaWiki-Vagrant +# knows how to configure a machine to fulfill various other roles which +# are not enabled by default. +# +# To enable an optional role, simply uncomment its delcaration below by +# removing the leading '#' symbol and saving this file. Then, run +# 'vagrant up' to ensure your machine is active, and then 'vagrant +# provision' to apply the updated configuration to your instance. +# +# +import 'base.pp' +import 'roles.pp' -# Declares a default search path for executables, allowing the path to -# be omitted from individual resources. Also configures Puppet to log -# the command's output if it was unsuccessful. -Exec { - path => [ '/bin', '/usr/bin', '/usr/sbin/', '/usr/local/bin' ], - logoutput => on_failure, +node 'mediawiki-vagrant' { + include role::generic + include role::mediawiki } - -# Ensure that apt-get update has ran in the last 24 hours before -# installing any packages. -Package { - require => Exec['update package index'], -} - -# Declare default uid / gid and permissions for file resources, and -# tells Puppet not to back up configuration files by default. -File { - backup => false, - owner => 'root', - group => 'root', - mode => '0644', -} - -# Emit a notice indicating whether or not the host's VirtualBox version -# was detected. See ../../Vagrantfile for the version detection code. -if ( $::virtualbox_version ) { - notice("Detected VirtualBox version ${::virtualbox_version}") -} else { - warning('Could not determine VirtualBox version.') -} - -# Run 'apt-get update' if the package index has not been updated in the -# last 24 hours. -exec { 'update package index': - command => 'apt-get update', - unless => 'bash -c \'(( $(date +%s) - $(stat -c %Y /var/lib/apt/periodic/update-success-stamp) < 86400 ))\'' -} - -# Configures a 'puppet' user group, required by Puppet. -group { 'puppet': - ensure => present, -} - -class { 'git': } -class { 'memcached': } -class { 'misc': } -class { 'mediawiki': } - -# Load any optional modules that have been enabled. -import 'extras.pp' diff --git a/puppet/modules/mediawiki/files/phpsh.sh b/puppet/modules/mediawiki/files/phpsh.sh index ede53f2..25126ba 100644 --- a/puppet/modules/mediawiki/files/phpsh.sh +++ b/puppet/modules/mediawiki/files/phpsh.sh @@ -8,7 +8,7 @@ [ -z "$BASH_VERSION" -o -z "$PS1" ] && return phpsh () { ( - cd /vagrant/mediawiki + cd $MW_INSTALL_PATH command phpsh "$@" ) } diff --git a/puppet/modules/mediawiki/manifests/init.pp b/puppet/modules/mediawiki/manifests/init.pp index 58e1730..93e15f1 100644 --- a/puppet/modules/mediawiki/manifests/init.pp +++ b/puppet/modules/mediawiki/manifests/init.pp @@ -1,55 +1,43 @@ # == Class: mediawiki # -# Provision a MediaWiki instance powered by MySQL, served by Apache, and -# customized for development work. +# Provision MediaWiki, MediaWiki is a free software open source wiki +# package written in PHP, originally for use on Wikipedia. # # === Parameters - -# [*wiki*] -# Wiki name (default: 'devwiki'). # -# [*admin*] -# User name for the initial admin account (default: 'admin'). +# [*wiki_name*] +# The name of your site (example: 'Wikipedia'). # -# [*pass*] -# Initial password for admin account (default: 'vagrant'). +# [*admin_user*] +# User name for the initial admin account. # -# [*dbname*] +# [*admin_pass*] +# Initial password for admin account. +# +# [*db_name*] # Logical MySQL database name. # -# [*dbuser*] -# MySQL user to use to connect to the database (default: 'root'). +# [*db_user*] +# MySQL user to use to connect to the database. # -# [*dbpass*] -# Password for MySQL account (default: 'vagrant'). +# [*db_pass*] +# Password for MySQL account. # -# [*server*] +# [*server_url*] # Full base URL of host (default: 'http://127.0.0.1:8080'). # -# === Examples -# -# class { 'mediawiki': -# wiki => 'mobiledevwiki', -# admin => 'mobiledev', -# pass => 'secret', -# } -# class mediawiki( - $wiki = 'devwiki', - $admin = 'admin', - $pass = 'vagrant', - $dbname = 'wiki', - $dbuser = 'root', - $dbpass = 'vagrant', - $server = 'http://127.0.0.1:8080', - $dir = '/vagrant/mediawiki', + $wiki_name, + $admin_user, + $admin_pass, + $db_name, + $db_pass, + $db_user, + $dir, + $server_url, ) { class { 'php': } class { 'phpsh': } - class { 'mysql': - dbname => $dbname, - password => $pass, - } apache::site { 'default': ensure => absent, @@ -59,7 +47,6 @@ remote => 'https://gerrit.wikimedia.org/r/p/mediawiki/core.git', directory => $dir, } - file { '/etc/apache2/sites-enabled/000-default': ensure => absent, @@ -77,7 +64,7 @@ before => Exec['mediawiki setup'], } - apache::site { 'wiki': + apache::site { $wiki_name: ensure => present, content => template('mediawiki/mediawiki-apache-site.erb'), } @@ -86,10 +73,9 @@ require => [ Exec['set mysql password'], Git::Clone['mediawiki'] ], creates => "${dir}/LocalSettings.php", cwd => "${dir}/maintenance/", - command => "php install.php ${wiki} ${admin} --pass ${pass} --dbname ${dbname} --dbuser ${dbuser} --dbpass ${dbpass} --server ${server} --scriptpath '/w'", + command => "php install.php ${wiki_name} ${admin} --pass ${pass} --dbname ${db_name} --dbuser ${db_user} --dbpass ${db_pass} --server ${server_url} --scriptpath '/w'", notify => Service['apache2'], } - exec { 'require extra settings': require => Exec['mediawiki setup'], @@ -103,9 +89,9 @@ } exec { 'set default database': - command => "echo 'database = \"${dbname}\"' >> /home/vagrant/.my.cnf", + command => "echo 'database = \"${db_name}\"' >> /home/vagrant/.my.cnf", require => [ Exec['mediawiki setup'], File['/home/vagrant/.my.cnf'] ], - unless => "grep 'database = \"${dbname}\"' /home/vagrant/.my.cnf", + unless => "grep 'database = \"${db_name}\"' /home/vagrant/.my.cnf", } apache::mod { 'alias': diff --git a/puppet/modules/mediawiki/manifests/phpsh.pp b/puppet/modules/mediawiki/manifests/phpsh.pp index e7dffde..a201d2c 100644 --- a/puppet/modules/mediawiki/manifests/phpsh.pp +++ b/puppet/modules/mediawiki/manifests/phpsh.pp @@ -4,6 +4,8 @@ # with the MediaWiki codebase. # class mediawiki::phpsh { + include mediawiki + package { [ 'python-pip', 'exuberant-ctags' ]: ensure => latest, } @@ -36,7 +38,7 @@ exec { 'generate-ctags': require => [ Package['exuberant-ctags'], Git::Clone['mediawiki'] ], - command => 'ctags --languages=php --recurse -f /vagrant/mediawiki/tags /vagrant/mediawiki', - creates => '/vagrant/mediawiki/tags', + command => "ctags --languages=php --recurse -f ${mediawiki::dir}/tags ${mediawiki::dir}", + creates => "${mediawiki::dir}/tags", } } diff --git a/puppet/modules/mediawiki/manifests/user.pp b/puppet/modules/mediawiki/manifests/user.pp index c671169..868486b 100644 --- a/puppet/modules/mediawiki/manifests/user.pp +++ b/puppet/modules/mediawiki/manifests/user.pp @@ -35,7 +35,7 @@ } exec { "mediawiki user ${username}": - cwd => '/vagrant/mediawiki/maintenance', + cwd => "${mediawiki::dir}/maintenance", command => "php createAndPromote.php ${options} ${username} ${password}", returns => [ 0, 1 ], require => Exec['mediawiki setup', 'set mediawiki install path'], diff --git a/puppet/modules/mediawiki/templates/mediawiki-apache-site.erb b/puppet/modules/mediawiki/templates/mediawiki-apache-site.erb index f6c08b1..beea814 100644 --- a/puppet/modules/mediawiki/templates/mediawiki-apache-site.erb +++ b/puppet/modules/mediawiki/templates/mediawiki-apache-site.erb @@ -16,7 +16,7 @@ <Directory /var/www/> </Directory> - <Directory /vagrant/mediawiki/> + <Directory <%= scope.lookupvar('mediawiki::dir') %>> # See <https://github.com/mitchellh/vagrant/issues/351>. EnableSendfile Off </Directory> @@ -26,8 +26,8 @@ RewriteEngine On RewriteRule ^/$ /w/index.php [R=301] - Alias /wiki "/vagrant/mediawiki/index.php" - Alias /w "/vagrant/mediawiki" + Alias /wiki "<%= scope.lookupvar('mediawiki::dir') %>/index.php" + Alias /w "<%= scope.lookupvar('mediawiki::dir') %>" ErrorLog "/vagrant/logs/apache-error.log" LogLevel error diff --git a/puppet/modules/mediawiki/templates/rc.php.erb b/puppet/modules/mediawiki/templates/rc.php.erb index 141a3d2..07b648d 100644 --- a/puppet/modules/mediawiki/templates/rc.php.erb +++ b/puppet/modules/mediawiki/templates/rc.php.erb @@ -11,9 +11,8 @@ switch ($___phpsh___codebase_mode) { case '': // MediaWiki codebase mode - chdir( '/vagrant/mediawiki' ); - include_once( '/etc/phpsh/rc.php' ); - include_once( '/vagrant/mediawiki/index.php' ); + chdir( '<%= scope.lookupvar("mediawiki::dir") %>' ); + include_once( '<%= scope.lookupvar("mediawiki::dir") %>/index.php' ); break; case 'none': // Vanilla PHP diff --git a/puppet/modules/misc/manifests/init.pp b/puppet/modules/misc/manifests/init.pp index c024cd6..702883a 100644 --- a/puppet/modules/misc/manifests/init.pp +++ b/puppet/modules/misc/manifests/init.pp @@ -28,10 +28,10 @@ } file { '/etc/update-motd.d/60-mediawiki-vagrant': - ensure => present, - mode => '0755', - source => 'puppet:///modules/misc/60-mediawiki-vagrant', - notify => Exec['update motd'], + ensure => present, + mode => '0755', + source => 'puppet:///modules/misc/60-mediawiki-vagrant', + notify => Exec['update motd'], } exec { 'update motd': diff --git a/puppet/modules/mysql/manifests/init.pp b/puppet/modules/mysql/manifests/init.pp index cd9778f..5ad81a1 100644 --- a/puppet/modules/mysql/manifests/init.pp +++ b/puppet/modules/mysql/manifests/init.pp @@ -5,23 +5,23 @@ # # === Parameters # -# [*password*] +# [*root_password*] # Password for the root MySQL account. # -# [*dbname*] +# [*default_db_name*] # If defined, the 'mysql' command-line client will be configured to # use this database by default (default: undefined). # # === Examples # # class { 'mysql': -# password => 'r00tp455w0rd', -# dbname => 'wiki', +# root_password => 'r00tp455w0rd', +# default_db_name => 'wiki', # } # class mysql( - $password = 'vagrant', - $dbname = undef, + $root_password, + $default_db_name = undef, ) { package { 'mysql-server': @@ -35,8 +35,8 @@ } exec { 'set mysql password': - command => "mysqladmin -u root password \"${password}\"", - unless => "mysqladmin -u root -p\"${password}\" status", + command => "mysqladmin -u root password \"${root_password}\"", + unless => "mysqladmin -u root -p\"${root_password}\" status", require => Service['mysql'], } diff --git a/puppet/modules/mysql/templates/my.cnf.erb b/puppet/modules/mysql/templates/my.cnf.erb index 9a382aa..6caa01f 100644 --- a/puppet/modules/mysql/templates/my.cnf.erb +++ b/puppet/modules/mysql/templates/my.cnf.erb @@ -3,14 +3,14 @@ # [client] user = "root" -password = "<%= @password %>" +password = "<%= @root_password %>" [mysql] pager = "less -inSFX" prompt = "(\R:\m)\_\u@\h:[\d]>\_" -<% if @dbname %> -database = "<%= @dbname %>" +<% if @default_db_name %> +database = "<%= @default_db_name %>" <% end %> comments = true show-warnings = true -- To view, visit https://gerrit.wikimedia.org/r/63262 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I88ca30607c0ff91f4fcafa49a01765eb1013bb31 Gerrit-PatchSet: 2 Gerrit-Project: mediawiki/vagrant Gerrit-Branch: master Gerrit-Owner: Ori.livneh <[email protected]> Gerrit-Reviewer: Mattflaschen <[email protected]> Gerrit-Reviewer: Ori.livneh <[email protected]> Gerrit-Reviewer: Spage <[email protected]> Gerrit-Reviewer: Yurik <[email protected]> Gerrit-Reviewer: jenkins-bot _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
