Ori.livneh has uploaded a new change for review.
https://gerrit.wikimedia.org/r/262846
Change subject: Add testreduce module and role
......................................................................
Add testreduce module and role
testreduce is Parsoid's round-trip test result aggregator and web UI. It
comprises a test coordination server that hands out test requests to
testing clients, accepts test results from them, and records the results
in a database, as well as a web interface for examining the test
results.
This patch adds a testreduce module and a minimal role. It hews closely
to how testreduce is currently configured on ruthenium.
Bug: T118778
Change-Id: Icd39e880ea4d0fec3e68c45d39521fb07f9dff7e
---
A manifests/role/testreduce.pp
M manifests/site.pp
A modules/testreduce/files/testreduce.upstart.conf
A modules/testreduce/manifests/init.pp
A modules/testreduce/templates/settings.js.erb
5 files changed, 197 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/46/262846/1
diff --git a/manifests/role/testreduce.pp b/manifests/role/testreduce.pp
new file mode 100644
index 0000000..12974c6
--- /dev/null
+++ b/manifests/role/testreduce.pp
@@ -0,0 +1,11 @@
+# == Class: role::testreduce
+#
+# Parsoid round-trip test result aggregator.
+#
+class role::testreduce {
+ class { 'testreduce':
+ db_name => 'testreduce_0715',
+ db_user => 'testreduce',
+ db_pass => '', # FIXME
+ }
+}
diff --git a/manifests/site.pp b/manifests/site.pp
index b90dc46..ef4cc32 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -2317,7 +2317,7 @@
# ruthenium is a parsoid regression test server
# https://www.mediawiki.org/wiki/Parsoid/Round-trip_testing
node 'ruthenium.eqiad.wmnet' {
- role testsystem
+ role testsystem, testreduce
}
# T95046 install/deploy scandium as zuul merger (ci) server
diff --git a/modules/testreduce/files/testreduce.upstart.conf
b/modules/testreduce/files/testreduce.upstart.conf
new file mode 100644
index 0000000..543788b
--- /dev/null
+++ b/modules/testreduce/files/testreduce.upstart.conf
@@ -0,0 +1,34 @@
+# vim: set ft=upstart:
+
+description "testreduce HTTP service"
+
+start on (local-filesystem and net-device-up IFACE!=lo)
+stop on runlevel [!2345]
+
+# up ulimit -n a bit
+limit nofile 10000 10000
+
+setuid "testreduce"
+setgid "testreduce"
+
+env DEFAULTFILE=/etc/default/testreduce
+
+# Basic built-in defaults. Overridden by whatever
+# is defined in the DEFAULTFILE defined above.
+env NODE_PATH="/srv/testreduce/node_modules"
+env BASE_PATH="/srv/testreduce/server"
+env LOG_FILE="/dev/null"
+env SETTINGS_FILE="/etc/testreduce/settings.js"
+env DAEMON_ARGS="--config $SETTINGS_FILE"
+env PORT="8001"
+
+respawn
+
+script
+ if [ -f "$DEFAULTFILE" ] ; then
+ . "$DEFAULTFILE"
+ fi
+ chdir "$BASE_PATH"
+ echo "Starting testreduce on port $PORT"
+ exec /usr/bin/nodejs server.js $DAEMON_ARGS < /dev/null >> "$LOG_FILE" 2>&1
+end script
diff --git a/modules/testreduce/manifests/init.pp
b/modules/testreduce/manifests/init.pp
new file mode 100644
index 0000000..a645d2d
--- /dev/null
+++ b/modules/testreduce/manifests/init.pp
@@ -0,0 +1,84 @@
+# == Class: testreduce
+#
+# Parsoid round-trip test result aggregator.
+#
+# === Parameters
+#
+# [*db_name*]
+# Database name for storing results.
+#
+# [*db_user*]
+# Database user.
+#
+# [*db_host*]
+# MySQL host. Default: 'localhost'.
+#
+# [*db_port*]
+# MySQL port. Default: 3306.
+#
+# [*coord_port*]
+# The result aggregator will listen on this port. Default: 8002.
+#
+# [*webapp_port*]
+# The user-facing webapp that displays test results will listen on
+# this port. Default: 8003.
+#
+class testreduce(
+ $db_name,
+ $db_user,
+ $db_pass,
+ $db_host = 'localhost',
+ $db_port = 3306,
+ $coord_port = 8002,
+ $webapp_port = 8003,
+) {
+ require_package('nodejs')
+ require_package('npm')
+
+ group { 'testreduce':
+ ensure => present,
+ system => true,
+ }
+
+ user { 'testreduce':
+ gid => 'testreduce',
+ home => '/srv/testreduce',
+ managehome => false,
+ system => true,
+ }
+
+ file { '/etc/testreduce':
+ ensure => directory,
+ owner => 'root',
+ group => 'root',
+ mode => '0755',
+ }
+
+ file { '/etc/testreduce/settings.js':
+ content => template('testreduce/settings.js.erb'),
+ owner => 'root',
+ group => 'root',
+ mode => '0444',
+ notify => Service['testreduce'],
+ }
+
+ file { '/etc/init/testreduce.conf':
+ source => 'puppet:///modules/testreduce/testreduce.upstart.conf',
+ owner => 'root',
+ group => 'root',
+ mode => '0444',
+ notify => Service['testreduce'],
+ }
+
+ service { 'testreduce':
+ ensure => running,
+ provider => upstart,
+ }
+
+ git::clone { 'mediawiki/services/parsoid/testreduce':
+ ensure => latest,
+ owner => 'root',
+ group => 'wikidev',
+ directory => '/srv/testreduce',
+ }
+}
diff --git a/modules/testreduce/templates/settings.js.erb
b/modules/testreduce/templates/settings.js.erb
new file mode 100644
index 0000000..608c376
--- /dev/null
+++ b/modules/testreduce/templates/settings.js.erb
@@ -0,0 +1,67 @@
+/*
+ * This is a sample configuration file.
+ * Copy this file to server.settings.js and edit that file to fit your
+ * MySQL connection and other settings.
+ *
+ * You can also override these settings with command line options, run
+ * $ node server.js --help
+ * to view them.
+ */
+
+module.exports = {
+ // Hostname of the database server.
+ host: "<%= @db_host =>",
+
+ // Port number to use for connection.
+ port: <%= @db_port =>,
+
+ // Database to use.
+ database: "<%= @db_name %>",
+
+ // User for MySQL login.
+ user: "<%= @db_user %>",
+
+ // Password.
+ password: "<%= @db_pass %>",
+
+ // Output MySQL debug data.
+ debug: false,
+
+ // Number of times to try fetching a page.
+ fetches: 6,
+
+ // Number of times an article will be sent for testing before it's
+ // considered an error.
+ tries: 6,
+
+ // Time in seconds to wait for a test result. If a test takes longer
than
+ // this, it will be considered a crash error. Be careful not to set
this to
+ // less than what any page takes to parse.
+ cutofftime: 600,
+
+ // Number of titles to fetch from database in one batch.
+ batch: 50,
+
+ // Ports for the webapp (for test results, regressions, etc)
+ // and the co-ordinator app (for clients requesting titles, posting
results)
+ // Use non-conflicting ports here.
+ webappPort: <%= @webapp_port %>,
+ coordPort: <%= @coord_port %>,
+
+ // (Optional) Remote server, if any, that will generate full results
+ resultServer: "http://parsoid-tests.wikimedia.org/parsoid/";,
+
+ // (Optional) Localhost server, if any, for generating the same results
+ localhostServer: "http://localhost:8000/";,
+
+ // (Optional)
+ generateTitleUrl: function(server, prefix, title) {
+ return server.replace(/\/$/, '') + "/_rt/" + prefix + "/" +
title;
+ },
+
+ // (Optional)
+ perfConfig:
require('/usr/lib/testreduce/server/server.perf_stats.js').perfConfig,
+
+ // (Optional)
+ parsoidRTConfig:
require('/usr/lib/testreduce/server/server.parsoid_rt.js').parsoidRTConfig,
+};
--
To view, visit https://gerrit.wikimedia.org/r/262846
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icd39e880ea4d0fec3e68c45d39521fb07f9dff7e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ori.livneh <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
