ArielGlenn has submitted this change and it was merged.
Change subject: beta: manage parsoid using upstart
......................................................................
beta: manage parsoid using upstart
On the beta cluster, we want to restart parsoid via a ssh command using
`/etc/init.d/parsoid restart`. That eventually calls a background task
which cause the file descriptors to be kept open by the spawned child
process. To prevent data loss, ssh does not close its connection and
hang.
This patch:
* converts the init script to an Ubuntu upstart service and make use it
on the beta cluster. Production is left unchanged, the old bits in
role::parsoid::common have been moved to the production class.
* logrotates /var/log/parsoid/parsoid.log. Limiting size to 1GB in case
some bugs cause a ton of spam ending up filling the harddrive.
* writes stdout/stderr to /var/log/parsoid/parsoid.log. Now configured
using /etc/default/parsoid variable PARSOID_LOG_FILE
Finally allow ssh connections from the deployment-bastion.pmtpa.wmflabs
instance (10.4.0.58) by adding a ferm::rule.
bug: 57233
Change-Id: I3d4a49b37e6759ae222ab2fdb698576228dcc96e
---
A files/misc/parsoid.upstart
M manifests/role/parsoid.pp
A templates/misc/parsoid.default.erb
A templates/misc/parsoid.logrotate.erb
4 files changed, 126 insertions(+), 16 deletions(-)
Approvals:
ArielGlenn: Looks good to me, approved
jenkins-bot: Verified
diff --git a/files/misc/parsoid.upstart b/files/misc/parsoid.upstart
new file mode 100644
index 0000000..a3b9a08
--- /dev/null
+++ b/files/misc/parsoid.upstart
@@ -0,0 +1,35 @@
+# vim: set ft=upstart:
+
+#####################################################################
+### THIS FILE IS MANAGED BY PUPPET
+### puppet:///files/misc/parsoid.upstart
+#####################################################################
+
+description "Parsoid HTTP service"
+
+start on (local-filesystem and net-device-up IFACE!=lo)
+stop on runlevel [!2345]
+
+# up ulimit -n a bit
+limit nofile 10000 10000
+
+setuid "parsoid"
+setgid "parsoid"
+
+env DEFAULTFILE=/etc/default/parsoid
+env VCAP_APP_PORT="8000"
+env NODE_PATH="/var/lib/parsoid/Parsoid/js/node_modules"
+
+respawn
+
+script
+ if [ -f "$DEFAULTFILE" ] ; then
+ . "$DEFAULTFILE"
+ fi
+ if [ "x$PARSOID_LOG_FILE" = "x" ] ; then
+ PARSOID_LOG_FILE='/dev/null'
+ fi
+
+ chdir "/var/lib/parsoid/Parsoid"
+ exec /usr/bin/nodejs js/api/server.js < /dev/null >> "$PARSOID_LOG_FILE"
2>&1
+end script
diff --git a/manifests/role/parsoid.pp b/manifests/role/parsoid.pp
index dcccff8..20a047c 100644
--- a/manifests/role/parsoid.pp
+++ b/manifests/role/parsoid.pp
@@ -18,14 +18,6 @@
mode => '2775',
}
- file { '/etc/init.d/parsoid':
- ensure => present,
- owner => root,
- group => root,
- mode => '0555',
- source => 'puppet:///files/misc/parsoid.init',
- }
-
file { '/usr/bin/parsoid':
ensure => present,
owner => root,
@@ -38,14 +30,6 @@
name => 'parsoid',
default_group => 'parsoid',
home => '/var/lib/parsoid',
- }
-
- service { 'parsoid':
- ensure => running,
- hasstatus => true,
- hasrestart => true,
- enable => true,
- require => File['/etc/init.d/parsoid'],
}
}
@@ -62,6 +46,22 @@
file { '/var/lib/parsoid/Parsoid':
ensure => link,
target => '/srv/deployment/parsoid/Parsoid',
+ }
+
+ # production uses an init script whereas labs experiments with upstart
+ file { '/etc/init.d/parsoid':
+ ensure => present,
+ owner => root,
+ group => root,
+ mode => '0555',
+ source => 'puppet:///files/misc/parsoid.init',
+ }
+ service { 'parsoid':
+ ensure => running,
+ hasstatus => true,
+ hasrestart => true,
+ enable => true,
+ require => File['/etc/init.d/parsoid'],
}
monitor_service { 'parsoid':
@@ -92,11 +92,65 @@
mode => '2775',
}
+ # beta uses upstart:
+ file { '/etc/init.d/parsoid':
+ ensure => 'link',
+ target => '/lib/init/upstart-job',
+ }
+ file { '/etc/init/parsoid.conf':
+ ensure => present,
+ owner => root,
+ group => root,
+ mode => '0444',
+ source => 'puppet:///files/misc/parsoid.upstart',
+ }
+ file { '/var/log/parsoid':
+ ensure => directory,
+ owner => parsoid,
+ group => parsoid,
+ mode => '0775',
+ }
+
+ $parsoid_log_file = '/var/log/parsoid/parsoid.log'
+
+ file { '/etc/default/parsoid':
+ ensure => present,
+ owner => root,
+ group => root,
+ mode => '0444',
+ content => template('misc/parsoid.default.erb'),
+ require => File['/var/log/parsoid'],
+ }
+
+ file { '/etc/logrotate.d/parsoid':
+ ensure => present,
+ owner => root,
+ group => root,
+ mode => '0444',
+ content => template('misc/parsoid.logrotate.erb'),
+ }
+
+ service { 'parsoid':
+ ensure => running,
+ hasstatus => true,
+ hasrestart => true,
+ enable => true,
+ provider => 'upstart',
+ subscribe => [
+ File['/etc/default/parsoid'],
+ File['/etc/init/parsoid.conf'],
+ ],
+ require => File['/etc/init.d/parsoid'],
+ }
+
# Beta parsoid server has some ferm DNAT rewriting rules (bug 45868) so we
# have to explicitly allow parsoid port 8000
ferm::service { 'http':
proto => 'tcp',
port => '8000'
}
+ ferm::rule { 'ssh-from-beta-bastion':
+ rule => 'proto tcp dport ssh { saddr 10.4.0.58 ACCEPT; }',
+ }
}
diff --git a/templates/misc/parsoid.default.erb
b/templates/misc/parsoid.default.erb
new file mode 100644
index 0000000..77643f4
--- /dev/null
+++ b/templates/misc/parsoid.default.erb
@@ -0,0 +1,7 @@
+#####################################################################
+### THIS FILE IS MANAGED BY PUPPET
+### puppet:///templates/misc/parsoid.default
+#####################################################################
+
+# File where the parsoid daemon will write stdin and stdout to
+PARSOID_LOG_FILE=<%= parsoid_log_file %>
diff --git a/templates/misc/parsoid.logrotate.erb
b/templates/misc/parsoid.logrotate.erb
new file mode 100644
index 0000000..1028a81
--- /dev/null
+++ b/templates/misc/parsoid.logrotate.erb
@@ -0,0 +1,14 @@
+#####################################################################
+### THIS FILE IS MANAGED BY PUPPET
+### puppet:///files/logrotate/parsoid
+#####################################################################
+
+<%= $parsoid_log_file %> {
+ daily
+ copytruncate
+ missingok
+ compress
+ notifempty
+ rotate 15
+ size 1GB
+}
--
To view, visit https://gerrit.wikimedia.org/r/99656
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3d4a49b37e6759ae222ab2fdb698576228dcc96e
Gerrit-PatchSet: 8
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Hashar <[email protected]>
Gerrit-Reviewer: ArielGlenn <[email protected]>
Gerrit-Reviewer: Catrope <[email protected]>
Gerrit-Reviewer: GWicke <[email protected]>
Gerrit-Reviewer: Hashar <[email protected]>
Gerrit-Reviewer: jenkins-bot
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
