[MediaWiki-commits] [Gerrit] operations/puppet[production]: Revert "Deploy MjoLniR with new deploy repository"
Gehel has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/391064 )
Change subject: Revert "Deploy MjoLniR with new deploy repository"
..
Revert "Deploy MjoLniR with new deploy repository"
There is still a mismatch between MjoLniR and mjolnir, which makes scap
fail to do init the repo on relforge. Let's revert and investigate further.
This reverts commit bc8f26a35d07e39fa32f356be590624d6d738f00.
Change-Id: I86cf0546dc9d4b367c1dcdd468466f6a31b1e9b3
---
M hieradata/role/common/deployment_server.yaml
D modules/mjolnir/manifests/init.pp
M modules/profile/manifests/mjolnir/kafka_daemon.pp
M modules/profile/templates/mjolnir/kafka-daemon.service.erb
M modules/role/manifests/elasticsearch/analytics.pp
5 files changed, 14 insertions(+), 21 deletions(-)
Approvals:
jenkins-bot: Verified
Gehel: Looks good to me, approved
diff --git a/hieradata/role/common/deployment_server.yaml
b/hieradata/role/common/deployment_server.yaml
index 9d82708..8eac2e3 100644
--- a/hieradata/role/common/deployment_server.yaml
+++ b/hieradata/role/common/deployment_server.yaml
@@ -173,8 +173,8 @@
# Librenms software
librenms/librenms:
repository: operations/software/librenms
- search/mjolnir/deploy:
-repository: search/MjoLniR/deploy
+ relforge/mjolnir:
+repository: search/MjoLniR
statsv/statsv:
repository: analytics/statsv
"docker-pkg/deploy":
diff --git a/modules/mjolnir/manifests/init.pp
b/modules/mjolnir/manifests/init.pp
deleted file mode 100644
index 5a3cbbd..000
--- a/modules/mjolnir/manifests/init.pp
+++ /dev/null
@@ -1,14 +0,0 @@
-# = Class: mjolnir
-#
-# This class installs the MjoLniR (Machine Learned Ranking) data
-# processing package.
-#
-class mjolnir {
-require_package('virtualenv', 'zip')
-
-scap::target { 'search/mjolnir/deploy':
-deploy_user => 'deploy-service',
-}
-}
-
-
diff --git a/modules/profile/manifests/mjolnir/kafka_daemon.pp
b/modules/profile/manifests/mjolnir/kafka_daemon.pp
index 479187c..c1dcaf9 100644
--- a/modules/profile/manifests/mjolnir/kafka_daemon.pp
+++ b/modules/profile/manifests/mjolnir/kafka_daemon.pp
@@ -9,11 +9,18 @@
# it is named just 'eqiad'.
$kafka_config = kafka_config('eqiad'),
) {
-class { 'mjolnir': }
+scap::target { 'relforge/mjolnir':
+ deploy_user => 'deploy-service',
+}
+
+# This is a limited subset of what the full mjolnir package requires
because
+# the daemon is a small part of the overall application. The daemon only
needs
+# to read/write kafka topics and send requests to localhost.
+require_package('python-kafka', 'python-requests')
systemd::service { 'mjolnir-kafka-daemon':
content => template('profile/mjolnir/kafka-daemon.service.erb'),
-require => Scap::Target['search/mjolnir/deploy'],
+require => Scap::Target['relforge/mjolnir'],
}
}
diff --git a/modules/profile/templates/mjolnir/kafka-daemon.service.erb
b/modules/profile/templates/mjolnir/kafka-daemon.service.erb
index 7e40bd8..b6947ba 100644
--- a/modules/profile/templates/mjolnir/kafka-daemon.service.erb
+++ b/modules/profile/templates/mjolnir/kafka-daemon.service.erb
@@ -5,7 +5,9 @@
[Service]
User=nobody
Group=nogroup
-ExecStart=/srv/deployment/search/mjolnir/venv/bin/mjolnir-utilities.py
kafka_daemon --brokers <%= @kafka_config['brokers']['string'] %>
+WorkingDirectory=/srv/deployment/relforge/mjolnir
+Environment=PYTHONPATH=/srv/deployment/relforge/mjolnir
+ExecStart=/usr/bin/python2
/srv/deployment/relforge/mjolnir/mjolnir/cli/kafka_daemon.py --brokers <%=
@kafka_config['brokers']['string'] %>
StandardInput=null
StandardOutput=journal
StandardError=journal
diff --git a/modules/role/manifests/elasticsearch/analytics.pp
b/modules/role/manifests/elasticsearch/analytics.pp
index 66d7789..a46391e 100644
--- a/modules/role/manifests/elasticsearch/analytics.pp
+++ b/modules/role/manifests/elasticsearch/analytics.pp
@@ -1,7 +1,5 @@
# Supports CirrusSearch usage on the analytics cluster
class role::elasticsearch::analytics {
-class { 'mjolnir': }
-
# wikimedia/discovery/analytics will be deployed to this node
scap::target { 'wikimedia/discovery/analytics':
deploy_user => 'deploy-service',
--
To view, visit https://gerrit.wikimedia.org/r/391064
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I86cf0546dc9d4b367c1dcdd468466f6a31b1e9b3
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel
Gerrit-Reviewer: EBernhardson
Gerrit-Reviewer: Gehel
Gerrit-Reviewer: jenkins-bot <>
___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
[MediaWiki-commits] [Gerrit] operations/puppet[production]: Revert "Deploy MjoLniR with new deploy repository"
Gehel has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/391064 )
Change subject: Revert "Deploy MjoLniR with new deploy repository"
..
Revert "Deploy MjoLniR with new deploy repository"
There is still a mismatch between MjoLniR and mjolnir, which makes scap
fail to do init the repo on relforge. Let's revert and investigate further.
This reverts commit bc8f26a35d07e39fa32f356be590624d6d738f00.
Change-Id: I86cf0546dc9d4b367c1dcdd468466f6a31b1e9b3
---
M hieradata/role/common/deployment_server.yaml
D modules/mjolnir/manifests/init.pp
M modules/profile/manifests/mjolnir/kafka_daemon.pp
M modules/profile/templates/mjolnir/kafka-daemon.service.erb
M modules/role/manifests/elasticsearch/analytics.pp
5 files changed, 14 insertions(+), 21 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/64/391064/1
diff --git a/hieradata/role/common/deployment_server.yaml
b/hieradata/role/common/deployment_server.yaml
index 9d82708..8eac2e3 100644
--- a/hieradata/role/common/deployment_server.yaml
+++ b/hieradata/role/common/deployment_server.yaml
@@ -173,8 +173,8 @@
# Librenms software
librenms/librenms:
repository: operations/software/librenms
- search/mjolnir/deploy:
-repository: search/MjoLniR/deploy
+ relforge/mjolnir:
+repository: search/MjoLniR
statsv/statsv:
repository: analytics/statsv
"docker-pkg/deploy":
diff --git a/modules/mjolnir/manifests/init.pp
b/modules/mjolnir/manifests/init.pp
deleted file mode 100644
index 5a3cbbd..000
--- a/modules/mjolnir/manifests/init.pp
+++ /dev/null
@@ -1,14 +0,0 @@
-# = Class: mjolnir
-#
-# This class installs the MjoLniR (Machine Learned Ranking) data
-# processing package.
-#
-class mjolnir {
-require_package('virtualenv', 'zip')
-
-scap::target { 'search/mjolnir/deploy':
-deploy_user => 'deploy-service',
-}
-}
-
-
diff --git a/modules/profile/manifests/mjolnir/kafka_daemon.pp
b/modules/profile/manifests/mjolnir/kafka_daemon.pp
index 479187c..c1dcaf9 100644
--- a/modules/profile/manifests/mjolnir/kafka_daemon.pp
+++ b/modules/profile/manifests/mjolnir/kafka_daemon.pp
@@ -9,11 +9,18 @@
# it is named just 'eqiad'.
$kafka_config = kafka_config('eqiad'),
) {
-class { 'mjolnir': }
+scap::target { 'relforge/mjolnir':
+ deploy_user => 'deploy-service',
+}
+
+# This is a limited subset of what the full mjolnir package requires
because
+# the daemon is a small part of the overall application. The daemon only
needs
+# to read/write kafka topics and send requests to localhost.
+require_package('python-kafka', 'python-requests')
systemd::service { 'mjolnir-kafka-daemon':
content => template('profile/mjolnir/kafka-daemon.service.erb'),
-require => Scap::Target['search/mjolnir/deploy'],
+require => Scap::Target['relforge/mjolnir'],
}
}
diff --git a/modules/profile/templates/mjolnir/kafka-daemon.service.erb
b/modules/profile/templates/mjolnir/kafka-daemon.service.erb
index 7e40bd8..b6947ba 100644
--- a/modules/profile/templates/mjolnir/kafka-daemon.service.erb
+++ b/modules/profile/templates/mjolnir/kafka-daemon.service.erb
@@ -5,7 +5,9 @@
[Service]
User=nobody
Group=nogroup
-ExecStart=/srv/deployment/search/mjolnir/venv/bin/mjolnir-utilities.py
kafka_daemon --brokers <%= @kafka_config['brokers']['string'] %>
+WorkingDirectory=/srv/deployment/relforge/mjolnir
+Environment=PYTHONPATH=/srv/deployment/relforge/mjolnir
+ExecStart=/usr/bin/python2
/srv/deployment/relforge/mjolnir/mjolnir/cli/kafka_daemon.py --brokers <%=
@kafka_config['brokers']['string'] %>
StandardInput=null
StandardOutput=journal
StandardError=journal
diff --git a/modules/role/manifests/elasticsearch/analytics.pp
b/modules/role/manifests/elasticsearch/analytics.pp
index 66d7789..a46391e 100644
--- a/modules/role/manifests/elasticsearch/analytics.pp
+++ b/modules/role/manifests/elasticsearch/analytics.pp
@@ -1,7 +1,5 @@
# Supports CirrusSearch usage on the analytics cluster
class role::elasticsearch::analytics {
-class { 'mjolnir': }
-
# wikimedia/discovery/analytics will be deployed to this node
scap::target { 'wikimedia/discovery/analytics':
deploy_user => 'deploy-service',
--
To view, visit https://gerrit.wikimedia.org/r/391064
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I86cf0546dc9d4b367c1dcdd468466f6a31b1e9b3
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel
___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
