Ottomata has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/344489 )
Change subject: Move camus jobs from analytics1027 to analytlics1003
......................................................................
Move camus jobs from analytics1027 to analytlics1003
Bug: T159527
Change-Id: Ib83276a4235fbc8adf4337efd97b8e9813dd5228
---
M manifests/site.pp
1 file changed, 17 insertions(+), 12 deletions(-)
Approvals:
Ottomata: Looks good to me, approved
jenkins-bot: Verified
diff --git a/manifests/site.pp b/manifests/site.pp
index 037e2bb..867fa25 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -64,7 +64,17 @@
# This node hosts Oozie and Hive servers,
# as well as a MySQL instance that stores
-# meta data associated with those services.
+# meta data associated with those and other
+# Analytics Cluster services.
+#
+# This node is also is a launch pad for various cron based Hadoop jobs.
+# Many ingestion jobs need a starting point. Oozie is a great
+# Hadoop job scheduler, but it is not better than cron
+# for some jobs that need to be launched at regular time
+# intervals. Cron is used for those. These crons
+# do not use local resources, instead, they launch
+# Hadoop jobs that run throughout the cluster.
+#
node 'analytics1003.eqiad.wmnet' {
role(analytics_cluster::client,
analytics_cluster::database::meta,
@@ -87,8 +97,11 @@
# is a little special and standalone, so we do it here.
analytics_cluster::hive::site_hdfs,
- # analytics1003 also runs various crons that launch
- # Hadoop jobs.
+ # Camus crons import data into
+ # from Kafka into HDFS.
+ analytics_cluster::refinery::job::camus,
+
+ # Various crons that launch Hadoop jobs.
analytics_cluster::refinery,
analytics_cluster::refinery::job::data_drop,
analytics_cluster::refinery::job::project_namespace_map,
@@ -115,15 +128,7 @@
# cron based Hadoop jobs.
node 'analytics1027.eqiad.wmnet' {
role(analytics_cluster::client,
- analytics_cluster::hue,
-
-
-
- # Include analytics/refinery deployment target.
- analytics_cluster::refinery,
- # Add cron jobs to run Camus to import data into
- # HDFS from Kafka.
- analytics_cluster::refinery::job::camus)
+ analytics_cluster::hue)
include ::standard
include ::base::firewall
--
To view, visit https://gerrit.wikimedia.org/r/344489
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib83276a4235fbc8adf4337efd97b8e9813dd5228
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <ao...@wikimedia.org>
Gerrit-Reviewer: Ottomata <ao...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
