from:"Gehel \(Code Review\)"

[MediaWiki-commits] [Gerrit] Revert "Increase the number of workers for osm2pgsql." - change (operations/puppet)

2016-06-03 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Revert "Increase the number of workers for osm2pgsql."
..


Revert "Increase the number of workers for osm2pgsql."

Causes lock contention and hangs the updater.

This reverts commit 71628dd4cd3a8dabdc95d8483fd33575ab84e418.

Change-Id: Icf5baa89cc7f3b0262ad6c5be54ced830a9d0839
---
M modules/role/manifests/maps/master.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  Yurik: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index a0bc12f..08be1da 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -41,7 +41,7 @@
 ensure=> present,
 flat_nodes=> true,
 expire_levels => '16',
-num_threads   => $::processorcount/2,
+num_threads   => 4,
 pg_password   => $osmupdater_pass,
 period=> 'day', # Remove thse as soon as we get down to minute
 hour  => '1',

-- 
To view, visit https://gerrit.wikimedia.org/r/292394
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Icf5baa89cc7f3b0262ad6c5be54ced830a9d0839
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: MaxSem 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Maps tables should all be owned by osmimporter - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293331

Change subject: Maps tables should all be owned by osmimporter
..

Maps tables should all be owned by osmimporter

Bug: T134901
Change-Id: Iccf42ba1be5c3b418b71cd899f006d7f4d432315
---
M modules/role/templates/maps/grants.sql.erb
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/31/293331/1

diff --git a/modules/role/templates/maps/grants.sql.erb 
b/modules/role/templates/maps/grants.sql.erb
index 1ab8f4e..355e6d9 100644
--- a/modules/role/templates/maps/grants.sql.erb
+++ b/modules/role/templates/maps/grants.sql.erb
@@ -14,6 +14,7 @@
 GRANT CREATE,TEMPORARY ON DATABASE gis TO osmimporter;
 
 ALTER DATABASE gis OWNER TO osmimporter;
+ALTER TABLE admin OWNER to osmimporter;
 ALTER TABLE geography_columns OWNER to osmimporter;
 ALTER TABLE geometry_columns OWNER to osmimporter;
 ALTER TABLE raster_columns OWNER to osmimporter;

-- 
To view, visit https://gerrit.wikimedia.org/r/293331
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iccf42ba1be5c3b418b71cd899f006d7f4d432315
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Scap3 config for Kartotherian - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Scap3 config for Kartotherian
..


Scap3 config for Kartotherian

Bug: T129150
Change-Id: I12a08831cfc07280f2021adedb00262b5e81686e
Depends-On: Ie2a956c5d14f068771f4cfbb0e673008870012cb
---
M hieradata/common/role/deployment.yaml
M hieradata/common/scap/server.yaml
M modules/kartotherian/manifests/init.pp
3 files changed, 5 insertions(+), 5 deletions(-)

Approvals:
  Gehel: Verified; Looks good to me, approved



diff --git a/hieradata/common/role/deployment.yaml 
b/hieradata/common/role/deployment.yaml
index 5b3137b..bf59741 100644
--- a/hieradata/common/role/deployment.yaml
+++ b/hieradata/common/role/deployment.yaml
@@ -91,9 +91,6 @@
 checkout_submodules: true
   sentry/sentry:
 upstream: https://gerrit.wikimedia.org/r/operations/software/sentry
-  kartotherian/deploy:
-upstream: https://gerrit.wikimedia.org/r/maps/kartotherian/deploy
-checkout_submodules: true
   tilerator/deploy:
 upstream: https://gerrit.wikimedia.org/r/maps/tilerator/deploy
 checkout_submodules: true
diff --git a/hieradata/common/scap/server.yaml 
b/hieradata/common/scap/server.yaml
index 3b0b5d3..5530687 100644
--- a/hieradata/common/scap/server.yaml
+++ b/hieradata/common/scap/server.yaml
@@ -54,6 +54,8 @@
   # The {} acts as an empty hash to use defaults.
   mobileapps/deploy: {}
 
+  kartotherian/deploy: {}
+
   # cxserver is the ContentTranslation server.
   cxserver/deploy:
 repository: cxserver/deploy
diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index 5b398d9..c5886c6 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -16,7 +16,8 @@
 $pgsql_kartotherian_pass = hiera('maps::postgresql_kartotherian_pass')
 
 service::node { 'kartotherian':
-port   => 6533,
-config => template('kartotherian/config.yaml.erb'),
+port   => 6533,
+config => template('kartotherian/config.yaml.erb'),
+deployment => 'scap3',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/291930
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I12a08831cfc07280f2021adedb00262b5e81686e
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Thcipriani 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Scap3 config for tilerator - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Scap3 config for tilerator
..


Scap3 config for tilerator

Bug: T129146
Change-Id: I7974ffe28b2f7858bccd90257ece27c409fa144a
Depends-On: Ifce82fba9bc29792d636b27721354bdb9ba8edf1
---
M hieradata/common/role/deployment.yaml
M hieradata/common/scap/server.yaml
M modules/tilerator/manifests/init.pp
3 files changed, 3 insertions(+), 3 deletions(-)

Approvals:
  Gehel: Verified; Looks good to me, approved
  Thcipriani: Looks good to me, but someone else must approve



diff --git a/hieradata/common/role/deployment.yaml 
b/hieradata/common/role/deployment.yaml
index bf59741..5b92a1e 100644
--- a/hieradata/common/role/deployment.yaml
+++ b/hieradata/common/role/deployment.yaml
@@ -91,9 +91,6 @@
 checkout_submodules: true
   sentry/sentry:
 upstream: https://gerrit.wikimedia.org/r/operations/software/sentry
-  tilerator/deploy:
-upstream: https://gerrit.wikimedia.org/r/maps/tilerator/deploy
-checkout_submodules: true
   cassandra/logstash-logback-encoder:
 gitfat_enabled: true
 upstream: 
https://gerrit.wikimedia.org/r/operations/software/logstash-logback-encoder
diff --git a/hieradata/common/scap/server.yaml 
b/hieradata/common/scap/server.yaml
index 5530687..5387b08 100644
--- a/hieradata/common/scap/server.yaml
+++ b/hieradata/common/scap/server.yaml
@@ -56,6 +56,8 @@
 
   kartotherian/deploy: {}
 
+  tilerator/deploy: {}
+
   # cxserver is the ContentTranslation server.
   cxserver/deploy:
 repository: cxserver/deploy
diff --git a/modules/tilerator/manifests/init.pp 
b/modules/tilerator/manifests/init.pp
index a211c4a..80c97c4 100644
--- a/modules/tilerator/manifests/init.pp
+++ b/modules/tilerator/manifests/init.pp
@@ -25,5 +25,6 @@
 port   => 6534,
 config => template('tilerator/config.yaml.erb'),
 no_workers => $::processorcount / 2,
+deployment => 'scap3',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/291268
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7974ffe28b2f7858bccd90257ece27c409fa144a
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Thcipriani 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: Thcipriani 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Maps tables should all be owned by osmimporter - change (operations/puppet)

2016-06-09 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Maps tables should all be owned by osmimporter
..


Maps tables should all be owned by osmimporter

Bug: T134901
Change-Id: Iccf42ba1be5c3b418b71cd899f006d7f4d432315
---
M modules/role/templates/maps/grants.sql.erb
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/role/templates/maps/grants.sql.erb 
b/modules/role/templates/maps/grants.sql.erb
index 1ab8f4e..355e6d9 100644
--- a/modules/role/templates/maps/grants.sql.erb
+++ b/modules/role/templates/maps/grants.sql.erb
@@ -14,6 +14,7 @@
 GRANT CREATE,TEMPORARY ON DATABASE gis TO osmimporter;
 
 ALTER DATABASE gis OWNER TO osmimporter;
+ALTER TABLE admin OWNER to osmimporter;
 ALTER TABLE geography_columns OWNER to osmimporter;
 ALTER TABLE geometry_columns OWNER to osmimporter;
 ALTER TABLE raster_columns OWNER to osmimporter;

-- 
To view, visit https://gerrit.wikimedia.org/r/293331
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Iccf42ba1be5c3b418b71cd899f006d7f4d432315
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Don't publish etags for WDQS - change (operations/puppet)

2016-06-09 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293492

Change subject: Don't publish etags for WDQS
..

Don't publish etags for WDQS

Etags in nginx are based on mtime, which is inconsistent between different
servers. This leads to unnecessary reload of resources. Removing etags should
allow if-modified-since to work better.

Bug: T137238
Change-Id: I272794341fd511766fcb5dc58b01dc4b8bc9a33e
---
M modules/wdqs/templates/nginx.erb
1 file changed, 5 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/92/293492/1

diff --git a/modules/wdqs/templates/nginx.erb b/modules/wdqs/templates/nginx.erb
index da3cd93..7cdf924 100644
--- a/modules/wdqs/templates/nginx.erb
+++ b/modules/wdqs/templates/nginx.erb
@@ -13,6 +13,11 @@
 
 add_header X-Served-By <%= @hostname %> always;
 
+# etag in nginx are based on mtime, which is not guaranteed to be the same
+# across multiple servers so let's disable etags and use more robust
+# caching mechanisms
+etag off;
+
 error_page 503 @maintenance;
 location @maintenance {
 root <%= @package_dir %>/gui;

-- 
To view, visit https://gerrit.wikimedia.org/r/293492
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I272794341fd511766fcb5dc58b01dc4b8bc9a33e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configure proxy for HTTPS as well as HTTP in replicate-osm. - change (operations/puppet)

2016-06-09 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293477

Change subject: Configure proxy for HTTPS as well as HTTP in replicate-osm.
..

Configure proxy for HTTPS as well as HTTP in replicate-osm.

Osmosis can download change sets over HTTP or HTTPS depending on
configuration. We need to ensure that proxy configuration allows both.

Change-Id: Iac59e537b566d8524882bc417ef678395040a8d1
---
M modules/osm/templates/replicate-osm.erb
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/77/293477/1

diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 421593e..827fc3e 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -4,7 +4,9 @@
 EXTRA_PARAMS=<%if @flat_nodes %>'--flat-nodes <%= @osmosis_dir %>/nodes.bin'<% 
end %>
 
 <% if @proxy -%>
-export JAVACMD_OPTIONS='-Dhttp.proxyHost=<%= @proxy.split(':')[0] %> 
-Dhttp.proxyPort=<%= @proxy.split(':')[1] %>'
+export JAVACMD_OPTIONS="\
+-Dhttp.proxyHost=<%= @proxy.split(':')[0] %>  -Dhttp.proxyPort=<%= 
@proxy.split(':')[1] %> \
+-Dhttps.proxyHost=<%= @proxy.split(':')[0] %> -Dhttps.proxyPort=<%= 
@proxy.split(':')[1] %>"
 <% end -%>
 
 /usr/bin/flock -xn <%= @osmosis_dir %>/replicate-osm.lck \

-- 
To view, visit https://gerrit.wikimedia.org/r/293477
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iac59e537b566d8524882bc417ef678395040a8d1
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] explicitely set input reader format in osm2pgsql - change (operations/puppet)

2016-06-09 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293475

Change subject: explicitely set input reader format in osm2pgsql
..

explicitely set input reader format in osm2pgsql

Latest version of osm2pgsql does not seem to autodetect format correctly
in our replicate-osm script. As the format is known in advanced, let's
make it explicit.

Bug: T112423
Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
---
M modules/osm/templates/replicate-osm.erb
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/75/293475/1

diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 421593e..b162754 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -16,6 +16,7 @@
 | /usr/bin/osm2pgsql \
   --hstore \
   --slim \
+  --input-reader xml \
   --cache <%= @memory_limit %> \
   --number-processes <%= @num_threads %> \
   --expire-tiles <%= @expire_levels %> \

-- 
To view, visit https://gerrit.wikimedia.org/r/293475
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Script to do the initial data load from OSM for Maps project - change (operations/puppet)

2016-06-09 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293105

Change subject: Script to do the initial data load from OSM for Maps project
..

Script to do the initial data load from OSM for Maps project

This script is a naive implementation of the steps taken to get Map's
postgres database in working shape.

Change-Id: Ibb7ce464636c380b5b7f1bed5ffe4a1fe1e1f177
---
A modules/role/files/maps/osm-initial-import.bash
1 file changed, 120 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/05/293105/2

diff --git a/modules/role/files/maps/osm-initial-import.bash 
b/modules/role/files/maps/osm-initial-import.bash
new file mode 100644
index 000..70bbcbe
--- /dev/null
+++ b/modules/role/files/maps/osm-initial-import.bash
@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+
+download_dir="/srv/downloads/initial-download/"
+osmosis_dir="/srv/osmosis"
+kartotherian_dir="/srv/deployment/kartotherian/deploy/"
+osm_bright_source_dir="${kartotherian_dir}/node_modules/osm-bright-source"
+proxy=
+database_host=`hostname -f`
+log_file=/var/log/osm-initial-import.log
+
+while getopts "dhHpx" opt; do
+  case "${opt}" in
+  d)  dump_date="${OPTARG}"
+  ;;
+  h)  show_help
+  exit 0
+  ;;
+  H)  database_host="${OPTARG}"
+  ;;
+  x)  password_file="${OPTARG}"
+  ;;
+  x)  proxy="-x ${OPTARG}"
+  ;;
+  esac
+done
+
+function show_help() {
+  echo "osm-initial-import.bash -d  [-H ] -p 
 -x "
+  echo "  date_of_import: find the latest dump at http://planet.osm.org/pbf/;
+  echo "  database_host: hostname of the postgresql database, default to 
`hostname -f`"
+  echo "  password_file: a file containing the postgresql password of the 
osmimporter user"
+  echo "  proxy: proxy used to download PBF files"
+  echo ""
+  echo "WARNING: the import is going to run for a long, long time. You should 
probably run this from inside screen or similar"
+  echo ""
+  echo "  example: osm-initial-import.bash -d 160530 -p ~/osmimporter_pass -x 
webproxy.eqiad.wmnet:8080"
+}
+
+function download_pbf() {
+  cd ${download_dir}
+  curl ${proxy} -C - -O 
http://planet.osm.org/pbf/planet-${dump_date}.osm.pbf.md5
+  curl ${proxy} -C - -O http://planet.osm.org/pbf/planet-${dump_date}.osm.pbf
+  md5sum -c planet-${dump_date}.osm.pbf.md5
+  if [ $? -ne 0 ] ; then
+echo "Download of PBF file failed, md5sum is invalid"
+exit -1
+  fi
+}
+
+function reset_postgres() {
+  puppet agent --disable "database reload in progress"
+  service postgresql@9.4-main stop
+
+  rm -rf /srv/postgresql/9.4/main
+
+  mkdir /srv/postgresql/9.4/main
+  chown postgres: /srv/postgresql/9.4/main/
+  chmod 700 /srv/postgresql/9.4/main/
+
+  sudo -u postgres /usr/lib/postgresql/9.4/bin/initdb -D 
/srv/postgresql/9.4/main
+  service postgresql@9.4-main start
+
+  # puppet creates multiple postgis resources
+  puppet agent --enable
+  puppet agent -t
+  cat /usr/local/bin/maps-grants.sql | sudo -u postgres psql -d gis -f -
+}
+
+function initial_osm_import() {
+  # disable replicate-osm cronjob
+  puppet apply -e "cron { 'planet_sync-gis': ensure => absent, user => 
'osmupdater' }"
+
+  puppet agent --disable "database reload in progress"
+  service tilerator stop
+  service tileratorui stop
+
+
+  cd ${osmosis_dir}
+  PGPASSWORD="$(< ${password_file})" sudo -E -u osmupdater osm2pgsql \
+--create --slim --flat-nodes nodes.bin -C 4 --number-processes 8 \
+--hstore --host ${database_host} -U osmimporter -d gis \
+${download_dir}/planet-${dump_date}.osm.pbf 2>&1 | tee ${log_file}
+
+  if [ ${PIPESTATUS[0]} -ne 0 ]; then
+echo "osm2pgsql failed to complete initial import"
+exit -1
+  fi
+  puppet agent --enable
+  puppet agent -t
+}
+
+function import_water_polygons() {
+  cd ${download_dir}
+
+  curl ${proxy} -C - -O 
http://data.openstreetmapdata.com/water-polygons-split-3857.zip
+  unzip water-polygons-split-3857.zip
+
+  cd water-polygons-split-3857
+  shp2pgsql -I -s 3857 -g way water_polygons.shp water_polygons | sudo -u 
postgres psql -Xqd gis
+}
+
+function custom_functions_and_indexes() {
+  sudo -u postgres psql -Xd gis -f 
"${osm_bright_source_dir}/node_modules/postgis-vt-util/lib.sql"
+  sudo -u postgres psql -Xd gis -f "${osm_bright_source_dir}/sql/admin.sql"
+  sudo -u postgres psql -Xd gis -f "${osm_bright_source_dir}/sql/functions.sql"
+  sudo -u postgres psql -Xd gis -f 
"${osm_bright_source_dir}/sql/water-indexes.sql"
+  sudo -u postgres psql -d gis -c 'select populate_admin();'
+}
+function cleanup() {
+  rm "${download_dir}/planet-${dump_date}.osm.pbf.md5"
+  rm "${download_dir}/planet-${dump_date}.osm.pbf"
+  rm "${download_dir}/water-polygons-split-3857.zip"
+}
+
+download_pbf
+reset_postgres
+initial_osm_import
+import_water_polygons
+custom_functions_and_indexes
+cleanup
\ No newline at end of file

-- 
To view, visit https://gerrit.wikimedia.org/r/293105
To unsubscribe,

[MediaWiki-commits] [Gerrit] Change expired file zoom level from 16 to 15. - change (operations/puppet)

2016-06-09 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Change expired file zoom level from 16 to 15.
..


Change expired file zoom level from 16 to 15.

Keeping on zoome level 15 still gives us the ability to oversample on level 16
but saves quite a lot of space.

Bug: T136483
Change-Id: If3b286f2540307a8b043cc13de7f710f3024f9cc
---
M modules/role/manifests/maps/master.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index 213d2a1..b5a0fc5 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -50,7 +50,7 @@
 osm::planet_sync { 'gis':
 ensure=> present,
 flat_nodes=> true,
-expire_levels => '16',
+expire_levels => '15',
 num_threads   => 4,
 pg_password   => $osmupdater_pass,
 period=> 'day', # Remove thse as soon as we get down to minute

-- 
To view, visit https://gerrit.wikimedia.org/r/291885
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If3b286f2540307a8b043cc13de7f710f3024f9cc
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Scap3 config for tilerator" - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293375

Change subject: Revert "Scap3 config for tilerator"
..

Revert "Scap3 config for tilerator"

This reverts commit e0a432e434053a867b9c60f09bd31f659569bc24.

Change-Id: I062fa7bdea9970b8a3914a3c540b75d48c166f29
---
M hieradata/common/role/deployment.yaml
M hieradata/common/scap/server.yaml
M modules/tilerator/manifests/init.pp
3 files changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/75/293375/1

diff --git a/hieradata/common/role/deployment.yaml 
b/hieradata/common/role/deployment.yaml
index 5b92a1e..bf59741 100644
--- a/hieradata/common/role/deployment.yaml
+++ b/hieradata/common/role/deployment.yaml
@@ -91,6 +91,9 @@
 checkout_submodules: true
   sentry/sentry:
 upstream: https://gerrit.wikimedia.org/r/operations/software/sentry
+  tilerator/deploy:
+upstream: https://gerrit.wikimedia.org/r/maps/tilerator/deploy
+checkout_submodules: true
   cassandra/logstash-logback-encoder:
 gitfat_enabled: true
 upstream: 
https://gerrit.wikimedia.org/r/operations/software/logstash-logback-encoder
diff --git a/hieradata/common/scap/server.yaml 
b/hieradata/common/scap/server.yaml
index 5387b08..5530687 100644
--- a/hieradata/common/scap/server.yaml
+++ b/hieradata/common/scap/server.yaml
@@ -56,8 +56,6 @@
 
   kartotherian/deploy: {}
 
-  tilerator/deploy: {}
-
   # cxserver is the ContentTranslation server.
   cxserver/deploy:
 repository: cxserver/deploy
diff --git a/modules/tilerator/manifests/init.pp 
b/modules/tilerator/manifests/init.pp
index 80c97c4..a211c4a 100644
--- a/modules/tilerator/manifests/init.pp
+++ b/modules/tilerator/manifests/init.pp
@@ -25,6 +25,5 @@
 port   => 6534,
 config => template('tilerator/config.yaml.erb'),
 no_workers => $::processorcount / 2,
-deployment => 'scap3',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/293375
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I062fa7bdea9970b8a3914a3c540b75d48c166f29
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Fixing lint issues - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Fixing lint issues
..


Fixing lint issues

those issue were probably introduced while Jenkins was taking a nap...

Change-Id: I98ebd391b1ca29d8023e0bbaf37a83e14ea8bca7
---
M modules/role/manifests/ci/master.pp
M modules/role/manifests/zookeeper/client.pp
2 files changed, 6 insertions(+), 6 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/ci/master.pp 
b/modules/role/manifests/ci/master.pp
index 3b351ab..0bbd5e3 100644
--- a/modules/role/manifests/ci/master.pp
+++ b/modules/role/manifests/ci/master.pp
@@ -71,10 +71,10 @@
 require contint::master_dir
 
 file { '/srv/ssd/jenkins':
-ensure  => 'directory',
-owner   => 'jenkins',
-group   => 'jenkins',
-mode=> '2775',  # group sticky bit
+ensure => 'directory',
+owner  => 'jenkins',
+group  => 'jenkins',
+mode   => '2775',  # group sticky bit
 }
 
 # Master does not run job anymore since June 2013. But better safe than
diff --git a/modules/role/manifests/zookeeper/client.pp 
b/modules/role/manifests/zookeeper/client.pp
index f0adf1e..66a0a53 100644
--- a/modules/role/manifests/zookeeper/client.pp
+++ b/modules/role/manifests/zookeeper/client.pp
@@ -29,8 +29,8 @@
 
 require_package('openjdk-7-jdk')
 class { '::zookeeper':
-hosts  => $hosts,
-version=> $version,
+hosts   => $hosts,
+version => $version,
 }
 }
 

-- 
To view, visit https://gerrit.wikimedia.org/r/293377
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I98ebd391b1ca29d8023e0bbaf37a83e14ea8bca7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Scap3 config for tilerator" - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Revert "Scap3 config for tilerator"
..


Revert "Scap3 config for tilerator"

This reverts commit e0a432e434053a867b9c60f09bd31f659569bc24.

Change-Id: I062fa7bdea9970b8a3914a3c540b75d48c166f29
---
M hieradata/common/role/deployment.yaml
M hieradata/common/scap/server.yaml
M modules/tilerator/manifests/init.pp
3 files changed, 3 insertions(+), 3 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/common/role/deployment.yaml 
b/hieradata/common/role/deployment.yaml
index 5b92a1e..bf59741 100644
--- a/hieradata/common/role/deployment.yaml
+++ b/hieradata/common/role/deployment.yaml
@@ -91,6 +91,9 @@
 checkout_submodules: true
   sentry/sentry:
 upstream: https://gerrit.wikimedia.org/r/operations/software/sentry
+  tilerator/deploy:
+upstream: https://gerrit.wikimedia.org/r/maps/tilerator/deploy
+checkout_submodules: true
   cassandra/logstash-logback-encoder:
 gitfat_enabled: true
 upstream: 
https://gerrit.wikimedia.org/r/operations/software/logstash-logback-encoder
diff --git a/hieradata/common/scap/server.yaml 
b/hieradata/common/scap/server.yaml
index 5387b08..5530687 100644
--- a/hieradata/common/scap/server.yaml
+++ b/hieradata/common/scap/server.yaml
@@ -56,8 +56,6 @@
 
   kartotherian/deploy: {}
 
-  tilerator/deploy: {}
-
   # cxserver is the ContentTranslation server.
   cxserver/deploy:
 repository: cxserver/deploy
diff --git a/modules/tilerator/manifests/init.pp 
b/modules/tilerator/manifests/init.pp
index 80c97c4..a211c4a 100644
--- a/modules/tilerator/manifests/init.pp
+++ b/modules/tilerator/manifests/init.pp
@@ -25,6 +25,5 @@
 port   => 6534,
 config => template('tilerator/config.yaml.erb'),
 no_workers => $::processorcount / 2,
-deployment => 'scap3',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/293375
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I062fa7bdea9970b8a3914a3c540b75d48c166f29
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Thcipriani 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Activate more logs on postgresql for maps. - change (operations/puppet)

2016-06-06 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Activate more logs on postgresql for maps.
..


Activate more logs on postgresql for maps.

test under load show that this generate a reasonable amount of logs as we are
only logging queries > 10 seconds.

Change-Id: Iba41209dc6ac23bc7bdc91a0a03d4abc2d45680f
---
M modules/role/files/maps/tuning.conf
1 file changed, 11 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/files/maps/tuning.conf 
b/modules/role/files/maps/tuning.conf
index 82677ca..1399060 100644
--- a/modules/role/files/maps/tuning.conf
+++ b/modules/role/files/maps/tuning.conf
@@ -5,3 +5,14 @@
 wal_buffers = 8MB
 shared_buffers = 7680MB
 max_connections = 120
+
+# activate more logging to investigate perf issues
+log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
+log_min_duration_statement = 1
+log_checkpoints = on
+log_connections = on
+log_disconnections = on
+log_lock_waits = on
+log_temp_files = 0
+log_autovacuum_min_duration = 0
+lc_messages='C'

-- 
To view, visit https://gerrit.wikimedia.org/r/292925
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Iba41209dc6ac23bc7bdc91a0a03d4abc2d45680f
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Activate more logs on postgresql for maps. - change (operations/puppet)

2016-06-06 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292925

Change subject: Activate more logs on postgresql for maps.
..

Activate more logs on postgresql for maps.

test under load show that this generate a reasonable amount of logs as we are
only logging queries > 10 seconds.

Change-Id: Iba41209dc6ac23bc7bdc91a0a03d4abc2d45680f
---
M modules/role/files/maps/tuning.conf
1 file changed, 11 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/25/292925/1

diff --git a/modules/role/files/maps/tuning.conf 
b/modules/role/files/maps/tuning.conf
index 82677ca..1399060 100644
--- a/modules/role/files/maps/tuning.conf
+++ b/modules/role/files/maps/tuning.conf
@@ -5,3 +5,14 @@
 wal_buffers = 8MB
 shared_buffers = 7680MB
 max_connections = 120
+
+# activate more logging to investigate perf issues
+log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
+log_min_duration_statement = 1
+log_checkpoints = on
+log_connections = on
+log_disconnections = on
+log_lock_waits = on
+log_temp_files = 0
+log_autovacuum_min_duration = 0
+lc_messages='C'

-- 
To view, visit https://gerrit.wikimedia.org/r/292925
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iba41209dc6ac23bc7bdc91a0a03d4abc2d45680f
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Send wmf.4 search and ttmserver traffic to codfw" - change (operations/mediawiki-config)

2016-06-06 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292938

Change subject: Revert "Send wmf.4 search and ttmserver traffic to codfw"
..

Revert "Send wmf.4 search and ttmserver traffic to codfw"

This reverts commit 869a7f9b278394a5d7146f0c03100dbe425dcc92.

Change-Id: Ic79b742359173d19998524795804a57b0f28d04e
---
M wmf-config/CirrusSearch-common.php
M wmf-config/CirrusSearch-production.php
M wmf-config/CommonSettings.php
3 files changed, 5 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/38/292938/1

diff --git a/wmf-config/CirrusSearch-common.php 
b/wmf-config/CirrusSearch-common.php
index 41453db..7eb218c 100644
--- a/wmf-config/CirrusSearch-common.php
+++ b/wmf-config/CirrusSearch-common.php
@@ -23,10 +23,7 @@
 
 # Set up the the default cluster to send queries to,
 # and the list of clusters to write to.
-if ( substr($IP, -12 ) === '1.28.0-wmf.4' ) {
-   // temporary hack for the elasticsearch 2.3 rollout
-   $wgCirrusSearchDefaultCluster = 'codfw';
-} elseif ( $wmgCirrusSearchDefaultCluster === 'local' ) {
+if ( $wmgCirrusSearchDefaultCluster === 'local' ) {
$wgCirrusSearchDefaultCluster = $wmfDatacenter;
 } else {
$wgCirrusSearchDefaultCluster = $wmgCirrusSearchDefaultCluster;
diff --git a/wmf-config/CirrusSearch-production.php 
b/wmf-config/CirrusSearch-production.php
index a00392c..3f307d4 100644
--- a/wmf-config/CirrusSearch-production.php
+++ b/wmf-config/CirrusSearch-production.php
@@ -122,6 +122,9 @@
 
 $wgCirrusSearchRecycleCompletionSuggesterIndex = 
$wmgCirrusSearchRecycleCompletionSuggesterIndex;
 
+// repoint morelike queries to codfw
+$wgCirrusSearchMoreLikeThisCluster = 'eqiad';
+
 // cache morelike queries to ObjectCache for 24 hours
 $wgCirrusSearchMoreLikeThisTTL = 86400;
 
diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index aebcaa1..7075d45 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -2366,12 +2366,7 @@
$wgExtraLanguageNames['qqq'] = 'Message documentation'; # No linguistic 
content. Used for documenting messages
 
// TODO: proper integration with new CirrusSearch config
-   if ( substr( $IP, -12 ) === '1.28.0-wmf.4' ) {
-   // temporary hack for the elasticsearch 2.3 rollout
-   $wgTranslateExtensionDefaultCluster = 'codfw';
-   } else {
-   $wgTranslateExtensionDefaultCluster = 'eqiad';
-   }
+   $wgTranslateExtensionDefaultCluster = 'eqiad';
$wgTranslateTranslationServices = [];
if ( $wmgUseTranslationMemory ) {
$servers = array_map(

-- 
To view, visit https://gerrit.wikimedia.org/r/292938
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic79b742359173d19998524795804a57b0f28d04e
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Activate more logs on postgresql for maps. - change (operations/puppet)

2016-06-06 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292931

Change subject: Activate more logs on postgresql for maps.
..

Activate more logs on postgresql for maps.

test under load show that this generate a reasonable amount of logs as we are
only logging queries > 10 seconds.

This additional logging is added only to the postgres master

Change-Id: If6d942ff4392d34b0f6b83b66fbe2d5a2a36afbf
---
M hieradata/role/common/maps/master.yaml
A modules/role/files/maps/logging.conf
M modules/role/manifests/maps/master.pp
3 files changed, 21 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/31/292931/1

diff --git a/hieradata/role/common/maps/master.yaml 
b/hieradata/role/common/maps/master.yaml
index 4fdf68f..f78 100644
--- a/hieradata/role/common/maps/master.yaml
+++ b/hieradata/role/common/maps/master.yaml
@@ -1,6 +1,8 @@
 postgresql::master::pgversion: 9.4
 postgresql::master::datadir: /srv/postgresql/9.4/main
-postgresql::master::includes: tuning.conf
+postgresql::master::includes:
+  - tuning.conf
+  - logging.conf
 postgresql::master::checkpoint_segments: 768
 postgresql::master::wal_keep_segments: 768
 debdeploy::grains:
diff --git a/modules/role/files/maps/logging.conf 
b/modules/role/files/maps/logging.conf
new file mode 100644
index 000..dbafdaf
--- /dev/null
+++ b/modules/role/files/maps/logging.conf
@@ -0,0 +1,10 @@
+# activate more logging to investigate perf issues
+log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
+log_min_duration_statement = 1
+log_checkpoints = on
+log_connections = on
+log_disconnections = on
+log_lock_waits = on
+log_temp_files = 0
+log_autovacuum_min_duration = 0
+lc_messages='C'
diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index 08be1da..69b08c8 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -31,6 +31,14 @@
 require => Class['::postgresql::postgis'],
 }
 
+file { '/etc/postgresql/9.4/main/logging.conf':
+ensure => 'present',
+owner  => 'root',
+group  => 'root',
+mode   => '0444',
+source => 'puppet:///modules/role/maps/logging.conf',
+}
+
 # PostgreSQL Replication
 $postgres_slaves = hiera('maps::postgres_slaves', undef)
 if $postgres_slaves {

-- 
To view, visit https://gerrit.wikimedia.org/r/292931
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If6d942ff4392d34b0f6b83b66fbe2d5a2a36afbf
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Activate more logs on postgresql for maps. - change (operations/puppet)

2016-06-06 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Activate more logs on postgresql for maps.
..


Activate more logs on postgresql for maps.

test under load show that this generate a reasonable amount of logs as we are
only logging queries > 10 seconds.

This additional logging is added only to the postgres master

Change-Id: If6d942ff4392d34b0f6b83b66fbe2d5a2a36afbf
---
M hieradata/role/common/maps/master.yaml
A modules/role/files/maps/logging.conf
M modules/role/files/maps/tuning.conf
M modules/role/manifests/maps/master.pp
4 files changed, 23 insertions(+), 12 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/role/common/maps/master.yaml 
b/hieradata/role/common/maps/master.yaml
index 4fdf68f..f78 100644
--- a/hieradata/role/common/maps/master.yaml
+++ b/hieradata/role/common/maps/master.yaml
@@ -1,6 +1,8 @@
 postgresql::master::pgversion: 9.4
 postgresql::master::datadir: /srv/postgresql/9.4/main
-postgresql::master::includes: tuning.conf
+postgresql::master::includes:
+  - tuning.conf
+  - logging.conf
 postgresql::master::checkpoint_segments: 768
 postgresql::master::wal_keep_segments: 768
 debdeploy::grains:
diff --git a/modules/role/files/maps/logging.conf 
b/modules/role/files/maps/logging.conf
new file mode 100644
index 000..dbafdaf
--- /dev/null
+++ b/modules/role/files/maps/logging.conf
@@ -0,0 +1,10 @@
+# activate more logging to investigate perf issues
+log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
+log_min_duration_statement = 1
+log_checkpoints = on
+log_connections = on
+log_disconnections = on
+log_lock_waits = on
+log_temp_files = 0
+log_autovacuum_min_duration = 0
+lc_messages='C'
diff --git a/modules/role/files/maps/tuning.conf 
b/modules/role/files/maps/tuning.conf
index 1399060..82677ca 100644
--- a/modules/role/files/maps/tuning.conf
+++ b/modules/role/files/maps/tuning.conf
@@ -5,14 +5,3 @@
 wal_buffers = 8MB
 shared_buffers = 7680MB
 max_connections = 120
-
-# activate more logging to investigate perf issues
-log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
-log_min_duration_statement = 1
-log_checkpoints = on
-log_connections = on
-log_disconnections = on
-log_lock_waits = on
-log_temp_files = 0
-log_autovacuum_min_duration = 0
-lc_messages='C'
diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index 08be1da..213d2a1 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -31,6 +31,16 @@
 require => Class['::postgresql::postgis'],
 }
 
+# some additional logging for the postgres master to help diagnose import
+# performance issues
+file { '/etc/postgresql/9.4/main/logging.conf':
+ensure => 'present',
+owner  => 'root',
+group  => 'root',
+mode   => '0444',
+source => 'puppet:///modules/role/maps/logging.conf',
+}
+
 # PostgreSQL Replication
 $postgres_slaves = hiera('maps::postgres_slaves', undef)
 if $postgres_slaves {

-- 
To view, visit https://gerrit.wikimedia.org/r/292931
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If6d942ff4392d34b0f6b83b66fbe2d5a2a36afbf
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Enable 'has_spec' on Kartotherian service. - change (operations/puppet)

2016-06-13 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294028

Change subject: Enable 'has_spec' on Kartotherian service.
..

Enable 'has_spec' on Kartotherian service.

Spec has been written and deployed for Kartotherian. We now need to use it.
To not spam everyone while testing, this is disabled on all servers except
maps2001. This will need to be cleaned once we confirm that all works
correctly.

Bug: T137617
Change-Id: Ibbd69c672b477d7d71326f12fb181c2201d39ab9
---
M hieradata/hosts/maps2001.yaml
M hieradata/role/common/maps/server.yaml
M modules/kartotherian/manifests/init.pp
3 files changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/28/294028/1

diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index db7b54b..adafca6 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -15,3 +15,5 @@
 tilerator::conf_sources: sources.prod2.yaml
 tilerator::ui::conf_sources: sources.prod2.yaml
 kartotherian::conf_sources: sources.prod2.yaml
+# enable specs only on maps2001 for testing
+kartotherian::enable_spec: true
diff --git a/hieradata/role/common/maps/server.yaml 
b/hieradata/role/common/maps/server.yaml
index 0249efa..47dab51 100644
--- a/hieradata/role/common/maps/server.yaml
+++ b/hieradata/role/common/maps/server.yaml
@@ -12,3 +12,6 @@
   - tilerator-admin
 service::configuration::logstash_host: logstash1001.eqiad.wmnet
 service::configuration::statsd_host: statsd.eqiad.wmnet
+
+# Let's keep spec disabled until it is correctly tested
+kartotherian::enable_spec: false
diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index c5886c6..b998a04 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -8,6 +8,7 @@
 # service
 class kartotherian(
 $conf_sources = 'sources.prod.yaml',
+$enable_spec  = true,
 ) {
 
 $cassandra_kartotherian_user = 'kartotherian'
@@ -19,5 +20,6 @@
 port   => 6533,
 config => template('kartotherian/config.yaml.erb'),
 deployment => 'scap3',
+has_spec   => $enable_spec,
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294028
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibbd69c672b477d7d71326f12fb181c2201d39ab9
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configure proxy for HTTPS as well as HTTP in replicate-osm. - change (operations/puppet)

2016-06-10 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Configure proxy for HTTPS as well as HTTP in replicate-osm.
..


Configure proxy for HTTPS as well as HTTP in replicate-osm.

Osmosis can download change sets over HTTP or HTTPS depending on
configuration. We need to ensure that proxy configuration allows both.

Bug: T134901
Change-Id: Iac59e537b566d8524882bc417ef678395040a8d1
---
M modules/osm/templates/replicate-osm.erb
1 file changed, 3 insertions(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 421593e..827fc3e 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -4,7 +4,9 @@
 EXTRA_PARAMS=<%if @flat_nodes %>'--flat-nodes <%= @osmosis_dir %>/nodes.bin'<% 
end %>
 
 <% if @proxy -%>
-export JAVACMD_OPTIONS='-Dhttp.proxyHost=<%= @proxy.split(':')[0] %> 
-Dhttp.proxyPort=<%= @proxy.split(':')[1] %>'
+export JAVACMD_OPTIONS="\
+-Dhttp.proxyHost=<%= @proxy.split(':')[0] %>  -Dhttp.proxyPort=<%= 
@proxy.split(':')[1] %> \
+-Dhttps.proxyHost=<%= @proxy.split(':')[0] %> -Dhttps.proxyPort=<%= 
@proxy.split(':')[1] %>"
 <% end -%>
 
 /usr/bin/flock -xn <%= @osmosis_dir %>/replicate-osm.lck \

-- 
To view, visit https://gerrit.wikimedia.org/r/293477
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Iac59e537b566d8524882bc417ef678395040a8d1
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Upgrade osm2pgsql to 0.90.0 - change (operations/puppet)

2016-06-10 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Upgrade osm2pgsql to 0.90.0
..


Upgrade osm2pgsql to 0.90.0

0.90.0 is available in Jessie backports.

I took eventlogging::service::service as an example of how we pull packages
from backports.

This new version has been tested on new maps2??? servers and show that initial
import and replication are working fine.

Bug: T112423
Depends-on: I0482b718a496cfabf647295b413bce5e2a453375
Change-Id: I13cc0a9566a32cdf8f4ce7a85e22bfdab1c50173
---
M modules/osm/manifests/init.pp
M modules/osm/spec/classes/osm_spec.rb
2 files changed, 61 insertions(+), 20 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/osm/manifests/init.pp b/modules/osm/manifests/init.pp
index 8512ed1..7592ca4 100644
--- a/modules/osm/manifests/init.pp
+++ b/modules/osm/manifests/init.pp
@@ -1,10 +1,30 @@
 #
 
-class osm($ensure='present') {
-package { [
-'osm2pgsql',
-'osmosis',
-]:
-ensure => $ensure,
+class osm (
+$ensure = present,
+){
+
+# osm2pgsql 0.90 is only available on jessie at the moment
+# there is no need for 0.90 on labs machines (precise)
+$osm2pgsql_ensure = $ensure ? {
+'present' => os_version('Debian >= Jessie') ? {
+true => '0.90.0+ds-1~bpo8+1',
+default => 'present',
+},
+default   => $ensure,
+}
+
+# Can't use require_package here because we need to specify version
+# from jessie-backports:
+if !defined(Package['osm2pgsql']) {
+package { 'osm2pgsql':
+ensure => $osm2pgsql_ensure,
+}
+}
+
+if !defined(Package['osmosis']) {
+package { 'osmosis':
+ensure => $ensure,
+}
 }
 }
diff --git a/modules/osm/spec/classes/osm_spec.rb 
b/modules/osm/spec/classes/osm_spec.rb
index a58f4c3..2803d56 100644
--- a/modules/osm/spec/classes/osm_spec.rb
+++ b/modules/osm/spec/classes/osm_spec.rb
@@ -1,24 +1,45 @@
 require 'spec_helper'
 
 describe 'osm', :type => :class do
-let(:params) { {
-:ensure   => 'present',
-}
-}
-
 context 'ensure present' do
-it { should contain_package('osm2pgsql').with_ensure('present') }
-it { should contain_package('osmosis').with_ensure('present') }
-end
-end
+let(:params) { {
+:ensure => 'present',
+} }
 
-describe 'osm', :type => :class do
-let(:params) { {
-:ensure   => 'absent',
-}
-}
+context 'on Ubuntu Precise' do
+let(:facts) { {
+:lsbdistrelease => 'Precise',
+:lsbdistid  => 'Ubuntu',
+} }
+
+context 'ensure present' do
+it { should 
contain_package('osm2pgsql').with_ensure('present') }
+it { should contain_package('osmosis').with_ensure('present') }
+end
+end
+
+context 'on Debian Jessie' do
+let(:facts) { {
+:lsbdistrelease => 'Jessie',
+:lsbdistid  => 'Debian',
+} }
+
+context 'ensure present' do
+it { should 
contain_package('osm2pgsql').with_ensure('0.90.0+ds-1~bpo8+1') }
+it { should contain_package('osmosis').with_ensure('present') }
+end
+end
+end
 
 context 'ensure absent' do
+let(:params) { {
+:ensure => 'absent',
+} }
+let(:facts) { {
+:lsbdistrelease => 'Precise',
+:lsbdistid  => 'Ubuntu',
+} }
+
 it { should contain_package('osm2pgsql').with_ensure('absent') }
 it { should contain_package('osmosis').with_ensure('absent') }
 end

-- 
To view, visit https://gerrit.wikimedia.org/r/287600
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I13cc0a9566a32cdf8f4ce7a85e22bfdab1c50173
Gerrit-PatchSet: 9
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] explicitely set input reader format in osm2pgsql - change (operations/puppet)

2016-06-10 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: explicitely set input reader format in osm2pgsql
..


explicitely set input reader format in osm2pgsql

Latest version of osm2pgsql does not seem to autodetect format correctly
in our replicate-osm script. As the format is known in advanced, let's
make it explicit.

Bug: T112423
Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
---
M modules/osm/manifests/planet_sync.pp
M modules/osm/spec/defines/osm_planet_sync_spec.rb
M modules/osm/templates/replicate-osm.erb
3 files changed, 37 insertions(+), 11 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/osm/manifests/planet_sync.pp 
b/modules/osm/manifests/planet_sync.pp
index 19a1651..e9b3ebf 100644
--- a/modules/osm/manifests/planet_sync.pp
+++ b/modules/osm/manifests/planet_sync.pp
@@ -30,6 +30,10 @@
 #  Memory in megabytes osm2pgsql should occupy
 #   $num_threads
 #  Number of threads to use during sync
+#   $input_reader_format
+#  Format passed to osm2pgsql as --input-reader parameter. osm2pgsql < 0.90
+#  needs 'libxml2' (which is default) and osm2pgsql >= 0.90 needs 'xml'.
+#  osm2pgsql == 0.90 is used on Jessie only at this point.
 #
 # Actions:
 #   sync with planet.osm
@@ -53,6 +57,10 @@
 $expire_levels='15',
 $memory_limit=floor($::memorysize_mb)/12,
 $num_threads=$::processorcount,
+$input_reader_format = os_version('Debian >= Jessie')? {
+true=> 'xml',
+default => 'libxml2',
+},
 ) {
 include ::osm::users
 
diff --git a/modules/osm/spec/defines/osm_planet_sync_spec.rb 
b/modules/osm/spec/defines/osm_planet_sync_spec.rb
index a669991..b5badb2 100644
--- a/modules/osm/spec/defines/osm_planet_sync_spec.rb
+++ b/modules/osm/spec/defines/osm_planet_sync_spec.rb
@@ -2,17 +2,34 @@
 
 describe 'osm::planet_sync', :type => :define do
 let(:title) { 'somedb' }
-let(:facts) { {
-:memoryfree => '1000 MB',
-}
-}
-let(:params) { {
-:osmosis_dir => '/srv/osmosis',
-:period => 'minute',
-}
-}
+
 context 'with ensure present' do
-it { should contain_cron('planet_sync-somedb') }
-it { should 
contain_file('/srv/osmosis/configuration.txt').with_content(/minute/) }
+let(:params) { {
+:osmosis_dir => '/srv/osmosis',
+:period  => 'minute',
+:pg_password => 'secret',
+} }
+
+context 'on Ubuntu Precise' do
+let(:facts) { {
+:lsbdistrelease => 'Precise',
+:lsbdistid  => 'Ubuntu',
+:memorysize_mb => 64420.94,
+}}
+
+it { should contain_cron('planet_sync-somedb') }
+it { should 
contain_file('/srv/osmosis/configuration.txt').with_content(/minute/) }
+it { should 
contain_file('/usr/local/bin/replicate-osm').with_content(/--input-reader 
libxml2/) }
+end
+
+context 'on Debian Jessie' do
+let(:facts) { {
+:lsbdistrelease => 'Jessie',
+:lsbdistid  => 'Debian',
+:memorysize_mb => 64420.94,
+}}
+it { should 
contain_file('/usr/local/bin/replicate-osm').with_content(/--input-reader xml/) 
}
+end
 end
+
 end
diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 827fc3e..875973b 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -18,6 +18,7 @@
 | /usr/bin/osm2pgsql \
   --hstore \
   --slim \
+  --input-reader <%= @input_reader_format %> \
   --cache <%= @memory_limit %> \
   --number-processes <%= @num_threads %> \
   --expire-tiles <%= @expire_levels %> \

-- 
To view, visit https://gerrit.wikimedia.org/r/293475
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I0482b718a496cfabf647295b413bce5e2a453375
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update configuration for elasticsearch 2.3 upgrade on eqiad - change (operations/puppet)

2016-06-03 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Update configuration for elasticsearch 2.3 upgrade on eqiad
..


Update configuration for elasticsearch 2.3 upgrade on eqiad

* bind_networks to _local_ and _site_
* publish_host _eth0_
* update mandatory plugins

Bug: T133126
Change-Id: I71fa30ecd6453dfd1d064aaf9cb76bb7d2fae64d
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/codfw/elasticsearch/server.yaml
M hieradata/role/common/elasticsearch/server.yaml
3 files changed, 8 insertions(+), 15 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  DCausse: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index f160014..629be2d 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -178,9 +178,6 @@
 - deployment-elastic06
 - deployment-elastic07
 - deployment-elastic08
-"elasticsearch::bind_networks":
-- _local_
-- _site_
 "elasticsearch::cluster_name": beta-search
 "elasticsearch::auto_create_index": '+apifeatureusage-*,-*'
 "elasticsearch::graylog_hosts":
diff --git a/hieradata/role/codfw/elasticsearch/server.yaml 
b/hieradata/role/codfw/elasticsearch/server.yaml
index 32945f7..5fde83b 100644
--- a/hieradata/role/codfw/elasticsearch/server.yaml
+++ b/hieradata/role/codfw/elasticsearch/server.yaml
@@ -9,17 +9,6 @@
   - elastic2007.codfw.wmnet
   - elastic2013.codfw.wmnet
 
-elasticsearch::bind_networks:
-  - _local_
-  - _site_
-
-elasticsearch::publish_host: _eth0_
-
-elasticsearch::plugins_mandatory:
-  - 'experimental-highlighter'
-  - 'extra'
-  - 'analysis-icu'
-
 lvs::realserver::realserver_ips: "10.2.1.30"
 
 elasticsearch::cluster_hosts:
diff --git a/hieradata/role/common/elasticsearch/server.yaml 
b/hieradata/role/common/elasticsearch/server.yaml
index 8593838..00e37b7 100644
--- a/hieradata/role/common/elasticsearch/server.yaml
+++ b/hieradata/role/common/elasticsearch/server.yaml
@@ -12,11 +12,18 @@
 # More than 30G isn't very useful
 elasticsearch::heap_memory: '30G'
 
+elasticsearch::bind_networks:
+  - _local_
+  - _site_
+
+elasticsearch::publish_host: _eth0_
+
 # Production elasticsearch needs these plugins to be loaded in order
 # to work properly.  This will keep elasticsearch from starting
 # if these plugins are  not available.
 elasticsearch::plugins_mandatory:
-  - 'experimental highlighter'
+  - 'experimental-highlighter'
+  - 'extra'
   - 'analysis-icu'
 
 # But it'd save time if we just waited for all of them to come back so

-- 
To view, visit https://gerrit.wikimedia.org/r/292541
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I71fa30ecd6453dfd1d064aaf9cb76bb7d2fae64d
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: DCausse 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update configuration for elasticsearch 2.3 upgrade on eqiad - change (operations/puppet)

2016-06-03 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292541

Change subject: Update configuration for elasticsearch 2.3 upgrade on eqiad
..

Update configuration for elasticsearch 2.3 upgrade on eqiad

* bind_networks to _local_ and _site_
* publish_host _eth0_
* update mandatory plugins

Change-Id: I71fa30ecd6453dfd1d064aaf9cb76bb7d2fae64d
---
M hieradata/role/eqiad/elasticsearch/server.yaml
1 file changed, 11 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/41/292541/1

diff --git a/hieradata/role/eqiad/elasticsearch/server.yaml 
b/hieradata/role/eqiad/elasticsearch/server.yaml
index 5507891..4fd9513 100644
--- a/hieradata/role/eqiad/elasticsearch/server.yaml
+++ b/hieradata/role/eqiad/elasticsearch/server.yaml
@@ -9,6 +9,17 @@
   - elastic1008.eqiad.wmnet
   - elastic1013.eqiad.wmnet
 
+elasticsearch::bind_networks:
+  - _local_
+  - _site_
+
+elasticsearch::publish_host: _eth0_
+
+elasticsearch::plugins_mandatory:
+  - 'experimental-highlighter'
+  - 'extra'
+  - 'analysis-icu'
+
 lvs::realserver::realserver_ips: "10.2.2.30"
 
 elasticsearch::cluster_hosts:

-- 
To view, visit https://gerrit.wikimedia.org/r/292541
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I71fa30ecd6453dfd1d064aaf9cb76bb7d2fae64d
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Move everything Postgres-related out of role::maps::server - change (operations/puppet)

2016-06-10 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Move everything Postgres-related out of role::maps::server
..


Move everything Postgres-related out of role::maps::server

This decouples database from Kartotherian and allows provisioning
them seaprately. I need this for testing in labs but this could
also be useful in prod.

Change-Id: I4bce77574b3492c3e3487efc6a304ceae6707a0f
---
M modules/role/manifests/maps/master.pp
A modules/role/manifests/maps/postgresql_common.pp
M modules/role/manifests/maps/server.pp
M modules/role/manifests/maps/slave.pp
4 files changed, 31 insertions(+), 24 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index b5a0fc5..b72ce08 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -1,6 +1,7 @@
 # Sets up a maps server master
 class role::maps::master {
 include ::postgresql::master
+include ::role::maps::postgresql_common
 include ::osm
 include ::osm::import_waterlines
 
diff --git a/modules/role/manifests/maps/postgresql_common.pp 
b/modules/role/manifests/maps/postgresql_common.pp
new file mode 100644
index 000..abdbca3
--- /dev/null
+++ b/modules/role/manifests/maps/postgresql_common.pp
@@ -0,0 +1,29 @@
+# Sets up Postgres settings common between master and slave roles
+class role::maps::postgresql_common {
+include ::postgresql::postgis
+
+# Tuning
+file { '/etc/postgresql/9.4/main/tuning.conf':
+  ensure => 'present',
+  owner  => 'root',
+  group  => 'root',
+  mode   => '0444',
+  source => 'puppet:///modules/role/maps/tuning.conf',
+}
+
+sysctl::parameters { 'postgres_shmem':
+  values => {
+# That is derived after tuning postgresql, deriving automatically is
+# not the safest idea yet.
+'kernel.shmmax' => 8388608000,
+  },
+}
+
+# TODO: Figure out a better way to do this
+# Ensure postgresql logs as maps-admin to allow maps-admin to read them
+# Rely on logrotate's copytruncate policy for postgres for the rest of the
+# log file
+file { '/var/log/postgresql/postgresql-9.4-main.log':
+  group => 'maps-admins',
+}
+}
diff --git a/modules/role/manifests/maps/server.pp 
b/modules/role/manifests/maps/server.pp
index 64232e1..4c188ad 100644
--- a/modules/role/manifests/maps/server.pp
+++ b/modules/role/manifests/maps/server.pp
@@ -2,7 +2,6 @@
 # the services kartotherian and tilerator
 class role::maps::server {
 include standard
-include ::postgresql::postgis
 include ::cassandra
 include ::cassandra::metrics
 include ::cassandra::logging
@@ -17,29 +16,6 @@
 
 if $::realm == 'production' {
 include lvs::realserver
-}
-
-# Tuning
-file { '/etc/postgresql/9.4/main/tuning.conf':
-ensure => 'present',
-owner  => 'root',
-group  => 'root',
-mode   => '0444',
-source => 'puppet:///modules/role/maps/tuning.conf',
-}
-sysctl::parameters { 'postgres_shmem':
-values => {
-# That is derived after tuning postgresql, deriving automatically 
is
-# not the safest idea yet.
-'kernel.shmmax' => 8388608000,
-},
-}
-# TODO: Figure out a better way to do this
-# Ensure postgresql logs as maps-admin to allow maps-admin to read them
-# Rely on logrotate's copytruncate policy for postgres for the rest of the
-# log file
-file { '/var/log/postgresql/postgresql-9.4-main.log':
-group => 'maps-admins',
 }
 
 $cassandra_hosts = hiera('cassandra::seeds')
diff --git a/modules/role/manifests/maps/slave.pp 
b/modules/role/manifests/maps/slave.pp
index 5ac2869..5d24736 100644
--- a/modules/role/manifests/maps/slave.pp
+++ b/modules/role/manifests/maps/slave.pp
@@ -1,6 +1,7 @@
 # Sets up a maps server slave
 class role::maps::slave {
 include ::postgresql::slave
+include ::role::maps::postgresql_common
 
 system::role { 'role::maps::slave':
 ensure  => 'present',

-- 
To view, visit https://gerrit.wikimedia.org/r/293540
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I4bce77574b3492c3e3487efc6a304ceae6707a0f
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: MaxSem 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Don't publish etags for WDQS - change (operations/puppet)

2016-06-10 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Don't publish etags for WDQS
..


Don't publish etags for WDQS

Etags in nginx are based on mtime, which is inconsistent between different
servers. This leads to unnecessary reload of resources. Removing etags should
allow if-modified-since to work better.

Bug: T137238
Change-Id: I272794341fd511766fcb5dc58b01dc4b8bc9a33e
---
M modules/wdqs/templates/nginx.erb
1 file changed, 5 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Smalyshev: Looks good to me, but someone else must approve
  BBlack: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/wdqs/templates/nginx.erb b/modules/wdqs/templates/nginx.erb
index da3cd93..7cdf924 100644
--- a/modules/wdqs/templates/nginx.erb
+++ b/modules/wdqs/templates/nginx.erb
@@ -13,6 +13,11 @@
 
 add_header X-Served-By <%= @hostname %> always;
 
+# etag in nginx are based on mtime, which is not guaranteed to be the same
+# across multiple servers so let's disable etags and use more robust
+# caching mechanisms
+etag off;
+
 error_page 503 @maintenance;
 location @maintenance {
 root <%= @package_dir %>/gui;

-- 
To view, visit https://gerrit.wikimedia.org/r/293492
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I272794341fd511766fcb5dc58b01dc4b8bc9a33e
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: BBlack 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Smalyshev 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Make all map sources public for admins of tileratorui - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Make all map sources public for admins of tileratorui
..


Make all map sources public for admins of tileratorui

Bug: T137053
Change-Id: I64aa688caba653cd0033b89646c120f56dcbddc6
---
M modules/tilerator/templates/config_ui.yaml.erb
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/tilerator/templates/config_ui.yaml.erb 
b/modules/tilerator/templates/config_ui.yaml.erb
index 4c0429b..447e97d 100644
--- a/modules/tilerator/templates/config_ui.yaml.erb
+++ b/modules/tilerator/templates/config_ui.yaml.erb
@@ -10,3 +10,4 @@
 
 sources: <%= @conf_sources %>
 uiOnly: true
+allSourcesPublic: true

-- 
To view, visit https://gerrit.wikimedia.org/r/294192
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I64aa688caba653cd0033b89646c120f56dcbddc6
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Yurik 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: RobH 
Gerrit-Reviewer: Yuvipanda 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Enable 'has_spec' on Kartotherian service. - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Enable 'has_spec' on Kartotherian service.
..


Enable 'has_spec' on Kartotherian service.

Spec has been written and deployed for Kartotherian. We now need to use it.
To not spam everyone while testing, this is disabled on all servers except
maps2001. This will need to be cleaned once we confirm that all works
correctly.

Bug: T137617
Change-Id: Ibbd69c672b477d7d71326f12fb181c2201d39ab9
---
M hieradata/hosts/maps2001.yaml
M hieradata/role/common/maps/server.yaml
M modules/kartotherian/manifests/init.pp
3 files changed, 7 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  MaxSem: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index db7b54b..adafca6 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -15,3 +15,5 @@
 tilerator::conf_sources: sources.prod2.yaml
 tilerator::ui::conf_sources: sources.prod2.yaml
 kartotherian::conf_sources: sources.prod2.yaml
+# enable specs only on maps2001 for testing
+kartotherian::enable_spec: true
diff --git a/hieradata/role/common/maps/server.yaml 
b/hieradata/role/common/maps/server.yaml
index 0249efa..47dab51 100644
--- a/hieradata/role/common/maps/server.yaml
+++ b/hieradata/role/common/maps/server.yaml
@@ -12,3 +12,6 @@
   - tilerator-admin
 service::configuration::logstash_host: logstash1001.eqiad.wmnet
 service::configuration::statsd_host: statsd.eqiad.wmnet
+
+# Let's keep spec disabled until it is correctly tested
+kartotherian::enable_spec: false
diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index c5886c6..b998a04 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -8,6 +8,7 @@
 # service
 class kartotherian(
 $conf_sources = 'sources.prod.yaml',
+$enable_spec  = true,
 ) {
 
 $cassandra_kartotherian_user = 'kartotherian'
@@ -19,5 +20,6 @@
 port   => 6533,
 config => template('kartotherian/config.yaml.erb'),
 deployment => 'scap3',
+has_spec   => $enable_spec,
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294028
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ibbd69c672b477d7d71326f12fb181c2201d39ab9
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Move service logs to /srv/log - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294285

Change subject: Move service logs to /srv/log
..

Move service logs to /srv/log

Sending logs to /var/log/kartotherian does not work, probably conflicting with
firejail. As suggested by mobrovac, let's try to move those logs somewhere
else to validate this.

Bug: T137618
Change-Id: I9efe0baaf50a54ec13d877e06a28dbb81ae094f7
---
M hieradata/hosts/maps2001.yaml
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/85/294285/1

diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index db7b54b..753100b 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -15,3 +15,5 @@
 tilerator::conf_sources: sources.prod2.yaml
 tilerator::ui::conf_sources: sources.prod2.yaml
 kartotherian::conf_sources: sources.prod2.yaml
+
+service::configuration::log_dir: /srv/log
\ No newline at end of file

-- 
To view, visit https://gerrit.wikimedia.org/r/294285
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9efe0baaf50a54ec13d877e06a28dbb81ae094f7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Move service logs to /srv/log" - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Revert "Move service logs to /srv/log"
..


Revert "Move service logs to /srv/log"

This reverts commit 2556fccc27cc40371d9d6748b8850dca181c93b4.

We will do a cleaner fix in the puppet-service module itself

Bug: T137618
Change-Id: If0a49b54af8c81f818b6bb80cd63f23ec17bf3e4
---
M hieradata/hosts/maps2001.yaml
1 file changed, 0 insertions(+), 2 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index 662e6366..adafca6 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -17,5 +17,3 @@
 kartotherian::conf_sources: sources.prod2.yaml
 # enable specs only on maps2001 for testing
 kartotherian::enable_spec: true
-
-service::configuration::log_dir: /srv/log
\ No newline at end of file

-- 
To view, visit https://gerrit.wikimedia.org/r/294286
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If0a49b54af8c81f818b6bb80cd63f23ec17bf3e4
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Send service logs by default to /srv/log. - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294288

Change subject: Send service logs by default to /srv/log.
..

Send service logs by default to /srv/log.

As firejail is used to start services and firejail mounts a tmpfs over
/var/log, using /var/log does not work as expected. /srv/log is not protected
by firejail and is already used by some services.

Change-Id: Iedae3ed12e3df4b2717c3d653ec89fc8220ca2c9
---
M hieradata/role/common/sca.yaml
M hieradata/role/common/scb.yaml
M modules/service/manifests/configuration.pp
3 files changed, 10 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/88/294288/1

diff --git a/hieradata/role/common/sca.yaml b/hieradata/role/common/sca.yaml
index d5d37e8..78e8f29 100644
--- a/hieradata/role/common/sca.yaml
+++ b/hieradata/role/common/sca.yaml
@@ -3,7 +3,6 @@
   - sc-admins
   - zotero-admin
   - apertium-admins
-service::configuration::log_dir: /srv/log
 debdeploy::grains:
   debdeploy-sca:
 value: standard
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index cef6d88..8b244ea 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -10,7 +10,6 @@
 - mobileapps-admin
 - changeprop-admin
 - ores-admin
-service::configuration::log_dir: /srv/log
 graphoid::allowed_domains:
   https:
 - mediawiki.org
diff --git a/modules/service/manifests/configuration.pp 
b/modules/service/manifests/configuration.pp
index c10c056..9602c36 100644
--- a/modules/service/manifests/configuration.pp
+++ b/modules/service/manifests/configuration.pp
@@ -29,7 +29,7 @@
 #
 # [*log_dir*]
 #   Local root log directory. The service's logs will be placed in its
-#   subdirectory. Default: /var/log
+#   subdirectory. Default: /srv/log
 #
 # [*use_dev_pkgs*]
 #   Whether the development packages declared for services should be installed.
@@ -43,8 +43,15 @@
 $statsd_port   = 8125,
 $logstash_host = 'localhost',
 $logstash_port = 12201,
-$log_dir   = '/var/log',
+$log_dir   = '/srv/log',
 $use_dev_pkgs  = false,
 ){
-# No op for now.
+
+file { $log_dir:
+ensure => present,
+owner  => 'root',
+group  => 'root',
+mode   => '0755',
+}
+
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294288
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iedae3ed12e3df4b2717c3d653ec89fc8220ca2c9
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Move service logs to /srv/log - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Move service logs to /srv/log
..


Move service logs to /srv/log

Sending logs to /var/log/kartotherian does not work, probably conflicting with
firejail. As suggested by mobrovac, let's try to move those logs somewhere
else to validate this. And let's do this test on a single server first,
cleanup will be required if moving the logs is the final solution.

Bug: T137618
Change-Id: I9efe0baaf50a54ec13d877e06a28dbb81ae094f7
---
M hieradata/hosts/maps2001.yaml
1 file changed, 2 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Mobrovac: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index adafca6..662e6366 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -17,3 +17,5 @@
 kartotherian::conf_sources: sources.prod2.yaml
 # enable specs only on maps2001 for testing
 kartotherian::enable_spec: true
+
+service::configuration::log_dir: /srv/log
\ No newline at end of file

-- 
To view, visit https://gerrit.wikimedia.org/r/294285
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I9efe0baaf50a54ec13d877e06a28dbb81ae094f7
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Move service logs to /srv/log" - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294286

Change subject: Revert "Move service logs to /srv/log"
..

Revert "Move service logs to /srv/log"

This reverts commit 2556fccc27cc40371d9d6748b8850dca181c93b4.

We will do a cleaner fix in the puppet-service module itself

Bug: T137618
Change-Id: If0a49b54af8c81f818b6bb80cd63f23ec17bf3e4
---
M hieradata/hosts/maps2001.yaml
1 file changed, 0 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/86/294286/1

diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index 662e6366..adafca6 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -17,5 +17,3 @@
 kartotherian::conf_sources: sources.prod2.yaml
 # enable specs only on maps2001 for testing
 kartotherian::enable_spec: true
-
-service::configuration::log_dir: /srv/log
\ No newline at end of file

-- 
To view, visit https://gerrit.wikimedia.org/r/294286
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If0a49b54af8c81f818b6bb80cd63f23ec17bf3e4
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] service::configuration: Set the default log_dir to /srv/log - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: service::configuration: Set the default log_dir to /srv/log
..


service::configuration: Set the default log_dir to /srv/log

Most services that use service::configuration store their logs in
/srv/log (changed in Hiera), but some still use /var/log. Make the
former the default for uniformity. Another point is that all of these
services are firejail-ed, which makes /var read-only, hence making it
pointless for services to try to log into /var/log.

Also fix a bug whereby service::node assumed
service::configuration::log_dir existed.

Change-Id: I2fd39e72f3b400a1b09320e92fa8105b847ab439
---
M hieradata/role/common/sca.yaml
M hieradata/role/common/scb.yaml
M modules/service/manifests/configuration.pp
M modules/service/manifests/node.pp
4 files changed, 16 insertions(+), 9 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Muehlenhoff: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/role/common/sca.yaml b/hieradata/role/common/sca.yaml
index d5d37e8..78e8f29 100644
--- a/hieradata/role/common/sca.yaml
+++ b/hieradata/role/common/sca.yaml
@@ -3,7 +3,6 @@
   - sc-admins
   - zotero-admin
   - apertium-admins
-service::configuration::log_dir: /srv/log
 debdeploy::grains:
   debdeploy-sca:
 value: standard
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index cef6d88..8b244ea 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -10,7 +10,6 @@
 - mobileapps-admin
 - changeprop-admin
 - ores-admin
-service::configuration::log_dir: /srv/log
 graphoid::allowed_domains:
   https:
 - mediawiki.org
diff --git a/modules/service/manifests/configuration.pp 
b/modules/service/manifests/configuration.pp
index c10c056..f6281cf 100644
--- a/modules/service/manifests/configuration.pp
+++ b/modules/service/manifests/configuration.pp
@@ -29,7 +29,7 @@
 #
 # [*log_dir*]
 #   Local root log directory. The service's logs will be placed in its
-#   subdirectory. Default: /var/log
+#   subdirectory. Default: /srv/log
 #
 # [*use_dev_pkgs*]
 #   Whether the development packages declared for services should be installed.
@@ -43,7 +43,7 @@
 $statsd_port   = 8125,
 $logstash_host = 'localhost',
 $logstash_port = 12201,
-$log_dir   = '/var/log',
+$log_dir   = '/srv/log',
 $use_dev_pkgs  = false,
 ){
 # No op for now.
diff --git a/modules/service/manifests/node.pp 
b/modules/service/manifests/node.pp
index 79d2b15..1ca33ea 100644
--- a/modules/service/manifests/node.pp
+++ b/modules/service/manifests/node.pp
@@ -247,12 +247,21 @@
 }
 
 if $local_logging {
+if !defined(File[$::service::configuration::log_dir]) {
+file { $::service::configuration::log_dir:
+ensure => directory,
+owner  => 'root',
+group  => 'root',
+mode   => '0755',
+}
+}
 file { $local_logdir:
-ensure => directory,
-owner  => $title,
-group  => 'root',
-mode   => '0755',
-before => Service[$title],
+ensure  => directory,
+owner   => $title,
+group   => 'root',
+mode=> '0755',
+before  => Service[$title],
+require => File[$::service::configuration::log_dir],
 }
 file { "/etc/logrotate.d/${title}":
 content => template('service/logrotate.erb'),

-- 
To view, visit https://gerrit.wikimedia.org/r/294289
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I2fd39e72f3b400a1b09320e92fa8105b847ab439
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Mobrovac 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: Muehlenhoff 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Kartotherian: Use the root URL for the checker script - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Kartotherian: Use the root URL for the checker script
..


Kartotherian: Use the root URL for the checker script

Change-Id: I85730e37caffdad50e8a78d60986ceaee92d5946
---
M modules/kartotherian/manifests/init.pp
1 file changed, 10 insertions(+), 4 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index b998a04..f04ac7d 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -16,10 +16,16 @@
 $pgsql_kartotherian_user = 'kartotherian'
 $pgsql_kartotherian_pass = hiera('maps::postgresql_kartotherian_pass')
 
+$monitor_url = $enable_spec ? {
+true=> '',
+default => '/_info',
+}
+
 service::node { 'kartotherian':
-port   => 6533,
-config => template('kartotherian/config.yaml.erb'),
-deployment => 'scap3',
-has_spec   => $enable_spec,
+port=> 6533,
+config  => template('kartotherian/config.yaml.erb'),
+deployment  => 'scap3',
+has_spec=> $enable_spec,
+healthcheck_url => $monitor_url,
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294313
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I85730e37caffdad50e8a78d60986ceaee92d5946
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Mobrovac 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Enable kartotherian service check for all maps servers - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294333

Change subject: Enable kartotherian service check for all maps servers
..

Enable kartotherian service check for all maps servers

Now that specs have been properly tested, we can enable them for all maps
servers.

Bug: T137617
Change-Id: I3b61be2311a9e3506d6b386c97e56026eee3be8f
---
M hieradata/hosts/maps2001.yaml
M hieradata/role/common/maps/server.yaml
M modules/kartotherian/manifests/init.pp
3 files changed, 2 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/33/294333/1

diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index adafca6..db7b54b 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -15,5 +15,3 @@
 tilerator::conf_sources: sources.prod2.yaml
 tilerator::ui::conf_sources: sources.prod2.yaml
 kartotherian::conf_sources: sources.prod2.yaml
-# enable specs only on maps2001 for testing
-kartotherian::enable_spec: true
diff --git a/hieradata/role/common/maps/server.yaml 
b/hieradata/role/common/maps/server.yaml
index 47dab51..0249efa 100644
--- a/hieradata/role/common/maps/server.yaml
+++ b/hieradata/role/common/maps/server.yaml
@@ -12,6 +12,3 @@
   - tilerator-admin
 service::configuration::logstash_host: logstash1001.eqiad.wmnet
 service::configuration::statsd_host: statsd.eqiad.wmnet
-
-# Let's keep spec disabled until it is correctly tested
-kartotherian::enable_spec: false
diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index f04ac7d..0638390 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -8,7 +8,6 @@
 # service
 class kartotherian(
 $conf_sources = 'sources.prod.yaml',
-$enable_spec  = true,
 ) {
 
 $cassandra_kartotherian_user = 'kartotherian'
@@ -16,16 +15,11 @@
 $pgsql_kartotherian_user = 'kartotherian'
 $pgsql_kartotherian_pass = hiera('maps::postgresql_kartotherian_pass')
 
-$monitor_url = $enable_spec ? {
-true=> '',
-default => '/_info',
-}
-
 service::node { 'kartotherian':
 port=> 6533,
 config  => template('kartotherian/config.yaml.erb'),
 deployment  => 'scap3',
-has_spec=> $enable_spec,
-healthcheck_url => $monitor_url,
+has_spec=> true,
+healthcheck_url => '',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294333
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3b61be2311a9e3506d6b386c97e56026eee3be8f
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Enable kartotherian service check for all maps servers - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Enable kartotherian service check for all maps servers
..


Enable kartotherian service check for all maps servers

Now that specs have been properly tested, we can enable them for all maps
servers.

Bug: T137617
Change-Id: I3b61be2311a9e3506d6b386c97e56026eee3be8f
---
M hieradata/hosts/maps2001.yaml
M hieradata/role/common/maps/server.yaml
M modules/kartotherian/manifests/init.pp
3 files changed, 2 insertions(+), 13 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Yurik: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index adafca6..db7b54b 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -15,5 +15,3 @@
 tilerator::conf_sources: sources.prod2.yaml
 tilerator::ui::conf_sources: sources.prod2.yaml
 kartotherian::conf_sources: sources.prod2.yaml
-# enable specs only on maps2001 for testing
-kartotherian::enable_spec: true
diff --git a/hieradata/role/common/maps/server.yaml 
b/hieradata/role/common/maps/server.yaml
index 47dab51..0249efa 100644
--- a/hieradata/role/common/maps/server.yaml
+++ b/hieradata/role/common/maps/server.yaml
@@ -12,6 +12,3 @@
   - tilerator-admin
 service::configuration::logstash_host: logstash1001.eqiad.wmnet
 service::configuration::statsd_host: statsd.eqiad.wmnet
-
-# Let's keep spec disabled until it is correctly tested
-kartotherian::enable_spec: false
diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index f04ac7d..0638390 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -8,7 +8,6 @@
 # service
 class kartotherian(
 $conf_sources = 'sources.prod.yaml',
-$enable_spec  = true,
 ) {
 
 $cassandra_kartotherian_user = 'kartotherian'
@@ -16,16 +15,11 @@
 $pgsql_kartotherian_user = 'kartotherian'
 $pgsql_kartotherian_pass = hiera('maps::postgresql_kartotherian_pass')
 
-$monitor_url = $enable_spec ? {
-true=> '',
-default => '/_info',
-}
-
 service::node { 'kartotherian':
 port=> 6533,
 config  => template('kartotherian/config.yaml.erb'),
 deployment  => 'scap3',
-has_spec=> $enable_spec,
-healthcheck_url => $monitor_url,
+has_spec=> true,
+healthcheck_url => '',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294333
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I3b61be2311a9e3506d6b386c97e56026eee3be8f
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add new maps servers to LVS - change (operations/puppet)

2016-06-14 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Add new maps servers to LVS
..


Add new maps servers to LVS

New maps servers will be ready to handle user traffic very soon. This change
is about getting ready to send that traffic.

Documentation on how to deploy this change:
https://wikitech.wikimedia.org/wiki/LVS#Deploy_a_change_to_an_existing_service

Bug: T137620
Change-Id: I86bbe007ef91d77aefa800112add4e1e70d72571
---
M conftool-data/nodes/codfw.yaml
1 file changed, 4 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/conftool-data/nodes/codfw.yaml b/conftool-data/nodes/codfw.yaml
index 3365b8e..08d553a 100644
--- a/conftool-data/nodes/codfw.yaml
+++ b/conftool-data/nodes/codfw.yaml
@@ -217,6 +217,10 @@
   acamar.wikimedia.org: [pdns_recursor]
   achernar.wikimedia.org: [pdns_recursor]
 maps:
+  maps2001.codfw.wmnet: [kartotherian]
+  maps2002.codfw.wmnet: [kartotherian]
+  maps2003.codfw.wmnet: [kartotherian]
+  maps2004.codfw.wmnet: [kartotherian]
   maps-test2001.codfw.wmnet: [kartotherian]
   maps-test2002.codfw.wmnet: [kartotherian]
   maps-test2003.codfw.wmnet: [kartotherian]

-- 
To view, visit https://gerrit.wikimedia.org/r/294068
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I86bbe007ef91d77aefa800112add4e1e70d72571
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: BBlack 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: RobH 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add the ability to configure contact group for check of serv... - change (operations/puppet)

2016-06-15 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294507

Change subject: Add the ability to configure contact group for check of 
services.
..

Add the ability to configure contact group for check of services.

For example, on Maps servers, it make sense to the interactive team to
be notified of service issues, but not to receive all notifications on
which they cannot act anyway.

The immediate need is on service::node, but I updated service::uwsgi as well
to keep coherence.

Bug: T137869
Bug: T137891
Change-Id: I1a1911263c04132d90f53e28ab8c5d2fc3a3a2c2
---
M modules/service/manifests/node.pp
M modules/service/manifests/uwsgi.pp
2 files changed, 23 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/07/294507/1

diff --git a/modules/service/manifests/node.pp 
b/modules/service/manifests/node.pp
index 1ca33ea..fa3dec4 100644
--- a/modules/service/manifests/node.pp
+++ b/modules/service/manifests/node.pp
@@ -84,6 +84,11 @@
 #   Whether Scap3 is used for deploying the config as well. Applicable only 
when
 #   $deployment == 'scap3'. Default: false
 #
+# [*contact_groups*]
+#   Contact groups for alerting.
+#   Default: hiera('contactgroups', 'admins') - use 'contactgroups' hiera
+#variable with a fallback to 'admins' if 'contactgroups' isn't set.
+#
 # === Examples
 #
 # To set up a service named myservice on port 8520 and with a templated
@@ -123,6 +128,7 @@
 $deployment  = undef,
 $deployment_user = 'deploy-service',
 $deployment_config = false,
+$contact_groups  = hiera('contactgroups', 'admins'),
 ) {
 case $deployment {
 'scap3': {
@@ -325,10 +331,11 @@
 mode=> '0755',
 }
 nrpe::monitor_service{ "endpoints_${title}":
-ensure   => $ensure_monitoring,
-description  => "${title} endpoints health",
-nrpe_command => "/usr/local/bin/check-${title}",
-subscribe=> File["/usr/local/bin/check-${title}"],
+ensure=> $ensure_monitoring,
+description   => "${title} endpoints health",
+nrpe_command  => "/usr/local/bin/check-${title}",
+subscribe => File["/usr/local/bin/check-${title}"],
+contact_group => $contact_groups,
 }
 # we also support smart-releases
 service::deployment_script { $name:
@@ -341,6 +348,7 @@
 ensure=> $ensure_monitoring,
 description   => $title,
 check_command => "check_http_port_url!${port}!${healthcheck_url}",
+contact_group => $contact_groups,
 }
 }
 
diff --git a/modules/service/manifests/uwsgi.pp 
b/modules/service/manifests/uwsgi.pp
index e8a8021..b29d3e9 100644
--- a/modules/service/manifests/uwsgi.pp
+++ b/modules/service/manifests/uwsgi.pp
@@ -55,6 +55,11 @@
 #   An array of string representing sudo rules in the sudoers format that you
 #   want the service to have. Default: empty array
 #
+# [*contact_groups*]
+#   Contact groups for alerting.
+#   Default: hiera('contactgroups', 'admins') - use 'contactgroups' hiera
+#variable with a fallback to 'admins' if 'contactgroups' isn't set.
+#
 # === Examples
 #
 #service::uwsgi { 'myservice':
@@ -81,6 +86,7 @@
 $deployment_manage_user = true,
 $deployment = 'scap3',
 $sudo_rules = [],
+$contact_groups = hiera('contactgroups', 'admins'),
 ) {
 if $deployment == 'scap3' {
 scap::target { $repo:
@@ -168,9 +174,10 @@
 
 $monitor_url = "http://${::ipaddress}:${port}${healthcheck_url};
 nrpe::monitor_service{ "endpoints_${title}":
-description  => "${title} endpoints health",
-nrpe_command => "/usr/local/lib/nagios/plugins/service_checker -t 
5 ${::ipaddress} ${monitor_url}",
-subscribe=> 
File['/usr/local/lib/nagios/plugins/service_checker'],
+description   => "${title} endpoints health",
+nrpe_command  => "/usr/local/lib/nagios/plugins/service_checker -t 
5 ${::ipaddress} ${monitor_url}",
+subscribe => 
File['/usr/local/lib/nagios/plugins/service_checker'],
+contact_group => $contact_groups,
 }
 # we also support smart-releases
 # TODO: Enable has_autorestart
@@ -182,6 +189,7 @@
 monitoring::service { $title:
 description   => $title,
 check_command => "check_http_port_url!${port}!${healthcheck_url}",
+contact_group => $contact_groups,
 }
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294507
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1a1911263c04132d90f53e28ab8c5d2fc3a3a2c2
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner:

[MediaWiki-commits] [Gerrit] Add interactive-team to default Icinga notification group fo... - change (operations/puppet)

2016-06-15 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294503

Change subject: Add interactive-team to default Icinga notification group for 
maps servers
..

Add interactive-team to default Icinga notification group for maps servers

This needs confirmation from yurik and MaxSem, but they probably want to
know when something happens to Maps.

Bug: T137869
Change-Id: Id2672c8cf9a65cac9a05747ec3926909f71d9ca6
---
M hieradata/role/common/maps/server.yaml
1 file changed, 3 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/03/294503/1

diff --git a/hieradata/role/common/maps/server.yaml 
b/hieradata/role/common/maps/server.yaml
index 0249efa..d20560d 100644
--- a/hieradata/role/common/maps/server.yaml
+++ b/hieradata/role/common/maps/server.yaml
@@ -12,3 +12,6 @@
   - tilerator-admin
 service::configuration::logstash_host: logstash1001.eqiad.wmnet
 service::configuration::statsd_host: statsd.eqiad.wmnet
+
+# Icinga notifications are sent by default to:
+contactgroups: 'admins,interactive-team'

-- 
To view, visit https://gerrit.wikimedia.org/r/294503
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id2672c8cf9a65cac9a05747ec3926909f71d9ca6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Adding gehel (Guillaume Lederrey) to icinga sms group - change (operations/puppet)

2016-06-15 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294486

Change subject: Adding gehel (Guillaume Lederrey) to icinga sms group
..

Adding gehel (Guillaume Lederrey) to icinga sms group

Guillaume should receive pages like any other ops.
SMS configuration has been done in private git repo.

Change-Id: I362e840d4df45310b16a3ed4d475b08039baed2a
---
M modules/nagios_common/files/contactgroups.cfg
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/86/294486/1

diff --git a/modules/nagios_common/files/contactgroups.cfg 
b/modules/nagios_common/files/contactgroups.cfg
index 90ea442..e603936 100644
--- a/modules/nagios_common/files/contactgroups.cfg
+++ b/modules/nagios_common/files/contactgroups.cfg
@@ -41,7 +41,7 @@
 
 define contactgroup {
 contactgroup_name   sms
-members 
akosiaris,andrew,ariel,bblack,cmjohnson,dzahn,ema,faidon,fgiunchedi,jgreen,mark,otto,volans,robh,springle,tstarling,rush,glavagetto,yuvipanda,jmm,jcrespo,team-operations
+members 
akosiaris,andrew,ariel,bblack,cmjohnson,dzahn,ema,faidon,fgiunchedi,jgreen,mark,otto,volans,robh,springle,tstarling,rush,glavagetto,yuvipanda,jmm,jcrespo,team-operations,gehel
 }
 
 define contactgroup {

-- 
To view, visit https://gerrit.wikimedia.org/r/294486
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I362e840d4df45310b16a3ed4d475b08039baed2a
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Adding a "interactive-team" icinga group for alerting. - change (operations/puppet)

2016-06-15 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294498

Change subject: Adding a "interactive-team" icinga group for alerting.
..

Adding a "interactive-team" icinga group for alerting.

This adds an icinga group for the interactive team (yurik, MaxSem and gehel).
It also sends notifications to #wikimedia-interactive.

This requires the contacts for yurik and MaxSem to be added in our private
git repo.

Bug: T137869
Change-Id: I5ad6fcb369985939bc6f4d9a6a87891bc0f0f7fb
---
M modules/icinga/manifests/ircbot.pp
M modules/nagios_common/files/contactgroups.cfg
M modules/nagios_common/templates/notification_commands.cfg.erb
3 files changed, 24 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/98/294498/1

diff --git a/modules/icinga/manifests/ircbot.pp 
b/modules/icinga/manifests/ircbot.pp
index 3b25e94..6cd2ae9 100644
--- a/modules/icinga/manifests/ircbot.pp
+++ b/modules/icinga/manifests/ircbot.pp
@@ -4,12 +4,13 @@
 class icinga::ircbot {
 
 $ircecho_logs   = {
-'/var/log/icinga/irc.log'   => '#wikimedia-operations',
-'/var/log/icinga/irc-wikidata.log'  => '#wikidata',
-'/var/log/icinga/irc-releng.log'=> '#wikimedia-releng',
-'/var/log/icinga/irc-labs.log'  => '#wikimedia-labs',
-'/var/log/icinga/irc-analytics.log' => '#wikimedia-analytics',
-'/var/log/icinga/irc-ores.log'  => '#wikimedia-ai',
+'/var/log/icinga/irc.log' => '#wikimedia-operations',
+'/var/log/icinga/irc-wikidata.log'=> '#wikidata',
+'/var/log/icinga/irc-releng.log'  => '#wikimedia-releng',
+'/var/log/icinga/irc-labs.log'=> '#wikimedia-labs',
+'/var/log/icinga/irc-analytics.log'   => '#wikimedia-analytics',
+'/var/log/icinga/irc-ores.log'=> '#wikimedia-ai',
+'/var/log/icinga/irc-interactive.log' => '#wikimedia-interactive',
 }
 $ircecho_nick   = 'icinga-wm'
 $ircecho_server = 'chat.freenode.net'
diff --git a/modules/nagios_common/files/contactgroups.cfg 
b/modules/nagios_common/files/contactgroups.cfg
index 90ea442..05c4da2 100644
--- a/modules/nagios_common/files/contactgroups.cfg
+++ b/modules/nagios_common/files/contactgroups.cfg
@@ -74,3 +74,8 @@
 contactgroup_name   team-discovery
 members discovery-alerts,irc
 }
+
+define contactgroup {
+contactgroup_name   team-interactive
+members irc-interactive,yurik,MaxSem,gehel
+}
diff --git a/modules/nagios_common/templates/notification_commands.cfg.erb 
b/modules/nagios_common/templates/notification_commands.cfg.erb
index 22b8bfb..607110d 100644
--- a/modules/nagios_common/templates/notification_commands.cfg.erb
+++ b/modules/nagios_common/templates/notification_commands.cfg.erb
@@ -68,7 +68,18 @@
command_lineecho "$NOTIFICATIONTYPE$ - $SERVICEDESC$ on $HOSTNAME$ 
is $SERVICESTATE$: $SERVICEOUTPUT$ $SERVICEACKAUTHOR$ $SERVICEACKCOMMENT$" >> 
<%= @irc_dir_path %>/irc-wikidata.log
}
 
-# IRC output for RelEng team
+# IRC output for interactive
+define command{
+   command_namenotify-host-by-irc-interactive
+   command_lineecho "$NOTIFICATIONTYPE$ - Host $HOSTALIAS$ is 
$HOSTSTATE$: $HOSTOUTPUT$ $HOSTACKAUTHOR$ $HOSTACKCOMMENT$" >> <%= 
@irc_dir_path %>/irc-interactive.log
+   }
+
+define command{
+   command_namenotify-service-by-irc-interactive
+   command_lineecho "$NOTIFICATIONTYPE$ - $SERVICEDESC$ on $HOSTNAME$ 
is $SERVICESTATE$: $SERVICEOUTPUT$ $SERVICEACKAUTHOR$ $SERVICEACKCOMMENT$" >> 
<%= @irc_dir_path %>/irc-interactive.log
+   }
+
+   # IRC output for RelEng team
 define command{
command_namenotify-host-by-irc-releng
command_lineecho "$NOTIFICATIONTYPE$ - Host $HOSTALIAS$ is 
$HOSTSTATE$: $HOSTOUTPUT$ $HOSTACKAUTHOR$ $HOSTACKCOMMENT$" >> <%= 
@irc_dir_path %>/irc-releng.log

-- 
To view, visit https://gerrit.wikimedia.org/r/294498
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5ad6fcb369985939bc6f4d9a6a87891bc0f0f7fb
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add new maps servers to LVS - change (operations/puppet)

2016-06-13 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294068

Change subject: Add new maps servers to LVS
..

Add new maps servers to LVS

New maps servers will be ready to handle user traffic very soon. This change
is about getting ready to send that traffic.

Documentation on how to deploy this change:
https://wikitech.wikimedia.org/wiki/LVS#Deploy_a_change_to_an_existing_service

Bug: T137620
Change-Id: I86bbe007ef91d77aefa800112add4e1e70d72571
---
M conftool-data/nodes/codfw.yaml
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/68/294068/1

diff --git a/conftool-data/nodes/codfw.yaml b/conftool-data/nodes/codfw.yaml
index 05c1b46..051ec9c 100644
--- a/conftool-data/nodes/codfw.yaml
+++ b/conftool-data/nodes/codfw.yaml
@@ -191,6 +191,10 @@
   acamar.wikimedia.org: [pdns_recursor]
   achernar.wikimedia.org: [pdns_recursor]
 maps:
+  maps2001.codfw.wmnet: [kartotherian]
+  maps2002.codfw.wmnet: [kartotherian]
+  maps2003.codfw.wmnet: [kartotherian]
+  maps2004.codfw.wmnet: [kartotherian]
   maps-test2001.codfw.wmnet: [kartotherian]
   maps-test2002.codfw.wmnet: [kartotherian]
   maps-test2003.codfw.wmnet: [kartotherian]

-- 
To view, visit https://gerrit.wikimedia.org/r/294068
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I86bbe007ef91d77aefa800112add4e1e70d72571
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Fixing lint issues - change (operations/puppet)

2016-06-08 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/293377

Change subject: Fixing lint issues
..

Fixing lint issues

those issue were probably introduced while Jenkins was taking a nap...

Change-Id: I98ebd391b1ca29d8023e0bbaf37a83e14ea8bca7
---
M modules/role/manifests/ci/master.pp
M modules/role/manifests/zookeeper/client.pp
2 files changed, 6 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/77/293377/1

diff --git a/modules/role/manifests/ci/master.pp 
b/modules/role/manifests/ci/master.pp
index 3b351ab..0bbd5e3 100644
--- a/modules/role/manifests/ci/master.pp
+++ b/modules/role/manifests/ci/master.pp
@@ -71,10 +71,10 @@
 require contint::master_dir
 
 file { '/srv/ssd/jenkins':
-ensure  => 'directory',
-owner   => 'jenkins',
-group   => 'jenkins',
-mode=> '2775',  # group sticky bit
+ensure => 'directory',
+owner  => 'jenkins',
+group  => 'jenkins',
+mode   => '2775',  # group sticky bit
 }
 
 # Master does not run job anymore since June 2013. But better safe than
diff --git a/modules/role/manifests/zookeeper/client.pp 
b/modules/role/manifests/zookeeper/client.pp
index f0adf1e..66a0a53 100644
--- a/modules/role/manifests/zookeeper/client.pp
+++ b/modules/role/manifests/zookeeper/client.pp
@@ -29,8 +29,8 @@
 
 require_package('openjdk-7-jdk')
 class { '::zookeeper':
-hosts  => $hosts,
-version=> $version,
+hosts   => $hosts,
+version => $version,
 }
 }
 

-- 
To view, visit https://gerrit.wikimedia.org/r/293377
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I98ebd391b1ca29d8023e0bbaf37a83e14ea8bca7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Elastic: update mandatory plugins for codfw - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Elastic: update mandatory plugins for codfw
..


Elastic: update mandatory plugins for codfw

Change-Id: I715711a02bfd9d85796c568205a5e47505f493f2
---
M hieradata/role/codfw/elasticsearch/server.yaml
1 file changed, 5 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/role/codfw/elasticsearch/server.yaml 
b/hieradata/role/codfw/elasticsearch/server.yaml
index 2a72393..4b17618 100644
--- a/hieradata/role/codfw/elasticsearch/server.yaml
+++ b/hieradata/role/codfw/elasticsearch/server.yaml
@@ -13,6 +13,11 @@
   - _local_
   - _site_
 
+elasticsearch::plugins_mandatory:
+  - 'experimental-highlighter'
+  - 'extra'
+  - 'analysis-icu'
+
 lvs::realserver::realserver_ips: "10.2.1.30"
 
 elasticsearch::cluster_hosts:

-- 
To view, visit https://gerrit.wikimedia.org/r/291694
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I715711a02bfd9d85796c568205a5e47505f493f2
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: DCausse 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Elasticsearch - configure bind_networks. - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Elasticsearch - configure bind_networks.
..


Elasticsearch - configure bind_networks.

New configuration required for upgrade to elasticsearch 2.3.x

Change-Id: Ia450d1f3a6b01494a238a254615335a74f89e580
---
M hieradata/role/codfw/elasticsearch/server.yaml
1 file changed, 4 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  DCausse: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/role/codfw/elasticsearch/server.yaml 
b/hieradata/role/codfw/elasticsearch/server.yaml
index 5fde83b..2a72393 100644
--- a/hieradata/role/codfw/elasticsearch/server.yaml
+++ b/hieradata/role/codfw/elasticsearch/server.yaml
@@ -9,6 +9,10 @@
   - elastic2007.codfw.wmnet
   - elastic2013.codfw.wmnet
 
+elasticsearch::bind_networks:
+  - _local_
+  - _site_
+
 lvs::realserver::realserver_ips: "10.2.1.30"
 
 elasticsearch::cluster_hosts:

-- 
To view, visit https://gerrit.wikimedia.org/r/291689
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia450d1f3a6b01494a238a254615335a74f89e580
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: DCausse 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Elasticsearch - configure bind_networks. - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/291689

Change subject: Elasticsearch - configure bind_networks.
..

Elasticsearch - configure bind_networks.

New configuration required for upgrade to elasticsearch 2.3.x

Change-Id: Ia450d1f3a6b01494a238a254615335a74f89e580
---
M hieradata/role/codfw/elasticsearch/server.yaml
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/89/291689/1

diff --git a/hieradata/role/codfw/elasticsearch/server.yaml 
b/hieradata/role/codfw/elasticsearch/server.yaml
index 5fde83b..2a72393 100644
--- a/hieradata/role/codfw/elasticsearch/server.yaml
+++ b/hieradata/role/codfw/elasticsearch/server.yaml
@@ -9,6 +9,10 @@
   - elastic2007.codfw.wmnet
   - elastic2013.codfw.wmnet
 
+elasticsearch::bind_networks:
+  - _local_
+  - _site_
+
 lvs::realserver::realserver_ips: "10.2.1.30"
 
 elasticsearch::cluster_hosts:

-- 
To view, visit https://gerrit.wikimedia.org/r/291689
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia450d1f3a6b01494a238a254615335a74f89e580
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Upgrade cluster plugins for elasticsearch 2.3.3 - change (operations...plugins)

2016-05-30 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Upgrade cluster plugins for elasticsearch 2.3.3
..


Upgrade cluster plugins for elasticsearch 2.3.3

Upgraded :
- analysis-icu
- experimental-highlighter-elasticsearch-plugin
- swift-repository-plugin
- extra

Removed :
- head
- whatson
- langdetect

Bug: T134937
Change-Id: Ia8cbc1a6336158a0b008f5a5c7eeece193089ab6
---
A analysis-icu/analysis-icu-2.3.3.jar
D analysis-icu/elasticsearch-analysis-icu-2.7.0.jar
D analysis-icu/icu4j-53.1.jar
A analysis-icu/icu4j-54.1.jar
D analysis-icu/lucene-analyzers-icu-4.10.4.jar
A analysis-icu/lucene-analyzers-icu-5.5.0.jar
A analysis-icu/plugin-descriptor.properties
D 
experimental-highlighter-elasticsearch-plugin/experimental-highlighter-core-1.7.0.jar
A 
experimental-highlighter-elasticsearch-plugin/experimental-highlighter-core-2.3.3.jar
D 
experimental-highlighter-elasticsearch-plugin/experimental-highlighter-elasticsearch-plugin-1.7.0.jar
A 
experimental-highlighter-elasticsearch-plugin/experimental-highlighter-elasticsearch-plugin-2.3.3.jar
D 
experimental-highlighter-elasticsearch-plugin/experimental-highlighter-lucene-1.7.0.jar
A 
experimental-highlighter-elasticsearch-plugin/experimental-highlighter-lucene-2.3.3.jar
A experimental-highlighter-elasticsearch-plugin/plugin-descriptor.properties
D extra/extra-1.7.1.jar
A extra/extra-2.3.3.jar
A extra/plugin-descriptor.properties
D head/_site/.gitignore
D head/_site/.jshintrc
D head/_site/Gruntfile.js
D head/_site/LICENCE
D head/_site/README.textile
D head/_site/dist/app.css
D head/_site/dist/app.js
D head/_site/dist/base/favicon.png
D head/_site/dist/base/loading.gif
D head/_site/dist/base/reset.css
D head/_site/dist/fonts/FontAwesome.otf
D head/_site/dist/fonts/fontawesome-webfont.eot
D head/_site/dist/fonts/fontawesome-webfont.svg
D head/_site/dist/fonts/fontawesome-webfont.ttf
D head/_site/dist/fonts/fontawesome-webfont.woff
D head/_site/dist/i18n.js
D head/_site/dist/lang/en_strings.js
D head/_site/dist/lang/fr_strings.js
D head/_site/dist/lang/pt_strings.js
D head/_site/dist/vendor.css
D head/_site/dist/vendor.js
D head/_site/elasticsearch-head.sublime-project
D head/_site/grunt_fileSets.js
D head/_site/index.html
D head/_site/package.json
D head/_site/src/app/app.css
D head/_site/src/app/app.js
D head/_site/src/app/base/boot.js
D head/_site/src/app/base/favicon.png
D head/_site/src/app/base/loading.gif
D head/_site/src/app/base/reset.css
D head/_site/src/app/data/boolQuery.js
D head/_site/src/app/data/dataSourceInterface.js
D head/_site/src/app/data/metaData.js
D head/_site/src/app/data/metaDataFactory.js
D head/_site/src/app/data/model/model.js
D head/_site/src/app/data/model/modelSpec.js
D head/_site/src/app/data/query.js
D head/_site/src/app/data/queryDataSourceInterface.js
D head/_site/src/app/data/resultDataSourceInterface.js
D head/_site/src/app/lang/en_strings.js
D head/_site/src/app/lang/fr_strings.js
D head/_site/src/app/lang/pt_strings.js
D head/_site/src/app/services/cluster.js
D head/_site/src/app/services/clusterState/clusterState.js
D head/_site/src/app/services/clusterState/clusterStateSpec.js
D head/_site/src/app/services/storage.js
D head/_site/src/app/ui/abstractField/abstractField.css
D head/_site/src/app/ui/abstractField/abstractField.js
D head/_site/src/app/ui/abstractPanel/abstractPanel.css
D head/_site/src/app/ui/abstractPanel/abstractPanel.js
D head/_site/src/app/ui/abstractWidget/abstractWidget.js
D head/_site/src/app/ui/anyRequest/anyRequest.css
D head/_site/src/app/ui/anyRequest/anyRequest.js
D head/_site/src/app/ui/browser/browser.css
D head/_site/src/app/ui/browser/browser.js
D head/_site/src/app/ui/button/button.css
D head/_site/src/app/ui/button/button.js
D head/_site/src/app/ui/button/buttonDemo.js
D head/_site/src/app/ui/checkField/checkField.js
D head/_site/src/app/ui/checkField/checkFieldDemo.js
D head/_site/src/app/ui/checkField/checkFieldSpec.js
D head/_site/src/app/ui/clusterConnect/clusterConnect.css
D head/_site/src/app/ui/clusterConnect/clusterConnect.js
D head/_site/src/app/ui/clusterOverview/clusterOverview.css
D head/_site/src/app/ui/clusterOverview/clusterOverview.js
D head/_site/src/app/ui/csvTable/csvTable.js
D head/_site/src/app/ui/dateHistogram/dateHistogram.js
D head/_site/src/app/ui/dialogPanel/dialogPanel.js
D head/_site/src/app/ui/draggablePanel/draggablePanel.js
D head/_site/src/app/ui/filterBrowser/filterBrowser.css
D head/_site/src/app/ui/filterBrowser/filterBrowser.js
D head/_site/src/app/ui/header/header.css
D head/_site/src/app/ui/header/header.js
D head/_site/src/app/ui/helpPanel/helpPanel.js
D head/_site/src/app/ui/indexSelector/indexSelector.js
D head/_site/src/app/ui/infoPanel/infoPanel.css
D head/_site/src/app/ui/infoPanel/infoPanel.js
D head/_site/src/app/ui/jsonPanel/jsonPanel.css
D head/_site/src/app/ui/jsonPanel/jsonPanel.js
D head/_site/src/app/ui/jsonPretty/jsonPretty.css
D

[MediaWiki-commits] [Gerrit] Elastic: add publish_host support - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Elastic: add publish_host support
..


Elastic: add publish_host support

Set publish_host to eth0 for codfw, elastic was confused by the lvs loopback
address.

Change-Id: I2728b8fadd1ab3f80d94129a578a193b8b86f346
---
M hieradata/role/codfw/elasticsearch/server.yaml
M modules/elasticsearch/manifests/init.pp
M modules/elasticsearch/templates/elasticsearch.yml.erb
3 files changed, 6 insertions(+), 1 deletion(-)

Approvals:
  Gehel: Verified; Looks good to me, approved



diff --git a/hieradata/role/codfw/elasticsearch/server.yaml 
b/hieradata/role/codfw/elasticsearch/server.yaml
index 4b17618..32945f7 100644
--- a/hieradata/role/codfw/elasticsearch/server.yaml
+++ b/hieradata/role/codfw/elasticsearch/server.yaml
@@ -13,6 +13,8 @@
   - _local_
   - _site_
 
+elasticsearch::publish_host: _eth0_
+
 elasticsearch::plugins_mandatory:
   - 'experimental-highlighter'
   - 'extra'
diff --git a/modules/elasticsearch/manifests/init.pp 
b/modules/elasticsearch/manifests/init.pp
index 0c00d66..44d3306 100644
--- a/modules/elasticsearch/manifests/init.pp
+++ b/modules/elasticsearch/manifests/init.pp
@@ -58,6 +58,8 @@
 #   its pings reach other servers.
 # - $bind_networks: networks to bind (both transport and http connectors)
 #   see 
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values
+# - $publish_host: host to publish (both transport and http connectors)
+#   see 
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html
 # - $filter_cache_size: size of the filter cache.  See
 #   
www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-cache.html
 #   for possible values.  Default is 10% like the Elasticsearch default.
@@ -99,6 +101,7 @@
 $multicast_enabled = true,
 $unicast_hosts = undef,
 $bind_networks = undef,
+$publish_host = undef,
 $filter_cache_size = '10%',
 $bulk_thread_pool_executors = undef,
 $bulk_thread_pool_capacity = undef,
diff --git a/modules/elasticsearch/templates/elasticsearch.yml.erb 
b/modules/elasticsearch/templates/elasticsearch.yml.erb
index c692bd9..aa36326 100644
--- a/modules/elasticsearch/templates/elasticsearch.yml.erb
+++ b/modules/elasticsearch/templates/elasticsearch.yml.erb
@@ -203,7 +203,7 @@
 # Set the address other nodes will use to communicate with this node. If not
 # set, it is automatically derived. It must point to an actual IP address.
 #
-#network.publish_host: 192.168.0.1
+<%= @publish_host ? "network.publish_host: #{@publish_host}" : 
"#network.publish_host: 192.168.0.1" %>
 
 # Set both 'bind_host' and 'publish_host':
 #

-- 
To view, visit https://gerrit.wikimedia.org/r/291698
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I2728b8fadd1ab3f80d94129a578a193b8b86f346
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: DCausse 
Gerrit-Reviewer: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Change expired file zoom level from 16 to 15. - change (operations/puppet)

2016-05-31 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/291885

Change subject: Change expired file zoom level from 16 to 15.
..

Change expired file zoom level from 16 to 15.

Keeping on zoome level 15 still gives us the ability to oversample on level 16
but saves quite a lot of space.

Bug: T136483
Change-Id: If3b286f2540307a8b043cc13de7f710f3024f9cc
---
M modules/role/manifests/maps/master.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/85/291885/1

diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index a0bc12f..5f4cb71 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -40,7 +40,7 @@
 osm::planet_sync { 'gis':
 ensure=> present,
 flat_nodes=> true,
-expire_levels => '16',
+expire_levels => '15',
 num_threads   => $::processorcount/2,
 pg_password   => $osmupdater_pass,
 period=> 'day', # Remove thse as soon as we get down to minute

-- 
To view, visit https://gerrit.wikimedia.org/r/291885
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If3b286f2540307a8b043cc13de7f710f3024f9cc
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Cirrus: disable the safeifier in labs - change (operations/mediawiki-config)

2016-05-26 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Cirrus: disable the safeifier in labs
..


Cirrus: disable the safeifier in labs

This was removed from the elastic 2.x plugins

Change-Id: I2eecae7f877dacf5f8915cd322c47e65f8765e18
---
M wmf-config/CirrusSearch-labs.php
1 file changed, 0 insertions(+), 7 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/wmf-config/CirrusSearch-labs.php b/wmf-config/CirrusSearch-labs.php
index 54ff204..af453b0 100644
--- a/wmf-config/CirrusSearch-labs.php
+++ b/wmf-config/CirrusSearch-labs.php
@@ -33,13 +33,6 @@
 # them off.
 $wgCirrusSearchMaxShardsPerNode = array();
 
-# Use the safer query from the extra extension that is currently only deployed
-# in beta.
-$wgCirrusSearchWikimediaExtraPlugin[ 'safer' ] = array(
-   'phrase' => array(
-   )
-);
-
 # Override prod configuration, there is only one cluster in beta
 $wgCirrusSearchDefaultCluster = 'eqiad';
 # Don't specially configure cluster for more like queries in beta

-- 
To view, visit https://gerrit.wikimedia.org/r/290886
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I2eecae7f877dacf5f8915cd322c47e65f8765e18
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: DCausse 
Gerrit-Reviewer: Florianschmidtwelzow 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Elastic: add support for network.host - change (operations/puppet)

2016-05-26 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Elastic: add support for network.host
..


Elastic: add support for network.host

Elastic 2.x will now bind localhost by default. We need to override this 
default.

Change-Id: I54c03daedf0f55158fea4058b4453d564b870044
---
M hieradata/labs/deployment-prep/common.yaml
M modules/elasticsearch/manifests/init.pp
M modules/elasticsearch/templates/elasticsearch.yml.erb
3 files changed, 7 insertions(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index baa6a54..5dabb41 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -177,6 +177,9 @@
 - deployment-elastic06
 - deployment-elastic07
 - deployment-elastic08
+"elasticsearch::bind_networks":
+- _local_
+- _site_
 "elasticsearch::cluster_name": beta-search
 "elasticsearch::auto_create_index": '+apifeatureusage-*,-*'
 "elasticsearch::graylog_hosts":
diff --git a/modules/elasticsearch/manifests/init.pp 
b/modules/elasticsearch/manifests/init.pp
index 95ad5ff..0c00d66 100644
--- a/modules/elasticsearch/manifests/init.pp
+++ b/modules/elasticsearch/manifests/init.pp
@@ -56,6 +56,8 @@
 #   hosts in the cluster to this list.  Elasticsearch will still use
 #   multicast discovery but this will keep it from getting lost if none of
 #   its pings reach other servers.
+# - $bind_networks: networks to bind (both transport and http connectors)
+#   see 
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values
 # - $filter_cache_size: size of the filter cache.  See
 #   
www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-cache.html
 #   for possible values.  Default is 10% like the Elasticsearch default.
@@ -96,6 +98,7 @@
 $rack = undef,
 $multicast_enabled = true,
 $unicast_hosts = undef,
+$bind_networks = undef,
 $filter_cache_size = '10%',
 $bulk_thread_pool_executors = undef,
 $bulk_thread_pool_capacity = undef,
diff --git a/modules/elasticsearch/templates/elasticsearch.yml.erb 
b/modules/elasticsearch/templates/elasticsearch.yml.erb
index 4b79a08..c692bd9 100644
--- a/modules/elasticsearch/templates/elasticsearch.yml.erb
+++ b/modules/elasticsearch/templates/elasticsearch.yml.erb
@@ -207,7 +207,7 @@
 
 # Set both 'bind_host' and 'publish_host':
 #
-#network.host: 192.168.0.1
+<%= @bind_networks ? "network.host: [ #{Array(@bind_networks).join(',')} ]" : 
"#network.host: 192.168.0.1" %>
 
 # Set a custom port for the node to node communication (9300 by default):
 #

-- 
To view, visit https://gerrit.wikimedia.org/r/290883
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I54c03daedf0f55158fea4058b4453d564b870044
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: DCausse 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Make sure that replicate-osm can not run in multiple instances. - change (operations/puppet)

2016-06-01 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292112

Change subject: Make sure that replicate-osm can not run in multiple instances.
..

Make sure that replicate-osm can not run in multiple instances.

Using flock seems to be a good way to ensure that at most one instance of
that script is running.

Change-Id: Ia9337935338695f96b01f654957bb9b590056378
---
M modules/osm/templates/replicate-osm.erb
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/12/292112/1

diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 1b4a623..421593e 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -7,7 +7,8 @@
 export JAVACMD_OPTIONS='-Dhttp.proxyHost=<%= @proxy.split(':')[0] %> 
-Dhttp.proxyPort=<%= @proxy.split(':')[1] %>'
 <% end -%>
 
-/usr/bin/osmosis \
+/usr/bin/flock -xn <%= @osmosis_dir %>/replicate-osm.lck \
+  /usr/bin/osmosis \
--read-replication-interval \
workingDirectory=<%= @osmosis_dir %> \
--simplify-change \

-- 
To view, visit https://gerrit.wikimedia.org/r/292112
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia9337935338695f96b01f654957bb9b590056378
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Make sure that replicate-osm can not run in multiple instances. - change (operations/puppet)

2016-06-01 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Make sure that replicate-osm can not run in multiple instances.
..


Make sure that replicate-osm can not run in multiple instances.

Using flock seems to be a good way to ensure that at most one instance of
that script is running.

Change-Id: Ia9337935338695f96b01f654957bb9b590056378
---
M modules/osm/templates/replicate-osm.erb
1 file changed, 2 insertions(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/osm/templates/replicate-osm.erb 
b/modules/osm/templates/replicate-osm.erb
index 1b4a623..421593e 100644
--- a/modules/osm/templates/replicate-osm.erb
+++ b/modules/osm/templates/replicate-osm.erb
@@ -7,7 +7,8 @@
 export JAVACMD_OPTIONS='-Dhttp.proxyHost=<%= @proxy.split(':')[0] %> 
-Dhttp.proxyPort=<%= @proxy.split(':')[1] %>'
 <% end -%>
 
-/usr/bin/osmosis \
+/usr/bin/flock -xn <%= @osmosis_dir %>/replicate-osm.lck \
+  /usr/bin/osmosis \
--read-replication-interval \
workingDirectory=<%= @osmosis_dir %> \
--simplify-change \

-- 
To view, visit https://gerrit.wikimedia.org/r/292112
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia9337935338695f96b01f654957bb9b590056378
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Increase the number of workers for osm2pgsql. - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/291825

Change subject: Increase the number of workers for osm2pgsql.
..

Increase the number of workers for osm2pgsql.

On new maps server, we have CPU underutilization during import. This change
raises the number of threads for new maps servers but keep the current
configuration for the old maps-test servers.

Bug: T136578
Change-Id: Ib9aee02b8a0e0a876bc713eccb1e64b185a82237
---
M hieradata/hosts/maps-test2001.yaml
M hieradata/hosts/maps2001.yaml
M modules/role/manifests/maps/master.pp
3 files changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/25/291825/1

diff --git a/hieradata/hosts/maps-test2001.yaml 
b/hieradata/hosts/maps-test2001.yaml
index c5dc28a..c9af666 100644
--- a/hieradata/hosts/maps-test2001.yaml
+++ b/hieradata/hosts/maps-test2001.yaml
@@ -1,2 +1,3 @@
 cassandra::rack: a5
 do_paging: false
+osm::planet_sync::num_threads: 4
diff --git a/hieradata/hosts/maps2001.yaml b/hieradata/hosts/maps2001.yaml
index db7b54b..a53e93b 100644
--- a/hieradata/hosts/maps2001.yaml
+++ b/hieradata/hosts/maps2001.yaml
@@ -15,3 +15,5 @@
 tilerator::conf_sources: sources.prod2.yaml
 tilerator::ui::conf_sources: sources.prod2.yaml
 kartotherian::conf_sources: sources.prod2.yaml
+
+osm::planet_sync::num_threads: 12
diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index 08be1da..f629357 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -41,7 +41,6 @@
 ensure=> present,
 flat_nodes=> true,
 expire_levels => '16',
-num_threads   => 4,
 pg_password   => $osmupdater_pass,
 period=> 'day', # Remove thse as soon as we get down to minute
 hour  => '1',

-- 
To view, visit https://gerrit.wikimedia.org/r/291825
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib9aee02b8a0e0a876bc713eccb1e64b185a82237
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Increase the number of workers for osm2pgsql. - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Increase the number of workers for osm2pgsql.
..


Increase the number of workers for osm2pgsql.

On new maps server, we have CPU underutilization during import. This change
raises the number of threads for new maps servers but keep the current
configuration for the old maps-test servers.

Bug: T136578
Change-Id: Ib9aee02b8a0e0a876bc713eccb1e64b185a82237
---
M modules/role/manifests/maps/master.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/maps/master.pp 
b/modules/role/manifests/maps/master.pp
index 08be1da..a0bc12f 100644
--- a/modules/role/manifests/maps/master.pp
+++ b/modules/role/manifests/maps/master.pp
@@ -41,7 +41,7 @@
 ensure=> present,
 flat_nodes=> true,
 expire_levels => '16',
-num_threads   => 4,
+num_threads   => $::processorcount/2,
 pg_password   => $osmupdater_pass,
 period=> 'day', # Remove thse as soon as we get down to minute
 hour  => '1',

-- 
To view, visit https://gerrit.wikimedia.org/r/291825
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ib9aee02b8a0e0a876bc713eccb1e64b185a82237
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Keep osmosis osm_expire files for a month - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/291788

Change subject: Keep osmosis osm_expire files for a month
..

Keep osmosis osm_expire files for a month

Bug: T136577
Change-Id: Ie84aaa60a7b2932cf274fcf2dcbbd449a3da86ed
---
M modules/osm/manifests/planet_sync.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/88/291788/1

diff --git a/modules/osm/manifests/planet_sync.pp 
b/modules/osm/manifests/planet_sync.pp
index 005f116..19a1651 100644
--- a/modules/osm/manifests/planet_sync.pp
+++ b/modules/osm/manifests/planet_sync.pp
@@ -103,7 +103,7 @@
 }
 cron { "expire_old_planet_syncs-${name}":
 ensure  => $ensure,
-command => "/usr/bin/find ${expire_dir} -mtime +5 -exec rm {} \\;",
+command => "/usr/bin/find ${expire_dir} -mtime +30 -exec rm {} \\;",
 user=> 'osmupdater',
 hour=> $hour,
 minute  => $minute,

-- 
To view, visit https://gerrit.wikimedia.org/r/291788
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie84aaa60a7b2932cf274fcf2dcbbd449a3da86ed
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Keep osmosis osm_expire files for a month - change (operations/puppet)

2016-05-30 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Keep osmosis osm_expire files for a month
..


Keep osmosis osm_expire files for a month

Bug: T136577
Change-Id: Ie84aaa60a7b2932cf274fcf2dcbbd449a3da86ed
---
M modules/osm/manifests/planet_sync.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/osm/manifests/planet_sync.pp 
b/modules/osm/manifests/planet_sync.pp
index 005f116..19a1651 100644
--- a/modules/osm/manifests/planet_sync.pp
+++ b/modules/osm/manifests/planet_sync.pp
@@ -103,7 +103,7 @@
 }
 cron { "expire_old_planet_syncs-${name}":
 ensure  => $ensure,
-command => "/usr/bin/find ${expire_dir} -mtime +5 -exec rm {} \\;",
+command => "/usr/bin/find ${expire_dir} -mtime +30 -exec rm {} \\;",
 user=> 'osmupdater',
 hour=> $hour,
 minute  => $minute,

-- 
To view, visit https://gerrit.wikimedia.org/r/291788
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ie84aaa60a7b2932cf274fcf2dcbbd449a3da86ed
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuring new elastic1043-1047 servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295524

Change subject: Configuring new elastic1043-1047 servers
..

Configuring new elastic1043-1047 servers

Bug: T138329
Change-Id: I244ab839945e1aec63b758a1f1741124a233dff7
---
M manifests/site.pp
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/24/295524/1

diff --git a/manifests/site.pp b/manifests/site.pp
index 550fe59..e24f5dc 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -953,7 +953,8 @@
 include base::firewall
 include standard
 }
-node /^elastic104[0-2]\.eqiad\.wmnet/ {
+
+node /^elastic104[0-7]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard

-- 
To view, visit https://gerrit.wikimedia.org/r/295524
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I244ab839945e1aec63b758a1f1741124a233dff7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add new MediaWiki appserver to the scap DSH list. - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295525

Change subject: Add new MediaWiki appserver to the scap DSH list.
..

Add new MediaWiki appserver to the scap DSH list.

mw1292.eqiad.wmnet

Change-Id: I30da87fb48ae09f730471e53f63e2aa7b797e7c6
---
M modules/scap/files/dsh/group/mediawiki-installation
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/25/295525/1

diff --git a/modules/scap/files/dsh/group/mediawiki-installation 
b/modules/scap/files/dsh/group/mediawiki-installation
index 83e900c..79d04fd 100644
--- a/modules/scap/files/dsh/group/mediawiki-installation
+++ b/modules/scap/files/dsh/group/mediawiki-installation
@@ -237,6 +237,7 @@
 mw1289.eqiad.wmnet
 mw1290.eqiad.wmnet
 mw1291.eqiad.wmnet
+mw1292.eqiad.wmnet
 mw1299.eqiad.wmnet
 
 # codfw

-- 
To view, visit https://gerrit.wikimedia.org/r/295525
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I30da87fb48ae09f730471e53f63e2aa7b797e7c6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add new MediaWiki appserver to the scap DSH list. - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Add new MediaWiki appserver to the scap DSH list.
..


Add new MediaWiki appserver to the scap DSH list.

mw1292.eqiad.wmnet

Change-Id: I30da87fb48ae09f730471e53f63e2aa7b797e7c6
---
M modules/scap/files/dsh/group/mediawiki-installation
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Elukey: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/scap/files/dsh/group/mediawiki-installation 
b/modules/scap/files/dsh/group/mediawiki-installation
index 83e900c..79d04fd 100644
--- a/modules/scap/files/dsh/group/mediawiki-installation
+++ b/modules/scap/files/dsh/group/mediawiki-installation
@@ -237,6 +237,7 @@
 mw1289.eqiad.wmnet
 mw1290.eqiad.wmnet
 mw1291.eqiad.wmnet
+mw1292.eqiad.wmnet
 mw1299.eqiad.wmnet
 
 # codfw

-- 
To view, visit https://gerrit.wikimedia.org/r/295525
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I30da87fb48ae09f730471e53f63e2aa7b797e7c6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Elukey 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Manage Postgresql data dir with Puppet - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Manage Postgresql data dir with Puppet
..


Manage Postgresql data dir with Puppet

Default data directory (/var/lib/postgresql/...) is created during
installation of debian package. In case we want to use another data directory,
like it is the case for maps servers, this directory is not created and need
to be managed by Puppet.

Minor refactoring of existing tests was done to ensure all tests are green.

Bug: T138092
Change-Id: Iec59cf0290a5ea34cdab26d3604b32a9edde4967
---
M hieradata/role/common/maps/master.yaml
M hieradata/role/common/maps/slave.yaml
M modules/postgresql/manifests/master.pp
M modules/postgresql/manifests/server.pp
M modules/postgresql/manifests/slave.pp
M modules/postgresql/spec/classes/postgresql_server_spec.rb
M modules/postgresql/spec/classes/postgresql_slave_spec.rb
M modules/postgresql/spec/defines/postgresql_user_spec.rb
M modules/postgresql/templates/postgresql.conf.erb
M modules/role/manifests/osm/common.pp
M modules/role/manifests/osm/master.pp
M modules/role/manifests/osm/slave.pp
M modules/role/manifests/postgres/common.pp
M modules/role/manifests/postgres/master.pp
M modules/role/manifests/postgres/slave.pp
15 files changed, 99 insertions(+), 66 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/role/common/maps/master.yaml 
b/hieradata/role/common/maps/master.yaml
index f78..4e2860e 100644
--- a/hieradata/role/common/maps/master.yaml
+++ b/hieradata/role/common/maps/master.yaml
@@ -1,5 +1,5 @@
 postgresql::master::pgversion: 9.4
-postgresql::master::datadir: /srv/postgresql/9.4/main
+postgresql::master::root_dir: /srv/postgresql
 postgresql::master::includes:
   - tuning.conf
   - logging.conf
diff --git a/hieradata/role/common/maps/slave.yaml 
b/hieradata/role/common/maps/slave.yaml
index 4454cb9..1ef1126 100644
--- a/hieradata/role/common/maps/slave.yaml
+++ b/hieradata/role/common/maps/slave.yaml
@@ -1,5 +1,5 @@
 postgresql::slave::pgversion: 9.4
-postgresql::slave::datadir: /srv/postgresql/9.4/main
+postgresql::slave::root_dir: /srv/postgresql
 postgresql::slave::includes: tuning.conf
 # Defined in the private repo
 #postgresql::slave::replication_pass: justapass
diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 1869fff..148a67c 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -17,6 +17,8 @@
 #   Defaults to 64. Refer to postgresql documentation for its meaning
 #   wal_keep_segments
 #   Defaults to 128. Refer to postgresql documentation for its meaning
+#   root_dir
+#   See $postgresql::server::root_dir
 #
 # Actions:
 #  Install/configure postgresql as a master. Also create replication users
@@ -38,14 +40,14 @@
 $max_wal_senders=3,
 $checkpoint_segments=64,
 $wal_keep_segments=128,
-$datadir=undef
-) {
+$root_dir='/var/lib/postgresql',
+) {
 
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
 includes  => [ $includes, 'master.conf'],
-datadir   => $datadir,
+root_dir  => $root_dir,
 }
 
 file { "/etc/postgresql/${pgversion}/main/master.conf":
diff --git a/modules/postgresql/manifests/server.pp 
b/modules/postgresql/manifests/server.pp
index 9ef223b..1bbf031 100644
--- a/modules/postgresql/manifests/server.pp
+++ b/modules/postgresql/manifests/server.pp
@@ -13,6 +13,9 @@
 #   includes
 #   An array of files that will be included in the config. It is
 #   the caller's responsibility to provide these
+#   root_dir
+#   The root directory for postgresql data. The actual directory will be
+#   "${root_dir}/${pgversion}/main".
 #
 # Actions:
 #  Install/configure postgresql
@@ -32,7 +35,7 @@
 $includes = [],
 $listen_addresses = '*',
 $port = '5432',
-$datadir  = undef,
+$root_dir = '/var/lib/postgresql',
 ) {
 package { [
 "postgresql-${pgversion}",
@@ -45,6 +48,23 @@
 ensure => $ensure,
 }
 
+$data_dir = "${root_dir}/${pgversion}/main"
+
+file {  [ $root_dir, "${root_dir}/${pgversion}" ] :
+ensure  => ensure_directory($ensure),
+owner   => 'postgres',
+group   => 'postgres',
+mode=> '0755',
+require => Package["postgresql-${pgversion}"],
+}
+
+file { $data_dir:
+ensure => ensure_directory($ensure),
+owner  => 'postgres',
+group  => 'postgres',
+mode   => '0700',
+}
+
 exec { 'pgreload':
 command => "/usr/bin/pg_ctlcluster ${pgversion} main reload",
 user=> 'postgres',
diff --git a/modules/postgresql/manifests/slave.pp 
b/modules/postgresql/manifests/slave.pp
index

[MediaWiki-commits] [Gerrit] Configuring new elastic1043-1047 servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Configuring new elastic1043-1047 servers
..


Configuring new elastic1043-1047 servers

Bug: T138329
Change-Id: I244ab839945e1aec63b758a1f1741124a233dff7
---
M manifests/site.pp
1 file changed, 2 insertions(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/manifests/site.pp b/manifests/site.pp
index 550fe59..e24f5dc 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -953,7 +953,8 @@
 include base::firewall
 include standard
 }
-node /^elastic104[0-2]\.eqiad\.wmnet/ {
+
+node /^elastic104[0-7]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard

-- 
To view, visit https://gerrit.wikimedia.org/r/295524
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I244ab839945e1aec63b758a1f1741124a233dff7
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: DCausse 
Gerrit-Reviewer: EBernhardson 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Adding missing dependency in exposing puppet SSL certs on el... - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Adding missing dependency in exposing puppet SSL certs on 
elasticsearch
..


Adding missing dependency in exposing puppet SSL certs on elasticsearch

Since those certs are used by nginx and published to /etc/nginx, the nginx
package needs to be installed before exposing the certs.

Bug: T138329
Change-Id: If53eedef7de92b726e6e2697f9e6b8a48341b844
---
M modules/elasticsearch/manifests/https.pp
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/elasticsearch/manifests/https.pp 
b/modules/elasticsearch/manifests/https.pp
index bdf673f..1de60f5 100644
--- a/modules/elasticsearch/manifests/https.pp
+++ b/modules/elasticsearch/manifests/https.pp
@@ -31,6 +31,7 @@
 ::base::expose_puppet_certs { '/etc/nginx':
 ensure  => $ensure,
 provide_private => true,
+require => Class['nginx'],
 }
 
 ::nginx::site { 'elasticsearch-ssl-termination':

-- 
To view, visit https://gerrit.wikimedia.org/r/295369
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If53eedef7de92b726e6e2697f9e6b8a48341b844
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: DCausse 
Gerrit-Reviewer: EBernhardson 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuring new elastic1033-1037 servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Configuring new elastic1033-1037 servers
..


Configuring new elastic1033-1037 servers

Bug: T138329
Change-Id: I3734402bacb11e36f3f0ae3e9ed37b4509783676
---
M manifests/site.pp
1 file changed, 1 insertion(+), 9 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/manifests/site.pp b/manifests/site.pp
index 304cad3..6230d96 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -953,19 +953,11 @@
 include base::firewall
 include standard
 }
-node /^elastic103[0-1]\.eqiad\.wmnet/ {
+node /^elastic103[0-7]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard
 }
-
-# Let's start by installing a single new node, add the others in a second step
-node 'elastic1032.eqiad.wmnet' {
-role elasticsearch::server
-include base::firewall
-include standard
-}
-
 
 node /^elastic20[0-3][0-9]\.codfw\.wmnet/ {
 role elasticsearch::server

-- 
To view, visit https://gerrit.wikimedia.org/r/295473
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I3734402bacb11e36f3f0ae3e9ed37b4509783676
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuring new elastic1033-1037 servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295473

Change subject: Configuring new elastic1033-1037 servers
..

Configuring new elastic1033-1037 servers

Bug: T138329
Change-Id: I3734402bacb11e36f3f0ae3e9ed37b4509783676
---
M manifests/site.pp
1 file changed, 1 insertion(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/73/295473/1

diff --git a/manifests/site.pp b/manifests/site.pp
index 75982cd..e5191d3 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -953,19 +953,11 @@
 include base::firewall
 include standard
 }
-node /^elastic103[0-1]\.eqiad\.wmnet/ {
+node /^elastic103[0-7]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard
 }
-
-# Let's start by installing a single new node, add the others in a second step
-node 'elastic1032.eqiad.wmnet' {
-role elasticsearch::server
-include base::firewall
-include standard
-}
-
 
 node /^elastic20[0-3][0-9]\.codfw\.wmnet/ {
 role elasticsearch::server

-- 
To view, visit https://gerrit.wikimedia.org/r/295473
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3734402bacb11e36f3f0ae3e9ed37b4509783676
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuring new elastic1038-1042 servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295490

Change subject: Configuring new elastic1038-1042 servers
..

Configuring new elastic1038-1042 servers

Bug: T138329
Change-Id: I4463385b41e9b29b02fee17630b7c28f0fcd47f6
---
M manifests/site.pp
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/90/295490/1

diff --git a/manifests/site.pp b/manifests/site.pp
index 6230d96..1fd6d29 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -948,12 +948,12 @@
 include base::firewall
 }
 
-node /^elastic10[0-2][0-9]\.eqiad\.wmnet/ {
+node /^elastic10[0-3][0-9]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard
 }
-node /^elastic103[0-7]\.eqiad\.wmnet/ {
+node /^elastic104[0-2]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard

-- 
To view, visit https://gerrit.wikimedia.org/r/295490
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4463385b41e9b29b02fee17630b7c28f0fcd47f6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuring new elastic1038-1042 servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Configuring new elastic1038-1042 servers
..


Configuring new elastic1038-1042 servers

Bug: T138329
Change-Id: I4463385b41e9b29b02fee17630b7c28f0fcd47f6
---
M manifests/site.pp
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/manifests/site.pp b/manifests/site.pp
index 6230d96..1fd6d29 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -948,12 +948,12 @@
 include base::firewall
 }
 
-node /^elastic10[0-2][0-9]\.eqiad\.wmnet/ {
+node /^elastic10[0-3][0-9]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard
 }
-node /^elastic103[0-7]\.eqiad\.wmnet/ {
+node /^elastic104[0-2]\.eqiad\.wmnet/ {
 role elasticsearch::server
 include base::firewall
 include standard

-- 
To view, visit https://gerrit.wikimedia.org/r/295490
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I4463385b41e9b29b02fee17630b7c28f0fcd47f6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Adding rack location of new elasticsearch servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Adding rack location of new elasticsearch servers
..


Adding rack location of new elasticsearch servers

Bug: T138329
Change-Id: I5d3e6ea2bd5117f6a4ece4f7de37960aac0666f8
---
M hieradata/regex.yaml
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index faa8315..bc2ea4d 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -59,7 +59,7 @@
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp /^elastic10(05|06|23|24|25|26|27|28|29)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|46|47)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295536
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5d3e6ea2bd5117f6a4ece4f7de37960aac0666f8
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Adding rack location of new elasticsearch servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295536

Change subject: Adding rack location of new elasticsearch servers
..

Adding rack location of new elasticsearch servers

Bug: T138329
Change-Id: I5d3e6ea2bd5117f6a4ece4f7de37960aac0666f8
---
M hieradata/regex.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/36/295536/1

diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index faa8315..bc2ea4d 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -59,7 +59,7 @@
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp /^elastic10(05|06|23|24|25|26|27|28|29)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|46|47)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295536
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5d3e6ea2bd5117f6a4ece4f7de37960aac0666f8
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Fixed missing location of elastic1045 - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Fixed missing location of elastic1045
..


Fixed missing location of elastic1045

Bug: T138329
Change-Id: I588966a102806dcd67abbf10416007f59c99791c
---
M hieradata/regex.yaml
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index bc2ea4d..c31960a 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -59,7 +59,7 @@
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|46|47)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|45|46|47)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295537
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I588966a102806dcd67abbf10416007f59c99791c
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Fixed missing location of elastic1045 - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295537

Change subject: Fixed missing location of elastic1045
..

Fixed missing location of elastic1045

Bug: T138329
Change-Id: I588966a102806dcd67abbf10416007f59c99791c
---
M hieradata/regex.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/37/295537/1

diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index bc2ea4d..c31960a 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -59,7 +59,7 @@
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|46|47)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|45|46|47)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295537
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I588966a102806dcd67abbf10416007f59c99791c
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Interactive team would like to be notified of issues with Maps. - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Interactive team would like to be notified of issues with Maps.
..


Interactive team would like to be notified of issues with Maps.

This was discussed with yurik and MaxSem. It make sense for the owner of the
service to be notified of problems, in case they can do something about it.

Bug: T137869
Change-Id: I10cf2f787397e2a42baf99abbe99c0827e16da00
---
M modules/role/manifests/graphite/alerts/reqstats.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  Giuseppe Lavagetto: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/role/manifests/graphite/alerts/reqstats.pp 
b/modules/role/manifests/graphite/alerts/reqstats.pp
index e420637..1a1ff3b 100644
--- a/modules/role/manifests/graphite/alerts/reqstats.pp
+++ b/modules/role/manifests/graphite/alerts/reqstats.pp
@@ -76,6 +76,6 @@
 critical=> $settings['critical'],
 from=> $settings['cron'],
 nagios_critical => false,
+contact_group   => 'admins,interactive-team',
 }
 }
-

-- 
To view, visit https://gerrit.wikimedia.org/r/294676
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I10cf2f787397e2a42baf99abbe99c0827e16da00
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Dzahn 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: MaxSem 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Interactive team would like to be notified of issues with Maps. - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294676

Change subject: Interactive team would like to be notified of issues with Maps.
..

Interactive team would like to be notified of issues with Maps.

This was discussed with yurik and MaxSem. It make sense for the owner of the
service to be notified of problems, in case they can do something about it.

Bug: T137869
Change-Id: I10cf2f787397e2a42baf99abbe99c0827e16da00
---
M modules/role/manifests/graphite/alerts/reqstats.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/76/294676/1

diff --git a/modules/role/manifests/graphite/alerts/reqstats.pp 
b/modules/role/manifests/graphite/alerts/reqstats.pp
index e420637..1a1ff3b 100644
--- a/modules/role/manifests/graphite/alerts/reqstats.pp
+++ b/modules/role/manifests/graphite/alerts/reqstats.pp
@@ -76,6 +76,6 @@
 critical=> $settings['critical'],
 from=> $settings['cron'],
 nagios_critical => false,
+contact_group   => 'admins,interactive-team',
 }
 }
-

-- 
To view, visit https://gerrit.wikimedia.org/r/294676
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I10cf2f787397e2a42baf99abbe99c0827e16da00
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Fix team name for icinga alert - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294700

Change subject: Fix team name for icinga alert
..

Fix team name for icinga alert

Name of team is team-interactive and not interactive-team

Change-Id: I09b37aedc805ec69950b23672cacbd11586e249a
---
M modules/role/manifests/graphite/alerts/reqstats.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/00/294700/1

diff --git a/modules/role/manifests/graphite/alerts/reqstats.pp 
b/modules/role/manifests/graphite/alerts/reqstats.pp
index 1a1ff3b..4e8970d 100644
--- a/modules/role/manifests/graphite/alerts/reqstats.pp
+++ b/modules/role/manifests/graphite/alerts/reqstats.pp
@@ -76,6 +76,6 @@
 critical=> $settings['critical'],
 from=> $settings['cron'],
 nagios_critical => false,
-contact_group   => 'admins,interactive-team',
+contact_group   => 'admins,team-interactive',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294700
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I09b37aedc805ec69950b23672cacbd11586e249a
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] maps caches: remove referrer checks - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: maps caches: remove referrer checks
..


maps caches: remove referrer checks

Bug: T137848
Change-Id: I2a5ff946f3183c6f150aed67a53e5368acf31a0e
---
M templates/varnish/maps-frontend.inc.vcl.erb
1 file changed, 1 insertion(+), 13 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/templates/varnish/maps-frontend.inc.vcl.erb 
b/templates/varnish/maps-frontend.inc.vcl.erb
index 482fb4d..8430c93 100644
--- a/templates/varnish/maps-frontend.inc.vcl.erb
+++ b/templates/varnish/maps-frontend.inc.vcl.erb
@@ -2,19 +2,7 @@
 
 sub cluster_fe_recv_pre_purge { }
 
-sub cluster_fe_recv {
-   if (req.http.referer
-   && req.url != "/"
-   && req.http.referer !~ 
"(?i)^https?://([-a-zA-Z0-9.]+\.)?(mediawiki|wikivoyage|wikivoyage-ev|wmflabs|wikidata)\.org(/|$)"
-   && req.http.referer !~ 
"(?i)^https?://(maps|phabricator|wikitech|incubator|pl)\.(m\.)?wikimedia\.org(/|$)"
-   && req.http.referer !~ 
"(?i)^https?://test2?\.(m\.)?wikipedia\.org(/|$)"
-   // Vagrant support
-   && req.http.referer !~ 
"(?i)^https?://([-a-zA-Z0-9.]+\.)?wmftest\.net(:\d+)?(/|$)"
-   && req.http.referer !~ 
"(?i)^https?://(localhost|127\.0\.0\.1)(:\d+)?(/|$)"
-   ) {
-   return (synth(403, "Access Denied"));
-   }
-}
+sub cluster_fe_recv { }
 
 sub cluster_fe_hash { }
 sub cluster_fe_hit { }

-- 
To view, visit https://gerrit.wikimedia.org/r/294390
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I2a5ff946f3183c6f150aed67a53e5368acf31a0e
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: MaxSem 
Gerrit-Reviewer: BBlack 
Gerrit-Reviewer: Ema 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuration for new maps cluster in eqiad - change (operations/puppet)

2016-06-17 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294914

Change subject: Configuration for new maps cluster in eqiad
..

Configuration for new maps cluster in eqiad

Base configuration added. The maps slaves are still commented out in site.pp
to ensure we install the master first.

Change-Id: I398157c77a1c278056191bcc223fc91b977fb65b
---
A hieradata/hosts/maps1001.yaml
A hieradata/hosts/maps1002.yaml
A hieradata/hosts/maps1003.yaml
A hieradata/hosts/maps1004.yaml
A hieradata/role/eqiad/maps/master.yaml
A hieradata/role/eqiad/maps/server.yaml
A hieradata/role/eqiad/maps/slave.yaml
M manifests/site.pp
8 files changed, 62 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/14/294914/1

diff --git a/hieradata/hosts/maps1001.yaml b/hieradata/hosts/maps1001.yaml
new file mode 100644
index 000..9f2723a
--- /dev/null
+++ b/hieradata/hosts/maps1001.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: a4
+do_paging: false
diff --git a/hieradata/hosts/maps1002.yaml b/hieradata/hosts/maps1002.yaml
new file mode 100644
index 000..160db87
--- /dev/null
+++ b/hieradata/hosts/maps1002.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: b4
+do_paging: false
diff --git a/hieradata/hosts/maps1003.yaml b/hieradata/hosts/maps1003.yaml
new file mode 100644
index 000..4e53a74
--- /dev/null
+++ b/hieradata/hosts/maps1003.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: c5
+do_paging: false
diff --git a/hieradata/hosts/maps1004.yaml b/hieradata/hosts/maps1004.yaml
new file mode 100644
index 000..7e4068c
--- /dev/null
+++ b/hieradata/hosts/maps1004.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: d3
+do_paging: false
diff --git a/hieradata/role/eqiad/maps/master.yaml 
b/hieradata/role/eqiad/maps/master.yaml
new file mode 100644
index 000..4f9e59c
--- /dev/null
+++ b/hieradata/role/eqiad/maps/master.yaml
@@ -0,0 +1,32 @@
+# Defined in the private repo
+#cassandra::super_password: some_password
+#maps::postgresql_tilerator_pass: some_password
+#maps::postgresql_tileratorui_pass: some_password
+#maps::postgresql_kartotherian_pass: some_password
+#maps::postgresql_osmimporter_pass: some_password
+#maps::postgresql_osmupdater_pass: some_password
+#maps::postgres_slaves:
+#  replication@maps1002-v4:
+#user: replication
+#password: some_password
+#cidr: 10.64.16.42
+#pgversion: 9.4
+#attrs: REPLICATION
+#database: replication
+#  replication@maps1003-v4:
+#user: replication
+#password: some_password
+#cidr: 10.64.32.117
+#pgversion: 9.4
+#attrs: REPLICATION
+#database: replication
+#  replication@maps1004-v4:
+#user: replication
+#password: some_password
+#cidr: 10.64.48.154
+#pgversion: 9.4
+#attrs: REPLICATION
+#database: replication
+#maps::cassandra_tilerator_pass: some_password
+#maps::cassandra_tileratorui_pass: some_password
+#maps::cassandra_kartotherian_pass: some_password
diff --git a/hieradata/role/eqiad/maps/server.yaml 
b/hieradata/role/eqiad/maps/server.yaml
new file mode 100644
index 000..72a3287
--- /dev/null
+++ b/hieradata/role/eqiad/maps/server.yaml
@@ -0,0 +1,10 @@
+cassandra::seeds: [ 10.64.0.79, 10.64.16.42, 10.64.32.117, 10.64.48.154 ]
+cassandra::dc: eqiad
+#lvs::realserver::realserver_ips:
+#  - 1.2.3.4 TODO: define LVS ip once it is configured
+maps::hosts:
+  - maps1001.eqiad.wmnet
+  - maps1002.eqiad.wmnet
+  - maps1003.eqiad.wmnet
+  - maps1004.eqiad.wmnet
+maps::redis_server: redis://maps1001.codfw.wmnet
diff --git a/hieradata/role/eqiad/maps/slave.yaml 
b/hieradata/role/eqiad/maps/slave.yaml
new file mode 100644
index 000..e9d4771
--- /dev/null
+++ b/hieradata/role/eqiad/maps/slave.yaml
@@ -0,0 +1 @@
+postgresql::slave::master_server: maps1001.eqiad.wmnet
diff --git a/manifests/site.pp b/manifests/site.pp
index ab9e343..0b53d3f 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -1944,6 +1944,17 @@
 include base::firewall
 }
 
+node 'maps1001.eqiad.wmnet' {
+role maps::server, maps::master
+include base::firewall
+}
+
+# Waiting for master to be configured before configuring the slaves
+# node /^maps100[2-4]\.eqiad\.wmnet/ {
+# role maps::server, maps::slave
+# include base::firewall
+# }
+
 node 'maps2001.codfw.wmnet' {
 role maps::server, maps::master
 include base::firewall

-- 
To view, visit https://gerrit.wikimedia.org/r/294914
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I398157c77a1c278056191bcc223fc91b977fb65b
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuration for new elasticsearch servers in eqiad. - change (operations/puppet)

2016-06-17 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294918

Change subject: Configuration for new elasticsearch servers in eqiad.
..

Configuration for new elasticsearch servers in eqiad.

Only maps1032 is configured in site.pp. Let's check first on a single server
that the installation works and replicate to the 15 others afterward.

Change-Id: I5ef95b53dcfd79fd0eee7bd245feee210b7cd8f8
---
M hieradata/role/eqiad/elasticsearch/server.yaml
M manifests/site.pp
2 files changed, 25 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/18/294918/1

diff --git a/hieradata/role/eqiad/elasticsearch/server.yaml 
b/hieradata/role/eqiad/elasticsearch/server.yaml
index 5507891..a343e20 100644
--- a/hieradata/role/eqiad/elasticsearch/server.yaml
+++ b/hieradata/role/eqiad/elasticsearch/server.yaml
@@ -43,7 +43,24 @@
   - elastic1029.eqiad.wmnet
   - elastic1030.eqiad.wmnet
   - elastic1031.eqiad.wmnet
+  - elastic1032.eqiad.wmnet
+  - elastic1033.eqiad.wmnet
+  - elastic1034.eqiad.wmnet
+  - elastic1035.eqiad.wmnet
+  - elastic1036.eqiad.wmnet
+  - elastic1037.eqiad.wmnet
+  - elastic1038.eqiad.wmnet
+  - elastic1039.eqiad.wmnet
+  - elastic1040.eqiad.wmnet
+  - elastic1041.eqiad.wmnet
+  - elastic1042.eqiad.wmnet
+  - elastic1043.eqiad.wmnet
+  - elastic1044.eqiad.wmnet
+  - elastic1045.eqiad.wmnet
+  - elastic1046.eqiad.wmnet
+  - elastic1047.eqiad.wmnet
 
+# TODO at what point do we need to update this number?
 elasticsearch::expected_nodes: 31
 
 # We should be able to run "OK" with 24 servers.
diff --git a/manifests/site.pp b/manifests/site.pp
index ab9e343..7eba37f 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -959,6 +959,14 @@
 include standard
 }
 
+# Let's start by installing a single new node, add the others in a second step
+node 'elastic1032.eqiad.wmnet' {
+role elasticsearch::server
+include base::firewall
+include standard
+}
+
+
 node /^elastic20[0-3][0-9]\.codfw\.wmnet/ {
 role elasticsearch::server
 include base::firewall

-- 
To view, visit https://gerrit.wikimedia.org/r/294918
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5ef95b53dcfd79fd0eee7bd245feee210b7cd8f8
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Team-interactive receives maps alerts - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Team-interactive receives maps alerts
..


Team-interactive receives maps alerts

Adding team-interactive to the following alerts:

* service check on kartotherian
* service check on tilerator
* service check on tileratorui
* LVS check on kartotherian service

Bug: T137869
Bug: T137851
Change-Id: I02fca2da8ac90366d6be1c67d1e985c1b2c46228
---
M hieradata/role/common/maps/server.yaml
M modules/kartotherian/manifests/init.pp
M modules/lvs/manifests/monitor_services.pp
M modules/tilerator/manifests/init.pp
M modules/tilerator/manifests/ui.pp
5 files changed, 67 insertions(+), 15 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Yurik: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/role/common/maps/server.yaml 
b/hieradata/role/common/maps/server.yaml
index 0249efa..1586e40 100644
--- a/hieradata/role/common/maps/server.yaml
+++ b/hieradata/role/common/maps/server.yaml
@@ -12,3 +12,7 @@
   - tilerator-admin
 service::configuration::logstash_host: logstash1001.eqiad.wmnet
 service::configuration::statsd_host: statsd.eqiad.wmnet
+
+kartotherian::contact_groups: 'admins,team-interactive'
+tilerator::contact_groups: 'admins,team-interactive'
+tilerator::ui::contact_groups: 'admins,team-interactive'
diff --git a/modules/kartotherian/manifests/init.pp 
b/modules/kartotherian/manifests/init.pp
index 0638390..2f00788 100644
--- a/modules/kartotherian/manifests/init.pp
+++ b/modules/kartotherian/manifests/init.pp
@@ -6,8 +6,23 @@
 # accomodate future kartotherian needs that are not suited for the service 
module
 # classes as well as conform to a de-facto standard of having a module for 
every
 # service
+#
+# === Parameters
+#
+# [*conf_sources*]
+#   Sources that will be added to the configuration file of the service. This
+#   defines the data transformation pipeline for the tile services. The actual
+#   file is loaded from the root of the source code directory.
+#   (/srv/deployment/kartotherian/deploy/src/)
+#   Default: 'sources.prod.yaml'
+#
+# [*contact_groups*]
+#   Contact groups for alerting.
+#   Default: 'admins'
+#
 class kartotherian(
-$conf_sources = 'sources.prod.yaml',
+$conf_sources   = 'sources.prod.yaml',
+$contact_groups = 'admins',
 ) {
 
 $cassandra_kartotherian_user = 'kartotherian'
@@ -21,5 +36,6 @@
 deployment  => 'scap3',
 has_spec=> true,
 healthcheck_url => '',
+contact_groups  => $contact_groups,
 }
 }
diff --git a/modules/lvs/manifests/monitor_services.pp 
b/modules/lvs/manifests/monitor_services.pp
index c92281f..7d39a28 100644
--- a/modules/lvs/manifests/monitor_services.pp
+++ b/modules/lvs/manifests/monitor_services.pp
@@ -106,8 +106,9 @@
 host  => 'kartotherian.svc.codfw.wmnet',
 group => 'lvs',
 description   => 'Kartotherian LVS codfw',
-check_command =>
-'check_wmf_service!http://kartotherian.svc.codfw.wmnet:6533!15',
+check_command => 
'check_wmf_service!http://kartotherian.svc.codfw.wmnet:6533!15',
+contact_group => 'admins,team-interactive',
+critical  => true,
 }
 
 }
diff --git a/modules/tilerator/manifests/init.pp 
b/modules/tilerator/manifests/init.pp
index 80c97c4..967843e 100644
--- a/modules/tilerator/manifests/init.pp
+++ b/modules/tilerator/manifests/init.pp
@@ -6,10 +6,24 @@
 # accomodate future tilerator needs that are not suited for the service module
 # classes as well as conform to a de-facto standard of having a module for 
every
 # service
+#
+# === Parameters
+#
+# [*conf_sources*]
+#   Sources that will be added to the configuration file of the service. This
+#   defines the data transformation pipeline for the tile services. The actual
+#   file is loaded from the root of the source code directory.
+#   (/srv/deployment/tilerator/deploy/src/)
+#   Default: 'sources.prod.yaml'
+#
+# [*contact_groups*]
+#   Contact groups for alerting.
+#   Default: 'admins'
 class tilerator(
-$conf_sources = 'sources.prod.yaml',
+$conf_sources   = 'sources.prod.yaml',
+$contact_groups = 'admins',
 ) {
-include tilerator::ui
+include ::tilerator::ui
 
 $cassandra_tilerator_user = 'tilerator'
 $cassandra_tilerator_pass = hiera('maps::cassandra_tilerator_pass')
@@ -22,9 +36,10 @@
 # So there will never be LVS or anything else than health check requests to
 # this port
 service::node { 'tilerator':
-port   => 6534,
-config => template('tilerator/config.yaml.erb'),
-no_workers => $::processorcount / 2,
-deployment => 'scap3',
+port   => 6534,
+config => template('tilerator/config.yaml.erb'),
+no_workers => $::processorcount / 2,
+deployment => 'scap3',
+contact_groups => $contact_groups,

[MediaWiki-commits] [Gerrit] Adding gehel (Guillaume Lederrey) to icinga sms group - change (operations/puppet)

2016-06-17 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Adding gehel (Guillaume Lederrey) to icinga sms group
..


Adding gehel (Guillaume Lederrey) to icinga sms group

Guillaume should receive pages like any other ops.
SMS configuration has been done in private git repo.

Change-Id: I362e840d4df45310b16a3ed4d475b08039baed2a
---
M modules/nagios_common/files/contactgroups.cfg
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/nagios_common/files/contactgroups.cfg 
b/modules/nagios_common/files/contactgroups.cfg
index 05c4da2..09f1052 100644
--- a/modules/nagios_common/files/contactgroups.cfg
+++ b/modules/nagios_common/files/contactgroups.cfg
@@ -41,7 +41,7 @@
 
 define contactgroup {
 contactgroup_name   sms
-members 
akosiaris,andrew,ariel,bblack,cmjohnson,dzahn,ema,faidon,fgiunchedi,jgreen,mark,otto,volans,robh,springle,tstarling,rush,glavagetto,yuvipanda,jmm,jcrespo,team-operations
+members 
akosiaris,andrew,ariel,bblack,cmjohnson,dzahn,ema,faidon,fgiunchedi,jgreen,mark,otto,volans,robh,springle,tstarling,rush,glavagetto,yuvipanda,jmm,jcrespo,team-operations,gehel
 }
 
 define contactgroup {

-- 
To view, visit https://gerrit.wikimedia.org/r/294486
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I362e840d4df45310b16a3ed4d475b08039baed2a
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Postgresql: init database with Puppet - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295343

Change subject: Postgresql: init database with Puppet
..

Postgresql: init database with Puppet

Database is initialized by postgresql package, but if we move the data
directory somewhere else, it needs to be initialized.

Bug: T138092
Change-Id: If7240e6d8ca20b5bbe11a4f411f43d7773433f00
---
M modules/postgresql/manifests/master.pp
1 file changed, 12 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/43/295343/1

diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 148a67c..5a273ed 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -43,6 +43,8 @@
 $root_dir='/var/lib/postgresql',
 ) {
 
+$data_dir = "${root_dir}/${pgversion}/main"
+
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
@@ -58,4 +60,14 @@
 content => template('postgresql/master.conf.erb'),
 require => Class['postgresql::server'],
 }
+
+if $ensure == 'present' {
+exec { 'pg-initdb':
+command => "/usr/lib/postgresql/${pgversion}/bin/initdb -D 
${data_dir}",
+user=> 'postgres',
+unless  => "/usr/bin/test -f ${data_dir}/PG_VERSION",
+require => Class['postgresql::server'],
+notify  => Exec['pgreload'],
+}
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/295343
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If7240e6d8ca20b5bbe11a4f411f43d7773433f00
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Deploy Graphoid with Scap3 - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Deploy Graphoid with Scap3
..


Deploy Graphoid with Scap3

Change-Id: Ic741c7e339a8f038ba9c08faa62360a070b254a0
---
M hieradata/common/role/deployment.yaml
M hieradata/common/scap/server.yaml
M modules/graphoid/manifests/init.pp
3 files changed, 2 insertions(+), 5 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Mobrovac: Looks good to me, but someone else must approve
  Yurik: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/common/role/deployment.yaml 
b/hieradata/common/role/deployment.yaml
index 5b92a1e..94f5135 100644
--- a/hieradata/common/role/deployment.yaml
+++ b/hieradata/common/role/deployment.yaml
@@ -44,10 +44,6 @@
 service_name: zotero
   zotero/translators:
 upstream: 
https://gerrit.wikimedia.org/r/mediawiki/services/zotero/translators
-  graphoid/deploy:
-upstream: https://gerrit.wikimedia.org/r/mediawiki/services/graphoid/deploy
-service_name: graphoid
-checkout_submodules: true
   rcstream/rcstream:
 upstream: https://gerrit.wikimedia.org/r/mediawiki/services/rcstream
 service_name: rcstream
diff --git a/hieradata/common/scap/server.yaml 
b/hieradata/common/scap/server.yaml
index 5387b08..c9579c0 100644
--- a/hieradata/common/scap/server.yaml
+++ b/hieradata/common/scap/server.yaml
@@ -55,8 +55,8 @@
   mobileapps/deploy: {}
 
   kartotherian/deploy: {}
-
   tilerator/deploy: {}
+  graphoid/deploy: {}
 
   # cxserver is the ContentTranslation server.
   cxserver/deploy:
diff --git a/modules/graphoid/manifests/init.pp 
b/modules/graphoid/manifests/init.pp
index c00bd24..652f526 100644
--- a/modules/graphoid/manifests/init.pp
+++ b/modules/graphoid/manifests/init.pp
@@ -48,6 +48,7 @@
 },
 has_spec=> true,
 healthcheck_url => '',
+deployment  => 'scap3',
 }
 
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294357
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ic741c7e339a8f038ba9c08faa62360a070b254a0
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Thcipriani 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Configuration for new elasticsearch servers in eqiad. - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Configuration for new elasticsearch servers in eqiad.
..


Configuration for new elasticsearch servers in eqiad.

Only elastic1032 is configured in site.pp. Let's check first on a single server
that the installation works and replicate to the 15 others afterward.

Changed regex.yaml and now use more stupid regex to improve readability.

Change-Id: I5ef95b53dcfd79fd0eee7bd245feee210b7cd8f8
---
M hieradata/regex.yaml
M hieradata/role/eqiad/elasticsearch/server.yaml
M manifests/site.pp
3 files changed, 42 insertions(+), 12 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  DCausse: Looks good to me, but someone else must approve
  EBernhardson: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index 813672e..34ea221 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -1,60 +1,65 @@
 es_codfw_rack_a5:
-  __regex: !ruby/regexp /^elastic200[1-3]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(01|02|03)\.codfw\.wmnet$/
   elasticsearch::rack: A5
   elasticsearch::row:  A
 
 es_codfw_rack_a8:
-  __regex: !ruby/regexp /^elastic200[4-6]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(04|05|06)\.codfw\.wmnet$/
   elasticsearch::rack: A8
   elasticsearch::row:  A
 
 es_codfw_rack_b5:
-  __regex: !ruby/regexp /^elastic200[7-9]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(07|08|09)\.codfw\.wmnet$/
   elasticsearch::rack: B5
   elasticsearch::row:  B
 
 es_codfw_rack_b8:
-  __regex: !ruby/regexp /^elastic201[0-2]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(10|11|12)\.codfw\.wmnet$/
   elasticsearch::rack: B8
   elasticsearch::row:  B
 
 es_codfw_rack_c1:
-  __regex: !ruby/regexp /^elastic201[3-5]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(13|14|15)\.codfw\.wmnet$/
   elasticsearch::rack: C1
   elasticsearch::row:  C
 
 es_codfw_rack_c5:
-  __regex: !ruby/regexp /^elastic201[6-8]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(16|17|18)\.codfw\.wmnet$/
   elasticsearch::rack: C5
   elasticsearch::row:  C
 
 es_codfw_rack_d1:
-  __regex: !ruby/regexp /^elastic20(1[9]|2[0-1])\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(19|20|21)\.codfw\.wmnet$/
   elasticsearch::rack: D1
   elasticsearch::row:  D
 
 es_codfw_rack_d5:
-  __regex: !ruby/regexp /^elastic202[2-4]\.codfw\.wmnet$/
+  __regex: !ruby/regexp /^elastic20(22|23|24)\.codfw\.wmnet$/
   elasticsearch::rack: D5
   elasticsearch::row:  D
 
 es_eqiad_rack_a3:
-  __regex: !ruby/regexp /^elastic10(0[0-4]|30|31)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(01|02|03|04|30|31|32|33|34|35)\.eqiad\.wmnet$/
   elasticsearch::rack: A3
   elasticsearch::row:  A
 
+es_eqiad_rack_b3:
+  __regex: !ruby/regexp /^elastic10(36|37|38|39)\.eqiad\.wmnet$/
+  elasticsearch::rack: B3
+  elasticsearch::row:  B
+
 es_eqiad_rack_c5:
-  __regex: !ruby/regexp /^elastic10(0[7-9]|1[0-2])\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(07|08|09|10|11|12|40|41|42|43)\.eqiad\.wmnet$/
   elasticsearch::rack: C5
   elasticsearch::row:  C
 
 es_eqiad_rack_d3:
-  __regex: !ruby/regexp /^elastic10(1[3-9]|2[0-2])\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(13|14|15|16|17|18|19|20|21|22)\.eqiad\.wmnet$/
   elasticsearch::rack: D3
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp /^elastic10(2[3-9]|05|06)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp /^elastic10(05|06|23|24|25|26|27|28|29)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 
diff --git a/hieradata/role/eqiad/elasticsearch/server.yaml 
b/hieradata/role/eqiad/elasticsearch/server.yaml
index 5507891..a343e20 100644
--- a/hieradata/role/eqiad/elasticsearch/server.yaml
+++ b/hieradata/role/eqiad/elasticsearch/server.yaml
@@ -43,7 +43,24 @@
   - elastic1029.eqiad.wmnet
   - elastic1030.eqiad.wmnet
   - elastic1031.eqiad.wmnet
+  - elastic1032.eqiad.wmnet
+  - elastic1033.eqiad.wmnet
+  - elastic1034.eqiad.wmnet
+  - elastic1035.eqiad.wmnet
+  - elastic1036.eqiad.wmnet
+  - elastic1037.eqiad.wmnet
+  - elastic1038.eqiad.wmnet
+  - elastic1039.eqiad.wmnet
+  - elastic1040.eqiad.wmnet
+  - elastic1041.eqiad.wmnet
+  - elastic1042.eqiad.wmnet
+  - elastic1043.eqiad.wmnet
+  - elastic1044.eqiad.wmnet
+  - elastic1045.eqiad.wmnet
+  - elastic1046.eqiad.wmnet
+  - elastic1047.eqiad.wmnet
 
+# TODO at what point do we need to update this number?
 elasticsearch::expected_nodes: 31
 
 # We should be able to run "OK" with 24 servers.
diff --git a/manifests/site.pp b/manifests/site.pp
index 0b53d3f..75982cd 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -959,6 +959,14 @@
 include standard
 }
 
+# Let's start by installing a single new node, add the others in a second step
+node 'elastic1032.eqiad.wmnet' {
+role elasticsearch::server
+include base::firewall
+include standard
+}
+
+
 node

[MediaWiki-commits] [Gerrit] Postgresql: init database with Puppet - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Postgresql: init database with Puppet
..


Postgresql: init database with Puppet

Database is initialized by postgresql package, but if we move the data
directory somewhere else, it needs to be initialized.

Bug: T138092
Change-Id: If7240e6d8ca20b5bbe11a4f411f43d7773433f00
---
M modules/postgresql/manifests/master.pp
1 file changed, 12 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 148a67c..5a273ed 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -43,6 +43,8 @@
 $root_dir='/var/lib/postgresql',
 ) {
 
+$data_dir = "${root_dir}/${pgversion}/main"
+
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
@@ -58,4 +60,14 @@
 content => template('postgresql/master.conf.erb'),
 require => Class['postgresql::server'],
 }
+
+if $ensure == 'present' {
+exec { 'pg-initdb':
+command => "/usr/lib/postgresql/${pgversion}/bin/initdb -D 
${data_dir}",
+user=> 'postgres',
+unless  => "/usr/bin/test -f ${data_dir}/PG_VERSION",
+require => Class['postgresql::server'],
+notify  => Exec['pgreload'],
+}
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/295343
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If7240e6d8ca20b5bbe11a4f411f43d7773433f00
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Adding missing dependency in exposing puppet SSL certs on el... - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295369

Change subject: Adding missing dependency in exposing puppet SSL certs on 
elasticsearch
..

Adding missing dependency in exposing puppet SSL certs on elasticsearch

Since those certs are used by nginx and published to /etc/nginx, the nginx
package needs to be installed before exposing the certs.

Bug: T138329
Change-Id: If53eedef7de92b726e6e2697f9e6b8a48341b844
---
M modules/elasticsearch/manifests/https.pp
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/69/295369/1

diff --git a/modules/elasticsearch/manifests/https.pp 
b/modules/elasticsearch/manifests/https.pp
index bdf673f..1de60f5 100644
--- a/modules/elasticsearch/manifests/https.pp
+++ b/modules/elasticsearch/manifests/https.pp
@@ -31,6 +31,7 @@
 ::base::expose_puppet_certs { '/etc/nginx':
 ensure  => $ensure,
 provide_private => true,
+require => Class['nginx'],
 }
 
 ::nginx::site { 'elasticsearch-ssl-termination':

-- 
To view, visit https://gerrit.wikimedia.org/r/295369
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If53eedef7de92b726e6e2697f9e6b8a48341b844
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Postgresql: init database with Puppet" - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295373

Change subject: Revert "Postgresql: init database with Puppet"
..

Revert "Postgresql: init database with Puppet"

This reverts commit 230c9fd41dc7536b7a45a01a3641b56ec081a98e.

Change-Id: I00f12c4a9c11ec065fa6378608f95d74d0190083
---
M modules/postgresql/manifests/master.pp
1 file changed, 0 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/73/295373/1

diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 5a273ed..148a67c 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -43,8 +43,6 @@
 $root_dir='/var/lib/postgresql',
 ) {
 
-$data_dir = "${root_dir}/${pgversion}/main"
-
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
@@ -59,15 +57,5 @@
 mode=> '0444',
 content => template('postgresql/master.conf.erb'),
 require => Class['postgresql::server'],
-}
-
-if $ensure == 'present' {
-exec { 'pg-initdb':
-command => "/usr/lib/postgresql/${pgversion}/bin/initdb -D 
${data_dir}",
-user=> 'postgres',
-unless  => "/usr/bin/test -f ${data_dir}/PG_VERSION",
-require => Class['postgresql::server'],
-notify  => Exec['pgreload'],
-}
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/295373
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I00f12c4a9c11ec065fa6378608f95d74d0190083
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Revert "Postgresql: init database with Puppet" - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Revert "Postgresql: init database with Puppet"
..


Revert "Postgresql: init database with Puppet"

This reverts commit 230c9fd41dc7536b7a45a01a3641b56ec081a98e.

Change-Id: I00f12c4a9c11ec065fa6378608f95d74d0190083
---
M modules/postgresql/manifests/master.pp
1 file changed, 0 insertions(+), 12 deletions(-)

Approvals:
  Gehel: Verified; Looks good to me, approved



diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 5a273ed..148a67c 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -43,8 +43,6 @@
 $root_dir='/var/lib/postgresql',
 ) {
 
-$data_dir = "${root_dir}/${pgversion}/main"
-
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
@@ -59,15 +57,5 @@
 mode=> '0444',
 content => template('postgresql/master.conf.erb'),
 require => Class['postgresql::server'],
-}
-
-if $ensure == 'present' {
-exec { 'pg-initdb':
-command => "/usr/lib/postgresql/${pgversion}/bin/initdb -D 
${data_dir}",
-user=> 'postgres',
-unless  => "/usr/bin/test -f ${data_dir}/PG_VERSION",
-require => Class['postgresql::server'],
-notify  => Exec['pgreload'],
-}
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/295373
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I00f12c4a9c11ec065fa6378608f95d74d0190083
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] postgres: Specify extra module in RSpec - change (operations/puppet)

2016-06-21 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: postgres: Specify extra module in RSpec
..


postgres: Specify extra module in RSpec

This should fix the missing functions when Rspecing the postgres module

Change-Id: I40fd23d24e3d964cc831a7b1b4c2e125243766fa
---
M modules/postgresql/Rakefile
1 file changed, 8 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/postgresql/Rakefile b/modules/postgresql/Rakefile
index d9226a8..652f46e 100644
--- a/modules/postgresql/Rakefile
+++ b/modules/postgresql/Rakefile
@@ -3,6 +3,8 @@
 
 require 'rspec/core/rake_task'
 
+extra_modules = ['wmflib']
+
 modulename = File.basename(File.expand_path(File.dirname(__FILE__)))
 
 symlinks = { 'spec/fixtures/modules/%s/files' % modulename => 
'../../../../files',
@@ -13,6 +15,9 @@
 
 task :setup do
   FileUtils.mkdir_p('spec/fixtures/modules/%s' % modulename)
+  extra_modules.each do |x|
+  FileUtils.ln_s('../../../../%s' % x, 'spec/fixtures/modules/%s' % x)
+  end
   symlinks.each do |x|
 if !File.exist?(x[0])
   FileUtils.ln_s(x[1], x[0])
@@ -23,6 +28,9 @@
 task :teardown do
   symlinks.each { |x| FileUtils.rm(x[0], :force => true) }
   FileUtils.rmdir('spec/fixtures/modules/%s' % modulename)
+  extra_modules.each do |x|
+  FileUtils.rm('spec/fixtures/modules/%s' % x, :force => true)
+  end
   FileUtils.rmdir('spec/fixtures/modules')
 end
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295327
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I40fd23d24e3d964cc831a7b1b4c2e125243766fa
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Postgresql: init database with Puppet - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295589

Change subject: Postgresql: init database with Puppet
..

Postgresql: init database with Puppet

Database is initialized by postgresql package, but if we move the data
directory somewhere else, it needs to be initialized.

To avoid dependency cycle, pg-reload is not notified, following the same
pattern used in postgresql::slave.

Bug: T138092
Change-Id: I8b5031306bf714a4455d9291ad32b3e6ae650d61
---
M modules/postgresql/manifests/master.pp
1 file changed, 11 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/89/295589/1

diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 148a67c..2b8fb61 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -43,6 +43,8 @@
 $root_dir='/var/lib/postgresql',
 ) {
 
+$data_dir = "${root_dir}/${pgversion}/main"
+
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
@@ -58,4 +60,13 @@
 content => template('postgresql/master.conf.erb'),
 require => Class['postgresql::server'],
 }
+
+if $ensure == 'present' {
+exec { 'pg-initdb':
+command => "/usr/lib/postgresql/${pgversion}/bin/initdb -D 
${data_dir}",
+ user=> 'postgres',
+ unless  => "/usr/bin/test -f ${data_dir}/PG_VERSION",
+ require => Class['postgresql::server'],
+ }
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/295589
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8b5031306bf714a4455d9291ad32b3e6ae650d61
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Moving elasticsearch masters to new servers - change (operations/puppet)

2016-06-22 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295585

Change subject: Moving elasticsearch masters to new servers
..

Moving elasticsearch masters to new servers

This is required to be able to decommission the old elasticsearch servers.

Bug: T138329
Change-Id: Idd7e769df8dc5ad4467cfc464af88d43778fa5ba
---
A hieradata/hosts/elastic1030.yaml
A hieradata/hosts/elastic1036.yaml
A hieradata/hosts/elastic1040.yaml
A hieradata/hosts/elastic1044.yaml
M hieradata/regex.yaml
M hieradata/role/eqiad/elasticsearch/server.yaml
6 files changed, 17 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/85/295585/1

diff --git a/hieradata/hosts/elastic1030.yaml b/hieradata/hosts/elastic1030.yaml
new file mode 100644
index 000..41eab79
--- /dev/null
+++ b/hieradata/hosts/elastic1030.yaml
@@ -0,0 +1,3 @@
+elasticsearch::master_eligible: true
+# make sure the elasticsearch::unicast_hosts property is updated if master
+# eligibility is modified
diff --git a/hieradata/hosts/elastic1036.yaml b/hieradata/hosts/elastic1036.yaml
new file mode 100644
index 000..41eab79
--- /dev/null
+++ b/hieradata/hosts/elastic1036.yaml
@@ -0,0 +1,3 @@
+elasticsearch::master_eligible: true
+# make sure the elasticsearch::unicast_hosts property is updated if master
+# eligibility is modified
diff --git a/hieradata/hosts/elastic1040.yaml b/hieradata/hosts/elastic1040.yaml
new file mode 100644
index 000..41eab79
--- /dev/null
+++ b/hieradata/hosts/elastic1040.yaml
@@ -0,0 +1,3 @@
+elasticsearch::master_eligible: true
+# make sure the elasticsearch::unicast_hosts property is updated if master
+# eligibility is modified
diff --git a/hieradata/hosts/elastic1044.yaml b/hieradata/hosts/elastic1044.yaml
new file mode 100644
index 000..41eab79
--- /dev/null
+++ b/hieradata/hosts/elastic1044.yaml
@@ -0,0 +1,3 @@
+elasticsearch::master_eligible: true
+# make sure the elasticsearch::unicast_hosts property is updated if master
+# eligibility is modified
diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index c31960a..b50957c 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -59,7 +59,7 @@
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|45|46|47)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|44|45|46|47)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 
diff --git a/hieradata/role/eqiad/elasticsearch/server.yaml 
b/hieradata/role/eqiad/elasticsearch/server.yaml
index a343e20..bc2f168 100644
--- a/hieradata/role/eqiad/elasticsearch/server.yaml
+++ b/hieradata/role/eqiad/elasticsearch/server.yaml
@@ -5,9 +5,10 @@
 elasticsearch::multicast_group: 224.2.2.5
 elasticsearch::multicast_enabled: false
 elasticsearch::unicast_hosts: # this should be set to the list of master 
eligible nodes
-  - elastic1001.eqiad.wmnet
-  - elastic1008.eqiad.wmnet
-  - elastic1013.eqiad.wmnet
+  - elastic1030.eqiad.wmnet
+  - elastic1036.eqiad.wmnet
+  - elastic1040.eqiad.wmnet
+  - elastic1044.eqiad.wmnet
 
 lvs::realserver::realserver_ips: "10.2.2.30"
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295585
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Idd7e769df8dc5ad4467cfc464af88d43778fa5ba
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Remove old maps-test servers from LVS config - change (operations/puppet)

2016-06-23 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295640

Change subject: Remove old maps-test servers from LVS config
..

Remove old maps-test servers from LVS config

Those servers are ready to be decommissioned. We have been running on new
servers without issue since June 14.

Change-Id: I583be5f7b69653fa85e1aa99444d77422cc315ee
---
M conftool-data/nodes/codfw.yaml
1 file changed, 0 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/40/295640/1

diff --git a/conftool-data/nodes/codfw.yaml b/conftool-data/nodes/codfw.yaml
index 08d553a..7fe49a6 100644
--- a/conftool-data/nodes/codfw.yaml
+++ b/conftool-data/nodes/codfw.yaml
@@ -221,10 +221,6 @@
   maps2002.codfw.wmnet: [kartotherian]
   maps2003.codfw.wmnet: [kartotherian]
   maps2004.codfw.wmnet: [kartotherian]
-  maps-test2001.codfw.wmnet: [kartotherian]
-  maps-test2002.codfw.wmnet: [kartotherian]
-  maps-test2003.codfw.wmnet: [kartotherian]
-  maps-test2004.codfw.wmnet: [kartotherian]
 elasticsearch:
   elastic2001.codfw.wmnet: [elasticsearch, elasticsearch-ssl]
   elastic2002.codfw.wmnet: [elasticsearch, elasticsearch-ssl]

-- 
To view, visit https://gerrit.wikimedia.org/r/295640
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I583be5f7b69653fa85e1aa99444d77422cc315ee
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Decommission old maps servers - change (operations/puppet)

2016-06-23 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295649

Change subject: Decommission old maps servers
..

Decommission old maps servers

Once new maps servers are in place and elasticsearch has been shutdown, we
still need to remove them from LVS and remove associated ferm rules.

This should not be merged before traffic is off those servers.

Bug: T138329
Change-Id: I8c1bdc1011a2bc656c7ba0fcf4c51ee58d71f08d
---
M conftool-data/nodes/eqiad.yaml
M hieradata/role/eqiad/elasticsearch/server.yaml
2 files changed, 0 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/49/295649/1

diff --git a/conftool-data/nodes/eqiad.yaml b/conftool-data/nodes/eqiad.yaml
index 0133f68..974f4a9 100644
--- a/conftool-data/nodes/eqiad.yaml
+++ b/conftool-data/nodes/eqiad.yaml
@@ -252,22 +252,6 @@
   restbase1014.eqiad.wmnet: [restbase]
   restbase1015.eqiad.wmnet: [restbase]
 elasticsearch:
-  elastic1001.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1002.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1003.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1004.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1005.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1006.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1007.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1008.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1009.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1010.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1011.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1012.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1013.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1014.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1015.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
-  elastic1016.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
   elastic1017.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
   elastic1018.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
   elastic1019.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
diff --git a/hieradata/role/eqiad/elasticsearch/server.yaml 
b/hieradata/role/eqiad/elasticsearch/server.yaml
index a343e20..538e63a 100644
--- a/hieradata/role/eqiad/elasticsearch/server.yaml
+++ b/hieradata/role/eqiad/elasticsearch/server.yaml
@@ -12,22 +12,6 @@
 lvs::realserver::realserver_ips: "10.2.2.30"
 
 elasticsearch::cluster_hosts:
-  - elastic1001.eqiad.wmnet
-  - elastic1002.eqiad.wmnet
-  - elastic1003.eqiad.wmnet
-  - elastic1004.eqiad.wmnet
-  - elastic1005.eqiad.wmnet
-  - elastic1006.eqiad.wmnet
-  - elastic1007.eqiad.wmnet
-  - elastic1008.eqiad.wmnet
-  - elastic1009.eqiad.wmnet
-  - elastic1010.eqiad.wmnet
-  - elastic1011.eqiad.wmnet
-  - elastic1012.eqiad.wmnet
-  - elastic1013.eqiad.wmnet
-  - elastic1014.eqiad.wmnet
-  - elastic1015.eqiad.wmnet
-  - elastic1016.eqiad.wmnet
   - elastic1017.eqiad.wmnet
   - elastic1018.eqiad.wmnet
   - elastic1019.eqiad.wmnet

-- 
To view, visit https://gerrit.wikimedia.org/r/295649
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8c1bdc1011a2bc656c7ba0fcf4c51ee58d71f08d
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add new elasticsearch servers to LVS - change (operations/puppet)

2016-06-23 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295657

Change subject: Add new elasticsearch servers to LVS
..

Add new elasticsearch servers to LVS

New elasticsearch servers are set up and have joined the cluster. They already
serve indirect traffic, but still need to be added to LVS to accept direct
traffic.

Bug: T138329
Change-Id: I007c6fb14754e310895c7adcd7490e71d6d45af6
---
M conftool-data/nodes/eqiad.yaml
1 file changed, 16 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/57/295657/1

diff --git a/conftool-data/nodes/eqiad.yaml b/conftool-data/nodes/eqiad.yaml
index 0133f68..ceb6e8e 100644
--- a/conftool-data/nodes/eqiad.yaml
+++ b/conftool-data/nodes/eqiad.yaml
@@ -283,6 +283,22 @@
   elastic1029.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
   elastic1030.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
   elastic1031.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1032.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1033.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1034.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1035.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1036.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1037.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1038.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1039.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1040.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1041.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1042.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1043.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1044.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1045.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1046.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
+  elastic1047.eqiad.wmnet: [elasticsearch, elasticsearch-ssl]
 rcstream:
   rcs1001.eqiad.wmnet: [nginx, nginx_ssl]
   rcs1002.eqiad.wmnet: [nginx, nginx_ssl]

-- 
To view, visit https://gerrit.wikimedia.org/r/295657
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I007c6fb14754e310895c7adcd7490e71d6d45af6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Moving elasticsearch masters to new servers - change (operations/puppet)

2016-06-23 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Moving elasticsearch masters to new servers
..


Moving elasticsearch masters to new servers

This is required to be able to decommission the old elasticsearch servers.

Bug: T138329
Change-Id: Idd7e769df8dc5ad4467cfc464af88d43778fa5ba
---
D hieradata/hosts/elastic1008.yaml
D hieradata/hosts/elastic1013.yaml
R hieradata/hosts/elastic1030.yaml
C hieradata/hosts/elastic1036.yaml
C hieradata/hosts/elastic1040.yaml
M hieradata/regex.yaml
M hieradata/role/eqiad/elasticsearch/server.yaml
7 files changed, 4 insertions(+), 10 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  DCausse: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/hieradata/hosts/elastic1008.yaml b/hieradata/hosts/elastic1008.yaml
deleted file mode 100644
index 41eab79..000
--- a/hieradata/hosts/elastic1008.yaml
+++ /dev/null
@@ -1,3 +0,0 @@
-elasticsearch::master_eligible: true
-# make sure the elasticsearch::unicast_hosts property is updated if master
-# eligibility is modified
diff --git a/hieradata/hosts/elastic1013.yaml b/hieradata/hosts/elastic1013.yaml
deleted file mode 100644
index 41eab79..000
--- a/hieradata/hosts/elastic1013.yaml
+++ /dev/null
@@ -1,3 +0,0 @@
-elasticsearch::master_eligible: true
-# make sure the elasticsearch::unicast_hosts property is updated if master
-# eligibility is modified
diff --git a/hieradata/hosts/elastic1001.yaml b/hieradata/hosts/elastic1030.yaml
similarity index 100%
rename from hieradata/hosts/elastic1001.yaml
rename to hieradata/hosts/elastic1030.yaml
diff --git a/hieradata/hosts/elastic1001.yaml b/hieradata/hosts/elastic1036.yaml
similarity index 100%
copy from hieradata/hosts/elastic1001.yaml
copy to hieradata/hosts/elastic1036.yaml
diff --git a/hieradata/hosts/elastic1001.yaml b/hieradata/hosts/elastic1040.yaml
similarity index 100%
copy from hieradata/hosts/elastic1001.yaml
copy to hieradata/hosts/elastic1040.yaml
diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index c31960a..b50957c 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -59,7 +59,7 @@
   elasticsearch::row:  D
 
 es_eqiad_rack_d4:
-  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|43|44|45|46|47)\.eqiad\.wmnet$/
+  __regex: !ruby/regexp 
/^elastic10(05|06|23|24|25|26|27|28|29|44|45|46|47)\.eqiad\.wmnet$/
   elasticsearch::rack: D4
   elasticsearch::row:  D
 
diff --git a/hieradata/role/eqiad/elasticsearch/server.yaml 
b/hieradata/role/eqiad/elasticsearch/server.yaml
index a343e20..e5fd413 100644
--- a/hieradata/role/eqiad/elasticsearch/server.yaml
+++ b/hieradata/role/eqiad/elasticsearch/server.yaml
@@ -5,9 +5,9 @@
 elasticsearch::multicast_group: 224.2.2.5
 elasticsearch::multicast_enabled: false
 elasticsearch::unicast_hosts: # this should be set to the list of master 
eligible nodes
-  - elastic1001.eqiad.wmnet
-  - elastic1008.eqiad.wmnet
-  - elastic1013.eqiad.wmnet
+  - elastic1030.eqiad.wmnet
+  - elastic1036.eqiad.wmnet
+  - elastic1040.eqiad.wmnet
 
 lvs::realserver::realserver_ips: "10.2.2.30"
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295585
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Idd7e769df8dc5ad4467cfc464af88d43778fa5ba
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: DCausse 
Gerrit-Reviewer: EBernhardson 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] fix puppet unit test in elastic search - change (operations/puppet)

2016-06-20 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: fix puppet unit test in elastic search
..


fix puppet unit test in elastic search

default values have changed in elastic search module but test was not updated

Change-Id: I78518f5e34e22d5974d0c9912a6c388d729305e2
---
M modules/elasticsearch/spec/defines/https_rspec.rb
1 file changed, 13 insertions(+), 13 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/elasticsearch/spec/defines/https_rspec.rb 
b/modules/elasticsearch/spec/defines/https_rspec.rb
index 49b94da..1d066e5 100644
--- a/modules/elasticsearch/spec/defines/https_rspec.rb
+++ b/modules/elasticsearch/spec/defines/https_rspec.rb
@@ -7,6 +7,19 @@
   } }
 
   context 'with default parameters' do
+it 'should ensure that certificates are present' do
+  should contain_file('/etc/nginx/ssl/cert.pem').with({ 'ensure' => 
'present' })
+  should contain_file('/etc/nginx/ssl/server.key').with({ 'ensure' => 
'present' })
+end
+
+it 'should ensure that nginx package is present' do
+  should contain_package('nginx-full').with({ 'ensure' => 'present' })
+end
+  end
+
+  context 'with ensure => absent' do
+let(:params) { { :ensure => 'absent' } }
+
 it 'should ensure that certificates are absent' do
   should contain_file('/etc/nginx/ssl/cert.pem').with({ 'ensure' => 
'absent' })
   should contain_file('/etc/nginx/ssl/server.key').with({ 'ensure' => 
'absent' })
@@ -14,19 +27,6 @@
 
 it 'should ensure that nginx package is absent' do
   should contain_package('nginx-full').with({ 'ensure' => 'absent' })
-end
-  end
-
-  context 'with ensure => present' do
-let(:params) { { :ensure => 'present' } }
-
-it 'should ensure that certificates are present' do
-  should contain_file('/etc/nginx/ssl/cert.pem').with({ 'ensure' => 
'present' })
-  should contain_file('/etc/nginx/ssl/server.key').with({ 'ensure' => 
'present' })
-end
-
-it 'should ensure that nginx package is installed' do
-  should contain_package('nginx-full').with({ 'ensure' => 'present' })
 end
   end
 

-- 
To view, visit https://gerrit.wikimedia.org/r/295127
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I78518f5e34e22d5974d0c9912a6c388d729305e2
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Maturain 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Manage Postgresql data dir with Puppet - change (operations/puppet)

2016-06-20 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295227

Change subject: Manage Postgresql data dir with Puppet
..

Manage Postgresql data dir with Puppet

Default data directory (/var/lib/postgresql/...) is created during
installation of debian package. In case we want to use another data directory,
like it is the case for maps servers, this directory is not created and need
to be managed by Puppet.

Minor refactoring of existing tests was done to ensure all tests are green.

Bug: T138092
Change-Id: Iec59cf0290a5ea34cdab26d3604b32a9edde4967
---
M hieradata/role/common/maps/master.yaml
M hieradata/role/common/maps/slave.yaml
M modules/postgresql/manifests/master.pp
M modules/postgresql/manifests/server.pp
M modules/postgresql/manifests/slave.pp
M modules/postgresql/spec/classes/postgresql_server_spec.rb
M modules/postgresql/spec/classes/postgresql_slave_spec.rb
M modules/postgresql/spec/defines/postgresql_user_spec.rb
M modules/postgresql/spec/spec_helper.rb
M modules/postgresql/templates/postgresql.conf.erb
10 files changed, 83 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/27/295227/1

diff --git a/hieradata/role/common/maps/master.yaml 
b/hieradata/role/common/maps/master.yaml
index f78..4e2860e 100644
--- a/hieradata/role/common/maps/master.yaml
+++ b/hieradata/role/common/maps/master.yaml
@@ -1,5 +1,5 @@
 postgresql::master::pgversion: 9.4
-postgresql::master::datadir: /srv/postgresql/9.4/main
+postgresql::master::root_dir: /srv/postgresql
 postgresql::master::includes:
   - tuning.conf
   - logging.conf
diff --git a/hieradata/role/common/maps/slave.yaml 
b/hieradata/role/common/maps/slave.yaml
index 4454cb9..1ef1126 100644
--- a/hieradata/role/common/maps/slave.yaml
+++ b/hieradata/role/common/maps/slave.yaml
@@ -1,5 +1,5 @@
 postgresql::slave::pgversion: 9.4
-postgresql::slave::datadir: /srv/postgresql/9.4/main
+postgresql::slave::root_dir: /srv/postgresql
 postgresql::slave::includes: tuning.conf
 # Defined in the private repo
 #postgresql::slave::replication_pass: justapass
diff --git a/modules/postgresql/manifests/master.pp 
b/modules/postgresql/manifests/master.pp
index 1869fff..2b4aa38 100644
--- a/modules/postgresql/manifests/master.pp
+++ b/modules/postgresql/manifests/master.pp
@@ -17,6 +17,8 @@
 #   Defaults to 64. Refer to postgresql documentation for its meaning
 #   wal_keep_segments
 #   Defaults to 128. Refer to postgresql documentation for its meaning
+#   root_dir
+#   See $postgresql::server::root_dir
 #
 # Actions:
 #  Install/configure postgresql as a master. Also create replication users
@@ -38,14 +40,14 @@
 $max_wal_senders=3,
 $checkpoint_segments=64,
 $wal_keep_segments=128,
-$datadir=undef
+$root_dir= '/var/lib/postgresql',
 ) {
 
 class { 'postgresql::server':
 ensure=> $ensure,
 pgversion => $pgversion,
 includes  => [ $includes, 'master.conf'],
-datadir   => $datadir,
+root_dir  => $root_dir,
 }
 
 file { "/etc/postgresql/${pgversion}/main/master.conf":
diff --git a/modules/postgresql/manifests/server.pp 
b/modules/postgresql/manifests/server.pp
index 9ef223b..6fa77c2 100644
--- a/modules/postgresql/manifests/server.pp
+++ b/modules/postgresql/manifests/server.pp
@@ -13,6 +13,9 @@
 #   includes
 #   An array of files that will be included in the config. It is
 #   the caller's responsibility to provide these
+#   root_dir
+#   The root directory for postgresql data. The actual directory will be
+#   "${root_dir}/${pgversion}/main".
 #
 # Actions:
 #  Install/configure postgresql
@@ -32,7 +35,7 @@
 $includes = [],
 $listen_addresses = '*',
 $port = '5432',
-$datadir  = undef,
+$root_dir = '/var/lib/postgresql',
 ) {
 package { [
 "postgresql-${pgversion}",
@@ -45,6 +48,21 @@
 ensure => $ensure,
 }
 
+$data_dir = "${root_dir}/${pgversion}/main"
+
+file {  [ $root_dir, "${root_dir}/${pgversion}" ] :
+ensure => ensure_directory($ensure),
+owner  => 'postgres',
+group  => 'postgres',
+mode   => '0755',
+}
+file { $data_dir:
+ensure => ensure_directory($ensure),
+owner  => 'postgres',
+group  => 'postgres',
+mode   => '0700',
+}
+
 exec { 'pgreload':
 command => "/usr/bin/pg_ctlcluster ${pgversion} main reload",
 user=> 'postgres',
diff --git a/modules/postgresql/manifests/slave.pp 
b/modules/postgresql/manifests/slave.pp
index 329a0f9..6399fb9 100644
--- a/modules/postgresql/manifests/slave.pp
+++ b/modules/postgresql/manifests/slave.pp
@@ -12,6 +12,8 @@
 #   Defaults to 9.1. Valid values 8.4, 9.1 in Ubuntu
 #   ensure
 #   Defaults to present
+#   root_dir
+#   See $postgresql::server::root_dir
 #
 #

[MediaWiki-commits] [Gerrit] Configuration for new maps cluster in eqiad - change (operations/puppet)

2016-06-20 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Configuration for new maps cluster in eqiad
..


Configuration for new maps cluster in eqiad

Base configuration added. The maps slaves are still commented out in site.pp
to ensure we install the master first.

Bug: T138092
Change-Id: I398157c77a1c278056191bcc223fc91b977fb65b
---
A hieradata/hosts/maps1001.yaml
A hieradata/hosts/maps1002.yaml
A hieradata/hosts/maps1003.yaml
A hieradata/hosts/maps1004.yaml
A hieradata/role/eqiad/maps/master.yaml
A hieradata/role/eqiad/maps/server.yaml
A hieradata/role/eqiad/maps/slave.yaml
M manifests/site.pp
8 files changed, 62 insertions(+), 0 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/hosts/maps1001.yaml b/hieradata/hosts/maps1001.yaml
new file mode 100644
index 000..9f2723a
--- /dev/null
+++ b/hieradata/hosts/maps1001.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: a4
+do_paging: false
diff --git a/hieradata/hosts/maps1002.yaml b/hieradata/hosts/maps1002.yaml
new file mode 100644
index 000..160db87
--- /dev/null
+++ b/hieradata/hosts/maps1002.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: b4
+do_paging: false
diff --git a/hieradata/hosts/maps1003.yaml b/hieradata/hosts/maps1003.yaml
new file mode 100644
index 000..4e53a74
--- /dev/null
+++ b/hieradata/hosts/maps1003.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: c5
+do_paging: false
diff --git a/hieradata/hosts/maps1004.yaml b/hieradata/hosts/maps1004.yaml
new file mode 100644
index 000..7e4068c
--- /dev/null
+++ b/hieradata/hosts/maps1004.yaml
@@ -0,0 +1,2 @@
+cassandra::rack: d3
+do_paging: false
diff --git a/hieradata/role/eqiad/maps/master.yaml 
b/hieradata/role/eqiad/maps/master.yaml
new file mode 100644
index 000..4f9e59c
--- /dev/null
+++ b/hieradata/role/eqiad/maps/master.yaml
@@ -0,0 +1,32 @@
+# Defined in the private repo
+#cassandra::super_password: some_password
+#maps::postgresql_tilerator_pass: some_password
+#maps::postgresql_tileratorui_pass: some_password
+#maps::postgresql_kartotherian_pass: some_password
+#maps::postgresql_osmimporter_pass: some_password
+#maps::postgresql_osmupdater_pass: some_password
+#maps::postgres_slaves:
+#  replication@maps1002-v4:
+#user: replication
+#password: some_password
+#cidr: 10.64.16.42
+#pgversion: 9.4
+#attrs: REPLICATION
+#database: replication
+#  replication@maps1003-v4:
+#user: replication
+#password: some_password
+#cidr: 10.64.32.117
+#pgversion: 9.4
+#attrs: REPLICATION
+#database: replication
+#  replication@maps1004-v4:
+#user: replication
+#password: some_password
+#cidr: 10.64.48.154
+#pgversion: 9.4
+#attrs: REPLICATION
+#database: replication
+#maps::cassandra_tilerator_pass: some_password
+#maps::cassandra_tileratorui_pass: some_password
+#maps::cassandra_kartotherian_pass: some_password
diff --git a/hieradata/role/eqiad/maps/server.yaml 
b/hieradata/role/eqiad/maps/server.yaml
new file mode 100644
index 000..fd3a78f
--- /dev/null
+++ b/hieradata/role/eqiad/maps/server.yaml
@@ -0,0 +1,10 @@
+cassandra::seeds: [ 10.64.0.79, 10.64.16.42, 10.64.32.117, 10.64.48.154 ]
+cassandra::dc: eqiad
+lvs::realserver::realserver_ips:
+  - 10.2.2.13
+maps::hosts:
+  - maps1001.eqiad.wmnet
+  - maps1002.eqiad.wmnet
+  - maps1003.eqiad.wmnet
+  - maps1004.eqiad.wmnet
+maps::redis_server: redis://maps1001.eqiad.wmnet
diff --git a/hieradata/role/eqiad/maps/slave.yaml 
b/hieradata/role/eqiad/maps/slave.yaml
new file mode 100644
index 000..e9d4771
--- /dev/null
+++ b/hieradata/role/eqiad/maps/slave.yaml
@@ -0,0 +1 @@
+postgresql::slave::master_server: maps1001.eqiad.wmnet
diff --git a/manifests/site.pp b/manifests/site.pp
index ab9e343..0b53d3f 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -1944,6 +1944,17 @@
 include base::firewall
 }
 
+node 'maps1001.eqiad.wmnet' {
+role maps::server, maps::master
+include base::firewall
+}
+
+# Waiting for master to be configured before configuring the slaves
+# node /^maps100[2-4]\.eqiad\.wmnet/ {
+# role maps::server, maps::slave
+# include base::firewall
+# }
+
 node 'maps2001.codfw.wmnet' {
 role maps::server, maps::master
 include base::firewall

-- 
To view, visit https://gerrit.wikimedia.org/r/294914
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I398157c77a1c278056191bcc223fc91b977fb65b
Gerrit-PatchSet: 6
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Yurik 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org

[MediaWiki-commits] [Gerrit] Fix team name for icinga alert - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Fix team name for icinga alert
..


Fix team name for icinga alert

Name of team is team-interactive and not interactive-team

Change-Id: I09b37aedc805ec69950b23672cacbd11586e249a
---
M modules/role/manifests/graphite/alerts/reqstats.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Gehel: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/graphite/alerts/reqstats.pp 
b/modules/role/manifests/graphite/alerts/reqstats.pp
index 1a1ff3b..4e8970d 100644
--- a/modules/role/manifests/graphite/alerts/reqstats.pp
+++ b/modules/role/manifests/graphite/alerts/reqstats.pp
@@ -76,6 +76,6 @@
 critical=> $settings['critical'],
 from=> $settings['cron'],
 nagios_critical => false,
-contact_group   => 'admins,interactive-team',
+contact_group   => 'admins,team-interactive',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294700
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I09b37aedc805ec69950b23672cacbd11586e249a
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel 
Gerrit-Reviewer: Faidon Liambotis 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add the ability to configure contact group for check of serv... - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has submitted this change and it was merged.

Change subject: Add the ability to configure contact group for check of 
services.
..


Add the ability to configure contact group for check of services.

For example, on Maps servers, it make sense to the interactive team to
be notified of service issues, but not to receive all notifications on
which they cannot act anyway.

The immediate need is on service::node, but I updated service::uwsgi as well
to keep coherence.

Bug: T137869
Bug: T137891
Change-Id: I1a1911263c04132d90f53e28ab8c5d2fc3a3a2c2
---
M modules/service/manifests/node.pp
M modules/service/manifests/uwsgi.pp
2 files changed, 23 insertions(+), 7 deletions(-)

Approvals:
  Gehel: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/service/manifests/node.pp 
b/modules/service/manifests/node.pp
index 1ca33ea..fa3dec4 100644
--- a/modules/service/manifests/node.pp
+++ b/modules/service/manifests/node.pp
@@ -84,6 +84,11 @@
 #   Whether Scap3 is used for deploying the config as well. Applicable only 
when
 #   $deployment == 'scap3'. Default: false
 #
+# [*contact_groups*]
+#   Contact groups for alerting.
+#   Default: hiera('contactgroups', 'admins') - use 'contactgroups' hiera
+#variable with a fallback to 'admins' if 'contactgroups' isn't set.
+#
 # === Examples
 #
 # To set up a service named myservice on port 8520 and with a templated
@@ -123,6 +128,7 @@
 $deployment  = undef,
 $deployment_user = 'deploy-service',
 $deployment_config = false,
+$contact_groups  = hiera('contactgroups', 'admins'),
 ) {
 case $deployment {
 'scap3': {
@@ -325,10 +331,11 @@
 mode=> '0755',
 }
 nrpe::monitor_service{ "endpoints_${title}":
-ensure   => $ensure_monitoring,
-description  => "${title} endpoints health",
-nrpe_command => "/usr/local/bin/check-${title}",
-subscribe=> File["/usr/local/bin/check-${title}"],
+ensure=> $ensure_monitoring,
+description   => "${title} endpoints health",
+nrpe_command  => "/usr/local/bin/check-${title}",
+subscribe => File["/usr/local/bin/check-${title}"],
+contact_group => $contact_groups,
 }
 # we also support smart-releases
 service::deployment_script { $name:
@@ -341,6 +348,7 @@
 ensure=> $ensure_monitoring,
 description   => $title,
 check_command => "check_http_port_url!${port}!${healthcheck_url}",
+contact_group => $contact_groups,
 }
 }
 
diff --git a/modules/service/manifests/uwsgi.pp 
b/modules/service/manifests/uwsgi.pp
index e8a8021..b29d3e9 100644
--- a/modules/service/manifests/uwsgi.pp
+++ b/modules/service/manifests/uwsgi.pp
@@ -55,6 +55,11 @@
 #   An array of string representing sudo rules in the sudoers format that you
 #   want the service to have. Default: empty array
 #
+# [*contact_groups*]
+#   Contact groups for alerting.
+#   Default: hiera('contactgroups', 'admins') - use 'contactgroups' hiera
+#variable with a fallback to 'admins' if 'contactgroups' isn't set.
+#
 # === Examples
 #
 #service::uwsgi { 'myservice':
@@ -81,6 +86,7 @@
 $deployment_manage_user = true,
 $deployment = 'scap3',
 $sudo_rules = [],
+$contact_groups = hiera('contactgroups', 'admins'),
 ) {
 if $deployment == 'scap3' {
 scap::target { $repo:
@@ -168,9 +174,10 @@
 
 $monitor_url = "http://${::ipaddress}:${port}${healthcheck_url};
 nrpe::monitor_service{ "endpoints_${title}":
-description  => "${title} endpoints health",
-nrpe_command => "/usr/local/lib/nagios/plugins/service_checker -t 
5 ${::ipaddress} ${monitor_url}",
-subscribe=> 
File['/usr/local/lib/nagios/plugins/service_checker'],
+description   => "${title} endpoints health",
+nrpe_command  => "/usr/local/lib/nagios/plugins/service_checker -t 
5 ${::ipaddress} ${monitor_url}",
+subscribe => 
File['/usr/local/lib/nagios/plugins/service_checker'],
+contact_group => $contact_groups,
 }
 # we also support smart-releases
 # TODO: Enable has_autorestart
@@ -182,6 +189,7 @@
 monitoring::service { $title:
 description   => $title,
 check_command => "check_http_port_url!${port}!${healthcheck_url}",
+contact_group => $contact_groups,
 }
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/294507
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1a1911263c04132d90f53e28ab8c5d2fc3a3a2c2
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch:

[MediaWiki-commits] [Gerrit] Fixed typo (cron instead of from). - change (operations/puppet)

2016-06-16 Thread Gehel (Code Review)

Gehel has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/294679

Change subject: Fixed typo (cron instead of from).
..

Fixed typo (cron instead of from).

This should be a noop as the default value for 'from' is '10min' anyway.
Still it make sense to fix it. Also used the more compact syntax for default
parameter values.

Change-Id: I71d90be11c93cd32fc0ddba371f21cf00925372c
---
M modules/role/manifests/graphite/alerts/reqstats.pp
1 file changed, 8 insertions(+), 33 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/79/294679/1

diff --git a/modules/role/manifests/graphite/alerts/reqstats.pp 
b/modules/role/manifests/graphite/alerts/reqstats.pp
index e420637..51b708f 100644
--- a/modules/role/manifests/graphite/alerts/reqstats.pp
+++ b/modules/role/manifests/graphite/alerts/reqstats.pp
@@ -2,80 +2,55 @@
 
 # Monitor production 5xx rates
 
-$settings = {'warning' => 250, 'critical' => 1000, from => '10min'}
+# Default values for all graphite threshold defined here. Can be
+# overwritten on specific instances below if needed.
+Monitoring::Graphite_threshold {
+warning => 250,
+critical=> 1000,
+from=> '10min',
+nagios_critical => false,
+}
 
 # sites aggregates
 monitoring::graphite_threshold { 'reqstats-5xx-eqiad':
 description => 'Eqiad HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.eqiad.*.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 monitoring::graphite_threshold { 'reqstats-5xx-esams':
 description => 'Esams HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.esams.*.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 monitoring::graphite_threshold { 'reqstats-5xx-codfw':
 description => 'Codfw HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.codfw.*.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 monitoring::graphite_threshold { 'reqstats-5xx-ulsfo':
 description => 'Ulsfo HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.ulsfo.*.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 # per-cache aggregates
 monitoring::graphite_threshold { 'reqstats-5xx-text':
 description => 'Text HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.*.text.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 monitoring::graphite_threshold { 'reqstats-5xx-uploads':
 description => 'Uploads HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.*.uploads.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 monitoring::graphite_threshold { 'reqstats-5xx-misc':
 description => 'Misc HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.*.misc.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 
 monitoring::graphite_threshold { 'reqstats-5xx-maps':
 description => 'Maps HTTP 5xx reqs/min',
 metric  => 
'sumSeries(varnish.*.maps.frontend.request.client.status.5xx.sum)',
-warning => $settings['warning'],
-critical=> $settings['critical'],
-from=> $settings['cron'],
-nagios_critical => false,
 }
 }
 

-- 
To view, visit https://gerrit.wikimedia.org/r/294679
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I71d90be11c93cd32fc0ddba371f21cf00925372c
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Gehel

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 1547 matches

Mail list logo