[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::docker::builder: add proxy settings to build config

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/379175 )

Change subject: profile::docker::builder: add proxy settings to build config
..


profile::docker::builder: add proxy settings to build config

Change-Id: I5a697620a3efc7fd2b24dd2119941748b8e5d5c8
---
M modules/profile/templates/docker/production-images-config.yaml.erb
1 file changed, 3 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/templates/docker/production-images-config.yaml.erb 
b/modules/profile/templates/docker/production-images-config.yaml.erb
index c83718a..57a4f08 100644
--- a/modules/profile/templates/docker/production-images-config.yaml.erb
+++ b/modules/profile/templates/docker/production-images-config.yaml.erb
@@ -1,3 +1,6 @@
+<%- if @proxy_address and @proxy_address != '' -%>
+http_proxy: "http://<%= @proxy_address %>:<%= @proxy_port %>"
+<%- end -%>
 registry: <%= @registry %>
 username: <%= @username %>
 password: <%= @password %>

-- 
To view, visit https://gerrit.wikimedia.org/r/379175
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5a697620a3efc7fd2b24dd2119941748b8e5d5c8
Gerrit-PatchSet: 6
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Add explicit management of http proxy for apt.

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/379258 )

Change subject: Add explicit management of http proxy for apt.
..

Add explicit management of http proxy for apt.

Change-Id: Ib28debd1caf2c7dc7db22cbefaac0a12a68ff6cb
---
M build
1 file changed, 21 insertions(+), 10 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/58/379258/1

diff --git a/build b/build
index d1b1da1..2574313 100755
--- a/build
+++ b/build
@@ -8,7 +8,7 @@
 import docker.errors
 import yaml
 
-from jinja2 import Environment, FileSystemLoader
+from jinja2 import Environment, FileSystemLoader, Template
 from debian.changelog import Changelog
 
 known_images = {}
@@ -31,15 +31,22 @@
 
 
 def apt_installer(opts):
+t = Template("""
+{%- if http_proxy -%}
+RUN echo 'Acquire::http::Proxy \"{{ http_proxy }}\";' > 
/etc/apt/apt.conf.d/80_proxy \\
+&& apt-get update {{ apt_options }} \\
+{%- else -%}
+RUN apt-get update {{ apt_options }} \\
+{%- endif %}
+&& DEBIAN_FRONTEND=noninteractive \\
+apt-get install {{ apt_options }} --yes {{ packages }} 
--no-install-recommends \\
+{%- if http_proxy %}
+&& rm -f /etc/apt/apt.conf.d/80_proxy \\
+{%- endif %}
+&& apt-get clean && rm -rf /var/lib/apt/lists/* """)
+
 def apt_install(pkgs):
-return """
-RUN apt-get update {apt_options} \
-&& DEBIAN_FRONTEND=noninteractive \
-apt-get install {apt_options} --yes {packages} --no-install-recommends \
-&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(
-apt_options=opts,
-packages=pkgs
-)
+return t.render(**opts, packages=pkgs)
 return apt_install
 
 
@@ -49,7 +56,7 @@
 self.path = path
 env = Environment(loader=FileSystemLoader(path))
 env.filters['image_tag'] = find_image_tag
-env.filters['apt_install'] = apt_installer(config['apt_options'])
+env.filters['apt_install'] = apt_installer(config)
 self.tpl = env.get_template('Dockerfile.template')
 self.config = config
 with open(os.path.join(path, 'changelog'), 'rb') as fh:
@@ -57,6 +64,9 @@
 changelog = Changelog(chlog)
 self.tag = str(changelog.get_version())
 self.name = str(changelog.get_package())
+
+def apt_installer(self):
+env = Environment()
 
 @property
 def dockerfile(self):
@@ -77,6 +87,7 @@
 'username': None, 'password': None,
 'seed_image': 'wikimedia-stretch:latest',
 'apt_options': '',
+'http_proxy': None,
 }
 self.config.update(self._read_config(configfile))
 self.client = docker.from_env(version='auto')

-- 
To view, visit https://gerrit.wikimedia.org/r/379258
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib28debd1caf2c7dc7db22cbefaac0a12a68ff6cb
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::docker::builder: add proxy settings to build config

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/379175 )

Change subject: profile::docker::builder: add proxy settings to build config
..

profile::docker::builder: add proxy settings to build config

Change-Id: I5a697620a3efc7fd2b24dd2119941748b8e5d5c8
---
M modules/profile/templates/docker/production-images-config.yaml.erb
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/75/379175/1

diff --git a/modules/profile/templates/docker/production-images-config.yaml.erb 
b/modules/profile/templates/docker/production-images-config.yaml.erb
index c83718a..b961f71 100644
--- a/modules/profile/templates/docker/production-images-config.yaml.erb
+++ b/modules/profile/templates/docker/production-images-config.yaml.erb
@@ -1,3 +1,5 @@
+<%- if @proxy_host and @proxy_host != '' -%>
+apt_options: "-o Acquire::http::Proxy::security.debian.org=<%= @proxy_host 
%>:<%= @proxy_port %> -o Acquire::http::Proxy::security-cdn.debian.org=<%= 
@proxy_host %>:<%= @proxy_port %>"
 registry: <%= @registry %>
 username: <%= @username %>
 password: <%= @password %>

-- 
To view, visit https://gerrit.wikimedia.org/r/379175
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5a697620a3efc7fd2b24dd2119941748b8e5d5c8
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::docker::builder: add build script for production-im...

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/379176 )

Change subject: profile::docker::builder: add build script for production-images
..

profile::docker::builder: add build script for production-images

For now this is just basically an alias, but in the future we might
consider expanding it.

Even if we end up not doing that, it will still make our lives easier.

Change-Id: I4cfed71e62583608c37d1681c15d1d0d9e554305
---
A modules/profile/files/docker/build-production-images.sh
M modules/profile/manifests/docker/builder.pp
2 files changed, 11 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/76/379176/1

diff --git a/modules/profile/files/docker/build-production-images.sh 
b/modules/profile/files/docker/build-production-images.sh
new file mode 100755
index 000..50681a3
--- /dev/null
+++ b/modules/profile/files/docker/build-production-images.sh
@@ -0,0 +1,3 @@
+#!/bin/bash
+cd /srv/images/production-images \
+   && .venv/bin/python ./build -c /etc/production-images/config.yaml images
diff --git a/modules/profile/manifests/docker/builder.pp 
b/modules/profile/manifests/docker/builder.pp
index a16952e..2dca04a 100644
--- a/modules/profile/manifests/docker/builder.pp
+++ b/modules/profile/manifests/docker/builder.pp
@@ -45,4 +45,12 @@
 group   => 'root',
 mode=> '0444'
 }
+
+file { '/usr/local/bin/build-production-images':
+ensure => present,
+source => 
'puppet:///modules/profile/docker/build-production-images.sh',
+owner  => 'root',
+group  => 'root',
+mode   => '0500'
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/379176
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4cfed71e62583608c37d1681c15d1d0d9e554305
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Add apt_options to apt-get update as well

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/379173 )

Change subject: Add apt_options to apt-get update as well
..


Add apt_options to apt-get update as well

Change-Id: Ia51bd8ca485b94eec102a6286545777d7564d18b
---
M build
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/build b/build
index 50c5dba..d1b1da1 100755
--- a/build
+++ b/build
@@ -33,7 +33,7 @@
 def apt_installer(opts):
 def apt_install(pkgs):
 return """
-RUN apt-get update \
+RUN apt-get update {apt_options} \
 && DEBIAN_FRONTEND=noninteractive \
 apt-get install {apt_options} --yes {packages} --no-install-recommends \
 && apt-get clean && rm -rf /var/lib/apt/lists/* """.format(

-- 
To view, visit https://gerrit.wikimedia.org/r/379173
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia51bd8ca485b94eec102a6286545777d7564d18b
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Add apt_options to apt-get update as well

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/379173 )

Change subject: Add apt_options to apt-get update as well
..

Add apt_options to apt-get update as well

Change-Id: Ia51bd8ca485b94eec102a6286545777d7564d18b
---
M build
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/73/379173/1

diff --git a/build b/build
index 50c5dba..d1b1da1 100755
--- a/build
+++ b/build
@@ -33,7 +33,7 @@
 def apt_installer(opts):
 def apt_install(pkgs):
 return """
-RUN apt-get update \
+RUN apt-get update {apt_options} \
 && DEBIAN_FRONTEND=noninteractive \
 apt-get install {apt_options} --yes {packages} --no-install-recommends \
 && apt-get clean && rm -rf /var/lib/apt/lists/* """.format(

-- 
To view, visit https://gerrit.wikimedia.org/r/379173
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia51bd8ca485b94eec102a6286545777d7564d18b
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Makefile: make "clean" fault-tolerant

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378953 )

Change subject: Makefile: make "clean" fault-tolerant
..


Makefile: make "clean" fault-tolerant

Change-Id: I5f4718037e29b63024d2e1d80f9c707749b419e9
---
M Makefile
1 file changed, 4 insertions(+), 4 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/Makefile b/Makefile
index a790203..2a49868 100644
--- a/Makefile
+++ b/Makefile
@@ -38,13 +38,13 @@
 clean-artifacts:
 ifeq ($(DOCKER), 1)
-docker rmi production-images-build:latest
-   rm production-image.created
+   -rm production-image.created
 endif
-   rm -rf .artifacts
+   -rm -rf .artifacts
 
 clean: clean-artifacts clean-dev
-   rm -rf .venv
-   rm -rf frozen-requirements.txt
+   -rm -rf .venv
+   -rm -rf frozen-requirements.txt
 
 clean-dev:
rm -rf .venv-dev

-- 
To view, visit https://gerrit.wikimedia.org/r/378953
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5f4718037e29b63024d2e1d80f9c707749b419e9
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Fixes to the build script:

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378952 )

Change subject: Fixes to the build script:
..


Fixes to the build script:

* Change the working directory to the one where the Dockerfile template
  is located
* Allow defining apt options in the config (for things like proxies)

Change-Id: If1f3c3a765819f96798e4ad8de2fd5a60558a4ce
---
M build
M config.yaml
2 files changed, 41 insertions(+), 23 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/build b/build
index a7b18c4..50c5dba 100755
--- a/build
+++ b/build
@@ -2,7 +2,7 @@
 import argparse
 import os
 
-from io import BytesIO
+from contextlib import contextmanager
 
 import docker
 import docker.errors
@@ -14,6 +14,14 @@
 known_images = {}
 
 
+@contextmanager
+def pushd(dirname):
+cur_dir = os.getcwd()
+os.chdir(dirname)
+yield
+os.chdir(cur_dir)
+
+
 def find_image_tag(image_name):
 if image_name not in known_images:
 print('WARNING: image {name} not found'.format(name=image_name))
@@ -22,22 +30,26 @@
 return "{}:{}".format(image.name, image.tag)
 
 
-def apt_install(pkgs):
-return """
-RUN apt-get update && \
-DEBIAN_FRONTEND=noninteractive \
-apt-get install --yes {packages} --no-install-recommends \
-&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(packages=pkgs)
+def apt_installer(opts):
+def apt_install(pkgs):
+return """
+RUN apt-get update \
+&& DEBIAN_FRONTEND=noninteractive \
+apt-get install {apt_options} --yes {packages} --no-install-recommends \
+&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(
+apt_options=opts,
+packages=pkgs
+)
+return apt_install
 
 
 class DockerImage(object):
 
-def __init__(self, path, config, base):
-if not base.endswith('/'):
-base += '/'
+def __init__(self, path, config):
+self.path = path
 env = Environment(loader=FileSystemLoader(path))
 env.filters['image_tag'] = find_image_tag
-env.filters['apt_install'] = apt_install
+env.filters['apt_install'] = apt_installer(config['apt_options'])
 self.tpl = env.get_template('Dockerfile.template')
 self.config = config
 with open(os.path.join(path, 'changelog'), 'rb') as fh:
@@ -53,17 +65,18 @@
 print('===')
 print(dockerfile)
 print('===')
-return BytesIO(bytes(dockerfile, 'utf8'))
+return dockerfile
 
 
 class DockerBuilder(object):
 
 def __init__(self, directory, configfile):
-self.base_directory = directory
+self.base_directory = os.path.join(os.getcwd(), directory)
 self.config = {
 'registry': 'docker-registry.wikimedia.org',
 'username': None, 'password': None,
-'seed_image': 'wikimedia-stretch'
+'seed_image': 'wikimedia-stretch:latest',
+'apt_options': '',
 }
 self.config.update(self._read_config(configfile))
 self.client = docker.from_env(version='auto')
@@ -84,7 +97,7 @@
 print(
 'Processing the dockerfile template in 
{base}'.format(base=root)
 )
-yield DockerImage(root, self.config, self.base_directory)
+yield DockerImage(root, self.config)
 
 def image_exists(self, image):
 try:
@@ -95,14 +108,19 @@
 
 def build(self, image):
 print('Building image {name}:{version}'.format(name=image.name, 
version=image.tag))
+print('Build context: {path}'.format(path=image.path))
 image_ref = "{name}:{tag}".format(name=image.name, tag=image.tag)
-self.client.images.build(
-fileobj=image.dockerfile,
-tag=image_ref,
-nocache=True,
-rm=True,
-pull=False,
-)
+with pushd(image.path):
+with open('Dockerfile', 'w') as fh:
+fh.write(image.dockerfile)
+self.client.images.build(
+path='.',
+tag=image_ref,
+nocache=True,
+rm=True,
+pull=False,
+)
+os.remove('Dockerfile')
 print("Image built.")
 fullname = os.path.join(self.config['registry'], image.name)
 for tag in [image.tag, 'latest']:
diff --git a/config.yaml b/config.yaml
index f75608c..fe1370b 100644
--- a/config.yaml
+++ b/config.yaml
@@ -1,2 +1,2 @@
 registry: docker-registry.wikimedia.org
-seed_image: wikimedia-jessie:latest
+seed_image: wikimedia-stretch:latest

-- 
To view, visit https://gerrit.wikimedia.org/r/378952
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If1f3c3a765819f96798e4ad8de2fd5a60558a4ce
Gerrit-PatchSet: 1
Gerrit-Project: 

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Fix container references

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378714 )

Change subject: Fix container references
..


Fix container references

Change-Id: If6d8c74f4163ecb2dc59527d6e78de51c2da4631
---
M images/fluent-bit/Dockerfile.template
M images/nodejs/devel/Dockerfile.template
2 files changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/images/fluent-bit/Dockerfile.template 
b/images/fluent-bit/Dockerfile.template
index 236dd41..127e578 100644
--- a/images/fluent-bit/Dockerfile.template
+++ b/images/fluent-bit/Dockerfile.template
@@ -1,4 +1,4 @@
-FROM {{ registry }}/{{ seed_image }}:latest
+FROM {{ registry }}/{{ seed_image }}
 LABEL Description="Fluent-bit image to run as a sidecar container" \
   maintainer="j...@wikimedia.org"
 
diff --git a/images/nodejs/devel/Dockerfile.template 
b/images/nodejs/devel/Dockerfile.template
index 8bd9da9..3f2d0db 100644
--- a/images/nodejs/devel/Dockerfile.template
+++ b/images/nodejs/devel/Dockerfile.template
@@ -1,5 +1,5 @@
 # TODO: allow to define a "latest" token that fetches the latest version of 
the parent
 # from this repository
-FROM {{ registry }}/{{ "nodejs-slim" | image_tag }}
+FROM {{ registry }}/{{ "nodejs" | image_tag }}
 
 {{ "npm" | apt_install}}

-- 
To view, visit https://gerrit.wikimedia.org/r/378714
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If6d8c74f4163ecb2dc59527d6e78de51c2da4631
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Makefile: make "clean" fault-tolerant

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378953 )

Change subject: Makefile: make "clean" fault-tolerant
..

Makefile: make "clean" fault-tolerant

Change-Id: I5f4718037e29b63024d2e1d80f9c707749b419e9
---
M Makefile
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/53/378953/1

diff --git a/Makefile b/Makefile
index a790203..2a49868 100644
--- a/Makefile
+++ b/Makefile
@@ -38,13 +38,13 @@
 clean-artifacts:
 ifeq ($(DOCKER), 1)
-docker rmi production-images-build:latest
-   rm production-image.created
+   -rm production-image.created
 endif
-   rm -rf .artifacts
+   -rm -rf .artifacts
 
 clean: clean-artifacts clean-dev
-   rm -rf .venv
-   rm -rf frozen-requirements.txt
+   -rm -rf .venv
+   -rm -rf frozen-requirements.txt
 
 clean-dev:
rm -rf .venv-dev

-- 
To view, visit https://gerrit.wikimedia.org/r/378953
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5f4718037e29b63024d2e1d80f9c707749b419e9
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Fixes to the build script:

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378952 )

Change subject: Fixes to the build script:
..

Fixes to the build script:

* Change the working directory to the one where the Dockerfile template
  is located
* Allow defining apt options in the config (for things like proxies)

Change-Id: If1f3c3a765819f96798e4ad8de2fd5a60558a4ce
---
M build
M config.yaml
2 files changed, 41 insertions(+), 23 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/52/378952/1

diff --git a/build b/build
index a7b18c4..50c5dba 100755
--- a/build
+++ b/build
@@ -2,7 +2,7 @@
 import argparse
 import os
 
-from io import BytesIO
+from contextlib import contextmanager
 
 import docker
 import docker.errors
@@ -14,6 +14,14 @@
 known_images = {}
 
 
+@contextmanager
+def pushd(dirname):
+cur_dir = os.getcwd()
+os.chdir(dirname)
+yield
+os.chdir(cur_dir)
+
+
 def find_image_tag(image_name):
 if image_name not in known_images:
 print('WARNING: image {name} not found'.format(name=image_name))
@@ -22,22 +30,26 @@
 return "{}:{}".format(image.name, image.tag)
 
 
-def apt_install(pkgs):
-return """
-RUN apt-get update && \
-DEBIAN_FRONTEND=noninteractive \
-apt-get install --yes {packages} --no-install-recommends \
-&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(packages=pkgs)
+def apt_installer(opts):
+def apt_install(pkgs):
+return """
+RUN apt-get update \
+&& DEBIAN_FRONTEND=noninteractive \
+apt-get install {apt_options} --yes {packages} --no-install-recommends \
+&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(
+apt_options=opts,
+packages=pkgs
+)
+return apt_install
 
 
 class DockerImage(object):
 
-def __init__(self, path, config, base):
-if not base.endswith('/'):
-base += '/'
+def __init__(self, path, config):
+self.path = path
 env = Environment(loader=FileSystemLoader(path))
 env.filters['image_tag'] = find_image_tag
-env.filters['apt_install'] = apt_install
+env.filters['apt_install'] = apt_installer(config['apt_options'])
 self.tpl = env.get_template('Dockerfile.template')
 self.config = config
 with open(os.path.join(path, 'changelog'), 'rb') as fh:
@@ -53,17 +65,18 @@
 print('===')
 print(dockerfile)
 print('===')
-return BytesIO(bytes(dockerfile, 'utf8'))
+return dockerfile
 
 
 class DockerBuilder(object):
 
 def __init__(self, directory, configfile):
-self.base_directory = directory
+self.base_directory = os.path.join(os.getcwd(), directory)
 self.config = {
 'registry': 'docker-registry.wikimedia.org',
 'username': None, 'password': None,
-'seed_image': 'wikimedia-stretch'
+'seed_image': 'wikimedia-stretch:latest',
+'apt_options': '',
 }
 self.config.update(self._read_config(configfile))
 self.client = docker.from_env(version='auto')
@@ -84,7 +97,7 @@
 print(
 'Processing the dockerfile template in 
{base}'.format(base=root)
 )
-yield DockerImage(root, self.config, self.base_directory)
+yield DockerImage(root, self.config)
 
 def image_exists(self, image):
 try:
@@ -95,14 +108,19 @@
 
 def build(self, image):
 print('Building image {name}:{version}'.format(name=image.name, 
version=image.tag))
+print('Build context: {path}'.format(path=image.path))
 image_ref = "{name}:{tag}".format(name=image.name, tag=image.tag)
-self.client.images.build(
-fileobj=image.dockerfile,
-tag=image_ref,
-nocache=True,
-rm=True,
-pull=False,
-)
+with pushd(image.path):
+with open('Dockerfile', 'w') as fh:
+fh.write(image.dockerfile)
+self.client.images.build(
+path='.',
+tag=image_ref,
+nocache=True,
+rm=True,
+pull=False,
+)
+os.remove('Dockerfile')
 print("Image built.")
 fullname = os.path.join(self.config['registry'], image.name)
 for tag in [image.tag, 'latest']:
diff --git a/config.yaml b/config.yaml
index f75608c..fe1370b 100644
--- a/config.yaml
+++ b/config.yaml
@@ -1,2 +1,2 @@
 registry: docker-registry.wikimedia.org
-seed_image: wikimedia-jessie:latest
+seed_image: wikimedia-stretch:latest

-- 
To view, visit https://gerrit.wikimedia.org/r/378952
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If1f3c3a765819f96798e4ad8de2fd5a60558a4ce

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: another dependency fix

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378873 )

Change subject: docker::baseimages: another dependency fix
..


docker::baseimages: another dependency fix

Change-Id: I02674fdca1153f4b69ac49dd6c64374c46b3f6d2
---
M modules/docker/manifests/baseimages.pp
1 file changed, 0 insertions(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 04fa3b4..def15d0 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -62,7 +62,6 @@
 owner  => 'root',
 group  => 'root',
 mode   => '0444',
-notify => Exec['apt-key add for wikimedia stretch'],
 }
 ## end stretch
 

-- 
To view, visit https://gerrit.wikimedia.org/r/378873
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I02674fdca1153f4b69ac49dd6c64374c46b3f6d2
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: another dependency fix

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378873 )

Change subject: docker::baseimages: another dependency fix
..

docker::baseimages: another dependency fix

Change-Id: I02674fdca1153f4b69ac49dd6c64374c46b3f6d2
---
M modules/docker/manifests/baseimages.pp
1 file changed, 0 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/73/378873/1

diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 04fa3b4..def15d0 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -62,7 +62,6 @@
 owner  => 'root',
 group  => 'root',
 mode   => '0444',
-notify => Exec['apt-key add for wikimedia stretch'],
 }
 ## end stretch
 

-- 
To view, visit https://gerrit.wikimedia.org/r/378873
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I02674fdca1153f4b69ac49dd6c64374c46b3f6d2
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: drop exec for apt-key add

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378872 )

Change subject: docker::baseimages: drop exec for apt-key add
..


docker::baseimages: drop exec for apt-key add

As stated in the apt-key manpage, the tool is deprecated. So, just
upload a gpg1 keyring containing the key we were previously copying.

Change-Id: I66d4601341ee709a8eb54b996e37291ab5340ce9
---
M modules/docker/files/wikimedia-stretch.pub.gpg
M modules/docker/manifests/baseimages.pp
2 files changed, 3 insertions(+), 8 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/docker/files/wikimedia-stretch.pub.gpg 
b/modules/docker/files/wikimedia-stretch.pub.gpg
index dccbbff..1f394a3 100644
--- a/modules/docker/files/wikimedia-stretch.pub.gpg
+++ b/modules/docker/files/wikimedia-stretch.pub.gpg
Binary files differ
diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index e1a5071..04fa3b4 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -48,26 +48,21 @@
 }
 
 ## Stretch
+$stretch_keyring = '/srv/images/base/wikimedia-stretch.pub.gpg'
 file { '/srv/images/base/stretch.yaml':
 content => template('docker/images/stretch.yaml.erb'),
 owner   => 'root',
 group   => 'root',
 mode=> '0544',
 }
-file { '/srv/images/base/wikimedia-stretch.pub.gpg':
+
+file { $stretch_keyring:
 ensure => present,
 source => 'puppet:///modules/docker/wikimedia-stretch.pub.gpg',
 owner  => 'root',
 group  => 'root',
 mode   => '0444',
 notify => Exec['apt-key add for wikimedia stretch'],
-}
-$stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
-exec { 'apt-key add for wikimedia stretch':
-command => "/usr/bin/apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
-user=> 'root',
-group   => 'root',
-refreshonly => true,
 }
 ## end stretch
 

-- 
To view, visit https://gerrit.wikimedia.org/r/378872
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I66d4601341ee709a8eb54b996e37291ab5340ce9
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: drop exec for apt-key add

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378872 )

Change subject: docker::baseimages: drop exec for apt-key add
..

docker::baseimages: drop exec for apt-key add

As stated in the apt-key manpage, the tool is deprecated. So, just
upload a gpg1 keyring containing the key we were previously copying.

Change-Id: I66d4601341ee709a8eb54b996e37291ab5340ce9
---
M modules/docker/files/wikimedia-stretch.pub.gpg
M modules/docker/manifests/baseimages.pp
2 files changed, 3 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/72/378872/1

diff --git a/modules/docker/files/wikimedia-stretch.pub.gpg 
b/modules/docker/files/wikimedia-stretch.pub.gpg
index dccbbff..1f394a3 100644
--- a/modules/docker/files/wikimedia-stretch.pub.gpg
+++ b/modules/docker/files/wikimedia-stretch.pub.gpg
Binary files differ
diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index e1a5071..04fa3b4 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -48,26 +48,21 @@
 }
 
 ## Stretch
+$stretch_keyring = '/srv/images/base/wikimedia-stretch.pub.gpg'
 file { '/srv/images/base/stretch.yaml':
 content => template('docker/images/stretch.yaml.erb'),
 owner   => 'root',
 group   => 'root',
 mode=> '0544',
 }
-file { '/srv/images/base/wikimedia-stretch.pub.gpg':
+
+file { $stretch_keyring:
 ensure => present,
 source => 'puppet:///modules/docker/wikimedia-stretch.pub.gpg',
 owner  => 'root',
 group  => 'root',
 mode   => '0444',
 notify => Exec['apt-key add for wikimedia stretch'],
-}
-$stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
-exec { 'apt-key add for wikimedia stretch':
-command => "/usr/bin/apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
-user=> 'root',
-group   => 'root',
-refreshonly => true,
 }
 ## end stretch
 

-- 
To view, visit https://gerrit.wikimedia.org/r/378872
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I66d4601341ee709a8eb54b996e37291ab5340ce9
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: brown paper bag fix

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378868 )

Change subject: docker::baseimages: brown paper bag fix
..


docker::baseimages: brown paper bag fix

Change-Id: I496e3552ffbb3cdf29f532c92e5f3ae0144675e7
---
M modules/docker/manifests/baseimages.pp
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 939c2af..e1a5071 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -63,8 +63,8 @@
 notify => Exec['apt-key add for wikimedia stretch'],
 }
 $stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
-exec { '/usr/bin/apt-key add for wikimedia stretch':
-command => "apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
+exec { 'apt-key add for wikimedia stretch':
+command => "/usr/bin/apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
 user=> 'root',
 group   => 'root',
 refreshonly => true,

-- 
To view, visit https://gerrit.wikimedia.org/r/378868
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I496e3552ffbb3cdf29f532c92e5f3ae0144675e7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: brown paper bag fix

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378868 )

Change subject: docker::baseimages: brown paper bag fix
..

docker::baseimages: brown paper bag fix

Change-Id: I496e3552ffbb3cdf29f532c92e5f3ae0144675e7
---
M modules/docker/manifests/baseimages.pp
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/68/378868/1

diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 939c2af..e1a5071 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -63,8 +63,8 @@
 notify => Exec['apt-key add for wikimedia stretch'],
 }
 $stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
-exec { '/usr/bin/apt-key add for wikimedia stretch':
-command => "apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
+exec { 'apt-key add for wikimedia stretch':
+command => "/usr/bin/apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
 user=> 'root',
 group   => 'root',
 refreshonly => true,

-- 
To view, visit https://gerrit.wikimedia.org/r/378868
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I496e3552ffbb3cdf29f532c92e5f3ae0144675e7
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: fully qualify exec command

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378866 )

Change subject: docker::baseimages: fully qualify exec command
..


docker::baseimages: fully qualify exec command

Change-Id: I17be1e9f7c39c8d12dcb88694e314951160a4c6b
---
M modules/docker/manifests/baseimages.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 7aa274c..939c2af 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -63,7 +63,7 @@
 notify => Exec['apt-key add for wikimedia stretch'],
 }
 $stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
-exec { 'apt-key add for wikimedia stretch':
+exec { '/usr/bin/apt-key add for wikimedia stretch':
 command => "apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
 user=> 'root',
 group   => 'root',

-- 
To view, visit https://gerrit.wikimedia.org/r/378866
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I17be1e9f7c39c8d12dcb88694e314951160a4c6b
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: fully qualify exec command

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378866 )

Change subject: docker::baseimages: fully qualify exec command
..

docker::baseimages: fully qualify exec command

Change-Id: I17be1e9f7c39c8d12dcb88694e314951160a4c6b
---
M modules/docker/manifests/baseimages.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/66/378866/1

diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 7aa274c..939c2af 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -63,7 +63,7 @@
 notify => Exec['apt-key add for wikimedia stretch'],
 }
 $stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
-exec { 'apt-key add for wikimedia stretch':
+exec { '/usr/bin/apt-key add for wikimedia stretch':
 command => "apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
 user=> 'root',
 group   => 'root',

-- 
To view, visit https://gerrit.wikimedia.org/r/378866
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I17be1e9f7c39c8d12dcb88694e314951160a4c6b
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: add stretch base image

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378860 )

Change subject: docker::baseimages: add stretch base image
..


docker::baseimages: add stretch base image

Change-Id: If2a00dfde08ddedbd23f48026e4f9d877feb6a7b
---
A modules/docker/files/wikimedia-stretch.pub.gpg
M modules/docker/manifests/baseimages.pp
M modules/docker/templates/images/build-base-images.erb
M modules/docker/templates/images/stretch.yaml.erb
M modules/profile/manifests/docker/builder.pp
5 files changed, 68 insertions(+), 8 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/docker/files/wikimedia-stretch.pub.gpg 
b/modules/docker/files/wikimedia-stretch.pub.gpg
new file mode 100644
index 000..dccbbff
--- /dev/null
+++ b/modules/docker/files/wikimedia-stretch.pub.gpg
@@ -0,0 +1,29 @@
+-BEGIN PGP PUBLIC KEY BLOCK-
+
+mQINBFieKQEBEACZXCSJEwJnXsEofPUIunQTqF8p1IipjkDF4sNSgyuA17AD235h
+EEll1Czzd13bajM4D9dO+Yz4q9lWcrdoaHXklTEnA1Dhjk0wgTRqEqa37PBxjm7p
+xTuSOEFAHnQfjM9ZzV6Bd3kzIlfxnTguiYbN1pf4KQ7u4TQSchKr8V4p+mf4Y+Xg
+6hhskLrvMRYO7mNGEzm0vdfWsGbbvPZlhrRMea4oyCrOGd2piXAIPLR6DFXpyrb9
+GdEKrg8evTUnntsIqQ60h8veFETGNUmtpSF9OwzVGfH8TKACV7qA30Bbp6wDSLeY
+JefpX4yBexY3FbNx1oxXbXTZbDR8RZ4olWjP+inKYJTwHxQnvYjXWgMeNR/BgX2s
+ontLXqK5BzBP+1E0vHdAAYSU/8vjr+zmy1qyDWYtIlqdMXzDkXTCZIkJE0WPp5Sx
+sEFqpLdO7ggqJV5UF7h/yIq9bcxoaNxYGJFRDWg3Lj6ZLCxiSN/QHuqascMijKAU
+2RHzQtXhfXzZA8PwZp92dalOZOguytEYyhzrPfJlysvOnXTigHa9WeybDcJgsOO8
+xFbgU9P98NgyE4ZIel6P1GpLnBY8j0KpbhWfCi5mmomwwK5OUWe0g9/u4z5FmN3z
+/WWA/8BjuJI9MFnS3YoU3safEUi0vD5pnuktVQGJOkSnR6gAN4dAoCcVEwARAQAB
+tDxXaWtpbWVkaWEgQXJjaGl2ZSBBdXRvbWF0aWMgU2lnbmluZyBLZXkgPHJvb3RA
+d2lraW1lZGlhLm9yZz6JAjcEEwEKACEFAlieKQECGwMFCwkIBwMFFQoJCAsFFgID
+AQACHgECF4AACgkQnTktP/rfGPtmohAAhQhGkOe0DX5ZSK6uEs2IpB1W3EvCPK87
+vSgI6FZyGEURH7s/MknQR2lMShgHZ1oqLwZV1DE5ao/D4khPw1JZFeYajXOxQ3Sr
+/Qjg6ZayrJ/r9sSchC4nd6m+rFpzuRsx22FWrVenzRBau6/J6OUz3WawsjX3Bwrg
+/Hw11/+kgsabkdsdK28ENWdVj0tLp5asSxrs4L4d2RjKpfiufBAGpkqFV5p2J8aw
+cJ7Wh3/1mMo+biTtuemHtWf4XfWmXv99ltSEVe8PpMYQ+FjNUHDzEZMS5K39em+V
+EOC7GsE+SHHF6lgjH2BtDcpkimbU9uROnGeIWtpRr+PNMu/yhFLUCxe7tf4yffIJ
+NIAL3RFcRhLj0xCbL3fd5DbQP4eB68k9pLHwYxpKba7DDmH0aMstvk5xrpphdHAP
+p03VbE6ZThKxA828yb2jWtumDhoYRlkHs2KrVufUPA411mUE+A53pwqBse/KM9Iq
+Pidc+TwMwy2nByRB9WFyyGS08wqhewpwp8mwIOejvRf+3yi+CGHM2Xi4bTmEqVL+
+Jm1D5W49O5/j1T6HyVPGsyH/0QgYwg2IVZwL5Jfy9w3974jTV3QAcCBafR9hcZCS
+S2wOIo0ckRI0APw6DQSFMyBrNs9rWy72Ixu9RC1GtzCCn5OaMRUM9FaqjRnTKX4m
+grWsU8i35fI=
+=ib96
+-END PGP PUBLIC KEY BLOCK-
diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 9925d09..7aa274c 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -34,11 +34,10 @@
 }
 
 file { '/srv/images/base':
-ensure  => directory,
-owner   => 'root',
-group   => 'root',
-mode=> '0755',
-require => File['/srv/images'],
+ensure => directory,
+owner  => 'root',
+group  => 'root',
+mode   => '0755',
 }
 
 file { '/srv/images/base/jessie.yaml':
@@ -46,9 +45,32 @@
 owner   => 'root',
 group   => 'root',
 mode=> '0544',
-require => File['/srv/images/base'],
 }
 
+## Stretch
+file { '/srv/images/base/stretch.yaml':
+content => template('docker/images/stretch.yaml.erb'),
+owner   => 'root',
+group   => 'root',
+mode=> '0544',
+}
+file { '/srv/images/base/wikimedia-stretch.pub.gpg':
+ensure => present,
+source => 'puppet:///modules/docker/wikimedia-stretch.pub.gpg',
+owner  => 'root',
+group  => 'root',
+mode   => '0444',
+notify => Exec['apt-key add for wikimedia stretch'],
+}
+$stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
+exec { 'apt-key add for wikimedia stretch':
+command => "apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
+user=> 'root',
+group   => 'root',
+refreshonly => true,
+}
+## end stretch
+
 if 'alpine' in $distributions {
 if $proxy_address {
 $env = ["https_proxy=http://${proxy_address}:${proxy_port};]
diff --git a/modules/docker/templates/images/build-base-images.erb 
b/modules/docker/templates/images/build-base-images.erb
index 1c65f00..5c374f6 100755
--- a/modules/docker/templates/images/build-base-images.erb
+++ b/modules/docker/templates/images/build-base-images.erb
@@ -11,6 +11,15 @@
 docker push <%= @docker_registry %>/wikimedia-jessie
 <%- end -%>
 
+<%- if @distributions.include?('stretch') -%>
+# Build debian stretch image!
+/usr/bin/bootstrap-vz /srv/images/base/stretch.yaml
+
+# Push it to the repository
+docker push <%= @docker_registry %>/wikimedia-stretch
+<%- 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: docker::baseimages: add stretch base image

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378860 )

Change subject: docker::baseimages: add stretch base image
..

docker::baseimages: add stretch base image

Change-Id: If2a00dfde08ddedbd23f48026e4f9d877feb6a7b
---
A modules/docker/files/wikimedia-stretch.pub.gpg
M modules/docker/manifests/baseimages.pp
M modules/docker/templates/images/build-base-images.erb
M modules/docker/templates/images/stretch.yaml.erb
4 files changed, 67 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/60/378860/1

diff --git a/modules/docker/files/wikimedia-stretch.pub.gpg 
b/modules/docker/files/wikimedia-stretch.pub.gpg
new file mode 100644
index 000..dccbbff
--- /dev/null
+++ b/modules/docker/files/wikimedia-stretch.pub.gpg
@@ -0,0 +1,29 @@
+-BEGIN PGP PUBLIC KEY BLOCK-
+
+mQINBFieKQEBEACZXCSJEwJnXsEofPUIunQTqF8p1IipjkDF4sNSgyuA17AD235h
+EEll1Czzd13bajM4D9dO+Yz4q9lWcrdoaHXklTEnA1Dhjk0wgTRqEqa37PBxjm7p
+xTuSOEFAHnQfjM9ZzV6Bd3kzIlfxnTguiYbN1pf4KQ7u4TQSchKr8V4p+mf4Y+Xg
+6hhskLrvMRYO7mNGEzm0vdfWsGbbvPZlhrRMea4oyCrOGd2piXAIPLR6DFXpyrb9
+GdEKrg8evTUnntsIqQ60h8veFETGNUmtpSF9OwzVGfH8TKACV7qA30Bbp6wDSLeY
+JefpX4yBexY3FbNx1oxXbXTZbDR8RZ4olWjP+inKYJTwHxQnvYjXWgMeNR/BgX2s
+ontLXqK5BzBP+1E0vHdAAYSU/8vjr+zmy1qyDWYtIlqdMXzDkXTCZIkJE0WPp5Sx
+sEFqpLdO7ggqJV5UF7h/yIq9bcxoaNxYGJFRDWg3Lj6ZLCxiSN/QHuqascMijKAU
+2RHzQtXhfXzZA8PwZp92dalOZOguytEYyhzrPfJlysvOnXTigHa9WeybDcJgsOO8
+xFbgU9P98NgyE4ZIel6P1GpLnBY8j0KpbhWfCi5mmomwwK5OUWe0g9/u4z5FmN3z
+/WWA/8BjuJI9MFnS3YoU3safEUi0vD5pnuktVQGJOkSnR6gAN4dAoCcVEwARAQAB
+tDxXaWtpbWVkaWEgQXJjaGl2ZSBBdXRvbWF0aWMgU2lnbmluZyBLZXkgPHJvb3RA
+d2lraW1lZGlhLm9yZz6JAjcEEwEKACEFAlieKQECGwMFCwkIBwMFFQoJCAsFFgID
+AQACHgECF4AACgkQnTktP/rfGPtmohAAhQhGkOe0DX5ZSK6uEs2IpB1W3EvCPK87
+vSgI6FZyGEURH7s/MknQR2lMShgHZ1oqLwZV1DE5ao/D4khPw1JZFeYajXOxQ3Sr
+/Qjg6ZayrJ/r9sSchC4nd6m+rFpzuRsx22FWrVenzRBau6/J6OUz3WawsjX3Bwrg
+/Hw11/+kgsabkdsdK28ENWdVj0tLp5asSxrs4L4d2RjKpfiufBAGpkqFV5p2J8aw
+cJ7Wh3/1mMo+biTtuemHtWf4XfWmXv99ltSEVe8PpMYQ+FjNUHDzEZMS5K39em+V
+EOC7GsE+SHHF6lgjH2BtDcpkimbU9uROnGeIWtpRr+PNMu/yhFLUCxe7tf4yffIJ
+NIAL3RFcRhLj0xCbL3fd5DbQP4eB68k9pLHwYxpKba7DDmH0aMstvk5xrpphdHAP
+p03VbE6ZThKxA828yb2jWtumDhoYRlkHs2KrVufUPA411mUE+A53pwqBse/KM9Iq
+Pidc+TwMwy2nByRB9WFyyGS08wqhewpwp8mwIOejvRf+3yi+CGHM2Xi4bTmEqVL+
+Jm1D5W49O5/j1T6HyVPGsyH/0QgYwg2IVZwL5Jfy9w3974jTV3QAcCBafR9hcZCS
+S2wOIo0ckRI0APw6DQSFMyBrNs9rWy72Ixu9RC1GtzCCn5OaMRUM9FaqjRnTKX4m
+grWsU8i35fI=
+=ib96
+-END PGP PUBLIC KEY BLOCK-
diff --git a/modules/docker/manifests/baseimages.pp 
b/modules/docker/manifests/baseimages.pp
index 9925d09..26b7519 100644
--- a/modules/docker/manifests/baseimages.pp
+++ b/modules/docker/manifests/baseimages.pp
@@ -34,11 +34,10 @@
 }
 
 file { '/srv/images/base':
-ensure  => directory,
-owner   => 'root',
-group   => 'root',
-mode=> '0755',
-require => File['/srv/images'],
+ensure => directory,
+owner  => 'root',
+group  => 'root',
+mode   => '0755',
 }
 
 file { '/srv/images/base/jessie.yaml':
@@ -46,9 +45,32 @@
 owner   => 'root',
 group   => 'root',
 mode=> '0544',
-require => File['/srv/images/base'],
 }
 
+## Stretch
+file { '/srv/images/base/stretch.yaml':
+content => template('docker/images/stretch.yaml.erb'),
+owner   => 'root',
+group   => 'root',
+mode=> '0544',
+}
+file { '/srv/images/base/wikimedia-stretch.pub.gpg':
+ensure => present,
+source => 'puppet:///modules/docker/wikimedia-stretch.pub.gpg',
+owner  => 'root',
+group  => 'root',
+mode   => '0444',
+notify => Exec['apt-key add for wikimedia stretch'],
+}
+$stretch_keyring = '/etc/apt/trusted.gpg.d/wikimedia-stretch.gpg'
+exec { 'apt-key add for wikimedia stretch':
+cmd => "apt-key add --keyring ${stretch_keyring} 
/srv/images/base/wikimedia-stretch.pub.gpg",
+user=> 'root',
+group   => 'root',
+refreshonly => true,
+}
+## end stretch
+
 if 'alpine' in $distributions {
 if $proxy_address {
 $env = ["https_proxy=http://${proxy_address}:${proxy_port};]
diff --git a/modules/docker/templates/images/build-base-images.erb 
b/modules/docker/templates/images/build-base-images.erb
index 1c65f00..7d8539a 100755
--- a/modules/docker/templates/images/build-base-images.erb
+++ b/modules/docker/templates/images/build-base-images.erb
@@ -11,6 +11,15 @@
 docker push <%= @docker_registry %>/wikimedia-jessie
 <%- end -%>
 
+<%- if @distributions.include?('stretch') -%>
+# Build debian stretch image!
+/usr/bin/bootstrap-vz /srv/images/base/jessie.yaml
+
+# Push it to the repository
+docker push <%= @docker_registry %>/wikimedia-stretch
+<%- end -%>
+
+
 <%- if 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: scap::conftool: fix home directory

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378847 )

Change subject: scap::conftool: fix home directory
..


scap::conftool: fix home directory

In I2840237 the deploy-service home dir was changed, but no care was
taken to grep the sources for occurences of '/var/lib/scap'.

Next time someone feels it's important to fix a labs-only limitation
that should be done with more care.

Bug: T176184
Change-Id: Ic244ce84dad875a16e7b31512becad1d1b6b510f
---
M modules/scap/manifests/conftool.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/scap/manifests/conftool.pp 
b/modules/scap/manifests/conftool.pp
index 612364b..201e0a4 100644
--- a/modules/scap/manifests/conftool.pp
+++ b/modules/scap/manifests/conftool.pp
@@ -7,6 +7,6 @@
 include ::conftool::scripts
 
 ::conftool::credentials { 'deploy-service':
-home => '/var/lib/scap',
+home => '/var/lib/deploy-service',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/378847
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ic244ce84dad875a16e7b31512becad1d1b6b510f
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: scap::conftool: fix home directory

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378847 )

Change subject: scap::conftool: fix home directory
..

scap::conftool: fix home directory

In I2840237 the deploy-service home dir was changed, but no care was
taken to grep the sources for occurences of '/var/lib/scap'.

Next time someone feels it's important to fix a labs-only limitation
that should be done with more care.

Change-Id: Ic244ce84dad875a16e7b31512becad1d1b6b510f
---
M modules/scap/manifests/conftool.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/47/378847/1

diff --git a/modules/scap/manifests/conftool.pp 
b/modules/scap/manifests/conftool.pp
index 612364b..201e0a4 100644
--- a/modules/scap/manifests/conftool.pp
+++ b/modules/scap/manifests/conftool.pp
@@ -7,6 +7,6 @@
 include ::conftool::scripts
 
 ::conftool::credentials { 'deploy-service':
-home => '/var/lib/scap',
+home => '/var/lib/deploy-service',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/378847
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic244ce84dad875a16e7b31512becad1d1b6b510f
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Fix container references

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378714 )

Change subject: Fix container references
..

Fix container references

Change-Id: If6d8c74f4163ecb2dc59527d6e78de51c2da4631
---
M images/fluent-bit/Dockerfile.template
M images/nodejs/devel/Dockerfile.template
2 files changed, 2 insertions(+), 2 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/14/378714/1

diff --git a/images/fluent-bit/Dockerfile.template 
b/images/fluent-bit/Dockerfile.template
index 236dd41..127e578 100644
--- a/images/fluent-bit/Dockerfile.template
+++ b/images/fluent-bit/Dockerfile.template
@@ -1,4 +1,4 @@
-FROM {{ registry }}/{{ seed_image }}:latest
+FROM {{ registry }}/{{ seed_image }}
 LABEL Description="Fluent-bit image to run as a sidecar container" \
   maintainer="j...@wikimedia.org"
 
diff --git a/images/nodejs/devel/Dockerfile.template 
b/images/nodejs/devel/Dockerfile.template
index 8bd9da9..3f2d0db 100644
--- a/images/nodejs/devel/Dockerfile.template
+++ b/images/nodejs/devel/Dockerfile.template
@@ -1,5 +1,5 @@
 # TODO: allow to define a "latest" token that fetches the latest version of 
the parent
 # from this repository
-FROM {{ registry }}/{{ "nodejs-slim" | image_tag }}
+FROM {{ registry }}/{{ "nodejs" | image_tag }}
 
 {{ "npm" | apt_install}}

-- 
To view, visit https://gerrit.wikimedia.org/r/378714
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If6d8c74f4163ecb2dc59527d6e78de51c2da4631
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Add fluent-bit image

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378260 )

Change subject: Add fluent-bit image
..


Add fluent-bit image

Bug: T175527
Change-Id: Ie43b9ca615a335e8cc25dc17eaffa48728780006
---
A images/fluent-bit/Dockerfile.template
A images/fluent-bit/changelog
A images/fluent-bit/fluent-bit.conf
3 files changed, 35 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/images/fluent-bit/Dockerfile.template 
b/images/fluent-bit/Dockerfile.template
new file mode 100644
index 000..236dd41
--- /dev/null
+++ b/images/fluent-bit/Dockerfile.template
@@ -0,0 +1,11 @@
+FROM {{ registry }}/{{ seed_image }}:latest
+LABEL Description="Fluent-bit image to run as a sidecar container" \
+  maintainer="j...@wikimedia.org"
+
+{{ "td-agent-bit" | apt_install }}
+
+
+COPY fluent-bit.conf /etc/td-agent-bit/td-agent-bit.conf
+
+# Entry point
+CMD ["/opt/td-agent-bit/bin/td-agent-bit", "-c", 
"/etc/td-agent-bit/td-agent-bit.conf"]
diff --git a/images/fluent-bit/changelog b/images/fluent-bit/changelog
new file mode 100644
index 000..a978ec1
--- /dev/null
+++ b/images/fluent-bit/changelog
@@ -0,0 +1,5 @@
+fluent-bit (0.12.2-1) wikimedia; urgency=medium
+
+  * Initial release.
+
+ -- Giuseppe Lavagetto   Fri, 15 Sep 2017 18:05:41 
+0200
diff --git a/images/fluent-bit/fluent-bit.conf 
b/images/fluent-bit/fluent-bit.conf
new file mode 100644
index 000..aa963d2
--- /dev/null
+++ b/images/fluent-bit/fluent-bit.conf
@@ -0,0 +1,19 @@
+[SERVICE]
+Flush1
+Daemon   Off
+Log_Levelinfo
+Log_File /var/log/fluent-bit.log
+Parsers_File parsers.conf
+
+[INPUT]
+Name tcp
+Listen 127.0.0.1
+Port 24224
+Chunk_Size 32
+Buffer_Size 64
+
+[OUTPUT]
+Name  forward
+Match ${FLUENTBIT_OUTPUT_MATCH}
+Host  ${K8S_NODE_IP}
+Port  24244

-- 
To view, visit https://gerrit.wikimedia.org/r/378260
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ie43b9ca615a335e8cc25dc17eaffa48728780006
Gerrit-PatchSet: 2
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Improvements to the build script

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/378259 )

Change subject: Improvements to the build script
..


Improvements to the build script

* Add --no-install-recommends to the apt command
* Use stretch as a base now
* Do not add a newline at the end of the apt command.

Change-Id: I551f87a695285fccc105fab284d826edb5eba355
---
M build
1 file changed, 3 insertions(+), 4 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/build b/build
index c9d2af8..a7b18c4 100755
--- a/build
+++ b/build
@@ -26,9 +26,8 @@
 return """
 RUN apt-get update && \
 DEBIAN_FRONTEND=noninteractive \
-apt-get install --yes {packages} \
-&& apt-get clean && rm -rf /var/lib/apt/lists/*
-""".format(packages=pkgs)
+apt-get install --yes {packages} --no-install-recommends \
+&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(packages=pkgs)
 
 
 class DockerImage(object):
@@ -64,7 +63,7 @@
 self.config = {
 'registry': 'docker-registry.wikimedia.org',
 'username': None, 'password': None,
-'seed_image': 'wikimedia-jessie'
+'seed_image': 'wikimedia-stretch'
 }
 self.config.update(self._read_config(configfile))
 self.client = docker.from_env(version='auto')

-- 
To view, visit https://gerrit.wikimedia.org/r/378259
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I551f87a695285fccc105fab284d826edb5eba355
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Improvements to the build script

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378259 )

Change subject: Improvements to the build script
..

Improvements to the build script

* Add --no-install-recommends to the apt command
* Use stretch as a base now
* Do not add a newline at the end of the apt command.

Change-Id: I551f87a695285fccc105fab284d826edb5eba355
---
M build
1 file changed, 3 insertions(+), 4 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/59/378259/1

diff --git a/build b/build
index c9d2af8..a7b18c4 100755
--- a/build
+++ b/build
@@ -26,9 +26,8 @@
 return """
 RUN apt-get update && \
 DEBIAN_FRONTEND=noninteractive \
-apt-get install --yes {packages} \
-&& apt-get clean && rm -rf /var/lib/apt/lists/*
-""".format(packages=pkgs)
+apt-get install --yes {packages} --no-install-recommends \
+&& apt-get clean && rm -rf /var/lib/apt/lists/* """.format(packages=pkgs)
 
 
 class DockerImage(object):
@@ -64,7 +63,7 @@
 self.config = {
 'registry': 'docker-registry.wikimedia.org',
 'username': None, 'password': None,
-'seed_image': 'wikimedia-jessie'
+'seed_image': 'wikimedia-stretch'
 }
 self.config.update(self._read_config(configfile))
 self.client = docker.from_env(version='auto')

-- 
To view, visit https://gerrit.wikimedia.org/r/378259
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I551f87a695285fccc105fab284d826edb5eba355
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations...production-images[master]: Add fluent-bit image

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378260 )

Change subject: Add fluent-bit image
..

Add fluent-bit image

Bug: T175527
Change-Id: Ie43b9ca615a335e8cc25dc17eaffa48728780006
---
A images/fluent-bit/Dockerfile.template
A images/fluent-bit/changelog
2 files changed, 19 insertions(+), 0 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/operations/docker-images/production-images 
refs/changes/60/378260/1

diff --git a/images/fluent-bit/Dockerfile.template 
b/images/fluent-bit/Dockerfile.template
new file mode 100644
index 000..7a7f030
--- /dev/null
+++ b/images/fluent-bit/Dockerfile.template
@@ -0,0 +1,14 @@
+FROM {{ registry }}/{{ seed_image }}:latest
+LABEL Description="Fluent-bit image to run as a sidecar container" \
+  maintainer="j...@wikimedia.org"
+
+{{ "td-agent-bit" | apt_install }} \
+&& mkdir -p /etc/fluent-bit/
+
+COPY fluent-bit.conf /etc/fluent-bit/
+COPY parsers.conf   /etc/fluent-bit/
+COPY parsers-wmf.conf /etc/fluent-bit/
+
+
+# Entry point
+CMD ["/opt/td-agent-bit/bin/td-agent-bit", "-c", 
"/etc/fluent-bit/fluent-bit.conf"]
diff --git a/images/fluent-bit/changelog b/images/fluent-bit/changelog
new file mode 100644
index 000..ee15d64
--- /dev/null
+++ b/images/fluent-bit/changelog
@@ -0,0 +1,5 @@
+fluent-bit (0.12.2) wikimedia; urgency=medium
+
+  * Initial release.
+
+ -- Giuseppe Lavagetto   Fri, 15 Sep 2017 18:05:41 
+0200

-- 
To view, visit https://gerrit.wikimedia.org/r/378260
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie43b9ca615a335e8cc25dc17eaffa48728780006
Gerrit-PatchSet: 1
Gerrit-Project: operations/docker-images/production-images
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: admin: add a new ed25519 key for myself

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378204 )

Change subject: admin: add a new ed25519 key for myself
..

admin: add a new ed25519 key for myself

Will remove the old one in a subsequent change

Change-Id: I027c73e416701b0264466553ec3cb31e099c0f4e
---
M modules/admin/data/data.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/04/378204/1

diff --git a/modules/admin/data/data.yaml b/modules/admin/data/data.yaml
index 0aa0e08..e94a3aa 100644
--- a/modules/admin/data/data.yaml
+++ b/modules/admin/data/data.yaml
@@ -1459,7 +1459,7 @@
 realname: Giuseppe Lavagetto
 ssh_keys:
   - ssh-rsa 
B3NzaC1yc2EDAQABAAACAQCf8yZc/+/LiiIXkweNACUDMYNezKpexv7W8d4xlUweCcnZeyVvdiVl/LoFySvf7G08shDXQrxhqnc9xA6ulGokfHLOjAoqEuwlVwA1Xjbyb7KSOA2PNLDG9kzQ36wQyaSIgSDZegA+3O1oi3Jp/oVfSaqN1zk+8/Jl1kUAsv5S2264lsQR1y34AHhL4txkn6lH4nxjBBl+nvspU/FPInsLrCprCvecR7OT5+kUJF4rEeiGqx1gu7Etd/4AQzb+csYXp3LKUYhW+6kxEhUbGhFRG+GYtwD5oYFwSR7CmqrYvthfaMmnG9JzvIJvGmEv9gIFAivNsodlJzR0ZZ1ya24UvEwpnHmgcPBgLI1tPGJT//p8P7Xd04KPHgbN1VG21s89gjNCvEErwR8lkTveFRBZ8I0D/ipVFP4fHcA+jx6Lit/krIcexZ3CN0Bfpr9rEiFOGMx081HvD6R3x2ZablNb3GA/lscUs3r18QdQDGazj2+4PkrgbCAl0pt52Pun9/uajQpfHEgkaO6CHMrdX9FR169D86ZDf3W7lV0SPjITDeHHFBFzzrVrju4xt01ybYsUiWaS+PZAeQEDgZ1156sH8RSm7/Oi18U++S/upN7csJfDwSwOss9ReJNn+tXMr+uEjnuIL+Z2v0oCfujAcJIauX5UpEt5oXlhK+8BClRzXQ==
 j...@wikimedia.org
-  - ssh-rsa 
B3NzaC1yc2EDAQABAAABAQC8qjtkdl7nDP0lD0oQQNcGCMelTFu9dlx+TEu+mo0p89wwBdtWuXEv7T0837aUYQ8FHyHUXnrgeCX43a8LnI0lM2tD8/5VT+aFqGDwETjLgUNhlK1JQp7ZDC4fS+mNczfT7ytJmTt/cmxjyPIQh7E/wdaMwF8ogvOsS9iy//wFF1fcS6rYSy5DYLF7s21YNjnqPuVtv3pqNseqrSkChtcGxEkrB2+/t8Nf01evutdsk0kUvUM24bdCtOlvGaW/+qIbr+YbR6omgAblmo9opuMVspn1B6/1NEZ/jK+zuQ008XEvJQCsm9ahlmD10MVbykktc1jBIS//N7Zs9RWQKcEN
 glavage...@wikimedia.org
+  - ssh-ed25519 
C3NzaC1lZDI1NTE5ILIByu4Mym+ToDBSnH9iKSJVrTcUYLLENBFt/oXTgzNA 
j...@wikimedia.org
 uid: 4816
 email: glavage...@wikimedia.org
 # T109521

-- 
To view, visit https://gerrit.wikimedia.org/r/378204
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I027c73e416701b0264466553ec3cb31e099c0f4e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: scap: do scope lookups in mw-deployment-vars.erb

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377689 )

Change subject: scap: do scope lookups in mw-deployment-vars.erb
..


scap: do scope lookups in mw-deployment-vars.erb

mw-deployment-vars.erb gets included from scap::scripts, but refers to
variables like $common_path, $rsync_host etc. from scap::master, without
doing an out-of-scope lookup. This is unsupported in the future parser.
Convert to scope.lookupvar()s for now as a quick fix while we think
about a longer-term fix (maybe fold scap::scripts into scap::master?)

Change-Id: I254e363b38359a202d5235a9b4c9bb8c5887d250
---
M modules/scap/templates/mw-deployment-vars.erb
1 file changed, 5 insertions(+), 5 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/scap/templates/mw-deployment-vars.erb 
b/modules/scap/templates/mw-deployment-vars.erb
index abd8da3..eb464e3 100644
--- a/modules/scap/templates/mw-deployment-vars.erb
+++ b/modules/scap/templates/mw-deployment-vars.erb
@@ -1,14 +1,14 @@
 # This file should be BASH / Python polyglot.
 
-MEDIAWIKI_DEPLOYMENT_DIR="<%= @common_path %>"
-MEDIAWIKI_STAGING_DIR="<%= @common_source_path %>"
+MEDIAWIKI_DEPLOYMENT_DIR="<%= scope.lookupvar('scap::master::common_path') %>"
+MEDIAWIKI_STAGING_DIR="<%= scope.lookupvar('scap::master::common_source_path') 
%>"
 
 # Back-compat aliases
 MW_COMMON="$MEDIAWIKI_DEPLOYMENT_DIR"
 MW_COMMON_SOURCE="$MEDIAWIKI_STAGING_DIR"
 
-MW_RSYNC_HOST="<%= @rsync_host %>"
+MW_RSYNC_HOST="<%= scope.lookupvar('scap::master::rsync_host') %>"
 MW_DSH_ARGS=('-cM' '-g' 'mediawiki-installation' '-o' '-oSetupTimeout=30' 
'-F30')
 MW_RSYNC_ARGS=('-a' '--delete-delay' '--delay-updates' '--compress' '--delete' 
'--exclude=**/.svn/lock' '--exclude=**/.git/objects' 
'--exclude=**/.git/**/objects' '--exclude=**/cache/l10n/*.cdb' '--no-perms')
-MW_STATSD_HOST="<%= @statsd_host %>"
-MW_STATSD_PORT=<%= @statsd_port %>
+MW_STATSD_HOST="<%= scope.lookupvar('scap::master::statsd_host') %>"
+MW_STATSD_PORT=<%= scope.lookupvar('scap::master::statsd_port') %>

-- 
To view, visit https://gerrit.wikimedia.org/r/377689
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I254e363b38359a202d5235a9b4c9bb8c5887d250
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Thcipriani 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: hhvm: use '', not undef for light_process_file_prefix

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377688 )

Change subject: hhvm: use '', not undef for light_process_file_prefix
..


hhvm: use '', not undef for light_process_file_prefix

Otherwise it is interpolated as
  hhvm.server.light_process_file_prefix = undef
with the future parser on the deployment servers.

Change-Id: I985129e3c959dfde4b8b449141ba8c7eaf833bdf
---
M hieradata/role/common/deployment_server.yaml
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/role/common/deployment_server.yaml 
b/hieradata/role/common/deployment_server.yaml
index 72ed5e0..ee5dfa7 100644
--- a/hieradata/role/common/deployment_server.yaml
+++ b/hieradata/role/common/deployment_server.yaml
@@ -27,12 +27,12 @@
   hhvm:
 server:
   light_process_count: 0
-  light_process_file_prefix:
+  light_process_file_prefix: ''
 hhvm::extra::cli:
   hhvm:
 server:
   light_process_count: 0
-  light_process_file_prefix:
+  light_process_file_prefix: ''
 # Default scap::server configuration.  This is used in production.
 # If you are setting up scap::server in labs, these will be used
 # unless you override them for your labs project.

-- 
To view, visit https://gerrit.wikimedia.org/r/377688
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I985129e3c959dfde4b8b449141ba8c7eaf833bdf
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: thumbor: fix weird integer interpolation

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377513 )

Change subject: thumbor: fix weird integer interpolation
..


thumbor: fix weird integer interpolation

This addresses a an issue where we used:
  range("${listen_port + 1}", ...)
instead of the more obvious:
  range($listen_port, ...)

There is an XXX in the code which suggests that this was done to avoid
an odd parsing error that happened at the time. Upon further
investigation, it looks like this was an issue with stdlib that was
later fixed (5c04d25), where:
  if start.match(/^\d+$/)
was replaced by:
  if start.to_s.match(/^\d+$/)

The current form causes a future parser error right now, which will
hopefully will be addressed by the more obvious/cleaner form.

Change-Id: I1aeba7fb961e6f838d75878a02eb546f66eb8908
---
M modules/thumbor/manifests/init.pp
1 file changed, 3 insertions(+), 4 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified
  Filippo Giunchedi: Looks good to me, but someone else must approve



diff --git a/modules/thumbor/manifests/init.pp 
b/modules/thumbor/manifests/init.pp
index f2747a9..5625db6 100644
--- a/modules/thumbor/manifests/init.pp
+++ b/modules/thumbor/manifests/init.pp
@@ -103,10 +103,9 @@
 before => Systemd::Unit['thumbor@'],
 }
 
-# XXX using a literal integer as the first argument results in
-# Error 400 on SERVER: undefined method `match' for 8801:Fixnum at
-# /etc/puppet/modules/thumbor/manifests/init.pp:62
-$ports = range("${listen_port + 1}", $listen_port + $instance_count)
+# use range(), which returns an array of integers, then interpolate it into
+# an array of strings, to use it as a parameter to thumbor::instance below
+$ports = prefix(range($listen_port + 1, $listen_port + $instance_count), 
'')
 
 nginx::site { 'thumbor':
 content => template('thumbor/nginx.conf.erb'),

-- 
To view, visit https://gerrit.wikimedia.org/r/377513
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1aeba7fb961e6f838d75878a02eb546f66eb8908
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Faidon Liambotis 
Gerrit-Reviewer: Filippo Giunchedi 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: toollabs: fix k8s classes that just include k8s::proxy

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377505 )

Change subject: toollabs: fix k8s classes that just include k8s::proxy
..


toollabs: fix k8s classes that just include k8s::proxy

Change-Id: If2ab6669c8941840dcf23600df4196349af71219
---
M modules/role/manifests/toollabs/k8s/bastion.pp
M modules/role/manifests/toollabs/k8s/webproxy.pp
2 files changed, 9 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/toollabs/k8s/bastion.pp 
b/modules/role/manifests/toollabs/k8s/bastion.pp
index ec922e2..9e3b511 100644
--- a/modules/role/manifests/toollabs/k8s/bastion.pp
+++ b/modules/role/manifests/toollabs/k8s/bastion.pp
@@ -13,6 +13,11 @@
 etcd_endpoints => $etcd_url,
 }
 
+
+class { '::k8s::infrastructure_config':
+master_host => $master_host,
+}
+
 class { '::k8s::proxy':
 master_host => $master_host,
 }
diff --git a/modules/role/manifests/toollabs/k8s/webproxy.pp 
b/modules/role/manifests/toollabs/k8s/webproxy.pp
index 28d973a..1f7256e 100644
--- a/modules/role/manifests/toollabs/k8s/webproxy.pp
+++ b/modules/role/manifests/toollabs/k8s/webproxy.pp
@@ -17,6 +17,10 @@
 master_host => $master_host,
 }
 
+class { '::k8s::infrastructure_config':
+master_host => $master_host,
+}
+
 class { '::k8s::proxy':
 master_host => $master_host,
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377505
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If2ab6669c8941840dcf23600df4196349af71219
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Rush 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: toollabs: fix k8s classes that just include k8s::proxy

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377505 )

Change subject: toollabs: fix k8s classes that just include k8s::proxy
..

toollabs: fix k8s classes that just include k8s::proxy

Change-Id: If2ab6669c8941840dcf23600df4196349af71219
---
M modules/role/manifests/toollabs/k8s/bastion.pp
M modules/role/manifests/toollabs/k8s/webproxy.pp
2 files changed, 9 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/05/377505/1

diff --git a/modules/role/manifests/toollabs/k8s/bastion.pp 
b/modules/role/manifests/toollabs/k8s/bastion.pp
index ec922e2..30e84db 100644
--- a/modules/role/manifests/toollabs/k8s/bastion.pp
+++ b/modules/role/manifests/toollabs/k8s/bastion.pp
@@ -13,6 +13,11 @@
 etcd_endpoints => $etcd_url,
 }
 
+
+class { 'k8s::infrastructure_config':
+master_host => $master_host,
+}
+
 class { '::k8s::proxy':
 master_host => $master_host,
 }
diff --git a/modules/role/manifests/toollabs/k8s/webproxy.pp 
b/modules/role/manifests/toollabs/k8s/webproxy.pp
index 28d973a..edd5daa 100644
--- a/modules/role/manifests/toollabs/k8s/webproxy.pp
+++ b/modules/role/manifests/toollabs/k8s/webproxy.pp
@@ -17,6 +17,10 @@
 master_host => $master_host,
 }
 
+class { 'k8s::infrastructure_config':
+master_host => $master_host,
+}
+
 class { '::k8s::proxy':
 master_host => $master_host,
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377505
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If2ab6669c8941840dcf23600df4196349af71219
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: role::snapshot::common: properly scope included classes

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377493 )

Change subject: role::snapshot::common: properly scope included classes
..

role::snapshot::common: properly scope included classes

While the old parser would search classes within the current namespace
if their name is not absolute, the future parser does not.

So, fix the unfortunate clash of namespaces we had here.

Bug: T171704
Change-Id: If41ff38295121de3b09cf97abda1edfd655825e6
---
M modules/role/manifests/snapshot/common.pp
1 file changed, 2 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/93/377493/1

diff --git a/modules/role/manifests/snapshot/common.pp 
b/modules/role/manifests/snapshot/common.pp
index a038d1a..7ee5723 100644
--- a/modules/role/manifests/snapshot/common.pp
+++ b/modules/role/manifests/snapshot/common.pp
@@ -6,9 +6,8 @@
 # mw packages and dependencies, dataset server nfs mount,
 # config files, stages files, dblists, html templates
 include ::role::mediawiki::common
-include snapshot::dumps
+include ::snapshot::dumps
 
 # scap3 deployment of dump scripts
-include snapshot::deployment
+include ::role::snapshot::deployment
 }
-

-- 
To view, visit https://gerrit.wikimedia.org/r/377493
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If41ff38295121de3b09cf97abda1edfd655825e6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: k8s: fix template scoping

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377459 )

Change subject: k8s: fix template scoping
..


k8s: fix template scoping

Since the future parser has different scoping rules, do as follows:
* Add a master_host parameter to k8s::infrastructure
* Explicitly declare k8s::infrastructure_config in k8s::node
* Change the "include" in a "require" for the config class in k8s::proxy
  and k8s::kubelet as it's the proper logical relationship

Bug: T171704
Change-Id: I7a59f3337d3f1d5c96eafbe3240a1544ef2366d8
---
M modules/k8s/manifests/infrastructure_config.pp
M modules/k8s/manifests/kubelet.pp
M modules/k8s/manifests/proxy.pp
M modules/profile/manifests/kubernetes/node.pp
4 files changed, 8 insertions(+), 3 deletions(-)

Approvals:
  Faidon Liambotis: Looks good to me, but someone else must approve
  Giuseppe Lavagetto: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/k8s/manifests/infrastructure_config.pp 
b/modules/k8s/manifests/infrastructure_config.pp
index 61c2c5c..3953c93 100644
--- a/modules/k8s/manifests/infrastructure_config.pp
+++ b/modules/k8s/manifests/infrastructure_config.pp
@@ -1,4 +1,4 @@
-class k8s::infrastructure_config {
+class k8s::infrastructure_config($master_host) {
 file { '/etc/kubernetes':
 ensure => directory,
 owner  => 'root',
diff --git a/modules/k8s/manifests/kubelet.pp b/modules/k8s/manifests/kubelet.pp
index 885c894..68b6d36 100644
--- a/modules/k8s/manifests/kubelet.pp
+++ b/modules/k8s/manifests/kubelet.pp
@@ -11,7 +11,7 @@
 $cni_bin_dir='/opt/cni/bin',
 $cni_conf_dir='/etc/cni/net.d',
 ) {
-include ::k8s::infrastructure_config
+require ::k8s::infrastructure_config
 
 require_package('kubernetes-node')
 
diff --git a/modules/k8s/manifests/proxy.pp b/modules/k8s/manifests/proxy.pp
index e7ad8fa..c313c76 100644
--- a/modules/k8s/manifests/proxy.pp
+++ b/modules/k8s/manifests/proxy.pp
@@ -3,7 +3,7 @@
 $proxy_mode = 'iptables',
 $masquerade_all = true,
 ) {
-include ::k8s::infrastructure_config
+require ::k8s::infrastructure_config
 
 $master_ip = ipresolve($master_host, 4, $::nameservers[0])
 
diff --git a/modules/profile/manifests/kubernetes/node.pp 
b/modules/profile/manifests/kubernetes/node.pp
index 881208c..90642c3 100644
--- a/modules/profile/manifests/kubernetes/node.pp
+++ b/modules/profile/manifests/kubernetes/node.pp
@@ -10,6 +10,11 @@
 user=> 'root',
 group   => 'root',
 }
+
+class { '::k8s::infrastructure_config':
+master_host => $master_fqdn,
+}
+
 class { '::k8s::kubelet':
 master_host   => $master_fqdn,
 listen_address=> '0.0.0.0',

-- 
To view, visit https://gerrit.wikimedia.org/r/377459
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7a59f3337d3f1d5c96eafbe3240a1544ef2366d8
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Faidon Liambotis 
Gerrit-Reviewer: Gehel 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: nagios_common: use the template if empty($content)

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377445 )

Change subject: nagios_common: use the template if empty($content)
..


nagios_common: use the template if empty($content)

We currently use a stock template if $content is undef, but $content is
set to secret('nagios/contacts.cfg') in the role class, which may return
'' in a test environment and seems to vary between current and future
parser. Change the if to vary on $content being empty() as well.

Change-Id: I5fc26b00f2314bc27fb2d1ba7110ba1040439ef1
---
M modules/monitoring/.fixtures.yml
M modules/nagios_common/.fixtures.yml
M modules/nagios_common/manifests/contacts.pp
3 files changed, 3 insertions(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/monitoring/.fixtures.yml b/modules/monitoring/.fixtures.yml
index dc4f9cc..b2567a1 100644
--- a/modules/monitoring/.fixtures.yml
+++ b/modules/monitoring/.fixtures.yml
@@ -6,3 +6,4 @@
 wmflib: "../../../../wmflib"
 logrotate: "../../../../logrotate"
 elasticsearch: "../../../../elasticsearch"
+stdlib: "../../../../stdlib"
diff --git a/modules/nagios_common/.fixtures.yml 
b/modules/nagios_common/.fixtures.yml
index 1cde35d..b8a4937 100644
--- a/modules/nagios_common/.fixtures.yml
+++ b/modules/nagios_common/.fixtures.yml
@@ -1,3 +1,4 @@
 fixtures:
 symlinks:
 nagios_common: "#{source_dir}"
+stdlib: "../../../../stdlib"
diff --git a/modules/nagios_common/manifests/contacts.pp 
b/modules/nagios_common/manifests/contacts.pp
index 7977b7e..a767645 100644
--- a/modules/nagios_common/manifests/contacts.pp
+++ b/modules/nagios_common/manifests/contacts.pp
@@ -47,7 +47,7 @@
 show_diff => false,
 }
 } else {
-if ($content == undef) {
+if ($content == undef or empty($content)) {
 $real_content = template('nagios_common/contacts.cfg.erb')
 } else {
 $real_content = $content

-- 
To view, visit https://gerrit.wikimedia.org/r/377445
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5fc26b00f2314bc27fb2d1ba7110ba1040439ef1
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: grafana: quote 'type' as the class' parameter

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377444 )

Change subject: grafana: quote 'type' as the class' parameter
..


grafana: quote 'type' as the class' parameter

As type is a reserved word in the future parser and can't appear as a
bareword.

Change-Id: Ifbb9a3b91222e00dca8fabe7208b12585ce64564
---
M modules/role/manifests/grafana/base.pp
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/grafana/base.pp 
b/modules/role/manifests/grafana/base.pp
index 8df412a..ba5c255 100644
--- a/modules/role/manifests/grafana/base.pp
+++ b/modules/role/manifests/grafana/base.pp
@@ -40,8 +40,8 @@
 # Grafana needs a database to store users and dashboards.
 # sqlite3 is the default, and it's perfectly adequate.
 'database'   => {
-type => 'sqlite3',
-path => 'grafana.db',
+'type' => 'sqlite3',
+'path' => 'grafana.db',
 },
 
 'security'   => {

-- 
To view, visit https://gerrit.wikimedia.org/r/377444
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ifbb9a3b91222e00dca8fabe7208b12585ce64564
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: openstack: move $ssl_settings near the template

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377442 )

Change subject: openstack: move $ssl_settings near the template
..


openstack: move $ssl_settings near the template

As passing it down from role::labs::openstack::nova::manager to
openstack::openstack_manager without scoping it is not supported in the
future parser.

Change-Id: I1b96beef7b18bad3ff2d008e83cf307cf9ebecec
---
M modules/openstack/manifests/openstack_manager.pp
M modules/role/manifests/labs/openstack/nova/manager.pp
2 files changed, 1 insertion(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/openstack/manifests/openstack_manager.pp 
b/modules/openstack/manifests/openstack_manager.pp
index 247993c..3e026c7 100644
--- a/modules/openstack/manifests/openstack_manager.pp
+++ b/modules/openstack/manifests/openstack_manager.pp
@@ -59,6 +59,7 @@
 'wikitech.wikimedia.org'=> '208.80.154.136',
 'labtestwikitech.wikimedia.org' => '208.80.153.14'
 }
+$ssl_settings = ssl_ciphersuite('apache', 'compat', true)
 apache::site { $webserver_hostname:
 content => template('openstack/common/wikitech.wikimedia.org.erb'),
 }
diff --git a/modules/role/manifests/labs/openstack/nova/manager.pp 
b/modules/role/manifests/labs/openstack/nova/manager.pp
index 3b4e898..137d998 100644
--- a/modules/role/manifests/labs/openstack/nova/manager.pp
+++ b/modules/role/manifests/labs/openstack/nova/manager.pp
@@ -27,8 +27,6 @@
 check_command => "check_ssl_http_letsencrypt!${sitename}",
 }
 
-$ssl_settings = ssl_ciphersuite('apache', 'compat', true)
-
 ferm::service { 'wikitech_http':
 proto => 'tcp',
 port  => '80',

-- 
To view, visit https://gerrit.wikimedia.org/r/377442
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1b96beef7b18bad3ff2d008e83cf307cf9ebecec
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Alex Monk 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: ganglia: fix class dependencies

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377441 )

Change subject: ganglia: fix class dependencies
..


ganglia: fix class dependencies

Depend on Class['::ganglia::web'] instead of Class['ganglia::web']
from role::ganglia::web, as in the current parser Class['ganglia::web']
is ambiguous and in this case, refers to role::ganglia::web.

Change-Id: I47e7727bdb05c3dfbfbc6f334f255456885c7cc5
---
M modules/role/manifests/ganglia/web.pp
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/role/manifests/ganglia/web.pp 
b/modules/role/manifests/ganglia/web.pp
index dd49adc..76db02e 100644
--- a/modules/role/manifests/ganglia/web.pp
+++ b/modules/role/manifests/ganglia/web.pp
@@ -97,6 +97,6 @@
 backup::set { 'var-lib-ganglia': }
 backup::set { 'srv-ganglia': }
 
-Class['ganglia::gmetad::rrdcached'] -> Class['ganglia::gmetad']
-Class['ganglia::gmetad'] -> Class['ganglia::web']
+Class['ganglia::gmetad::rrdcached'] -> Class['::ganglia::gmetad']
+Class['ganglia::gmetad'] -> Class['::ganglia::web']
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377441
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I47e7727bdb05c3dfbfbc6f334f255456885c7cc5
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Use String as redis::instance's $name (noop)

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377440 )

Change subject: Use String as redis::instance's $name (noop)
..


Use String as redis::instance's $name (noop)

Replace instances of:
  redis::instance { 6379:
with:
  redis::instance { '6379':
across the tree.

The former is odd syntax even with the current parser and actually
rejected in the future parser.

Also do the same with hieradata, for profile::redis::master::instances.

Change-Id: Id70162e5b56f102bcba38c3618e7a50719401d82
---
M modules/deployment/manifests/redis.pp
M modules/role/manifests/rcstream.pp
M modules/role/manifests/xenon.pp
M modules/sentry/manifests/init.pp
M modules/toollabs/manifests/redis.pp
5 files changed, 6 insertions(+), 6 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/deployment/manifests/redis.pp 
b/modules/deployment/manifests/redis.pp
index 28528ea..9b1ccbc 100644
--- a/modules/deployment/manifests/redis.pp
+++ b/modules/deployment/manifests/redis.pp
@@ -12,7 +12,7 @@
 if ($::fqdn != $deployment_server) {
 $deployment_ipv4 = ipresolve($deployment_server, 4)
 # Just a read-only slave for now
-redis::instance { 6379:
+redis::instance { '6379':
 settings => {
 daemonize   => $daemonize_redis,
 slave_read_only => true,
@@ -21,7 +21,7 @@
 },
 }
 } else {
-redis::instance{ 6379:
+redis::instance { '6379':
 settings => {
 daemonize => $daemonize_redis,
 bind  => '0.0.0.0',
diff --git a/modules/role/manifests/rcstream.pp 
b/modules/role/manifests/rcstream.pp
index 663e7e1..6c9a4fa 100644
--- a/modules/role/manifests/rcstream.pp
+++ b/modules/role/manifests/rcstream.pp
@@ -12,7 +12,7 @@
 description => 'MediaWiki Recent Changes stream',
 }
 
-redis::instance { 6379:
+redis::instance { '6379':
 settings => {
 maxmemory   => '100mb',
 maxmemory_policy=> 'volatile-lru',
diff --git a/modules/role/manifests/xenon.pp b/modules/role/manifests/xenon.pp
index dcdac19..f9e6d4c 100644
--- a/modules/role/manifests/xenon.pp
+++ b/modules/role/manifests/xenon.pp
@@ -10,7 +10,7 @@
 include ::apache::mod::proxy
 include ::apache::mod::proxy_http
 
-redis::instance { 6379:
+redis::instance { '6379':
 settings => {
 maxmemory   => '1Mb',
 stop_writes_on_bgsave_error => 'no',
diff --git a/modules/sentry/manifests/init.pp b/modules/sentry/manifests/init.pp
index dd46947..8f27816 100644
--- a/modules/sentry/manifests/init.pp
+++ b/modules/sentry/manifests/init.pp
@@ -37,7 +37,7 @@
 
 require sentry::packages
 
-redis::instance { 6379: }
+redis::instance { '6379': }
 
 git::clone { 'operations/software/sentry':
 ensure=> latest,
diff --git a/modules/toollabs/manifests/redis.pp 
b/modules/toollabs/manifests/redis.pp
index 480205a..d1bb994 100644
--- a/modules/toollabs/manifests/redis.pp
+++ b/modules/toollabs/manifests/redis.pp
@@ -27,7 +27,7 @@
 $slaveof = undef
 }
 
-redis::instance { 6379:
+redis::instance { '6379':
 settings => {
 client_output_buffer_limit  => 'slave 512mb 200mb 60',
 dbfilename  => "${::hostname}-6379.rdb",

-- 
To view, visit https://gerrit.wikimedia.org/r/377440
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id70162e5b56f102bcba38c3618e7a50719401d82
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Coren 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Merlijn van Deen 
Gerrit-Reviewer: Yuvipanda 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: statsd_proxy: use validate_numeric()

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377439 )

Change subject: statsd_proxy: use validate_numeric()
..


statsd_proxy: use validate_numeric()

Use validate_numeric() to validate the $server_port parameter as a
numeric and the $backend_ports parameter as an array of numerics,
instead of trying to validate them with regular expressions. Validating
Integers against REs breaks with the future parser and this is easier to
read anyway.

Change-Id: I9cf253bf84231767d064b39a306690f79f2f6c4b
---
M modules/statsd_proxy/manifests/init.pp
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/statsd_proxy/manifests/init.pp 
b/modules/statsd_proxy/manifests/init.pp
index 0a2f98e..793c0ac 100644
--- a/modules/statsd_proxy/manifests/init.pp
+++ b/modules/statsd_proxy/manifests/init.pp
@@ -33,8 +33,8 @@
 ) {
 validate_ensure($ensure)
 validate_array($backend_ports)
-validate_re(join($backend_ports, ' '), '^\d+( \d+)*$', '$backend_ports 
must be an array of port numbers')
-validate_re($server_port, '^\d+$', '$server_port must be a port number')
+validate_numeric($backend_ports)
+validate_numeric($server_port)
 
 package { 'statsd-proxy':
 ensure => $ensure,

-- 
To view, visit https://gerrit.wikimedia.org/r/377439
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I9cf253bf84231767d064b39a306690f79f2f6c4b
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: uwsgi: use validate_numeric()

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377438 )

Change subject: uwsgi: use validate_numeric()
..


uwsgi: use validate_numeric()

Use validate_numeric() to validate the $port parameter, instead of
trying to validate it with a regular expression. Validating Integers
against REs breaks with the future parser and this is easier to read
anyway.

Change-Id: Id21b3dce62db579a7e8c0df59912eeb39e9fe404
---
M modules/service/manifests/uwsgi.pp
1 file changed, 3 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  Mobrovac: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/service/manifests/uwsgi.pp 
b/modules/service/manifests/uwsgi.pp
index 25c2210..1ab8148 100644
--- a/modules/service/manifests/uwsgi.pp
+++ b/modules/service/manifests/uwsgi.pp
@@ -111,9 +111,10 @@
 }
 
 # sanity check since a default port cannot be assigned
-unless $port and $port =~ /^\d+$/ {
-fail('Service port must be specified and must be a number!')
+unless $port {
+fail('Service port must be specified!')
 }
+validate_numeric($port)
 
 # the local log file name
 $local_logdir = "${service::configuration::log_dir}/${title}"

-- 
To view, visit https://gerrit.wikimedia.org/r/377438
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id21b3dce62db579a7e8c0df59912eeb39e9fe404
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: nutcracker: use validate_numeric()

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377437 )

Change subject: nutcracker: use validate_numeric()
..


nutcracker: use validate_numeric()

Use validate_numeric() instead of validate_re() for the $verbosity
parameter. Besides being simpler to read, it is currently implicitly
defined as an Integer, and validating an RE against an Integer fails
with the future parser.

Change-Id: Ic804bb758dc9c846d5fef9e7a5ab8481e422ae89
---
M modules/nutcracker/manifests/init.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/nutcracker/manifests/init.pp 
b/modules/nutcracker/manifests/init.pp
index 3d6c833..d472b3a 100644
--- a/modules/nutcracker/manifests/init.pp
+++ b/modules/nutcracker/manifests/init.pp
@@ -41,7 +41,7 @@
 ) {
 validate_hash($pools)
 validate_re($ensure, '^(present|absent)$')
-validate_re($verbosity, '^(\d|10|11)$')
+validate_numeric($verbosity, 11, 0)
 
 package { 'nutcracker':
 ensure => $ensure,

-- 
To view, visit https://gerrit.wikimedia.org/r/377437
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ic804bb758dc9c846d5fef9e7a5ab8481e422ae89
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: swift: use !~ instead of ! $title =~ /.../

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377436 )

Change subject: swift: use !~ instead of ! $title =~ /.../
..


swift: use !~ instead of ! $title =~ /.../

Current form breaks with the future parser, probably due to operator
precedence.

Change-Id: I14e7f0db85c0f447b74301928d440795963f920c
---
M modules/swift/manifests/init_device.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/swift/manifests/init_device.pp 
b/modules/swift/manifests/init_device.pp
index 5d74f1f..2f1d72c 100644
--- a/modules/swift/manifests/init_device.pp
+++ b/modules/swift/manifests/init_device.pp
@@ -1,5 +1,5 @@
 define swift::init_device($partition_nr='1') {
-if (! $title =~ /^\/dev\/([hvs]d[a-z]+|md[0-9]+)$/) {
+if ($title !~ /^\/dev\/([hvs]d[a-z]+|md[0-9]+)$/) {
 fail("unable to init ${title} for swift")
 }
 

-- 
To view, visit https://gerrit.wikimedia.org/r/377436
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I14e7f0db85c0f447b74301928d440795963f920c
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Faidon Liambotis 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs/private[master]: Add missing secrets

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377465 )

Change subject: Add missing secrets
..


Add missing secrets

Change-Id: Ice290396057116af0d5d5462ec360a02c89ddf85
---
A 
modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
A 
modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem
2 files changed, 0 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git 
a/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
new file mode 100644
index 000..e69de29
--- /dev/null
+++ 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
diff --git 
a/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem
 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem
new file mode 100644
index 000..e69de29
--- /dev/null
+++ 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem

-- 
To view, visit https://gerrit.wikimedia.org/r/377465
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ice290396057116af0d5d5462ec360a02c89ddf85
Gerrit-PatchSet: 1
Gerrit-Project: labs/private
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs/private[master]: Add missing secrets

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377465 )

Change subject: Add missing secrets
..

Add missing secrets

Change-Id: Ice290396057116af0d5d5462ec360a02c89ddf85
---
A 
modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
A 
modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem
2 files changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/labs/private 
refs/changes/65/377465/1

diff --git 
a/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
new file mode 100644
index 000..e69de29
--- /dev/null
+++ 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_privkey.pem
diff --git 
a/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem
 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem
new file mode 100644
index 000..e69de29
--- /dev/null
+++ 
b/modules/secret/secrets/puppetmaster/labtest-puppetmaster.wikimedia.org_pubkey.pem

-- 
To view, visit https://gerrit.wikimedia.org/r/377465
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ice290396057116af0d5d5462ec360a02c89ddf85
Gerrit-PatchSet: 1
Gerrit-Project: labs/private
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: k8s: fix template scoping

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377459 )

Change subject: k8s: fix template scoping
..

k8s: fix template scoping

Since the future parser has different scoping rules, do as follows:
* Add a master_host parameter to k8s::infrastructure
* Explicitly declare k8s::infrastructure_config in k8s::node
* Change the "include" in a "require" for the config class in k8s::proxy
  and k8s::kubelet as it's the proper logical relationship

Bug: T171704
Change-Id: I7a59f3337d3f1d5c96eafbe3240a1544ef2366d8
---
M modules/k8s/manifests/infrastructure_config.pp
M modules/k8s/manifests/kubelet.pp
M modules/k8s/manifests/proxy.pp
M modules/profile/manifests/kubernetes/node.pp
4 files changed, 8 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/59/377459/1

diff --git a/modules/k8s/manifests/infrastructure_config.pp 
b/modules/k8s/manifests/infrastructure_config.pp
index 61c2c5c..3953c93 100644
--- a/modules/k8s/manifests/infrastructure_config.pp
+++ b/modules/k8s/manifests/infrastructure_config.pp
@@ -1,4 +1,4 @@
-class k8s::infrastructure_config {
+class k8s::infrastructure_config($master_host) {
 file { '/etc/kubernetes':
 ensure => directory,
 owner  => 'root',
diff --git a/modules/k8s/manifests/kubelet.pp b/modules/k8s/manifests/kubelet.pp
index 885c894..68b6d36 100644
--- a/modules/k8s/manifests/kubelet.pp
+++ b/modules/k8s/manifests/kubelet.pp
@@ -11,7 +11,7 @@
 $cni_bin_dir='/opt/cni/bin',
 $cni_conf_dir='/etc/cni/net.d',
 ) {
-include ::k8s::infrastructure_config
+require ::k8s::infrastructure_config
 
 require_package('kubernetes-node')
 
diff --git a/modules/k8s/manifests/proxy.pp b/modules/k8s/manifests/proxy.pp
index e7ad8fa..c313c76 100644
--- a/modules/k8s/manifests/proxy.pp
+++ b/modules/k8s/manifests/proxy.pp
@@ -3,7 +3,7 @@
 $proxy_mode = 'iptables',
 $masquerade_all = true,
 ) {
-include ::k8s::infrastructure_config
+require ::k8s::infrastructure_config
 
 $master_ip = ipresolve($master_host, 4, $::nameservers[0])
 
diff --git a/modules/profile/manifests/kubernetes/node.pp 
b/modules/profile/manifests/kubernetes/node.pp
index 881208c..90642c3 100644
--- a/modules/profile/manifests/kubernetes/node.pp
+++ b/modules/profile/manifests/kubernetes/node.pp
@@ -10,6 +10,11 @@
 user=> 'root',
 group   => 'root',
 }
+
+class { '::k8s::infrastructure_config':
+master_host => $master_fqdn,
+}
+
 class { '::k8s::kubelet':
 master_host   => $master_fqdn,
 listen_address=> '0.0.0.0',

-- 
To view, visit https://gerrit.wikimedia.org/r/377459
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7a59f3337d3f1d5c96eafbe3240a1544ef2366d8
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: role::scb: only include profiles, not roles

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377251 )

Change subject: role::scb: only include profiles, not roles
..


role::scb: only include profiles, not roles

Change-Id: I456d3cb3bcec2c7ba7340aa93e8f3a70aaae35c5
---
M modules/role/manifests/scb.pp
1 file changed, 28 insertions(+), 23 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  Alexandros Kosiaris: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/role/manifests/scb.pp b/modules/role/manifests/scb.pp
index 39c44af..8309a1c 100644
--- a/modules/role/manifests/scb.pp
+++ b/modules/role/manifests/scb.pp
@@ -1,33 +1,38 @@
 # "Compendium" class for nodes supporting various *oid services
 # This class is an intermediate step to better design
 class role::scb {
+
+$services = [
+'ORES', 'changeprop', 'citoid', 'cpjobqueue', 'cxserver',
+'eventstreams', 'graphoid', 'mathoid', 'mobileapps',
+'pdfrender', 'trendingedits',
+]
+$msg_services = join($services, "\n\t")
+
+system::role { 'scb':
+description => "Service cluster B; includes:\n\t${msg_services}"
+}
+
+include ::standard
+include ::base::firewall
+include role::lvs::realserver
+
+# Ores
 include ::profile::ores::worker
 include ::profile::ores::web
 include ::profile::nutcracker
 
+
 include ::profile::cpjobqueue
-system::role { 'cpjobqueue':
-description => 'ChangeProp instance for the JobQueue',
-}
-
 include ::profile::recommendation_api
-
-include role::mobileapps
-include role::mathoid
-include role::graphoid
-include role::citoid
-include role::cxserver
-include role::changeprop
-include role::apertium
-include role::eventstreams
-include role::pdfrender
-include role::trendingedits
-
-include ::standard
-include ::base::firewall
-
-if hiera('has_lvs', true) {
-include role::lvs::realserver
-}
-
+include ::profile::mobileapps
+include ::profile::mathoid
+include ::profile::graphoid
+include ::profile::citoid
+include ::profile::cxserver
+include ::profile::changeprop
+include ::profile::apertium
+include ::profile::eventstreams
+include ::profile::pdfrender
+include ::profile::trendingedits
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377251
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I456d3cb3bcec2c7ba7340aa93e8f3a70aaae35c5
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: citoid: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377250 )

Change subject: citoid: move to role/profile
..


citoid: move to role/profile

Change-Id: I349c1275f92372c6badb5c14ae4839851f665282
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/citoid/README
D modules/citoid/tests/Makefile
D modules/citoid/tests/init.pp
R modules/profile/manifests/citoid.pp
M modules/role/manifests/citoid.pp
7 files changed, 10 insertions(+), 36 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 9cb9e2b..66c6420 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -14,8 +14,8 @@
 #service::configuration::http_proxy:
 service::configuration::statsd_host: labmon1001.eqiad.wmnet
 service::configuration::logstash_host: 
deployment-logstash2.deployment-prep.eqiad.wmflabs
-citoid::zotero_host: deployment-zotero01.deployment-prep.eqiad.wmflabs
-citoid::zotero_port: 1969
+profile::citoid::zotero_host: deployment-zotero01.deployment-prep.eqiad.wmflabs
+profile::citoid::zotero_port: 1969
 profile::changeprop::purge_host: 
deployment-cache-text04.deployment-prep.eqiad.wmflabs
 profile::changeprop::ores_uris: 
['http://deployment-sca03.deployment-prep.eqiad.wmflabs:8081']
 profile::changeprop::restbase_uri: 
http://deployment-restbase02.deployment-prep.eqiad.wmflabs:7231
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index 29e3760..b6fb3b4 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -55,8 +55,8 @@
   'Cache-Control': 'public, s-maxage=300, max-age=300'
 ### END GRAPHOID ###
 
-citoid::zotero_port: 1969
-citoid::zotero_host: "zotero.svc.%{::site}.wmnet"
+profile::citoid::zotero_port: 1969
+profile::citoid::zotero_host: "zotero.svc.%{::site}.wmnet"
 profile::ores::web::redis_host: "oresrdb.svc.%{::site}.wmnet"
 # The password is in the private store, this is here for completeness
 # profile::ores::web::redis_password: nothing
diff --git a/modules/citoid/README b/modules/citoid/README
deleted file mode 100644
index 1091e3c..000
--- a/modules/citoid/README
+++ /dev/null
@@ -1,4 +0,0 @@
-While only being a thin wrapper around service::node, this module exists to
-accomodate future citoid needs that are not suited for the service module
-classes as well as conform to a de-facto standard of having a module for every
-service
diff --git a/modules/citoid/tests/Makefile b/modules/citoid/tests/Makefile
deleted file mode 100644
index 76cd656..000
--- a/modules/citoid/tests/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:   test
-
-test:  $(OBJS)
-
-%.po:  %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/citoid/tests/init.pp b/modules/citoid/tests/init.pp
deleted file mode 100644
index 0e8e091..000
--- a/modules/citoid/tests/init.pp
+++ /dev/null
@@ -1,4 +0,0 @@
-class {'::citoid':
-zotero_host => 'localhost',
-zotero_port => 8000,
-}
diff --git a/modules/citoid/manifests/init.pp 
b/modules/profile/manifests/citoid.pp
similarity index 64%
rename from modules/citoid/manifests/init.pp
rename to modules/profile/manifests/citoid.pp
index 99a4bc5..bd17228 100644
--- a/modules/citoid/manifests/init.pp
+++ b/modules/profile/manifests/citoid.pp
@@ -1,11 +1,6 @@
-# == Class: citoid
+# == Class: profile::citoid
 #
 # This class installs and configures citoid
-#
-# While only being a thin wrapper around service::node, this class exists to
-# accomodate future citoid needs that are not suited for the service module
-# classes as well as conform to a de-facto standard of having a module for 
every
-# service
 #
 # === Parameters
 #
@@ -18,10 +13,10 @@
 # [*wskey*]
 #   The WorldCat Search API key to use. Default: ''
 #
-class citoid(
-$zotero_host,
-$zotero_port,
-$wskey = '',
+class profile::citoid(
+$zotero_host=hiera('profile::citoid::zotero_host'),
+$zotero_port=hiera('profile::citoid::zotero_port'),
+$wskey = hiera('citoid::wskey', ''), # TODO: fix namespace
 ) {
 service::node { 'citoid':
 port  => 1970,
diff --git a/modules/role/manifests/citoid.pp b/modules/role/manifests/citoid.pp
index 8359e5a..f8ac29f 100644
--- a/modules/role/manifests/citoid.pp
+++ b/modules/role/manifests/citoid.pp
@@ -5,5 +5,5 @@
 
 system::role { 'citoid': }
 
-include ::citoid
+include ::profile::citoid
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377250
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged

[MediaWiki-commits] [Gerrit] operations/puppet[production]: citoid: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377250 )

Change subject: citoid: move to role/profile
..

citoid: move to role/profile

Change-Id: I349c1275f92372c6badb5c14ae4839851f665282
---
D modules/citoid/README
D modules/citoid/tests/Makefile
D modules/citoid/tests/init.pp
R modules/profile/manifests/citoid.pp
M modules/role/manifests/citoid.pp
5 files changed, 6 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/50/377250/1

diff --git a/modules/citoid/README b/modules/citoid/README
deleted file mode 100644
index 1091e3c..000
--- a/modules/citoid/README
+++ /dev/null
@@ -1,4 +0,0 @@
-While only being a thin wrapper around service::node, this module exists to
-accomodate future citoid needs that are not suited for the service module
-classes as well as conform to a de-facto standard of having a module for every
-service
diff --git a/modules/citoid/tests/Makefile b/modules/citoid/tests/Makefile
deleted file mode 100644
index 76cd656..000
--- a/modules/citoid/tests/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:   test
-
-test:  $(OBJS)
-
-%.po:  %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/citoid/tests/init.pp b/modules/citoid/tests/init.pp
deleted file mode 100644
index 0e8e091..000
--- a/modules/citoid/tests/init.pp
+++ /dev/null
@@ -1,4 +0,0 @@
-class {'::citoid':
-zotero_host => 'localhost',
-zotero_port => 8000,
-}
diff --git a/modules/citoid/manifests/init.pp 
b/modules/profile/manifests/citoid.pp
similarity index 64%
rename from modules/citoid/manifests/init.pp
rename to modules/profile/manifests/citoid.pp
index 99a4bc5..bd17228 100644
--- a/modules/citoid/manifests/init.pp
+++ b/modules/profile/manifests/citoid.pp
@@ -1,11 +1,6 @@
-# == Class: citoid
+# == Class: profile::citoid
 #
 # This class installs and configures citoid
-#
-# While only being a thin wrapper around service::node, this class exists to
-# accomodate future citoid needs that are not suited for the service module
-# classes as well as conform to a de-facto standard of having a module for 
every
-# service
 #
 # === Parameters
 #
@@ -18,10 +13,10 @@
 # [*wskey*]
 #   The WorldCat Search API key to use. Default: ''
 #
-class citoid(
-$zotero_host,
-$zotero_port,
-$wskey = '',
+class profile::citoid(
+$zotero_host=hiera('profile::citoid::zotero_host'),
+$zotero_port=hiera('profile::citoid::zotero_port'),
+$wskey = hiera('citoid::wskey', ''), # TODO: fix namespace
 ) {
 service::node { 'citoid':
 port  => 1970,
diff --git a/modules/role/manifests/citoid.pp b/modules/role/manifests/citoid.pp
index 8359e5a..f8ac29f 100644
--- a/modules/role/manifests/citoid.pp
+++ b/modules/role/manifests/citoid.pp
@@ -5,5 +5,5 @@
 
 system::role { 'citoid': }
 
-include ::citoid
+include ::profile::citoid
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377250
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I349c1275f92372c6badb5c14ae4839851f665282
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: role::scb: only include profiles, not roles

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377251 )

Change subject: role::scb: only include profiles, not roles
..

role::scb: only include profiles, not roles

Change-Id: I456d3cb3bcec2c7ba7340aa93e8f3a70aaae35c5
---
M modules/role/manifests/scb.pp
1 file changed, 27 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/51/377251/1

diff --git a/modules/role/manifests/scb.pp b/modules/role/manifests/scb.pp
index 39c44af..1ea9c2c 100644
--- a/modules/role/manifests/scb.pp
+++ b/modules/role/manifests/scb.pp
@@ -1,33 +1,37 @@
 # "Compendium" class for nodes supporting various *oid services
 # This class is an intermediate step to better design
 class role::scb {
+
+$services = [
+'ORES', 'changeprop', 'citoid', 'cpjobqueue', 'cxserver',
+'eventstreams', 'graphoid', 'mathoid', 'mobileapps',
+'pdfrender', 'trendingedits',
+]
+
+system::role { 'scb':
+description => inline_template('Service cluster B; includes:\n<%= 
@services.join "\n" -%>')
+}
+
+include ::standard
+include ::base::firewall
+include role::lvs::realserver
+
+# Ores
 include ::profile::ores::worker
 include ::profile::ores::web
 include ::profile::nutcracker
 
+
 include ::profile::cpjobqueue
-system::role { 'cpjobqueue':
-description => 'ChangeProp instance for the JobQueue',
-}
-
 include ::profile::recommendation_api
-
-include role::mobileapps
-include role::mathoid
-include role::graphoid
-include role::citoid
-include role::cxserver
-include role::changeprop
-include role::apertium
-include role::eventstreams
-include role::pdfrender
-include role::trendingedits
-
-include ::standard
-include ::base::firewall
-
-if hiera('has_lvs', true) {
-include role::lvs::realserver
-}
-
+include ::profile::mobileapps
+include ::profile::mathoid
+include ::profile::graphoid
+include ::profile::citoid
+include ::profile::cxserver
+include ::profile::changeprop
+include ::profile::apertium
+include ::profile::eventstreams
+include ::profile::pdfrender
+include ::profile::trendingedits
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377251
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I456d3cb3bcec2c7ba7340aa93e8f3a70aaae35c5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: CP-JobQueue: Add the service to SCB

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377213 )

Change subject: CP-JobQueue: Add the service to SCB
..


CP-JobQueue: Add the service to SCB

Bug: T175281
Change-Id: I1ebdb7eb4964e0ecb57c050ce46876bb8e72a1c7
---
M hieradata/role/common/scb.yaml
M modules/admin/data/data.yaml
M modules/role/manifests/scb.pp
3 files changed, 14 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index 5553030..29e3760 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -3,6 +3,7 @@
   - sc-admins
   - citoid-admin
   - citoid-users
+  - cpjobqueue-admin
   - cxserver-admin
   - apertium-admins
   - graphoid-admin
diff --git a/modules/admin/data/data.yaml b/modules/admin/data/data.yaml
index c368c39..86fe9b8 100644
--- a/modules/admin/data/data.yaml
+++ b/modules/admin/data/data.yaml
@@ -526,6 +526,7 @@
 privileges: ['ALL = NOPASSWD: /usr/bin/puppet agent *',
'ALL = NOPASSWD: /usr/sbin/service changeprop *',
'ALL = NOPASSWD: /usr/sbin/service citoid *',
+   'ALL = NOPASSWD: /usr/sbin/service cpjobqueue *',
'ALL = NOPASSWD: /usr/sbin/service cxserver *',
'ALL = NOPASSWD: /usr/sbin/service graphoid *',
'ALL = NOPASSWD: /usr/sbin/service mathoid *',
@@ -617,6 +618,12 @@
 members: [gwicke, ppchelko, eevans, mobrovac, nschaaf]
 privileges: ['ALL = NOPASSWD: /usr/sbin/service recommendation_api *',
  'ALL = (recommendation_api) NOPASSWD: ALL']
+  cpjobqueue-admin:
+description: Group of cpjobqueue admins
+gid: 795
+members: [mobrovac, gwicke, ppchelko, eevans]
+privileges: ['ALL = NOPASSWD: /usr/sbin/service cpjobqueue *',
+ 'ALL = (cpjobqueue) NOPASSWD: ALL']
 
 users:
   rush:
diff --git a/modules/role/manifests/scb.pp b/modules/role/manifests/scb.pp
index 1ae90c0..39c44af 100644
--- a/modules/role/manifests/scb.pp
+++ b/modules/role/manifests/scb.pp
@@ -5,6 +5,11 @@
 include ::profile::ores::web
 include ::profile::nutcracker
 
+include ::profile::cpjobqueue
+system::role { 'cpjobqueue':
+description => 'ChangeProp instance for the JobQueue',
+}
+
 include ::profile::recommendation_api
 
 include role::mobileapps
@@ -24,4 +29,5 @@
 if hiera('has_lvs', true) {
 include role::lvs::realserver
 }
+
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377213
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1ebdb7eb4964e0ecb57c050ce46876bb8e72a1c7
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Mobrovac 
Gerrit-Reviewer: Alex Monk 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: Muehlenhoff 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: ChangeProp: Separate packages into profile::changeprop::pack...

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377218 )

Change subject: ChangeProp: Separate packages into profile::changeprop::packages
..


ChangeProp: Separate packages into profile::changeprop::packages

Change-Id: I768a6000c7ae44017a71f2790b8c5894b88e4f33
---
M modules/profile/manifests/changeprop.pp
A modules/profile/manifests/changeprop/packages.pp
M modules/profile/manifests/cpjobqueue.pp
3 files changed, 12 insertions(+), 6 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/changeprop.pp 
b/modules/profile/manifests/changeprop.pp
index a9aed56..d825459 100644
--- a/modules/profile/manifests/changeprop.pp
+++ b/modules/profile/manifests/changeprop.pp
@@ -11,15 +11,12 @@
 
 include ::passwords::redis
 include ::service::configuration
+require ::profile::changeprop::packages
+
 $kafka_config = kafka_config('main')
 $broker_list = $kafka_config['brokers']['string']
 $redis_path = "/var/run/nutcracker/redis_${::site}.sock"
 $redis_pass = $::passwords::redis::main_password
-
-service::packages { 'changeprop':
-pkgs => ['librdkafka++1', 'librdkafka1'],
-dev_pkgs => ['librdkafka-dev'],
-}
 
 service::node { 'changeprop':
 enable=> true,
diff --git a/modules/profile/manifests/changeprop/packages.pp 
b/modules/profile/manifests/changeprop/packages.pp
new file mode 100644
index 000..1df50de
--- /dev/null
+++ b/modules/profile/manifests/changeprop/packages.pp
@@ -0,0 +1,9 @@
+# Packages required by changeprop and cpjobqueue
+class profile::changeprop::packages() {
+
+service::packages { 'changeprop':
+pkgs => ['librdkafka++1', 'librdkafka1'],
+dev_pkgs => ['librdkafka-dev'],
+}
+
+}
diff --git a/modules/profile/manifests/cpjobqueue.pp 
b/modules/profile/manifests/cpjobqueue.pp
index afc91a0..2fa2463 100644
--- a/modules/profile/manifests/cpjobqueue.pp
+++ b/modules/profile/manifests/cpjobqueue.pp
@@ -25,7 +25,7 @@
 ) {
 
 include ::passwords::redis
-require ::changeprop::packages
+require ::profile::changeprop::packages
 
 $kafka_config = kafka_config('main')
 

-- 
To view, visit https://gerrit.wikimedia.org/r/377218
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I768a6000c7ae44017a71f2790b8c5894b88e4f33
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Mobrovac 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: graphoid: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377212 )

Change subject: graphoid: convert to role/profile
..


graphoid: convert to role/profile

Change-Id: Ib15aaa3d86c825edb3d672331bcaf113852fb10c
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/graphoid/manifests/packages.pp
R modules/profile/manifests/graphoid.pp
M modules/role/manifests/graphoid.pp
5 files changed, 29 insertions(+), 44 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 6100459..9cb9e2b 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -22,7 +22,7 @@
 profile::changeprop::purge_port: 4827
 # Used to sync the setting between all Kafka clusters and clients.
 kafka_message_max_bytes: 4194304
-graphoid::allowed_domains:
+profile::graphoid::allowed_domains:
   http:
 - wmflabs.org
   https:
@@ -47,9 +47,9 @@
 - wdqs-test.wmflabs.org
   geoshape:
 - maps.wikimedia.org
-graphoid::headers:
+profile::graphoid::headers:
   'Cache-Control': 'public, s-maxage=360, max-age=360'
-graphoid::error_headers:
+profile::graphoid::error_headers:
   'Cache-Control': 'public, s-maxage=30, max-age=30'
 lvs::configuration::lvs_services:
   apaches:
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index bf4a56b..5553030 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -26,7 +26,9 @@
   pdfrender: {}
   trendingedits: {}
   "recommendation-api": {}
-graphoid::allowed_domains:
+
+### BEGIN GRAPHOID
+profile::graphoid::allowed_domains:
   https:
 - mediawiki.org
 - wikibooks.org
@@ -46,10 +48,12 @@
 - query.wikidata.org
   geoshape:
 - maps.wikimedia.org
-graphoid::headers:
+profile::graphoid::headers:
   'Cache-Control': 'public, s-maxage=3600, max-age=3600'
-graphoid::error_headers:
+profile::graphoid::error_headers:
   'Cache-Control': 'public, s-maxage=300, max-age=300'
+### END GRAPHOID ###
+
 citoid::zotero_port: 1969
 citoid::zotero_host: "zotero.svc.%{::site}.wmnet"
 profile::ores::web::redis_host: "oresrdb.svc.%{::site}.wmnet"
@@ -96,6 +100,7 @@
 profile::changeprop::ores_uris:
   - http://ores.svc.eqiad.wmnet:8081
   - http://ores.svc.codfw.wmnet:8081
+### END CHANGEPROP ###
 profile::nutcracker::memcached_pools: {}
 profile::nutcracker::monitor_port: 0 # we have nothing exposed via tcp
 
diff --git a/modules/graphoid/manifests/packages.pp 
b/modules/graphoid/manifests/packages.pp
deleted file mode 100644
index 05686a4..000
--- a/modules/graphoid/manifests/packages.pp
+++ /dev/null
@@ -1,19 +0,0 @@
-# == Class: graphoid::packages
-#
-# Installs the packages needed by graphoid
-#
-# NOTE: this is a temporary work-around for the CI to be able to install
-# development packages. In the future, we want to have more integration so as 
to
-# run tests as close to production as possible.
-#
-class graphoid::packages {
-
-require ::mediawiki::packages::fonts
-
-service::packages { 'graphoid':
-pkgs => ['libcairo2', 'libgif4', 'libjpeg62-turbo', 
'libpango1.0-0'],
-dev_pkgs => ['libcairo2-dev', 'libgif-dev', 'libpango1.0-dev',
-'libjpeg62-turbo-dev'],
-}
-
-}
diff --git a/modules/graphoid/manifests/init.pp 
b/modules/profile/manifests/graphoid.pp
similarity index 65%
rename from modules/graphoid/manifests/init.pp
rename to modules/profile/manifests/graphoid.pp
index 81f4af8..6f111cf 100644
--- a/modules/graphoid/manifests/init.pp
+++ b/modules/profile/manifests/graphoid.pp
@@ -1,4 +1,4 @@
-# == Class: graphoid
+# == Class: profile::graphoid
 #
 # This class installs and configures graphoid, a node.js service that
 # converts a graph definition into a PNG image
@@ -8,32 +8,31 @@
 # [*allowed_domains*]
 #   The protocol-to-list-of-domains map. Default: {}
 #   The protocols include http, https, as well as some custom graph-specific 
protocols.
-#   See 
https://www.mediawiki.org/wiki/Extension:Graph?venotify=restored#External_data
-#
-# [*domain_map*]
-#   The domain-to-domain alias map. Default: {}
-#
-# [*timeout*]
-#   The timeout (in ms) for requests. Default: 5000
-#
+#   See 
https://www.mediawiki.org/wiki/Extension:Graph?venotify=restored#External_data#
 # [*headers*]
 #   A map of headers that will be sent with each reply. Could be used for 
caching, etc. Default: false
 #
 # [*error_headers*]
 #   A map of headers that will be sent with each reply in case of an error. If 
not set, above headers will be used. Default: false
 #
-class graphoid(
-$allowed_domains = {},
-$domain_map= {},
-$timeout   = 5000,
-$headers   = false,
-$error_headers = false,
+class profile::graphoid(
+$allowed_domains = 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: changeprop: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377211 )

Change subject: changeprop: convert to role/profile
..


changeprop: convert to role/profile

Change-Id: I191a4f46aaffa13bca21bfbb3a00292ef10f46b1
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/changeprop/manifests/init.pp
D modules/changeprop/manifests/packages.pp
D modules/changeprop/tests/Makefile
D modules/changeprop/tests/init.pp
A modules/profile/manifests/changeprop.pp
M modules/role/manifests/changeprop.pp
8 files changed, 61 insertions(+), 131 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index a1fbef7..6100459 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -16,10 +16,10 @@
 service::configuration::logstash_host: 
deployment-logstash2.deployment-prep.eqiad.wmflabs
 citoid::zotero_host: deployment-zotero01.deployment-prep.eqiad.wmflabs
 citoid::zotero_port: 1969
-changeprop::purge_host: deployment-cache-text04.deployment-prep.eqiad.wmflabs
-changeprop::ores_uris: 
['http://deployment-sca03.deployment-prep.eqiad.wmflabs:8081']
-# Need to redefine RESTBase URI as Change-Prop redefines it in order to 
provide a different value for async updates
-changeprop::restbase_uri: 
http://deployment-restbase02.deployment-prep.eqiad.wmflabs:7231
+profile::changeprop::purge_host: 
deployment-cache-text04.deployment-prep.eqiad.wmflabs
+profile::changeprop::ores_uris: 
['http://deployment-sca03.deployment-prep.eqiad.wmflabs:8081']
+profile::changeprop::restbase_uri: 
http://deployment-restbase02.deployment-prep.eqiad.wmflabs:7231
+profile::changeprop::purge_port: 4827
 # Used to sync the setting between all Kafka clusters and clients.
 kafka_message_max_bytes: 4194304
 graphoid::allowed_domains:
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index df1763b..bf4a56b 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -89,7 +89,13 @@
   broker.version.fallback: '0.9.0.1'
 ### END EVENTSTREAMS ###
 
-changeprop::restbase_uri: http://restbase-async.discovery.wmnet:7231
+### BEGIN CHANGEPROP ###
+profile::changeprop::restbase_uri: http://restbase-async.discovery.wmnet:7231
+profile::changeprop::purge_host: 239.128.0.112
+profile::changeprop::purge_port: 4827
+profile::changeprop::ores_uris:
+  - http://ores.svc.eqiad.wmnet:8081
+  - http://ores.svc.codfw.wmnet:8081
 profile::nutcracker::memcached_pools: {}
 profile::nutcracker::monitor_port: 0 # we have nothing exposed via tcp
 
diff --git a/modules/changeprop/manifests/init.pp 
b/modules/changeprop/manifests/init.pp
deleted file mode 100644
index 53e31e9..000
--- a/modules/changeprop/manifests/init.pp
+++ /dev/null
@@ -1,83 +0,0 @@
-# == Class: changeprop
-#
-# This class installs and configures the change propagation service, a part of
-# the EventBus system responsible for reacting to events received via Kafka and
-# dispatching the appropriate requests.
-#
-# === Parameters
-#
-# [*broker_list*]
-#   Comma-separated list of Kafka broker URIs
-#
-# [*purge_host*]
-#   The vhtcpd daemon host to send purge requests to. Default: 239.128.0.112
-#
-# [*purge_port*]
-#   The port the vhtcp daemon listens to. Default: 4827
-#
-# [*restbase_uri*]
-#   RESTBase's URI. Note that this is redefined here so that async update
-#   requests can be sent to the inactive DC. Default:
-#   'http://restbase.svc.eqiad.wmnet:7231'
-#
-# [*ores_uris*]
-#   A list of urls for the ORES service. Defaults to:
-#   [http://ores.svc.eqiad.wmnet:8081, http://ores.svc.codfw.wmnet:8081]
-#
-# [*redis_path*]
-#   The UNIX socket file path of the Redis/Nutcracker server. Default:
-#   "/var/run/nutcracker/redis_${::site}.sock"
-#
-# [*redis_pass*]
-#   The password to use when authenticating with Redis/Nutcracker. Default:
-#   'abc1234'
-#
-# [*kafka_msg_max_bytes*]
-#   The maximum number of bytes allowed in a Kafka message. Default:
-#   '1048576'
-#
-class changeprop(
-$broker_list,
-$purge_host  = '239.128.0.112',
-$purge_port  = 4827,
-$restbase_uri= 'http://restbase.svc.eqiad.wmnet:7231',
-$ores_uris   = [
-'http://ores.svc.eqiad.wmnet:8081',
-'http://ores.svc.codfw.wmnet:8081',
-],
-$redis_path  = "/var/run/nutcracker/redis_${::site}.sock",
-$redis_pass  = 'abc1234',
-$kafka_msg_max_bytes = 1048576,
-) {
-
-include ::service::configuration
-
-require ::changeprop::packages
-
-service::node { 'changeprop':
-enable=> true,
-port  => 7272,
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-

[MediaWiki-commits] [Gerrit] operations/puppet[production]: pdfrender: switch to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377210 )

Change subject: pdfrender: switch to role/profile
..


pdfrender: switch to role/profile

In this case, since it's not incapsulating service::node and collects
quite a few resources, we prefer not to remove the named class.

Change-Id: I735701d4ea4069c307b328d8fcb5fab782aad165
---
A modules/profile/manifests/pdfrender.pp
M modules/role/manifests/pdfrender.pp
2 files changed, 20 insertions(+), 16 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/pdfrender.pp 
b/modules/profile/manifests/pdfrender.pp
new file mode 100644
index 000..28f5c7f
--- /dev/null
+++ b/modules/profile/manifests/pdfrender.pp
@@ -0,0 +1,19 @@
+class profile::pdfrender(
+$is_active = hiera('profile::pdfrender::is_active', true)
+) {
+
+$port = 5252
+
+class { '::pdfrender':
+port=> $port,
+no_browsers => 4,
+running => $is_active,
+}
+
+ferm::service { "pdfrender_http_${port}":
+proto  => 'tcp',
+port   => $port,
+srange => '$DOMAIN_NETWORKS',
+}
+
+}
diff --git a/modules/role/manifests/pdfrender.pp 
b/modules/role/manifests/pdfrender.pp
index 657def0c..42f99e2 100644
--- a/modules/role/manifests/pdfrender.pp
+++ b/modules/role/manifests/pdfrender.pp
@@ -1,22 +1,7 @@
 class role::pdfrender {
-$is_active = hiera('role::pdfrender::is_active', true)
-
 system::role { 'pdfrender':
 description => 'A PDF render service based on Electron',
 }
 
-$port = 5252
-
-class { '::pdfrender':
-port=> $port,
-no_browsers => 4,
-running => $is_active,
-}
-
-ferm::service { "pdfrender_http_${port}":
-proto  => 'tcp',
-port   => $port,
-srange => '$DOMAIN_NETWORKS',
-}
-
+include ::profile::pdfrender
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377210
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I735701d4ea4069c307b328d8fcb5fab782aad165
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: eventstreams: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377209 )

Change subject: eventstreams: convert to role/profile
..


eventstreams: convert to role/profile

Change-Id: I94dad66103ce344a04b85a581203ecc5540052d0
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/eventstreams/manifests/init.pp
A modules/profile/manifests/eventstreams.pp
M modules/role/manifests/eventstreams.pp
5 files changed, 84 insertions(+), 128 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 7a35d08..a1fbef7 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -341,3 +341,14 @@
 prometheus_nodes:
   - deployment-prometheus01.deployment-prep.eqiad.wmflabs
 profile::recommendation_api::wdqs_uri: http://wdqs-test.wmflabs.org
+
+# Eventstreams config
+profile::eventstreams::kafka_cluster_name: main
+profile::eventstreams::streams:
+  test:
+topics: ["%{::site}.test.event"]
+  revision-create:
+topics: ["%{::site}.mediawiki.revision-create"]
+  recentchange:
+topics: ["%{::site}.mediawiki.recentchange"]
+profile::eventstreams::rdkafka_config: {}
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index e6134b5..df1763b 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -56,9 +56,12 @@
 # The password is in the private store, this is here for completeness
 # profile::ores::web::redis_password: nothing
 
+
+### BEGIN EVENTSTREAMS ###
+profile::eventstreams::kafka_cluster_name: analytics
 # Stream configuration for Public EventStreams service
 # Maps stream route names to composite Kafka topics.
-role::eventstreams::streams:
+profile::eventstreams::streams:
   recentchange:
 description: "Mediawiki RecentChanges feed. Schema: 
https://github.com/wikimedia/mediawiki-event-schemas/tree/master/jsonschema/mediawiki/recentchange;
 topics:
@@ -76,7 +79,7 @@
   - codfw.test.event
 
 # rdkafka config for Public EventStreams service.
-role::eventstreams::rdkafka_config:
+profile::eventstreams::rdkafka_config:
   # Send rdkafka stats to statsd once per minute.
   statistics.interval.ms: 6
   # Specify Kafka API version as workaround for Brokers < 0.10
@@ -84,6 +87,7 @@
   # This will not be necessary when the target Kafka cluster will be running 
0.10
   # librdkafka 0.9.4.x default for api.version.request is false, no need to 
set it
   broker.version.fallback: '0.9.0.1'
+### END EVENTSTREAMS ###
 
 changeprop::restbase_uri: http://restbase-async.discovery.wmnet:7231
 profile::nutcracker::memcached_pools: {}
diff --git a/modules/eventstreams/manifests/init.pp 
b/modules/eventstreams/manifests/init.pp
deleted file mode 100644
index 011dec8..000
--- a/modules/eventstreams/manifests/init.pp
+++ /dev/null
@@ -1,57 +0,0 @@
-# == Class: eventstreams
-#
-# === Parameters
-#
-# [*broker_list*]
-#   Comma-separated list of Kafka broker URIs
-#
-# [*streams*]
-#   Hash of stream route config and their composite topics. E.g.
-#
-#   streamName1:
-#   topics: [topicA, topicB]
-#   streamName2:
-#   topics: [topicC, topicD]
-#
-# [*port*]
-#   Default: 8092
-#
-# [*log_level*]
-#   Log level for service logger. Default: info
-#
-# [*rdkafka_config*]
-#   Extra librdkafka configuration to provide to node-rdkafka.  Default: {}
-#
-class eventstreams(
-$broker_list,
-$streams,
-$port   = 8092,
-$log_level  = 'info',
-$rdkafka_config = {},
-) {
-service::packages { 'eventstreams':
-pkgs => ['librdkafka++1', 'librdkafka1'],
-}
-
-service::node { 'eventstreams':
-enable=> true,
-port  => $port,
-has_spec  => false, # TODO: figure out how to monitor stream 
with spec x-amples
-deployment=> 'scap3',
-deployment_config => true,
-deployment_vars   => {
-log_level  => $log_level,
-site   => $::site,
-broker_list=> $broker_list,
-rdkafka_config => $rdkafka_config,
-streams=> $streams,
-},
-auto_refresh  => false,
-init_restart  => false,
-environment   => {
-'UV_THREADPOOL_SIZE' => 128,
-},
-require   => Service::Packages['eventstreams'],
-}
-
-}
diff --git a/modules/profile/manifests/eventstreams.pp 
b/modules/profile/manifests/eventstreams.pp
new file mode 100644
index 000..d340984
--- /dev/null
+++ b/modules/profile/manifests/eventstreams.pp
@@ -0,0 +1,66 @@
+# == Class profile::eventstreams
+#
+# Profile that installs EventStreams HTTP service.
+# This class includes the ::eventstreams role, and 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: trendingedits: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377208 )

Change subject: trendingedits: move to role/profile
..


trendingedits: move to role/profile

Change-Id: Ia0a2248c7282583a8c2fcc87e94d86fd749c9c33
---
A modules/profile/manifests/trendingedits.pp
M modules/role/manifests/trendingedits.pp
D modules/trendingedits/manifests/init.pp
D modules/trendingedits/manifests/packages.pp
4 files changed, 28 insertions(+), 64 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/trendingedits.pp 
b/modules/profile/manifests/trendingedits.pp
new file mode 100644
index 000..ca57360
--- /dev/null
+++ b/modules/profile/manifests/trendingedits.pp
@@ -0,0 +1,27 @@
+# Profile class for trendingedits
+class profile::trendingedits {
+
+$kafka_config = kafka_config('main')
+$port = 6699
+
+service::packages { 'trendingedits':
+pkgs => ['librdkafka++1', 'librdkafka1'],
+dev_pkgs => ['librdkafka-dev'],
+}
+
+service::node { 'trendingedits':
+port  => $port,
+repo  => 'trending-edits/deploy',
+healthcheck_url   => '',
+has_spec  => true,
+deployment=> 'scap3',
+deployment_config => true,
+deployment_vars   => {
+broker_list => $kafka_config['brokers']['string'],
+site=> $::site,
+},
+environment   => {
+'UV_THREADPOOL_SIZE' => 16
+},
+}
+}
diff --git a/modules/role/manifests/trendingedits.pp 
b/modules/role/manifests/trendingedits.pp
index 7716f0c..90a1205 100644
--- a/modules/role/manifests/trendingedits.pp
+++ b/modules/role/manifests/trendingedits.pp
@@ -1,17 +1,8 @@
 # Role class for trendingedits
 class role::trendingedits {
-
-$kafka_config = kafka_config('main')
-$port = 6699
-
 system::role { 'trendingedits':
 description => 'computes the list of currently-trending articles',
 }
 
-class { '::trendingedits':
-port=> $port,
-broker_list => $kafka_config['brokers']['string'],
-}
-
+include ::profile::trendingedits
 }
-
diff --git a/modules/trendingedits/manifests/init.pp 
b/modules/trendingedits/manifests/init.pp
deleted file mode 100644
index 075cf75..000
--- a/modules/trendingedits/manifests/init.pp
+++ /dev/null
@@ -1,38 +0,0 @@
-# == Class: trendingedits
-#
-# This class installs and configures the trending edits service, which follows
-# events from the EventBus system in real time and computes the list of
-# currently-trending articles based on the number of edits.
-#
-# === Parameters
-#
-# [*port*]
-#   The port to bind the service to
-#
-# [*broker_list*]
-#   Comma-separated list of Kafka broker URIs
-#
-class trendingedits(
-$port,
-$broker_list,
-) {
-
-require ::trendingedits::packages
-
-service::node { 'trendingedits':
-port  => $port,
-repo  => 'trending-edits/deploy',
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-deployment_config => true,
-deployment_vars   => {
-broker_list => $broker_list,
-site=> $::site,
-},
-environment   => {
-'UV_THREADPOOL_SIZE' => 16
-},
-}
-
-}
diff --git a/modules/trendingedits/manifests/packages.pp 
b/modules/trendingedits/manifests/packages.pp
deleted file mode 100644
index 71b4530..000
--- a/modules/trendingedits/manifests/packages.pp
+++ /dev/null
@@ -1,16 +0,0 @@
-# == Class: trendingedits::packages
-#
-# Installs the packages needed by the trending edits service
-#
-# NOTE: this is a temporary work-around for the CI to be able to install
-# development packages. In the future, we want to have more integration so as 
to
-# run tests as close to production as possible.
-#
-class trendingedits::packages {
-
-  service::packages { 'trendingedits':
-pkgs => ['librdkafka++1', 'librdkafka1'],
-dev_pkgs => ['librdkafka-dev'],
-  }
-
-}

-- 
To view, visit https://gerrit.wikimedia.org/r/377208
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia0a2248c7282583a8c2fcc87e94d86fd749c9c33
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: apertium: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377207 )

Change subject: apertium: move to role/profile
..


apertium: move to role/profile

Change-Id: I65f9890bad802dedee36296bf1b4bbfe309d59e1
---
M hieradata/labs/deployment-prep/common.yaml
D modules/apertium/tests/Makefile
D modules/apertium/tests/init.pp
R modules/profile/manifests/apertium.pp
R modules/profile/templates/initscripts/apertium-apy.systemd.erb
R modules/profile/templates/initscripts/apertium-apy.upstart.erb
M modules/role/manifests/apertium.pp
7 files changed, 28 insertions(+), 47 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 26242c7..7a35d08 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -168,7 +168,6 @@
   explicit_macs: false
   disable_agent_forwarding: false
   challenge_response_auth: false
-"role::apertium::apertium_port": 2737
 "role::cxserver::cxserver_port": 8080
 "misc::syslog-server::basepath": /data/project/syslog
 "profile::cxserver::apertium_uri": http://apertium-beta.wmflabs.org
diff --git a/modules/apertium/tests/Makefile b/modules/apertium/tests/Makefile
deleted file mode 100644
index 76cd656..000
--- a/modules/apertium/tests/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:   test
-
-test:  $(OBJS)
-
-%.po:  %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/apertium/tests/init.pp b/modules/apertium/tests/init.pp
deleted file mode 100644
index b35284d..000
--- a/modules/apertium/tests/init.pp
+++ /dev/null
@@ -1 +0,0 @@
-include ::apertium
diff --git a/modules/apertium/manifests/init.pp 
b/modules/profile/manifests/apertium.pp
similarity index 83%
rename from modules/apertium/manifests/init.pp
rename to modules/profile/manifests/apertium.pp
index 0d97f3e..4345fb1 100644
--- a/modules/apertium/manifests/init.pp
+++ b/modules/profile/manifests/apertium.pp
@@ -6,21 +6,21 @@
 # === Parameters
 #
 # [*num_of_processes*]
-#   Number of APY instance processes to run.
-# [*max_idle_seconds*]
-#   Seconds to wait before shutdown idle process.
-# [*uid*]
-#   The username apertium-apy will run with.
-# [*gid*]
-#   The group apertium-apy will run with.
-class apertium(
-$num_of_processes = 1,
-$max_idle_seconds = 300,
-$uid = 'apertium',
-$gid = 'apertium',
-) {
 
-include ::service::configuration
+# [*max_idle_seconds*]
+#
+class profile::apertium {
+require ::service::configuration
+
+# Port we're listening on
+$port = 2737
+# Number of APY instance processes to run.
+$num_of_processes = 1
+# Seconds to wait before shutting down an idle process.
+$max_idle_seconds = 300
+# User and group
+$uid = 'apertium'
+$gid = 'apertium'
 
 $log_dir = "${::service::configuration::log_dir}/apertium"
 
@@ -134,4 +134,14 @@
 not_if_empty  => true,
 rotate=> 15,
 }
+
+ferm::service { 'apertium_http':
+proto => 'tcp',
+port  => $port,
+}
+
+monitoring::service { 'apertium':
+description   => 'apertium apy',
+check_command => 
"check_http_hostheader_port_url!apertium.svc.${::site}.wmnet!${port}!/listPairs",
+}
 }
diff --git a/modules/apertium/templates/initscripts/apertium-apy.systemd.erb 
b/modules/profile/templates/initscripts/apertium-apy.systemd.erb
similarity index 91%
rename from modules/apertium/templates/initscripts/apertium-apy.systemd.erb
rename to modules/profile/templates/initscripts/apertium-apy.systemd.erb
index 0e81513..da17036 100644
--- a/modules/apertium/templates/initscripts/apertium-apy.systemd.erb
+++ b/modules/profile/templates/initscripts/apertium-apy.systemd.erb
@@ -1,7 +1,7 @@
 # NOTE: This file is managed by Puppet
-# Systemd unit for <%= @title %>-apy
+# Systemd unit for apertium-apy
 [Unit]
-Description="<%= @title %>-apy service"
+Description="apertium-apy service"
 
 [Service]
 User=<%= @uid %>
diff --git a/modules/apertium/templates/initscripts/apertium-apy.upstart.erb 
b/modules/profile/templates/initscripts/apertium-apy.upstart.erb
similarity index 100%
rename from modules/apertium/templates/initscripts/apertium-apy.upstart.erb
rename to modules/profile/templates/initscripts/apertium-apy.upstart.erb
diff --git a/modules/role/manifests/apertium.pp 
b/modules/role/manifests/apertium.pp
index 613bb40..55671d4 100644
--- a/modules/role/manifests/apertium.pp
+++ b/modules/role/manifests/apertium.pp
@@ -1,23 +1,9 @@
 # vim: set ts=4 et sw=4:
 #
 # filtertags: labs-project-deployment-prep
-class role::apertium(
-$port = '2737',

[MediaWiki-commits] [Gerrit] operations/puppet[production]: mathoid: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377205 )

Change subject: mathoid: move to role/profile
..


mathoid: move to role/profile

Change-Id: I5756122ac1f1363ee7239c43ecb403e8ba1e54c2
---
D modules/mathoid/manifests/init.pp
D modules/mathoid/manifests/packages.pp
D modules/mathoid/tests/Makefile
D modules/mathoid/tests/mathoid.pp
A modules/profile/manifests/mathoid.pp
M modules/role/manifests/mathoid.pp
6 files changed, 25 insertions(+), 49 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/mathoid/manifests/init.pp 
b/modules/mathoid/manifests/init.pp
deleted file mode 100644
index 5f714b0..000
--- a/modules/mathoid/manifests/init.pp
+++ /dev/null
@@ -1,18 +0,0 @@
-# == Class: mathoid
-#
-# Mathoid is an application which takes various forms of math input and
-# converts it to MathML + SVG output. It is a web-service implemented
-# in node.js.
-#
-class mathoid {
-
-require ::mathoid::packages
-
-service::node { 'mathoid':
-port  => 10042,
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-deployment_config => true,
-}
-}
diff --git a/modules/mathoid/manifests/packages.pp 
b/modules/mathoid/manifests/packages.pp
deleted file mode 100644
index 529d3a3..000
--- a/modules/mathoid/manifests/packages.pp
+++ /dev/null
@@ -1,16 +0,0 @@
-# == Class: mathoid::packages
-#
-# Installs the packages needed by Mathoid
-#
-# NOTE: this is a temporary work-around for the CI to be able to install
-# development packages. In the future, we want to have more integration so as 
to
-# run tests as close to production as possible.
-#
-class mathoid::packages {
-
-service::packages { 'mathoid':
-pkgs => ['librsvg2-2'],
-dev_pkgs => ['librsvg2-dev'],
-}
-
-}
diff --git a/modules/mathoid/tests/Makefile b/modules/mathoid/tests/Makefile
deleted file mode 100644
index 76cd656..000
--- a/modules/mathoid/tests/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:   test
-
-test:  $(OBJS)
-
-%.po:  %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/mathoid/tests/mathoid.pp b/modules/mathoid/tests/mathoid.pp
deleted file mode 100644
index 1f030cd..000
--- a/modules/mathoid/tests/mathoid.pp
+++ /dev/null
@@ -1 +0,0 @@
-class { 'mathoid': }
diff --git a/modules/profile/manifests/mathoid.pp 
b/modules/profile/manifests/mathoid.pp
new file mode 100644
index 000..9d4601e
--- /dev/null
+++ b/modules/profile/manifests/mathoid.pp
@@ -0,0 +1,24 @@
+# == Class: mathoid
+#
+# Mathoid is an application which takes various forms of math input and
+# converts it to MathML + SVG output. It is a web-service implemented
+# in node.js.
+#
+class profile::mathoid {
+# NOTE: this is a temporary work-around for the CI to be able to install
+# development packages. In the future, we want to have more integration so 
as to
+# run tests as close to production as possible.
+#
+service::packages { 'mathoid':
+pkgs => ['librsvg2-2'],
+dev_pkgs => ['librsvg2-dev'],
+}
+
+service::node { 'mathoid':
+port  => 10042,
+healthcheck_url   => '',
+has_spec  => true,
+deployment=> 'scap3',
+deployment_config => true,
+}
+}
diff --git a/modules/role/manifests/mathoid.pp 
b/modules/role/manifests/mathoid.pp
index b10f5c4..809acb8 100644
--- a/modules/role/manifests/mathoid.pp
+++ b/modules/role/manifests/mathoid.pp
@@ -6,5 +6,5 @@
 description => 'mathoid server'
 }
 
-include ::mathoid
+include ::profile::mathoid
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377205
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5756122ac1f1363ee7239c43ecb403e8ba1e54c2
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: mobileapps: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377204 )

Change subject: mobileapps: move to role/profile
..


mobileapps: move to role/profile

Change-Id: I579fd89846f54aab6762e709edcc076f4efec948
---
D modules/mobileapps/tests/Makefile
D modules/mobileapps/tests/init.pp
R modules/profile/manifests/mobileapps.pp
M modules/role/manifests/mobileapps.pp
4 files changed, 3 insertions(+), 18 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/mobileapps/tests/Makefile 
b/modules/mobileapps/tests/Makefile
deleted file mode 100644
index 3551657..000
--- a/modules/mobileapps/tests/Makefile
+++ /dev/null
@@ -1,14 +0,0 @@
-# Test automator
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:test
-
-test:   $(OBJS)
-
-%.po:   %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/mobileapps/tests/init.pp b/modules/mobileapps/tests/init.pp
deleted file mode 100644
index 0733291..000
--- a/modules/mobileapps/tests/init.pp
+++ /dev/null
@@ -1 +0,0 @@
-include ::mobileapps
diff --git a/modules/mobileapps/manifests/init.pp 
b/modules/profile/manifests/mobileapps.pp
similarity index 89%
rename from modules/mobileapps/manifests/init.pp
rename to modules/profile/manifests/mobileapps.pp
index 13db326..edc703e 100644
--- a/modules/mobileapps/manifests/init.pp
+++ b/modules/profile/manifests/mobileapps.pp
@@ -1,4 +1,4 @@
-# Class: mobileapps
+# Class: profile::mobileapps
 #
 # This class installs and configures mobileapps
 #
@@ -7,7 +7,7 @@
 # classes as well as conform to a de-facto standard of having a module for 
every
 # service
 #
-class mobileapps() {
+class profile::mobileapps {
 service::node { 'mobileapps':
 port  => ,
 has_spec  => true,
diff --git a/modules/role/manifests/mobileapps.pp 
b/modules/role/manifests/mobileapps.pp
index d1de20f..e553686 100644
--- a/modules/role/manifests/mobileapps.pp
+++ b/modules/role/manifests/mobileapps.pp
@@ -5,5 +5,5 @@
 description => 'A service for use by mobile apps. Provides DOM 
manipulation, aggregation, JSON flattening'
 }
 
-include ::mobileapps
+include ::profile::mobileapps
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377204
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I579fd89846f54aab6762e709edcc076f4efec948
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: BearND 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Mholloway 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: cxserver: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/377206 )

Change subject: cxserver: convert to role/profile
..


cxserver: convert to role/profile

Change-Id: I04872168708db65d316c7fb32bfc4dd6f536f97b
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/cxserver/manifests/init.pp
D modules/cxserver/tests/Makefile
D modules/cxserver/tests/init.pp
A modules/profile/manifests/cxserver.pp
M modules/role/manifests/cxserver.pp
7 files changed, 31 insertions(+), 63 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index e2f9602..26242c7 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -171,7 +171,7 @@
 "role::apertium::apertium_port": 2737
 "role::cxserver::cxserver_port": 8080
 "misc::syslog-server::basepath": /data/project/syslog
-"cxserver::apertium": http://apertium-beta.wmflabs.org
+"profile::cxserver::apertium_uri": http://apertium-beta.wmflabs.org
 role::deployment::mediawiki::key_fingerprint: 
f0:54:06:fa:17:27:97:a2:cc:69:a0:a7:df:4c:0a:e3
 "role::deployment::salt_masters::deployment_server": 
deployment-tin.deployment-prep.eqiad.wmflabs
 "hhvm::extra::fcgi":
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index f2e4eec..e6134b5 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -95,3 +95,6 @@
 # ChangePropagation for JobQueue
 profile::cpjobqueue::redis_path: "/var/run/nutcracker/redis_%{::site}.sock"
 profile::cpjobqueue::jobrunner_host: https://jobrunner.discovery.wmnet
+
+#CXserver
+profile::cxserver::apertium_uri: "http://apertium.svc.%{::site}.wmnet:2737;
diff --git a/modules/cxserver/manifests/init.pp 
b/modules/cxserver/manifests/init.pp
deleted file mode 100644
index 206214a..000
--- a/modules/cxserver/manifests/init.pp
+++ /dev/null
@@ -1,36 +0,0 @@
-# == Class: cxserver
-#
-# cxserver is a node.js backend for the Content Translation tool.
-# https://www.mediawiki.org/wiki/Content_translation
-#
-# === Parameters
-#
-# [*apertium*]
-#   Url to Apertium service.
-# [*yandex_api_key*]
-#   API key for Yandex service.
-# [*youdao_api_key*]
-#   API key for Youdao service.
-# [*jwt_secret*]
-#   JWT secret token.
-class cxserver(
-$apertium = "http://apertium.svc.${::site}.wmnet:2737;,
-$yandex_api_key = undef,
-$youdao_api_key = undef,
-$jwt_secret = undef,
-) {
-
-service::node { 'cxserver':
-port  => 8080,
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-deployment_config => true,
-deployment_vars   => {
-jwt_token=> $jwt_secret,
-apertium_uri => $apertium,
-yandex_key   => $yandex_api_key,
-youdao_key   => $youdao_api_key,
-},
-}
-}
diff --git a/modules/cxserver/tests/Makefile b/modules/cxserver/tests/Makefile
deleted file mode 100644
index 3551657..000
--- a/modules/cxserver/tests/Makefile
+++ /dev/null
@@ -1,14 +0,0 @@
-# Test automator
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:test
-
-test:   $(OBJS)
-
-%.po:   %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/cxserver/tests/init.pp b/modules/cxserver/tests/init.pp
deleted file mode 100644
index c17a7b0..000
--- a/modules/cxserver/tests/init.pp
+++ /dev/null
@@ -1 +0,0 @@
-include ::cxserver
diff --git a/modules/profile/manifests/cxserver.pp 
b/modules/profile/manifests/cxserver.pp
new file mode 100644
index 000..a778d0c
--- /dev/null
+++ b/modules/profile/manifests/cxserver.pp
@@ -0,0 +1,26 @@
+# vim: set ts=4 et sw=4:
+#
+# filtertags: labs-project-deployment-prep
+
+class profile::cxserver(
+$apertium_uri=hiera('profile::cxserver::apertium_uri')
+) {
+include ::passwords::cxserver
+$yandex_api_key = $::passwords::cxserver::yandex_api_key
+$youdao_api_key = $::passwords::cxserver::youdao_api_key
+$jwt_secret = $::passwords::cxserver::jwt_secret
+
+service::node { 'cxserver':
+port  => 8080,
+healthcheck_url   => '',
+has_spec  => true,
+deployment=> 'scap3',
+deployment_config => true,
+deployment_vars   => {
+jwt_token=> $jwt_secret,
+apertium_uri => $apertium_uri,
+yandex_key   => $yandex_api_key,
+youdao_key   => $youdao_api_key,
+},
+}
+}
diff --git a/modules/role/manifests/cxserver.pp 
b/modules/role/manifests/cxserver.pp
index 6b877ce..c4441f1 100644
--- 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: cxserver: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377206 )

Change subject: cxserver: convert to role/profile
..

cxserver: convert to role/profile

Change-Id: I04872168708db65d316c7fb32bfc4dd6f536f97b
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/cxserver/manifests/init.pp
D modules/cxserver/tests/Makefile
D modules/cxserver/tests/init.pp
A modules/profile/manifests/cxserver.pp
M modules/role/manifests/cxserver.pp
7 files changed, 31 insertions(+), 63 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/06/377206/1

diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index e2f9602..26242c7 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -171,7 +171,7 @@
 "role::apertium::apertium_port": 2737
 "role::cxserver::cxserver_port": 8080
 "misc::syslog-server::basepath": /data/project/syslog
-"cxserver::apertium": http://apertium-beta.wmflabs.org
+"profile::cxserver::apertium_uri": http://apertium-beta.wmflabs.org
 role::deployment::mediawiki::key_fingerprint: 
f0:54:06:fa:17:27:97:a2:cc:69:a0:a7:df:4c:0a:e3
 "role::deployment::salt_masters::deployment_server": 
deployment-tin.deployment-prep.eqiad.wmflabs
 "hhvm::extra::fcgi":
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index f2e4eec..e6134b5 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -95,3 +95,6 @@
 # ChangePropagation for JobQueue
 profile::cpjobqueue::redis_path: "/var/run/nutcracker/redis_%{::site}.sock"
 profile::cpjobqueue::jobrunner_host: https://jobrunner.discovery.wmnet
+
+#CXserver
+profile::cxserver::apertium_uri: "http://apertium.svc.%{::site}.wmnet:2737;
diff --git a/modules/cxserver/manifests/init.pp 
b/modules/cxserver/manifests/init.pp
deleted file mode 100644
index 206214a..000
--- a/modules/cxserver/manifests/init.pp
+++ /dev/null
@@ -1,36 +0,0 @@
-# == Class: cxserver
-#
-# cxserver is a node.js backend for the Content Translation tool.
-# https://www.mediawiki.org/wiki/Content_translation
-#
-# === Parameters
-#
-# [*apertium*]
-#   Url to Apertium service.
-# [*yandex_api_key*]
-#   API key for Yandex service.
-# [*youdao_api_key*]
-#   API key for Youdao service.
-# [*jwt_secret*]
-#   JWT secret token.
-class cxserver(
-$apertium = "http://apertium.svc.${::site}.wmnet:2737;,
-$yandex_api_key = undef,
-$youdao_api_key = undef,
-$jwt_secret = undef,
-) {
-
-service::node { 'cxserver':
-port  => 8080,
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-deployment_config => true,
-deployment_vars   => {
-jwt_token=> $jwt_secret,
-apertium_uri => $apertium,
-yandex_key   => $yandex_api_key,
-youdao_key   => $youdao_api_key,
-},
-}
-}
diff --git a/modules/cxserver/tests/Makefile b/modules/cxserver/tests/Makefile
deleted file mode 100644
index 3551657..000
--- a/modules/cxserver/tests/Makefile
+++ /dev/null
@@ -1,14 +0,0 @@
-# Test automator
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:test
-
-test:   $(OBJS)
-
-%.po:   %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/cxserver/tests/init.pp b/modules/cxserver/tests/init.pp
deleted file mode 100644
index c17a7b0..000
--- a/modules/cxserver/tests/init.pp
+++ /dev/null
@@ -1 +0,0 @@
-include ::cxserver
diff --git a/modules/profile/manifests/cxserver.pp 
b/modules/profile/manifests/cxserver.pp
new file mode 100644
index 000..a778d0c
--- /dev/null
+++ b/modules/profile/manifests/cxserver.pp
@@ -0,0 +1,26 @@
+# vim: set ts=4 et sw=4:
+#
+# filtertags: labs-project-deployment-prep
+
+class profile::cxserver(
+$apertium_uri=hiera('profile::cxserver::apertium_uri')
+) {
+include ::passwords::cxserver
+$yandex_api_key = $::passwords::cxserver::yandex_api_key
+$youdao_api_key = $::passwords::cxserver::youdao_api_key
+$jwt_secret = $::passwords::cxserver::jwt_secret
+
+service::node { 'cxserver':
+port  => 8080,
+healthcheck_url   => '',
+has_spec  => true,
+deployment=> 'scap3',
+deployment_config => true,
+deployment_vars   => {
+jwt_token=> $jwt_secret,
+apertium_uri => $apertium_uri,
+yandex_key   => $yandex_api_key,
+youdao_key   => $youdao_api_key,
+},
+}
+}
diff --git a/modules/role/manifests/cxserver.pp 
b/modules/role/manifests/cxserver.pp
index 6b877ce..c4441f1 100644
--- 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: apertium: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377207 )

Change subject: apertium: move to role/profile
..

apertium: move to role/profile

Change-Id: I65f9890bad802dedee36296bf1b4bbfe309d59e1
---
M hieradata/labs/deployment-prep/common.yaml
D modules/apertium/tests/Makefile
D modules/apertium/tests/init.pp
R modules/profile/manifests/apertium.pp
R modules/profile/templates/initscripts/apertium-apy.systemd.erb
R modules/profile/templates/initscripts/apertium-apy.upstart.erb
M modules/role/manifests/apertium.pp
7 files changed, 26 insertions(+), 45 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/07/377207/1

diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 26242c7..7a35d08 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -168,7 +168,6 @@
   explicit_macs: false
   disable_agent_forwarding: false
   challenge_response_auth: false
-"role::apertium::apertium_port": 2737
 "role::cxserver::cxserver_port": 8080
 "misc::syslog-server::basepath": /data/project/syslog
 "profile::cxserver::apertium_uri": http://apertium-beta.wmflabs.org
diff --git a/modules/apertium/tests/Makefile b/modules/apertium/tests/Makefile
deleted file mode 100644
index 76cd656..000
--- a/modules/apertium/tests/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:   test
-
-test:  $(OBJS)
-
-%.po:  %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/apertium/tests/init.pp b/modules/apertium/tests/init.pp
deleted file mode 100644
index b35284d..000
--- a/modules/apertium/tests/init.pp
+++ /dev/null
@@ -1 +0,0 @@
-include ::apertium
diff --git a/modules/apertium/manifests/init.pp 
b/modules/profile/manifests/apertium.pp
similarity index 83%
rename from modules/apertium/manifests/init.pp
rename to modules/profile/manifests/apertium.pp
index 0d97f3e..4345fb1 100644
--- a/modules/apertium/manifests/init.pp
+++ b/modules/profile/manifests/apertium.pp
@@ -6,21 +6,21 @@
 # === Parameters
 #
 # [*num_of_processes*]
-#   Number of APY instance processes to run.
-# [*max_idle_seconds*]
-#   Seconds to wait before shutdown idle process.
-# [*uid*]
-#   The username apertium-apy will run with.
-# [*gid*]
-#   The group apertium-apy will run with.
-class apertium(
-$num_of_processes = 1,
-$max_idle_seconds = 300,
-$uid = 'apertium',
-$gid = 'apertium',
-) {
 
-include ::service::configuration
+# [*max_idle_seconds*]
+#
+class profile::apertium {
+require ::service::configuration
+
+# Port we're listening on
+$port = 2737
+# Number of APY instance processes to run.
+$num_of_processes = 1
+# Seconds to wait before shutting down an idle process.
+$max_idle_seconds = 300
+# User and group
+$uid = 'apertium'
+$gid = 'apertium'
 
 $log_dir = "${::service::configuration::log_dir}/apertium"
 
@@ -134,4 +134,14 @@
 not_if_empty  => true,
 rotate=> 15,
 }
+
+ferm::service { 'apertium_http':
+proto => 'tcp',
+port  => $port,
+}
+
+monitoring::service { 'apertium':
+description   => 'apertium apy',
+check_command => 
"check_http_hostheader_port_url!apertium.svc.${::site}.wmnet!${port}!/listPairs",
+}
 }
diff --git a/modules/apertium/templates/initscripts/apertium-apy.systemd.erb 
b/modules/profile/templates/initscripts/apertium-apy.systemd.erb
similarity index 100%
rename from modules/apertium/templates/initscripts/apertium-apy.systemd.erb
rename to modules/profile/templates/initscripts/apertium-apy.systemd.erb
diff --git a/modules/apertium/templates/initscripts/apertium-apy.upstart.erb 
b/modules/profile/templates/initscripts/apertium-apy.upstart.erb
similarity index 100%
rename from modules/apertium/templates/initscripts/apertium-apy.upstart.erb
rename to modules/profile/templates/initscripts/apertium-apy.upstart.erb
diff --git a/modules/role/manifests/apertium.pp 
b/modules/role/manifests/apertium.pp
index 613bb40..55671d4 100644
--- a/modules/role/manifests/apertium.pp
+++ b/modules/role/manifests/apertium.pp
@@ -1,23 +1,9 @@
 # vim: set ts=4 et sw=4:
 #
 # filtertags: labs-project-deployment-prep
-class role::apertium(
-$port = '2737',
-) {
+class role::apertium {
 system::role { 'apertium':
 description => 'Apertium APY server'
 }
-
-include ::apertium
-
-ferm::service { 'apertium_http':
-proto => 'tcp',
-port  => $port,
-}
-
-monitoring::service { 'apertium':
-description   => 'apertium apy',
-check_command => 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: mathoid: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377205 )

Change subject: mathoid: move to role/profile
..

mathoid: move to role/profile

Change-Id: I5756122ac1f1363ee7239c43ecb403e8ba1e54c2
---
D modules/mathoid/manifests/init.pp
D modules/mathoid/manifests/packages.pp
D modules/mathoid/tests/Makefile
D modules/mathoid/tests/mathoid.pp
A modules/profile/manifests/mathoid.pp
M modules/role/manifests/mathoid.pp
6 files changed, 25 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/05/377205/1

diff --git a/modules/mathoid/manifests/init.pp 
b/modules/mathoid/manifests/init.pp
deleted file mode 100644
index 5f714b0..000
--- a/modules/mathoid/manifests/init.pp
+++ /dev/null
@@ -1,18 +0,0 @@
-# == Class: mathoid
-#
-# Mathoid is an application which takes various forms of math input and
-# converts it to MathML + SVG output. It is a web-service implemented
-# in node.js.
-#
-class mathoid {
-
-require ::mathoid::packages
-
-service::node { 'mathoid':
-port  => 10042,
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-deployment_config => true,
-}
-}
diff --git a/modules/mathoid/manifests/packages.pp 
b/modules/mathoid/manifests/packages.pp
deleted file mode 100644
index 529d3a3..000
--- a/modules/mathoid/manifests/packages.pp
+++ /dev/null
@@ -1,16 +0,0 @@
-# == Class: mathoid::packages
-#
-# Installs the packages needed by Mathoid
-#
-# NOTE: this is a temporary work-around for the CI to be able to install
-# development packages. In the future, we want to have more integration so as 
to
-# run tests as close to production as possible.
-#
-class mathoid::packages {
-
-service::packages { 'mathoid':
-pkgs => ['librsvg2-2'],
-dev_pkgs => ['librsvg2-dev'],
-}
-
-}
diff --git a/modules/mathoid/tests/Makefile b/modules/mathoid/tests/Makefile
deleted file mode 100644
index 76cd656..000
--- a/modules/mathoid/tests/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:   test
-
-test:  $(OBJS)
-
-%.po:  %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/mathoid/tests/mathoid.pp b/modules/mathoid/tests/mathoid.pp
deleted file mode 100644
index 1f030cd..000
--- a/modules/mathoid/tests/mathoid.pp
+++ /dev/null
@@ -1 +0,0 @@
-class { 'mathoid': }
diff --git a/modules/profile/manifests/mathoid.pp 
b/modules/profile/manifests/mathoid.pp
new file mode 100644
index 000..9d4601e
--- /dev/null
+++ b/modules/profile/manifests/mathoid.pp
@@ -0,0 +1,24 @@
+# == Class: mathoid
+#
+# Mathoid is an application which takes various forms of math input and
+# converts it to MathML + SVG output. It is a web-service implemented
+# in node.js.
+#
+class profile::mathoid {
+# NOTE: this is a temporary work-around for the CI to be able to install
+# development packages. In the future, we want to have more integration so 
as to
+# run tests as close to production as possible.
+#
+service::packages { 'mathoid':
+pkgs => ['librsvg2-2'],
+dev_pkgs => ['librsvg2-dev'],
+}
+
+service::node { 'mathoid':
+port  => 10042,
+healthcheck_url   => '',
+has_spec  => true,
+deployment=> 'scap3',
+deployment_config => true,
+}
+}
diff --git a/modules/role/manifests/mathoid.pp 
b/modules/role/manifests/mathoid.pp
index b10f5c4..809acb8 100644
--- a/modules/role/manifests/mathoid.pp
+++ b/modules/role/manifests/mathoid.pp
@@ -6,5 +6,5 @@
 description => 'mathoid server'
 }
 
-include ::mathoid
+include ::profile::mathoid
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377205
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5756122ac1f1363ee7239c43ecb403e8ba1e54c2
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: pdfrender: switch to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377210 )

Change subject: pdfrender: switch to role/profile
..

pdfrender: switch to role/profile

In this case, since it's not incapsulating service::node and collects
quite a few resources, we prefer not to remove the named class.

Change-Id: I735701d4ea4069c307b328d8fcb5fab782aad165
---
A modules/profile/manifests/pdfrender.pp
M modules/role/manifests/pdfrender.pp
2 files changed, 20 insertions(+), 16 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/10/377210/1

diff --git a/modules/profile/manifests/pdfrender.pp 
b/modules/profile/manifests/pdfrender.pp
new file mode 100644
index 000..28f5c7f
--- /dev/null
+++ b/modules/profile/manifests/pdfrender.pp
@@ -0,0 +1,19 @@
+class profile::pdfrender(
+$is_active = hiera('profile::pdfrender::is_active', true)
+) {
+
+$port = 5252
+
+class { '::pdfrender':
+port=> $port,
+no_browsers => 4,
+running => $is_active,
+}
+
+ferm::service { "pdfrender_http_${port}":
+proto  => 'tcp',
+port   => $port,
+srange => '$DOMAIN_NETWORKS',
+}
+
+}
diff --git a/modules/role/manifests/pdfrender.pp 
b/modules/role/manifests/pdfrender.pp
index 657def0c..42f99e2 100644
--- a/modules/role/manifests/pdfrender.pp
+++ b/modules/role/manifests/pdfrender.pp
@@ -1,22 +1,7 @@
 class role::pdfrender {
-$is_active = hiera('role::pdfrender::is_active', true)
-
 system::role { 'pdfrender':
 description => 'A PDF render service based on Electron',
 }
 
-$port = 5252
-
-class { '::pdfrender':
-port=> $port,
-no_browsers => 4,
-running => $is_active,
-}
-
-ferm::service { "pdfrender_http_${port}":
-proto  => 'tcp',
-port   => $port,
-srange => '$DOMAIN_NETWORKS',
-}
-
+include ::profile::pdfrender
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377210
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I735701d4ea4069c307b328d8fcb5fab782aad165
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: trendingedits: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377208 )

Change subject: trendingedits: move to role/profile
..

trendingedits: move to role/profile

Change-Id: Ia0a2248c7282583a8c2fcc87e94d86fd749c9c33
---
A modules/profile/manifests/trendingedits.pp
M modules/role/manifests/trendingedits.pp
D modules/trendingedits/manifests/init.pp
D modules/trendingedits/manifests/packages.pp
4 files changed, 34 insertions(+), 64 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/08/377208/1

diff --git a/modules/profile/manifests/trendingedits.pp 
b/modules/profile/manifests/trendingedits.pp
new file mode 100644
index 000..066ea0b
--- /dev/null
+++ b/modules/profile/manifests/trendingedits.pp
@@ -0,0 +1,33 @@
+# Profile class for trendingedits
+class profile::trendingedits {
+
+$kafka_config = kafka_config('main')
+$port = 6699
+
+service::packages { 'trendingedits':
+pkgs => ['librdkafka++1', 'librdkafka1'],
+dev_pkgs => ['librdkafka-dev'],
+}
+
+service::node { 'trendingedits':
+port  => $port,
+repo  => 'trending-edits/deploy',
+healthcheck_url   => '',
+has_spec  => true,
+deployment=> 'scap3',
+deployment_config => true,
+deployment_vars   => {
+broker_list => $kafka_config['brokers']['string'],
+site=> $::site,
+},
+environment   => {
+'UV_THREADPOOL_SIZE' => 16
+},
+}
+
+class { '::trendingedits':
+port=> $port,
+broker_list =>
+}
+
+}
diff --git a/modules/role/manifests/trendingedits.pp 
b/modules/role/manifests/trendingedits.pp
index 7716f0c..90a1205 100644
--- a/modules/role/manifests/trendingedits.pp
+++ b/modules/role/manifests/trendingedits.pp
@@ -1,17 +1,8 @@
 # Role class for trendingedits
 class role::trendingedits {
-
-$kafka_config = kafka_config('main')
-$port = 6699
-
 system::role { 'trendingedits':
 description => 'computes the list of currently-trending articles',
 }
 
-class { '::trendingedits':
-port=> $port,
-broker_list => $kafka_config['brokers']['string'],
-}
-
+include ::profile::trendingedits
 }
-
diff --git a/modules/trendingedits/manifests/init.pp 
b/modules/trendingedits/manifests/init.pp
deleted file mode 100644
index 075cf75..000
--- a/modules/trendingedits/manifests/init.pp
+++ /dev/null
@@ -1,38 +0,0 @@
-# == Class: trendingedits
-#
-# This class installs and configures the trending edits service, which follows
-# events from the EventBus system in real time and computes the list of
-# currently-trending articles based on the number of edits.
-#
-# === Parameters
-#
-# [*port*]
-#   The port to bind the service to
-#
-# [*broker_list*]
-#   Comma-separated list of Kafka broker URIs
-#
-class trendingedits(
-$port,
-$broker_list,
-) {
-
-require ::trendingedits::packages
-
-service::node { 'trendingedits':
-port  => $port,
-repo  => 'trending-edits/deploy',
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-deployment_config => true,
-deployment_vars   => {
-broker_list => $broker_list,
-site=> $::site,
-},
-environment   => {
-'UV_THREADPOOL_SIZE' => 16
-},
-}
-
-}
diff --git a/modules/trendingedits/manifests/packages.pp 
b/modules/trendingedits/manifests/packages.pp
deleted file mode 100644
index 71b4530..000
--- a/modules/trendingedits/manifests/packages.pp
+++ /dev/null
@@ -1,16 +0,0 @@
-# == Class: trendingedits::packages
-#
-# Installs the packages needed by the trending edits service
-#
-# NOTE: this is a temporary work-around for the CI to be able to install
-# development packages. In the future, we want to have more integration so as 
to
-# run tests as close to production as possible.
-#
-class trendingedits::packages {
-
-  service::packages { 'trendingedits':
-pkgs => ['librdkafka++1', 'librdkafka1'],
-dev_pkgs => ['librdkafka-dev'],
-  }
-
-}

-- 
To view, visit https://gerrit.wikimedia.org/r/377208
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0a2248c7282583a8c2fcc87e94d86fd749c9c33
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: changeprop: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377211 )

Change subject: changeprop: convert to role/profile
..

changeprop: convert to role/profile

Change-Id: I191a4f46aaffa13bca21bfbb3a00292ef10f46b1
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/changeprop/manifests/init.pp
D modules/changeprop/manifests/packages.pp
D modules/changeprop/tests/Makefile
D modules/changeprop/tests/init.pp
A modules/profile/manifests/changeprop.pp
M modules/role/manifests/changeprop.pp
8 files changed, 61 insertions(+), 131 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/11/377211/1

diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index a1fbef7..6100459 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -16,10 +16,10 @@
 service::configuration::logstash_host: 
deployment-logstash2.deployment-prep.eqiad.wmflabs
 citoid::zotero_host: deployment-zotero01.deployment-prep.eqiad.wmflabs
 citoid::zotero_port: 1969
-changeprop::purge_host: deployment-cache-text04.deployment-prep.eqiad.wmflabs
-changeprop::ores_uris: 
['http://deployment-sca03.deployment-prep.eqiad.wmflabs:8081']
-# Need to redefine RESTBase URI as Change-Prop redefines it in order to 
provide a different value for async updates
-changeprop::restbase_uri: 
http://deployment-restbase02.deployment-prep.eqiad.wmflabs:7231
+profile::changeprop::purge_host: 
deployment-cache-text04.deployment-prep.eqiad.wmflabs
+profile::changeprop::ores_uris: 
['http://deployment-sca03.deployment-prep.eqiad.wmflabs:8081']
+profile::changeprop::restbase_uri: 
http://deployment-restbase02.deployment-prep.eqiad.wmflabs:7231
+profile::changeprop::purge_port: 4827
 # Used to sync the setting between all Kafka clusters and clients.
 kafka_message_max_bytes: 4194304
 graphoid::allowed_domains:
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index df1763b..bf4a56b 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -89,7 +89,13 @@
   broker.version.fallback: '0.9.0.1'
 ### END EVENTSTREAMS ###
 
-changeprop::restbase_uri: http://restbase-async.discovery.wmnet:7231
+### BEGIN CHANGEPROP ###
+profile::changeprop::restbase_uri: http://restbase-async.discovery.wmnet:7231
+profile::changeprop::purge_host: 239.128.0.112
+profile::changeprop::purge_port: 4827
+profile::changeprop::ores_uris:
+  - http://ores.svc.eqiad.wmnet:8081
+  - http://ores.svc.codfw.wmnet:8081
 profile::nutcracker::memcached_pools: {}
 profile::nutcracker::monitor_port: 0 # we have nothing exposed via tcp
 
diff --git a/modules/changeprop/manifests/init.pp 
b/modules/changeprop/manifests/init.pp
deleted file mode 100644
index 53e31e9..000
--- a/modules/changeprop/manifests/init.pp
+++ /dev/null
@@ -1,83 +0,0 @@
-# == Class: changeprop
-#
-# This class installs and configures the change propagation service, a part of
-# the EventBus system responsible for reacting to events received via Kafka and
-# dispatching the appropriate requests.
-#
-# === Parameters
-#
-# [*broker_list*]
-#   Comma-separated list of Kafka broker URIs
-#
-# [*purge_host*]
-#   The vhtcpd daemon host to send purge requests to. Default: 239.128.0.112
-#
-# [*purge_port*]
-#   The port the vhtcp daemon listens to. Default: 4827
-#
-# [*restbase_uri*]
-#   RESTBase's URI. Note that this is redefined here so that async update
-#   requests can be sent to the inactive DC. Default:
-#   'http://restbase.svc.eqiad.wmnet:7231'
-#
-# [*ores_uris*]
-#   A list of urls for the ORES service. Defaults to:
-#   [http://ores.svc.eqiad.wmnet:8081, http://ores.svc.codfw.wmnet:8081]
-#
-# [*redis_path*]
-#   The UNIX socket file path of the Redis/Nutcracker server. Default:
-#   "/var/run/nutcracker/redis_${::site}.sock"
-#
-# [*redis_pass*]
-#   The password to use when authenticating with Redis/Nutcracker. Default:
-#   'abc1234'
-#
-# [*kafka_msg_max_bytes*]
-#   The maximum number of bytes allowed in a Kafka message. Default:
-#   '1048576'
-#
-class changeprop(
-$broker_list,
-$purge_host  = '239.128.0.112',
-$purge_port  = 4827,
-$restbase_uri= 'http://restbase.svc.eqiad.wmnet:7231',
-$ores_uris   = [
-'http://ores.svc.eqiad.wmnet:8081',
-'http://ores.svc.codfw.wmnet:8081',
-],
-$redis_path  = "/var/run/nutcracker/redis_${::site}.sock",
-$redis_pass  = 'abc1234',
-$kafka_msg_max_bytes = 1048576,
-) {
-
-include ::service::configuration
-
-require ::changeprop::packages
-
-service::node { 'changeprop':
-enable=> true,
-port  => 7272,
-healthcheck_url   => '',
-has_spec  => true,
-deployment=> 'scap3',
-

[MediaWiki-commits] [Gerrit] operations/puppet[production]: eventstreams: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377209 )

Change subject: eventstreams: convert to role/profile
..

eventstreams: convert to role/profile

Change-Id: I94dad66103ce344a04b85a581203ecc5540052d0
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/eventstreams/manifests/init.pp
A modules/profile/manifests/eventstreams.pp
M modules/role/manifests/eventstreams.pp
5 files changed, 84 insertions(+), 128 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/09/377209/1

diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 7a35d08..a1fbef7 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -341,3 +341,14 @@
 prometheus_nodes:
   - deployment-prometheus01.deployment-prep.eqiad.wmflabs
 profile::recommendation_api::wdqs_uri: http://wdqs-test.wmflabs.org
+
+# Eventstreams config
+profile::eventstreams::kafka_cluster_name: main
+profile::eventstreams::streams:
+  test:
+topics: ["%{::site}.test.event"]
+  revision-create:
+topics: ["%{::site}.mediawiki.revision-create"]
+  recentchange:
+topics: ["%{::site}.mediawiki.recentchange"]
+profile::eventstreams::rdkafka_config: {}
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index e6134b5..df1763b 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -56,9 +56,12 @@
 # The password is in the private store, this is here for completeness
 # profile::ores::web::redis_password: nothing
 
+
+### BEGIN EVENTSTREAMS ###
+profile::eventstreams::kafka_cluster_name: analytics
 # Stream configuration for Public EventStreams service
 # Maps stream route names to composite Kafka topics.
-role::eventstreams::streams:
+profile::eventstreams::streams:
   recentchange:
 description: "Mediawiki RecentChanges feed. Schema: 
https://github.com/wikimedia/mediawiki-event-schemas/tree/master/jsonschema/mediawiki/recentchange;
 topics:
@@ -76,7 +79,7 @@
   - codfw.test.event
 
 # rdkafka config for Public EventStreams service.
-role::eventstreams::rdkafka_config:
+profile::eventstreams::rdkafka_config:
   # Send rdkafka stats to statsd once per minute.
   statistics.interval.ms: 6
   # Specify Kafka API version as workaround for Brokers < 0.10
@@ -84,6 +87,7 @@
   # This will not be necessary when the target Kafka cluster will be running 
0.10
   # librdkafka 0.9.4.x default for api.version.request is false, no need to 
set it
   broker.version.fallback: '0.9.0.1'
+### END EVENTSTREAMS ###
 
 changeprop::restbase_uri: http://restbase-async.discovery.wmnet:7231
 profile::nutcracker::memcached_pools: {}
diff --git a/modules/eventstreams/manifests/init.pp 
b/modules/eventstreams/manifests/init.pp
deleted file mode 100644
index 011dec8..000
--- a/modules/eventstreams/manifests/init.pp
+++ /dev/null
@@ -1,57 +0,0 @@
-# == Class: eventstreams
-#
-# === Parameters
-#
-# [*broker_list*]
-#   Comma-separated list of Kafka broker URIs
-#
-# [*streams*]
-#   Hash of stream route config and their composite topics. E.g.
-#
-#   streamName1:
-#   topics: [topicA, topicB]
-#   streamName2:
-#   topics: [topicC, topicD]
-#
-# [*port*]
-#   Default: 8092
-#
-# [*log_level*]
-#   Log level for service logger. Default: info
-#
-# [*rdkafka_config*]
-#   Extra librdkafka configuration to provide to node-rdkafka.  Default: {}
-#
-class eventstreams(
-$broker_list,
-$streams,
-$port   = 8092,
-$log_level  = 'info',
-$rdkafka_config = {},
-) {
-service::packages { 'eventstreams':
-pkgs => ['librdkafka++1', 'librdkafka1'],
-}
-
-service::node { 'eventstreams':
-enable=> true,
-port  => $port,
-has_spec  => false, # TODO: figure out how to monitor stream 
with spec x-amples
-deployment=> 'scap3',
-deployment_config => true,
-deployment_vars   => {
-log_level  => $log_level,
-site   => $::site,
-broker_list=> $broker_list,
-rdkafka_config => $rdkafka_config,
-streams=> $streams,
-},
-auto_refresh  => false,
-init_restart  => false,
-environment   => {
-'UV_THREADPOOL_SIZE' => 128,
-},
-require   => Service::Packages['eventstreams'],
-}
-
-}
diff --git a/modules/profile/manifests/eventstreams.pp 
b/modules/profile/manifests/eventstreams.pp
new file mode 100644
index 000..c535922
--- /dev/null
+++ b/modules/profile/manifests/eventstreams.pp
@@ -0,0 +1,66 @@
+# == Class profile::eventstreams
+#
+# Profile that installs EventStreams HTTP service.
+# This class includes the ::eventstreams role, and configures

[MediaWiki-commits] [Gerrit] operations/puppet[production]: graphoid: convert to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377212 )

Change subject: graphoid: convert to role/profile
..

graphoid: convert to role/profile

Change-Id: Ib15aaa3d86c825edb3d672331bcaf113852fb10c
---
M hieradata/labs/deployment-prep/common.yaml
M hieradata/role/common/scb.yaml
D modules/graphoid/manifests/packages.pp
R modules/profile/manifests/graphoid.pp
M modules/role/manifests/graphoid.pp
5 files changed, 29 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/12/377212/1

diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 6100459..9cb9e2b 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -22,7 +22,7 @@
 profile::changeprop::purge_port: 4827
 # Used to sync the setting between all Kafka clusters and clients.
 kafka_message_max_bytes: 4194304
-graphoid::allowed_domains:
+profile::graphoid::allowed_domains:
   http:
 - wmflabs.org
   https:
@@ -47,9 +47,9 @@
 - wdqs-test.wmflabs.org
   geoshape:
 - maps.wikimedia.org
-graphoid::headers:
+profile::graphoid::headers:
   'Cache-Control': 'public, s-maxage=360, max-age=360'
-graphoid::error_headers:
+profile::graphoid::error_headers:
   'Cache-Control': 'public, s-maxage=30, max-age=30'
 lvs::configuration::lvs_services:
   apaches:
diff --git a/hieradata/role/common/scb.yaml b/hieradata/role/common/scb.yaml
index bf4a56b..5553030 100644
--- a/hieradata/role/common/scb.yaml
+++ b/hieradata/role/common/scb.yaml
@@ -26,7 +26,9 @@
   pdfrender: {}
   trendingedits: {}
   "recommendation-api": {}
-graphoid::allowed_domains:
+
+### BEGIN GRAPHOID
+profile::graphoid::allowed_domains:
   https:
 - mediawiki.org
 - wikibooks.org
@@ -46,10 +48,12 @@
 - query.wikidata.org
   geoshape:
 - maps.wikimedia.org
-graphoid::headers:
+profile::graphoid::headers:
   'Cache-Control': 'public, s-maxage=3600, max-age=3600'
-graphoid::error_headers:
+profile::graphoid::error_headers:
   'Cache-Control': 'public, s-maxage=300, max-age=300'
+### END GRAPHOID ###
+
 citoid::zotero_port: 1969
 citoid::zotero_host: "zotero.svc.%{::site}.wmnet"
 profile::ores::web::redis_host: "oresrdb.svc.%{::site}.wmnet"
@@ -96,6 +100,7 @@
 profile::changeprop::ores_uris:
   - http://ores.svc.eqiad.wmnet:8081
   - http://ores.svc.codfw.wmnet:8081
+### END CHANGEPROP ###
 profile::nutcracker::memcached_pools: {}
 profile::nutcracker::monitor_port: 0 # we have nothing exposed via tcp
 
diff --git a/modules/graphoid/manifests/packages.pp 
b/modules/graphoid/manifests/packages.pp
deleted file mode 100644
index 05686a4..000
--- a/modules/graphoid/manifests/packages.pp
+++ /dev/null
@@ -1,19 +0,0 @@
-# == Class: graphoid::packages
-#
-# Installs the packages needed by graphoid
-#
-# NOTE: this is a temporary work-around for the CI to be able to install
-# development packages. In the future, we want to have more integration so as 
to
-# run tests as close to production as possible.
-#
-class graphoid::packages {
-
-require ::mediawiki::packages::fonts
-
-service::packages { 'graphoid':
-pkgs => ['libcairo2', 'libgif4', 'libjpeg62-turbo', 
'libpango1.0-0'],
-dev_pkgs => ['libcairo2-dev', 'libgif-dev', 'libpango1.0-dev',
-'libjpeg62-turbo-dev'],
-}
-
-}
diff --git a/modules/graphoid/manifests/init.pp 
b/modules/profile/manifests/graphoid.pp
similarity index 65%
rename from modules/graphoid/manifests/init.pp
rename to modules/profile/manifests/graphoid.pp
index 81f4af8..6f111cf 100644
--- a/modules/graphoid/manifests/init.pp
+++ b/modules/profile/manifests/graphoid.pp
@@ -1,4 +1,4 @@
-# == Class: graphoid
+# == Class: profile::graphoid
 #
 # This class installs and configures graphoid, a node.js service that
 # converts a graph definition into a PNG image
@@ -8,32 +8,31 @@
 # [*allowed_domains*]
 #   The protocol-to-list-of-domains map. Default: {}
 #   The protocols include http, https, as well as some custom graph-specific 
protocols.
-#   See 
https://www.mediawiki.org/wiki/Extension:Graph?venotify=restored#External_data
-#
-# [*domain_map*]
-#   The domain-to-domain alias map. Default: {}
-#
-# [*timeout*]
-#   The timeout (in ms) for requests. Default: 5000
-#
+#   See 
https://www.mediawiki.org/wiki/Extension:Graph?venotify=restored#External_data#
 # [*headers*]
 #   A map of headers that will be sent with each reply. Could be used for 
caching, etc. Default: false
 #
 # [*error_headers*]
 #   A map of headers that will be sent with each reply in case of an error. If 
not set, above headers will be used. Default: false
 #
-class graphoid(
-$allowed_domains = {},
-$domain_map= {},
-$timeout   = 5000,
-$headers   = false,
-$error_headers = false,
+class profile::graphoid(
+$allowed_domains = 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: mobileapps: move to role/profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377204 )

Change subject: mobileapps: move to role/profile
..

mobileapps: move to role/profile

Change-Id: I579fd89846f54aab6762e709edcc076f4efec948
---
D modules/mobileapps/tests/Makefile
D modules/mobileapps/tests/init.pp
R modules/profile/manifests/mobileapps.pp
M modules/role/manifests/mobileapps.pp
4 files changed, 3 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/04/377204/1

diff --git a/modules/mobileapps/tests/Makefile 
b/modules/mobileapps/tests/Makefile
deleted file mode 100644
index 3551657..000
--- a/modules/mobileapps/tests/Makefile
+++ /dev/null
@@ -1,14 +0,0 @@
-# Test automator
-MANIFESTS=$(wildcard *.pp)
-OBJS=$(MANIFESTS:.pp=.po)
-TESTS_DIR=$(dir $(CURDIR))
-MODULE_DIR=$(TESTS_DIR:/=)
-MODULES_DIR=$(dir $(MODULE_DIR))
-
-all:test
-
-test:   $(OBJS)
-
-%.po:   %.pp
-   puppet parser validate $<
-   puppet apply --noop --modulepath $(MODULES_DIR) $<
diff --git a/modules/mobileapps/tests/init.pp b/modules/mobileapps/tests/init.pp
deleted file mode 100644
index 0733291..000
--- a/modules/mobileapps/tests/init.pp
+++ /dev/null
@@ -1 +0,0 @@
-include ::mobileapps
diff --git a/modules/mobileapps/manifests/init.pp 
b/modules/profile/manifests/mobileapps.pp
similarity index 89%
rename from modules/mobileapps/manifests/init.pp
rename to modules/profile/manifests/mobileapps.pp
index 13db326..edc703e 100644
--- a/modules/mobileapps/manifests/init.pp
+++ b/modules/profile/manifests/mobileapps.pp
@@ -1,4 +1,4 @@
-# Class: mobileapps
+# Class: profile::mobileapps
 #
 # This class installs and configures mobileapps
 #
@@ -7,7 +7,7 @@
 # classes as well as conform to a de-facto standard of having a module for 
every
 # service
 #
-class mobileapps() {
+class profile::mobileapps {
 service::node { 'mobileapps':
 port  => ,
 has_spec  => true,
diff --git a/modules/role/manifests/mobileapps.pp 
b/modules/role/manifests/mobileapps.pp
index d1de20f..e553686 100644
--- a/modules/role/manifests/mobileapps.pp
+++ b/modules/role/manifests/mobileapps.pp
@@ -5,5 +5,5 @@
 description => 'A service for use by mobile apps. Provides DOM 
manipulation, aggregation, JSON flattening'
 }
 
-include ::mobileapps
+include ::profile::mobileapps
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/377204
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I579fd89846f54aab6762e709edcc076f4efec948
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: etcd: limit RAID resync speed if on linux software raid

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376712 )

Change subject: etcd: limit RAID resync speed if on linux software raid
..


etcd: limit RAID resync speed if on linux software raid

This should help avoid the loss of consensus we experienced on the codfw
cluster whenever the nodes were resyncing their RAID partitions.

Bug: T162013
Change-Id: Ia0e331d2a3ad2cbb78fe18dda3959175580f9890
---
M modules/profile/manifests/etcd.pp
1 file changed, 8 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified
  Volans: Looks good to me, but someone else must approve



diff --git a/modules/profile/manifests/etcd.pp 
b/modules/profile/manifests/etcd.pp
index 49dd6ca..d721670 100644
--- a/modules/profile/manifests/etcd.pp
+++ b/modules/profile/manifests/etcd.pp
@@ -107,4 +107,12 @@
 srv_domain => $srv_dns,
 }
 
+# T162013 - reduce raid resync speeds on clustered etcd noes with software 
RAID
+# in order to mitigate the risk of losing consensus due to I/O latencies
+if 'md' in $facts['raid'] {
+sysctl::parameters { 'raid_resync_speed':
+ensure => present,
+values => { 'dev.raid.speed_limit_max' => '2' },
+}
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/376712
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia0e331d2a3ad2cbb78fe18dda3959175580f9890
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Volans 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: etcd: limit RAID resync speed if on linux software raid

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376712 )

Change subject: etcd: limit RAID resync speed if on linux software raid
..

etcd: limit RAID resync speed if on linux software raid

This should help avoid the loss of consensus we experienced on the codfw
cluster whenever the nodes were resyncing their RAID partitions.

Bug: T162013
Change-Id: Ia0e331d2a3ad2cbb78fe18dda3959175580f9890
---
M modules/profile/manifests/etcd.pp
1 file changed, 8 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/12/376712/1

diff --git a/modules/profile/manifests/etcd.pp 
b/modules/profile/manifests/etcd.pp
index 49dd6ca..d721670 100644
--- a/modules/profile/manifests/etcd.pp
+++ b/modules/profile/manifests/etcd.pp
@@ -107,4 +107,12 @@
 srv_domain => $srv_dns,
 }
 
+# T162013 - reduce raid resync speeds on clustered etcd noes with software 
RAID
+# in order to mitigate the risk of losing consensus due to I/O latencies
+if 'md' in $facts['raid'] {
+sysctl::parameters { 'raid_resync_speed':
+ensure => present,
+values => { 'dev.raid.speed_limit_max' => '2' },
+}
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/376712
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0e331d2a3ad2cbb78fe18dda3959175580f9890
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/dns[master]: Add discovery entry for jobrunner, active/passive

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376518 )

Change subject: Add discovery entry for jobrunner, active/passive
..


Add discovery entry for jobrunner, active/passive

Bug: T174599
Change-Id: If5b5e0732876ac011ac275381f7f3b4b314a6ede
---
M config-geo-test
M templates/wmnet
2 files changed, 2 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/config-geo-test b/config-geo-test
index d4aaa20..107c1fb 100644
--- a/config-geo-test
+++ b/config-geo-test
@@ -37,6 +37,7 @@
 # mock active-passive entries by copying here and changing name
 disc-appservers-rw  => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 disc-api-rw => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
+disc-jobrunner  => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 disc-imagescaler-rw => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 disc-swift-rw   => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 }
diff --git a/templates/wmnet b/templates/wmnet
index de0fcf2..07c5cd3 100644
--- a/templates/wmnet
+++ b/templates/wmnet
@@ -4809,6 +4809,7 @@
 appservers-rw  300/10 IN DYNA metafo!disc-appservers-rw
 api-ro 300/10 IN DYNA geoip!disc-api-ro
 api-rw 300/10 IN DYNA metafo!disc-api-rw
+jobrunner  300/10 IN DYNA metafo!disc-jobrunner
 imagescaler-ro 300/10 IN DYNA geoip!disc-imagescaler-ro
 imagescaler-rw 300/10 IN DYNA metafo!disc-imagescaler-rw
 swift-ro   300/10 IN DYNA geoip!disc-swift-ro

-- 
To view, visit https://gerrit.wikimedia.org/r/376518
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If5b5e0732876ac011ac275381f7f3b4b314a6ede
Gerrit-PatchSet: 1
Gerrit-Project: operations/dns
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: add discovery data

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376516 )

Change subject: jobrunner: add discovery data
..


jobrunner: add discovery data

Bug: T174599
Change-Id: I0db2528ba95ab7cf509f8178e8ff3fe34c7cb09d
---
M conftool-data/discovery/mediawiki.yaml
M hieradata/common/discovery.yaml
2 files changed, 4 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/conftool-data/discovery/mediawiki.yaml 
b/conftool-data/discovery/mediawiki.yaml
index bbc8666..803cc5c 100644
--- a/conftool-data/discovery/mediawiki.yaml
+++ b/conftool-data/discovery/mediawiki.yaml
@@ -7,3 +7,4 @@
 swift-rw: [eqiad, codfw]
 swift-ro: [eqiad, codfw]
 thumbor: [eqiad, codfw]
+jobrunner: [eqiad, codfw]
diff --git a/hieradata/common/discovery.yaml b/hieradata/common/discovery.yaml
index c0a14ca..165d456 100644
--- a/hieradata/common/discovery.yaml
+++ b/hieradata/common/discovery.yaml
@@ -20,6 +20,9 @@
   api-rw:
 lvs: api
 active_active: false
+  jobrunner:
+lvs: jobrunner
+active_active: false
   imagescaler-rw:
 lvs: rendering
 active_active: false

-- 
To view, visit https://gerrit.wikimedia.org/r/376516
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I0db2528ba95ab7cf509f8178e8ff3fe34c7cb09d
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/dns[master]: Add discovery entry for jobrunner, active/passive

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376518 )

Change subject: Add discovery entry for jobrunner, active/passive
..

Add discovery entry for jobrunner, active/passive

Bug: T174599
Change-Id: If5b5e0732876ac011ac275381f7f3b4b314a6ede
---
M config-geo-test
M templates/wmnet
2 files changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/dns 
refs/changes/18/376518/1

diff --git a/config-geo-test b/config-geo-test
index d4aaa20..107c1fb 100644
--- a/config-geo-test
+++ b/config-geo-test
@@ -37,6 +37,7 @@
 # mock active-passive entries by copying here and changing name
 disc-appservers-rw  => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 disc-api-rw => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
+disc-jobrunner  => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 disc-imagescaler-rw => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 disc-swift-rw   => { datacenters => mock, dcmap => { mock => 
192.0.2.1 } }
 }
diff --git a/templates/wmnet b/templates/wmnet
index de0fcf2..07c5cd3 100644
--- a/templates/wmnet
+++ b/templates/wmnet
@@ -4809,6 +4809,7 @@
 appservers-rw  300/10 IN DYNA metafo!disc-appservers-rw
 api-ro 300/10 IN DYNA geoip!disc-api-ro
 api-rw 300/10 IN DYNA metafo!disc-api-rw
+jobrunner  300/10 IN DYNA metafo!disc-jobrunner
 imagescaler-ro 300/10 IN DYNA geoip!disc-imagescaler-ro
 imagescaler-rw 300/10 IN DYNA metafo!disc-imagescaler-rw
 swift-ro   300/10 IN DYNA geoip!disc-swift-ro

-- 
To view, visit https://gerrit.wikimedia.org/r/376518
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If5b5e0732876ac011ac275381f7f3b4b314a6ede
Gerrit-PatchSet: 1
Gerrit-Project: operations/dns
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: add discovery data

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376516 )

Change subject: jobrunner: add discovery data
..

jobrunner: add discovery data

Bug: T174599
Change-Id: I0db2528ba95ab7cf509f8178e8ff3fe34c7cb09d
---
M conftool-data/discovery/mediawiki.yaml
M hieradata/common/discovery.yaml
2 files changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/16/376516/1

diff --git a/conftool-data/discovery/mediawiki.yaml 
b/conftool-data/discovery/mediawiki.yaml
index bbc8666..803cc5c 100644
--- a/conftool-data/discovery/mediawiki.yaml
+++ b/conftool-data/discovery/mediawiki.yaml
@@ -7,3 +7,4 @@
 swift-rw: [eqiad, codfw]
 swift-ro: [eqiad, codfw]
 thumbor: [eqiad, codfw]
+jobrunner: [eqiad, codfw]
diff --git a/hieradata/common/discovery.yaml b/hieradata/common/discovery.yaml
index c0a14ca..165d456 100644
--- a/hieradata/common/discovery.yaml
+++ b/hieradata/common/discovery.yaml
@@ -20,6 +20,9 @@
   api-rw:
 lvs: api
 active_active: false
+  jobrunner:
+lvs: jobrunner
+active_active: false
   imagescaler-rw:
 lvs: rendering
 active_active: false

-- 
To view, visit https://gerrit.wikimedia.org/r/376516
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0db2528ba95ab7cf509f8178e8ff3fe34c7cb09d
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner_tls: add monitoring

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376023 )

Change subject: profile::mediawiki::jobrunner_tls: add monitoring
..


profile::mediawiki::jobrunner_tls: add monitoring

Change-Id: I891444ab7baec34cd7b3d0403e9695ae43b7d016
---
M modules/profile/manifests/mediawiki/jobrunner_tls.pp
1 file changed, 7 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/mediawiki/jobrunner_tls.pp 
b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
index be4646f..a50bb63 100644
--- a/modules/profile/manifests/mediawiki/jobrunner_tls.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
@@ -25,4 +25,11 @@
 srange  => '$DOMAIN_NETWORKS',
 }
 
+monitoring::service { 'jobrunner https':
+description=> 'Nginx local proxy to apache',
+check_command  => 
'check_https_url!jobrunner.discovery.wmnet!/rpc/RunJobs.php',
+retries=> 2,
+retry_interval => 2,
+}
+
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/376023
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I891444ab7baec34cd7b3d0403e9695ae43b7d016
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: fix ProxyPass directives for LVS vhost

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376211 )

Change subject: jobrunner: fix ProxyPass directives for LVS vhost
..


jobrunner: fix ProxyPass directives for LVS vhost

Change-Id: I95d868d2dea21fb838046e6d41c606e676c13008
---
M modules/profile/templates/mediawiki/jobrunner/site.conf.erb
1 file changed, 3 insertions(+), 3 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb 
b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
index 6a57995..e8e9a59 100644
--- a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
+++ b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
@@ -13,9 +13,9 @@
 DocumentRoot "/srv/mediawiki"
 # Only selected paths are allowed here
 # TODO: move monitoring directly to RunSingleJob.php when it's ready?
-ProxyPass /wiki/health-check.php 
fcgi://127.0.0.1:9000/srv/mediawiki/w/health-check.php retry=0
-ProxyPass /rpc/RunJobs.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
-ProxyPass /rpc/RunSingleJob.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
+ProxyPass /w/health-check.php 
fcgi://127.0.0.1:9000/srv/mediawiki/w/health-check.php retry=0
+ProxyPass /rpc/RunJobs.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/RunJobs.php retry=0
+ProxyPass /rpc/RunSingleJob.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/RunSingleJob.php retry=0
 ErrorLog ${APACHE_LOG_DIR}/jobqueue-error.log
 CustomLog ${APACHE_LOG_DIR}/jobqueue-access.log wmf
 

-- 
To view, visit https://gerrit.wikimedia.org/r/376211
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I95d868d2dea21fb838046e6d41c606e676c13008
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: fix ProxyPass directives for LVS vhost

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376211 )

Change subject: jobrunner: fix ProxyPass directives for LVS vhost
..

jobrunner: fix ProxyPass directives for LVS vhost

Change-Id: I95d868d2dea21fb838046e6d41c606e676c13008
---
M modules/profile/templates/mediawiki/jobrunner/site.conf.erb
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/11/376211/1

diff --git a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb 
b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
index 6a57995..e8e9a59 100644
--- a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
+++ b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
@@ -13,9 +13,9 @@
 DocumentRoot "/srv/mediawiki"
 # Only selected paths are allowed here
 # TODO: move monitoring directly to RunSingleJob.php when it's ready?
-ProxyPass /wiki/health-check.php 
fcgi://127.0.0.1:9000/srv/mediawiki/w/health-check.php retry=0
-ProxyPass /rpc/RunJobs.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
-ProxyPass /rpc/RunSingleJob.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
+ProxyPass /w/health-check.php 
fcgi://127.0.0.1:9000/srv/mediawiki/w/health-check.php retry=0
+ProxyPass /rpc/RunJobs.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/RunJobs.php retry=0
+ProxyPass /rpc/RunSingleJob.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/RunSingleJob.php retry=0
 ErrorLog ${APACHE_LOG_DIR}/jobqueue-error.log
 CustomLog ${APACHE_LOG_DIR}/jobqueue-access.log wmf
 

-- 
To view, visit https://gerrit.wikimedia.org/r/376211
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I95d868d2dea21fb838046e6d41c606e676c13008
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner_tls: relay requests to the loc...

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376022 )

Change subject: profile::mediawiki::jobrunner_tls: relay requests to the 
local-only port
..


profile::mediawiki::jobrunner_tls: relay requests to the local-only port

Bug: T174599
Change-Id: I67e2eac339aef5c860d5d474b1af4aef44a7733a
---
M modules/profile/manifests/mediawiki/jobrunner_tls.pp
1 file changed, 2 insertions(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/mediawiki/jobrunner_tls.pp 
b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
index 97ddda7..be4646f 100644
--- a/modules/profile/manifests/mediawiki/jobrunner_tls.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
@@ -14,9 +14,10 @@
 certs_active   => [$certname],
 default_server => true,
 do_ocsp=> false,
-upstream_ports => [$::profile::mediawiki::jobrunner::port],
+upstream_ports => [$::profile::mediawiki::jobrunner::local_only_port],
 access_log => false,
 }
+
 ::ferm::service { 'mediawiki-jobrunner-https':
 proto   => 'tcp',
 port=> 'https',

-- 
To view, visit https://gerrit.wikimedia.org/r/376022
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I67e2eac339aef5c860d5d474b1af4aef44a7733a
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: add missing newline in template

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376204 )

Change subject: jobrunner: add missing newline in template
..


jobrunner: add missing newline in template

Change-Id: Ib61608fa12dde11b4ba639ff5bb0fb584a90d900
---
M modules/profile/manifests/mediawiki/jobrunner.pp
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index 0917e22..14af8e8 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -32,7 +32,7 @@
 
 apache::conf { 'hhvm_jobrunner_port':
 priority => 1,
-content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>Listen <%= @local_only_port %>\n"),
+content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\nListen <%= @local_only_port %>\n"),
 }
 
 apache::site{ 'hhvm_jobrunner':

-- 
To view, visit https://gerrit.wikimedia.org/r/376204
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ib61608fa12dde11b4ba639ff5bb0fb584a90d900
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: add missing newline in template

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376204 )

Change subject: jobrunner: add missing newline in template
..

jobrunner: add missing newline in template

Change-Id: Ib61608fa12dde11b4ba639ff5bb0fb584a90d900
---
M modules/profile/manifests/mediawiki/jobrunner.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/04/376204/1

diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index 0917e22..14af8e8 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -32,7 +32,7 @@
 
 apache::conf { 'hhvm_jobrunner_port':
 priority => 1,
-content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>Listen <%= @local_only_port %>\n"),
+content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\nListen <%= @local_only_port %>\n"),
 }
 
 apache::site{ 'hhvm_jobrunner':

-- 
To view, visit https://gerrit.wikimedia.org/r/376204
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib61608fa12dde11b4ba639ff5bb0fb584a90d900
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner: Add local-only port

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376021 )

Change subject: profile::mediawiki::jobrunner: Add local-only port
..


profile::mediawiki::jobrunner: Add local-only port

This second vhost on a different port will only allow to reach the
desired endpoints, and will only be available on localhost.

Bug: T174599
Change-Id: I0aecb5a3f8be5547dd486a630e8eb61fe7873f6c
---
M modules/profile/manifests/mediawiki/jobrunner.pp
M modules/profile/templates/mediawiki/jobrunner/site.conf.erb
2 files changed, 16 insertions(+), 2 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index be4fef6..0917e22 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -6,6 +6,7 @@
 ) {
 # Parameters we don't need to override
 $port = 9005
+$local_only_port = 9006
 
 # The jobrunner script that submits jobs to hhvm
 $active = ($::mw_primary == $::site)
@@ -31,7 +32,7 @@
 
 apache::conf { 'hhvm_jobrunner_port':
 priority => 1,
-content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\n"),
+content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>Listen <%= @local_only_port %>\n"),
 }
 
 apache::site{ 'hhvm_jobrunner':
@@ -55,11 +56,11 @@
 source => 
'puppet:///modules/diamond/collector/nf_conntrack_counter.py',
 }
 
+# TODO: restrict this to monitoring and localhost only.
 ::ferm::service { 'mediawiki-jobrunner':
 proto   => 'tcp',
 port=> $port,
 notrack => true,
 srange  => '$DOMAIN_NETWORKS',
 }
-
 }
diff --git a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb 
b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
index e081230..6a57995 100644
--- a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
+++ b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
@@ -6,3 +6,16 @@
 ErrorLog ${APACHE_LOG_DIR}/jobqueue-error.log
 CustomLog ${APACHE_LOG_DIR}/jobqueue-access.log wmf
 
+
+>
+ServerName jobrunner.svc.<%= scope['::site'] %>.wmnet
+ServerAlias jobrunner.discovery.wmnet
+DocumentRoot "/srv/mediawiki"
+# Only selected paths are allowed here
+# TODO: move monitoring directly to RunSingleJob.php when it's ready?
+ProxyPass /wiki/health-check.php 
fcgi://127.0.0.1:9000/srv/mediawiki/w/health-check.php retry=0
+ProxyPass /rpc/RunJobs.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
+ProxyPass /rpc/RunSingleJob.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
+ErrorLog ${APACHE_LOG_DIR}/jobqueue-error.log
+CustomLog ${APACHE_LOG_DIR}/jobqueue-access.log wmf
+

-- 
To view, visit https://gerrit.wikimedia.org/r/376021
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I0aecb5a3f8be5547dd486a630e8eb61fe7873f6c
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner: refactor things to the profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/376020 )

Change subject: profile::mediawiki::jobrunner: refactor things to the profile
..


profile::mediawiki::jobrunner: refactor things to the profile

All the setup of the web service belonged in the profile.

Change-Id: I2ba00a2d60644d4846fa753e70708edcd4bfed36
---
M modules/mediawiki/manifests/jobrunner.pp
M modules/profile/manifests/mediawiki/jobrunner.pp
R modules/profile/templates/mediawiki/jobrunner/site.conf.erb
3 files changed, 16 insertions(+), 29 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/mediawiki/manifests/jobrunner.pp 
b/modules/mediawiki/manifests/jobrunner.pp
index 66506df..b71f7d6 100644
--- a/modules/mediawiki/manifests/jobrunner.pp
+++ b/modules/mediawiki/manifests/jobrunner.pp
@@ -105,33 +105,4 @@
 ensure  => present,
 content => template('mediawiki/jobrunner/logrotate.conf.erb'),
 }
-
-include ::apache::mod::proxy_fcgi
-
-class { '::apache::mpm':
-mpm => 'worker',
-}
-
-apache::conf { 'hhvm_jobrunner_port':
-priority => 1,
-content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\n"),
-}
-
-apache::site{ 'hhvm_jobrunner':
-priority => 1,
-content  => template('mediawiki/jobrunner/site.conf.erb'),
-}
-
-# Hack for T122069: on servers running GWT jobs, restart HHVM
-# once it occupies more than 60% of the available memory
-if ($runners_gwt > 0) {
-cron { 'periodic_hhvm_restart':
-command => '/bin/ps -C hhvm -o pmem= | awk \'{sum+=$1} END { if 
(sum <= 50.0) exit 1  }\'  && /usr/sbin/service hhvm restart >/dev/null 
2>/dev/null',
-minute  => fqdn_rand(60, 'periodic_hhvm_restart'),
-}
-} else {
-cron { 'periodic_hhvm_restart':
-ensure => absent,
-}
-}
 }
diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index baa802c..be4fef6 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -6,6 +6,8 @@
 ) {
 # Parameters we don't need to override
 $port = 9005
+
+# The jobrunner script that submits jobs to hhvm
 $active = ($::mw_primary == $::site)
 class { '::mediawiki::jobrunner':
 port  => $port,
@@ -22,6 +24,20 @@
 runners_translate => pick($runners['translate'], 0)
 }
 
+# Special HHVM setup
+class { '::apache::mpm':
+mpm => 'worker',
+}
+
+apache::conf { 'hhvm_jobrunner_port':
+priority => 1,
+content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\n"),
+}
+
+apache::site{ 'hhvm_jobrunner':
+priority => 1,
+content  => template('profile/mediawiki/jobrunner/site.conf.erb'),
+}
 
 ::monitoring::service { 'jobrunner_http_hhvm':
 description   => 'HHVM jobrunner',
diff --git a/modules/mediawiki/templates/jobrunner/site.conf.erb 
b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
similarity index 100%
rename from modules/mediawiki/templates/jobrunner/site.conf.erb
rename to modules/profile/templates/mediawiki/jobrunner/site.conf.erb

-- 
To view, visit https://gerrit.wikimedia.org/r/376020
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I2ba00a2d60644d4846fa753e70708edcd4bfed36
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Alex Monk 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner: Add local-only port

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376021 )

Change subject: profile::mediawiki::jobrunner: Add local-only port
..

profile::mediawiki::jobrunner: Add local-only port

This second vhost on a different port will only allow to reach the
desired endpoints, and will only be available on localhost.

Bug: T174599
Change-Id: I0aecb5a3f8be5547dd486a630e8eb61fe7873f6c
---
M modules/profile/manifests/mediawiki/jobrunner.pp
M modules/profile/templates/mediawiki/jobrunner/site.conf.erb
2 files changed, 16 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/21/376021/1

diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index f279323..ef80c3a 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -6,6 +6,7 @@
 ) {
 # Parameters we don't need to override
 $port = 9005
+$local_only_port = 9006
 
 # The jobrunner script that submits jobs to hhvm
 $active = ($::mw_primary == $::site)
@@ -33,7 +34,7 @@
 
 apache::conf { 'hhvm_jobrunner_port':
 priority => 1,
-content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\n"),
+content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>Listen <%= @local_only_port %>\n"),
 }
 
 apache::site{ 'hhvm_jobrunner':
@@ -57,11 +58,11 @@
 source => 
'puppet:///modules/diamond/collector/nf_conntrack_counter.py',
 }
 
+# TODO: restrict this to monitoring and localhost only.
 ::ferm::service { 'mediawiki-jobrunner':
 proto   => 'tcp',
 port=> $port,
 notrack => true,
 srange  => '$DOMAIN_NETWORKS',
 }
-
 }
diff --git a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb 
b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
index e081230..6a57995 100644
--- a/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
+++ b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
@@ -6,3 +6,16 @@
 ErrorLog ${APACHE_LOG_DIR}/jobqueue-error.log
 CustomLog ${APACHE_LOG_DIR}/jobqueue-access.log wmf
 
+
+>
+ServerName jobrunner.svc.<%= scope['::site'] %>.wmnet
+ServerAlias jobrunner.discovery.wmnet
+DocumentRoot "/srv/mediawiki"
+# Only selected paths are allowed here
+# TODO: move monitoring directly to RunSingleJob.php when it's ready?
+ProxyPass /wiki/health-check.php 
fcgi://127.0.0.1:9000/srv/mediawiki/w/health-check.php retry=0
+ProxyPass /rpc/RunJobs.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
+ProxyPass /rpc/RunSingleJob.php 
fcgi://127.0.0.1:9000/srv/mediawiki/rpc/runJobs.php retry=0
+ErrorLog ${APACHE_LOG_DIR}/jobqueue-error.log
+CustomLog ${APACHE_LOG_DIR}/jobqueue-access.log wmf
+

-- 
To view, visit https://gerrit.wikimedia.org/r/376021
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0aecb5a3f8be5547dd486a630e8eb61fe7873f6c
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner: refactor things to the profile

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376020 )

Change subject: profile::mediawiki::jobrunner: refactor things to the profile
..

profile::mediawiki::jobrunner: refactor things to the profile

All the setup of the web service belonged in the profile.

Change-Id: I2ba00a2d60644d4846fa753e70708edcd4bfed36
---
M modules/mediawiki/manifests/jobrunner.pp
M modules/profile/manifests/mediawiki/jobrunner.pp
R modules/profile/templates/mediawiki/jobrunner/site.conf.erb
3 files changed, 18 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/20/376020/1

diff --git a/modules/mediawiki/manifests/jobrunner.pp 
b/modules/mediawiki/manifests/jobrunner.pp
index 66506df..b71f7d6 100644
--- a/modules/mediawiki/manifests/jobrunner.pp
+++ b/modules/mediawiki/manifests/jobrunner.pp
@@ -105,33 +105,4 @@
 ensure  => present,
 content => template('mediawiki/jobrunner/logrotate.conf.erb'),
 }
-
-include ::apache::mod::proxy_fcgi
-
-class { '::apache::mpm':
-mpm => 'worker',
-}
-
-apache::conf { 'hhvm_jobrunner_port':
-priority => 1,
-content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\n"),
-}
-
-apache::site{ 'hhvm_jobrunner':
-priority => 1,
-content  => template('mediawiki/jobrunner/site.conf.erb'),
-}
-
-# Hack for T122069: on servers running GWT jobs, restart HHVM
-# once it occupies more than 60% of the available memory
-if ($runners_gwt > 0) {
-cron { 'periodic_hhvm_restart':
-command => '/bin/ps -C hhvm -o pmem= | awk \'{sum+=$1} END { if 
(sum <= 50.0) exit 1  }\'  && /usr/sbin/service hhvm restart >/dev/null 
2>/dev/null',
-minute  => fqdn_rand(60, 'periodic_hhvm_restart'),
-}
-} else {
-cron { 'periodic_hhvm_restart':
-ensure => absent,
-}
-}
 }
diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index baa802c..f279323 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -6,6 +6,8 @@
 ) {
 # Parameters we don't need to override
 $port = 9005
+
+# The jobrunner script that submits jobs to hhvm
 $active = ($::mw_primary == $::site)
 class { '::mediawiki::jobrunner':
 port  => $port,
@@ -22,6 +24,22 @@
 runners_translate => pick($runners['translate'], 0)
 }
 
+# Special HHVM setup
+class { '::apache::mod::proxy_fcgi': }
+
+class { '::apache::mpm':
+mpm => 'worker',
+}
+
+apache::conf { 'hhvm_jobrunner_port':
+priority => 1,
+content  => inline_template("# This file is managed by Puppet\nListen 
<%= @port %>\n"),
+}
+
+apache::site{ 'hhvm_jobrunner':
+priority => 1,
+content  => template('profile/mediawiki/jobrunner/site.conf.erb'),
+}
 
 ::monitoring::service { 'jobrunner_http_hhvm':
 description   => 'HHVM jobrunner',
diff --git a/modules/mediawiki/templates/jobrunner/site.conf.erb 
b/modules/profile/templates/mediawiki/jobrunner/site.conf.erb
similarity index 100%
rename from modules/mediawiki/templates/jobrunner/site.conf.erb
rename to modules/profile/templates/mediawiki/jobrunner/site.conf.erb

-- 
To view, visit https://gerrit.wikimedia.org/r/376020
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2ba00a2d60644d4846fa753e70708edcd4bfed36
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner_tls: add monitoring

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376023 )

Change subject: profile::mediawiki::jobrunner_tls: add monitoring
..

profile::mediawiki::jobrunner_tls: add monitoring

Change-Id: I891444ab7baec34cd7b3d0403e9695ae43b7d016
---
M modules/profile/manifests/mediawiki/jobrunner_tls.pp
1 file changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/23/376023/1

diff --git a/modules/profile/manifests/mediawiki/jobrunner_tls.pp 
b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
index be4646f..b37c63a 100644
--- a/modules/profile/manifests/mediawiki/jobrunner_tls.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
@@ -25,4 +25,11 @@
 srange  => '$DOMAIN_NETWORKS',
 }
 
+monitoring::service { 'jobrunner https':
+description=> 'Nginx local proxy to apache',
+check_command  => 
'check_https_url!jobrunner.discovery.wmnet!/rpc/runJobs.php',
+retries=> 2,
+retry_interval => 2,
+}
+
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/376023
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I891444ab7baec34cd7b3d0403e9695ae43b7d016
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner: restrict firewall rules

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376024 )

Change subject: profile::mediawiki::jobrunner: restrict firewall rules
..

profile::mediawiki::jobrunner: restrict firewall rules

We don't need anything besides localhost and the monitoring hosts to
connect to port 9005, so let's just acknowledge that.

Change-Id: I2808525665b65ef8506637aa4cc39eb88cfd951d
---
M modules/profile/manifests/mediawiki/jobrunner.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/24/376024/1

diff --git a/modules/profile/manifests/mediawiki/jobrunner.pp 
b/modules/profile/manifests/mediawiki/jobrunner.pp
index ef80c3a..2012387 100644
--- a/modules/profile/manifests/mediawiki/jobrunner.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner.pp
@@ -63,6 +63,6 @@
 proto   => 'tcp',
 port=> $port,
 notrack => true,
-srange  => '$DOMAIN_NETWORKS',
+srange  => '$MONITORING_HOSTS',
 }
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/376024
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2808525665b65ef8506637aa4cc39eb88cfd951d
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner_tls: relay requests to the loc...

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/376022 )

Change subject: profile::mediawiki::jobrunner_tls: relay requests to the 
local-only port
..

profile::mediawiki::jobrunner_tls: relay requests to the local-only port

Bug: T174599
Change-Id: I67e2eac339aef5c860d5d474b1af4aef44a7733a
---
M modules/profile/manifests/mediawiki/jobrunner_tls.pp
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/22/376022/1

diff --git a/modules/profile/manifests/mediawiki/jobrunner_tls.pp 
b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
index 97ddda7..be4646f 100644
--- a/modules/profile/manifests/mediawiki/jobrunner_tls.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
@@ -14,9 +14,10 @@
 certs_active   => [$certname],
 default_server => true,
 do_ocsp=> false,
-upstream_ports => [$::profile::mediawiki::jobrunner::port],
+upstream_ports => [$::profile::mediawiki::jobrunner::local_only_port],
 access_log => false,
 }
+
 ::ferm::service { 'mediawiki-jobrunner-https':
 proto   => 'tcp',
 port=> 'https',

-- 
To view, visit https://gerrit.wikimedia.org/r/376022
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I67e2eac339aef5c860d5d474b1af4aef44a7733a
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner_tls: add ferm rule

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/375828 )

Change subject: profile::mediawiki::jobrunner_tls: add ferm rule
..


profile::mediawiki::jobrunner_tls: add ferm rule

Change-Id: If60f3cd0769eed52d32cdc38da071b8241c47a23
---
M modules/profile/manifests/mediawiki/jobrunner_tls.pp
1 file changed, 7 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/profile/manifests/mediawiki/jobrunner_tls.pp 
b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
index c849db4..97ddda7 100644
--- a/modules/profile/manifests/mediawiki/jobrunner_tls.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
@@ -17,4 +17,11 @@
 upstream_ports => [$::profile::mediawiki::jobrunner::port],
 access_log => false,
 }
+::ferm::service { 'mediawiki-jobrunner-https':
+proto   => 'tcp',
+port=> 'https',
+notrack => true,
+srange  => '$DOMAIN_NETWORKS',
+}
+
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/375828
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If60f3cd0769eed52d32cdc38da071b8241c47a23
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: profile::mediawiki::jobrunner_tls: add ferm rule

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/375828 )

Change subject: profile::mediawiki::jobrunner_tls: add ferm rule
..

profile::mediawiki::jobrunner_tls: add ferm rule

Change-Id: If60f3cd0769eed52d32cdc38da071b8241c47a23
---
M modules/profile/manifests/mediawiki/jobrunner_tls.pp
1 file changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/28/375828/1

diff --git a/modules/profile/manifests/mediawiki/jobrunner_tls.pp 
b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
index c849db4..97ddda7 100644
--- a/modules/profile/manifests/mediawiki/jobrunner_tls.pp
+++ b/modules/profile/manifests/mediawiki/jobrunner_tls.pp
@@ -17,4 +17,11 @@
 upstream_ports => [$::profile::mediawiki::jobrunner::port],
 access_log => false,
 }
+::ferm::service { 'mediawiki-jobrunner-https':
+proto   => 'tcp',
+port=> 'https',
+notrack => true,
+srange  => '$DOMAIN_NETWORKS',
+}
+
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/375828
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If60f3cd0769eed52d32cdc38da071b8241c47a23
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: add LVS service configuration

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/375801 )

Change subject: jobrunner: add LVS service configuration
..


jobrunner: add LVS service configuration

Bug: T174599
Change-Id: I65d0372a7ca3bb2e5e613cfc94feab9602030267
---
M hieradata/common/lvs/configuration.yaml
M hieradata/role/common/mediawiki/jobrunner.yaml
M modules/role/manifests/mediawiki/jobrunner.pp
3 files changed, 34 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/hieradata/common/lvs/configuration.yaml 
b/hieradata/common/lvs/configuration.yaml
index 9aba416..d367f9f 100644
--- a/hieradata/common/lvs/configuration.yaml
+++ b/hieradata/common/lvs/configuration.yaml
@@ -132,6 +132,9 @@
   "recommendation-api": _block038
 eqiad: 10.2.2.37
 codfw: 10.2.1.37
+  jobrunner: _block039
+eqiad: 10.2.2.26
+codfw: 10.2.1.26
 
 lvs::configuration::lvs_services:
   text:
@@ -1332,3 +1335,30 @@
   hostname: recommendation-api.svc.eqiad.wmnet
 codfw:
   hostname: recommendation-api.svc.codfw.wmnet
+  jobrunner:
+description: "JobRunner LVS interface (https)"
+class: low-traffic
+sites:
+- eqiad
+- codfw
+ip: *ip_block039
+port: 443
+# It's ok to lose most of the service capacity in this case
+depool-threshold: '.2'
+monitors:
+  ProxyFetch:
+url:
+- https://jobrunner.discovery.wmnet/w/health-check.php
+  IdleConnection:
+timeout-clean-reconnect: 3
+max-delay: 300
+conftool:
+  cluster: jobrunner
+  service: nginx
+icinga:
+  check_command: "check_https_url!/w/health-check.php"
+  sites:
+eqiad:
+  hostname: jobrunner.svc.eqiad.wmnet
+codfw:
+  hostname: jobrunner.svc.codfw.wmnet
diff --git a/hieradata/role/common/mediawiki/jobrunner.yaml 
b/hieradata/role/common/mediawiki/jobrunner.yaml
index 848a024..5e13e9c 100644
--- a/hieradata/role/common/mediawiki/jobrunner.yaml
+++ b/hieradata/role/common/mediawiki/jobrunner.yaml
@@ -17,6 +17,9 @@
   size: 30
 "namedPools.cirrus-eqiad":
   size: 30
+role::lvs::realserver::pools:
+  hhvm:
+lvs_name: jobrunner
 
 # Use the future parser here
 profile::base::environment: "future"
diff --git a/modules/role/manifests/mediawiki/jobrunner.pp 
b/modules/role/manifests/mediawiki/jobrunner.pp
index af7b793..81abbcc 100644
--- a/modules/role/manifests/mediawiki/jobrunner.pp
+++ b/modules/role/manifests/mediawiki/jobrunner.pp
@@ -12,6 +12,7 @@
 
 # TODO: change role used in beta
 if hiera('has_lvs', true) {
+include ::role::lvs::realserver
 include ::profile::mediawiki::jobrunner_tls
 }
 

-- 
To view, visit https://gerrit.wikimedia.org/r/375801
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I65d0372a7ca3bb2e5e613cfc94feab9602030267
Gerrit-PatchSet: 4
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: jobrunner: add nginx service

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/375800 )

Change subject: jobrunner: add nginx service
..


jobrunner: add nginx service

Bug: T174599
Change-Id: I8a70f41d128d288fc5e23cf19c8cf82f1efda5d2
---
M conftool-data/node/codfw.yaml
M conftool-data/node/eqiad.yaml
M conftool-data/service/mediawiki.yaml
A files/ssl/jobrunner.svc.codfw.wmnet.crt
A files/ssl/jobrunner.svc.eqiad.wmnet.crt
A modules/profile/manifests/mediawiki/jobrunner_tls.pp
M modules/role/manifests/mediawiki/jobrunner.pp
7 files changed, 116 insertions(+), 30 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/conftool-data/node/codfw.yaml b/conftool-data/node/codfw.yaml
index b36675d..9246606 100644
--- a/conftool-data/node/codfw.yaml
+++ b/conftool-data/node/codfw.yaml
@@ -142,21 +142,21 @@
 mw2244.codfw.wmnet: [apache2,nginx]
 mw2245.codfw.wmnet: [apache2,nginx]
   jobrunner:
-mw2153.codfw.wmnet: [apache2]
-mw2154.codfw.wmnet: [apache2]
-mw2155.codfw.wmnet: [apache2]
-mw2156.codfw.wmnet: [apache2]
-mw2157.codfw.wmnet: [apache2]
-mw2158.codfw.wmnet: [apache2]
-mw2159.codfw.wmnet: [apache2]
-mw2160.codfw.wmnet: [apache2]
-mw2161.codfw.wmnet: [apache2]
-mw2162.codfw.wmnet: [apache2]
-mw2243.codfw.wmnet: [apache2]
-mw2247.codfw.wmnet: [apache2]
-mw2248.codfw.wmnet: [apache2]
-mw2249.codfw.wmnet: [apache2]
-mw2250.codfw.wmnet: [apache2]
+mw2153.codfw.wmnet: [apache2,nginx]
+mw2154.codfw.wmnet: [apache2,nginx]
+mw2155.codfw.wmnet: [apache2,nginx]
+mw2156.codfw.wmnet: [apache2,nginx]
+mw2157.codfw.wmnet: [apache2,nginx]
+mw2158.codfw.wmnet: [apache2,nginx]
+mw2159.codfw.wmnet: [apache2,nginx]
+mw2160.codfw.wmnet: [apache2,nginx]
+mw2161.codfw.wmnet: [apache2,nginx]
+mw2162.codfw.wmnet: [apache2,nginx]
+mw2243.codfw.wmnet: [apache2,nginx]
+mw2247.codfw.wmnet: [apache2,nginx]
+mw2248.codfw.wmnet: [apache2,nginx]
+mw2249.codfw.wmnet: [apache2,nginx]
+mw2250.codfw.wmnet: [apache2,nginx]
   videoscaler:
 mw2118.codfw.wmnet: [apache2]
 mw2119.codfw.wmnet: [apache2]
diff --git a/conftool-data/node/eqiad.yaml b/conftool-data/node/eqiad.yaml
index 358c376..1b2a252 100644
--- a/conftool-data/node/eqiad.yaml
+++ b/conftool-data/node/eqiad.yaml
@@ -8,21 +8,21 @@
 mw1259.eqiad.wmnet: [apache2]
 mw1260.eqiad.wmnet: [apache2]
   jobrunner:
-mw1161.eqiad.wmnet: [apache2]
-mw1162.eqiad.wmnet: [apache2]
-mw1163.eqiad.wmnet: [apache2]
-mw1164.eqiad.wmnet: [apache2]
-mw1165.eqiad.wmnet: [apache2]
-mw1166.eqiad.wmnet: [apache2]
-mw1167.eqiad.wmnet: [apache2]
-mw1299.eqiad.wmnet: [apache2]
-mw1300.eqiad.wmnet: [apache2]
-mw1301.eqiad.wmnet: [apache2]
-mw1302.eqiad.wmnet: [apache2]
-mw1303.eqiad.wmnet: [apache2]
-mw1304.eqiad.wmnet: [apache2]
-mw1305.eqiad.wmnet: [apache2]
-mw1306.eqiad.wmnet: [apache2]
+mw1161.eqiad.wmnet: [apache2,nginx]
+mw1162.eqiad.wmnet: [apache2,nginx]
+mw1163.eqiad.wmnet: [apache2,nginx]
+mw1164.eqiad.wmnet: [apache2,nginx]
+mw1165.eqiad.wmnet: [apache2,nginx]
+mw1166.eqiad.wmnet: [apache2,nginx]
+mw1167.eqiad.wmnet: [apache2,nginx]
+mw1299.eqiad.wmnet: [apache2,nginx]
+mw1300.eqiad.wmnet: [apache2,nginx]
+mw1301.eqiad.wmnet: [apache2,nginx]
+mw1302.eqiad.wmnet: [apache2,nginx]
+mw1303.eqiad.wmnet: [apache2,nginx]
+mw1304.eqiad.wmnet: [apache2,nginx]
+mw1305.eqiad.wmnet: [apache2,nginx]
+mw1306.eqiad.wmnet: [apache2,nginx]
   api_appserver:
 mw1189.eqiad.wmnet: [apache2,nginx]
 mw1190.eqiad.wmnet: [apache2,nginx]
diff --git a/conftool-data/service/mediawiki.yaml 
b/conftool-data/service/mediawiki.yaml
index fd50d17..2b601bd 100644
--- a/conftool-data/service/mediawiki.yaml
+++ b/conftool-data/service/mediawiki.yaml
@@ -67,6 +67,14 @@
 datacenters:
   - eqiad
   - codfw
+  nginx:
+port: 443
+default_values:
+  "pooled": "no"
+  "weight": 10
+datacenters:
+  - eqiad
+  - codfw
 testserver:
   apache2:
 port: 80
diff --git a/files/ssl/jobrunner.svc.codfw.wmnet.crt 
b/files/ssl/jobrunner.svc.codfw.wmnet.crt
new file mode 100644
index 000..962556d
--- /dev/null
+++ b/files/ssl/jobrunner.svc.codfw.wmnet.crt
@@ -0,0 +1,26 @@
+-BEGIN CERTIFICATE-
+MIIEXDCCAkSgAwIBAgICDIQwDQYJKoZIhvcNAQELBQAwKzEpMCcGA1UEAwwgUHVw
+cGV0IENBOiBwYWxsYWRpdW0uZXFpYWQud21uZXQwHhcNMTcwOTAzMDk0ODExWhcN
+MjIwOTAzMDk0ODExWjCBgzEiMCAGA1UEAwwZam9icnVubmVyLnN2Yy5jb2Rmdy53
+bW5ldDEjMCEGA1UECgwaV2lraW1lZGlhIEZvdW5kYXRpb24sIEluYy4xCzAJBgNV
+BAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRYwFAYDVQQHDA1TYW4gRnJhbmNp
+c2NvMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEmCEu5sFU1306AVv1vuuI0aBs
+9VcUkz1KwhrP49HXJAn7KWT6UvOj/cSVSpy4ywiBQcabqCqcQ0vmjQ1KXwFwyKOB

[MediaWiki-commits] [Gerrit] operations/dns[master]: Add entries for the jobrunner LVS service

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/375747 )

Change subject: Add entries for the jobrunner LVS service
..


Add entries for the jobrunner LVS service

Bug: T174599
Change-Id: Ic60fc292c5b286e9efcca3335f532bfd24af4487
---
M templates/10.in-addr.arpa
M templates/wmnet
2 files changed, 4 insertions(+), 4 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/templates/10.in-addr.arpa b/templates/10.in-addr.arpa
index 50b7137..a925da5 100644
--- a/templates/10.in-addr.arpa
+++ b/templates/10.in-addr.arpa
@@ -35,7 +35,7 @@
 23  1H  IN PTR  eventbus.svc.codfw.wmnet.
 24  1H  IN PTR  thumbor.svc.codfw.wmnet.
 25  1H  IN PTR  prometheus.svc.codfw.wmnet.
-
+26  1H  IN PTR  jobrunner.svc.codfw.wmnet.
 27  1H  IN PTR  ms-fe.svc.codfw.wmnet.
 28  1H  IN PTR  parsoid.svc.codfw.wmnet.
 30  1H  IN PTR  search.svc.codfw.wmnet.
@@ -71,7 +71,7 @@
 23  1H  IN PTR  eventbus.svc.eqiad.wmnet.
 24  1H  IN PTR  thumbor.svc.eqiad.wmnet.
 25  1H  IN PTR  prometheus.svc.eqiad.wmnet.
-
+26  1H  IN PTR  jobrunner.svc.eqiad.wmnet.
 27  1H  IN PTR  ms-fe.svc.eqiad.wmnet.
 28  1H  IN PTR  parsoid.svc.eqiad.wmnet.
 
diff --git a/templates/wmnet b/templates/wmnet
index c6d5c56..423a87f 100644
--- a/templates/wmnet
+++ b/templates/wmnet
@@ -4717,7 +4717,7 @@
 eventbus1H  IN A10.2.2.23
 thumbor 1H  IN A10.2.2.24
 prometheus  1H  IN A10.2.2.25
-
+jobrunner   1H  IN A10.2.2.26
 ms-fe   1H  IN A10.2.2.27
 swift   1H  IN CNAMEms-fe.svc.eqiad.wmnet.
 ms-fe-thumbs 1H IN A10.2.2.27
@@ -4775,7 +4775,7 @@
 eventbus1H  IN A10.2.1.23
 thumbor 1H  IN A10.2.1.24
 prometheus  1H  IN A10.2.1.25
-
+jobrunner   1H  IN A10.2.1.26
 ms-fe   1H  IN A10.2.1.27
 swift   1H  IN CNAMEms-fe.svc.codfw.wmnet.
 ms-fe-thumbs1H  IN A10.2.1.27

-- 
To view, visit https://gerrit.wikimedia.org/r/375747
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ic60fc292c5b286e9efcca3335f532bfd24af4487
Gerrit-PatchSet: 1
Gerrit-Project: operations/dns
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs/private[master]: Add secrets for jobrunner.svc

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/375809 )

Change subject: Add secrets for jobrunner.svc
..


Add secrets for jobrunner.svc

Change-Id: I3e009239d005a02cea53eadc9e971fe850be4054
---
A modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key
A modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key
2 files changed, 6 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key 
b/modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key
new file mode 100644
index 000..e8d3e5c
--- /dev/null
+++ b/modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key
@@ -0,0 +1,3 @@
+-BEGIN RSA PRIVATE KEY-
+SNAKEOIL
+-END RSA PRIVATE KEY-
diff --git a/modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key 
b/modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key
new file mode 100644
index 000..e8d3e5c
--- /dev/null
+++ b/modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key
@@ -0,0 +1,3 @@
+-BEGIN RSA PRIVATE KEY-
+SNAKEOIL
+-END RSA PRIVATE KEY-

-- 
To view, visit https://gerrit.wikimedia.org/r/375809
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I3e009239d005a02cea53eadc9e971fe850be4054
Gerrit-PatchSet: 1
Gerrit-Project: labs/private
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs/private[master]: Add secrets for jobrunner.svc

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/375809 )

Change subject: Add secrets for jobrunner.svc
..

Add secrets for jobrunner.svc

Change-Id: I3e009239d005a02cea53eadc9e971fe850be4054
---
A modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key
A modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key
2 files changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/labs/private 
refs/changes/09/375809/1

diff --git a/modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key 
b/modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key
new file mode 100644
index 000..e8d3e5c
--- /dev/null
+++ b/modules/secret/secrets/ssl/jobrunner.svc.codfw.wmnet.key
@@ -0,0 +1,3 @@
+-BEGIN RSA PRIVATE KEY-
+SNAKEOIL
+-END RSA PRIVATE KEY-
diff --git a/modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key 
b/modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key
new file mode 100644
index 000..e8d3e5c
--- /dev/null
+++ b/modules/secret/secrets/ssl/jobrunner.svc.eqiad.wmnet.key
@@ -0,0 +1,3 @@
+-BEGIN RSA PRIVATE KEY-
+SNAKEOIL
+-END RSA PRIVATE KEY-

-- 
To view, visit https://gerrit.wikimedia.org/r/375809
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3e009239d005a02cea53eadc9e971fe850be4054
Gerrit-PatchSet: 1
Gerrit-Project: labs/private
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs/private[master]: Add profile::openstack::main::rabbit_monitor_pass

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/375807 )

Change subject: Add profile::openstack::main::rabbit_monitor_pass
..


Add profile::openstack::main::rabbit_monitor_pass

Change-Id: I7d7589a95dd9911dc83428d44845be17746e96ee
---
A hieradata/common/profile/openstack/main.yaml
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Verified; Looks good to me, approved



diff --git a/hieradata/common/profile/openstack/main.yaml 
b/hieradata/common/profile/openstack/main.yaml
new file mode 100644
index 000..3c53fb4
--- /dev/null
+++ b/hieradata/common/profile/openstack/main.yaml
@@ -0,0 +1 @@
+profile::openstack::main::rabbit_monitor_pass: a_password

-- 
To view, visit https://gerrit.wikimedia.org/r/375807
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7d7589a95dd9911dc83428d44845be17746e96ee
Gerrit-PatchSet: 1
Gerrit-Project: labs/private
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs/private[master]: Add profile::openstack::main::rabbit_monitor_pass

[Giuseppe Lavagetto (Code Review)]

Giuseppe Lavagetto has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/375807 )

Change subject: Add profile::openstack::main::rabbit_monitor_pass
..

Add profile::openstack::main::rabbit_monitor_pass

Change-Id: I7d7589a95dd9911dc83428d44845be17746e96ee
---
A hieradata/common/profile/openstack/main.yaml
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/labs/private 
refs/changes/07/375807/1

diff --git a/hieradata/common/profile/openstack/main.yaml 
b/hieradata/common/profile/openstack/main.yaml
new file mode 100644
index 000..3c53fb4
--- /dev/null
+++ b/hieradata/common/profile/openstack/main.yaml
@@ -0,0 +1 @@
+profile::openstack::main::rabbit_monitor_pass: a_password

-- 
To view, visit https://gerrit.wikimedia.org/r/375807
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7d7589a95dd9911dc83428d44845be17746e96ee
Gerrit-PatchSet: 1
Gerrit-Project: labs/private
Gerrit-Branch: master
Gerrit-Owner: Giuseppe Lavagetto 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits