Plan for tomorrow's Fedora Infrastructure Meeting 2018-02-08

[Stephen John Smoogen]

= Preamble =
The infrastructure team will be having its weekly meeting tomorrow,
2018-02-08 at 18:00 UTC in #fedora-meeting on the freenode network.

We have a gobby document
(see: https://fedoraproject.org/wiki/Gobby )

fedora-infrastructure-meeting-next is the document.

Please try and review and edit that document before the meeting and we
will use it to have our agenda of things to discuss. A copy as of today
is included in this email.

If you have something to discuss, add the topic to the discussion area
with your name. If you would like to teach other folks about some
application or setup in our infrastructure, please add that topic and
your name to the learn about section.

= Introduction =
We will use it over the week before the meeting to gather status and info and
discussion items and so forth, then use it in the irc meeting to transfer
information to the meetbot logs.

= Meeting start stuff =

#startmeeting Infrastructure (2018-02-08)
#meetingname infrastructure
#topic aloha
#chair smooge relrod nirik dgilmore threebean pingou puiterwijk
pbrobinson maxamillion

= Let new people say hello =

#topic New folks introductions
#info This is a place where people who are interested in Fedora
Infrastructure can introduce themselves

= Status / Information / Trivia / Announcements =

(We put things here we want others on the team to know, but don't need
to discuss)
(Please use #info  - your name)

#topic announcements and information

#info Mass rebuild began on 2018-02-06
#info Infrastructure hackfest in april:
https://fedoraproject.org/wiki/Infrastructure_Hackathon_2018
#info

= Things we should discuss =

We use this section to bring up discussion topics. Things we want to talk about
as a group and come up with some consensus /suor decision or just brainstorm a
problem or issue. If there are none of these we skip this section.
(Use #topic your discussion topic - your username)

#topic Ticket cleanup
#info none this week.

#topic

= Apprentice office hours =

#topic Apprentice Open office hours

Here we will discuss any apprentice questions, try and match up people looking
for things to do with things to do, progress, testing anything like that.

= Learn about some application or setup in infrastructure =

(This section, each week we get 1 person to talk about an application or setup
that we have. Just going over what it is, how to contribute, ideas for
improvement,
etc. Whoever would like to do this, just add the i/nfo in this section. In the
event we don't find someone to teach about something, we skip this section
and just move on to open floor.)

#topic Learn about:
#info none this week

= Meeting end stuff =

#topic Open Floor

#endmeeting

-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Deploying JS-based application (Hubs)

[Kevin Fenzi]

On 02/05/2018 01:58 AM, Aurelien Bompard wrote:
> Hey, sorry for the silence on this subject, I was experimenting with
> deployment options.
> So far, here's what I've come up with.
> 
> Npm has a command called "shrinkwrap" which will list all the installed
> dependencies and their versions recursively and dump that in a file. It's
> pretty much like "pip freeze". Then all subsequent calls to "npm install"
> will use those versions regardless of what the most current one is.
> So I've generated and committed this version in git, and then I've cleaned
> up the local dependencies dir, run "npm install" again, and tarred up this
> big directory. I'm using this tarball as an additional source for the Hubs
> RPM that I wrote, untar it in %prep, build the JS in %build, install the
> resulting files in %install.
> This adresses the issues of reproductible builds, and the "bundled" sources
> (which are only bundled in the source RPM, not the binary one) are easily
> re-generated on any machine with npm installed. It also means that the RPM
> can build in mock because all dependencies are locally provided.
> 
> Would this method of deployment work for you?
> 
> Concerning services location (all local or split), I'm thinking on the
> following layout:
> - shared PostgreSQL
> - everything else on the same host.
> It's simple and it would be sufficient for a start because we're only going
> to setup a few group hubs for "early adopter" teams (however anyone with a
> FAS account can login and have their hub autocreated). Then if we need to
> loadbalance we'll be able to split the services, that shouldn't be a
> problem.
> 
> What do you think?

I think this could work ok, but you might run into problems with updates
if the versions shift around... but I guess as long as you re
'shrinkwrap' on updates it should work. We don't have any way to track
security issues for all the frozen set do we? How often do you see us
updating that set? everytime the hubs rpm updates? Or just when we want
to try and roll up to latest?

> By the way, Ryan has open a RFR (with an ominous number):
> https://pagure.io/fedora-infrastructure/issue/
> Is it missing any info?
I don't think so off hand. Looks like pingou is listed as the sponsor so
when things are ready he would be the one to tell you are a go for a
staging instance(s) and all the other next steps (sops, playbooks,
nagios monitoring, etc).

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

[PATCH] Added script to check age of mirrorlist pkl

[Adrian Reber]

The following script can be used for nagios to contact the mirrolist
servers and query the age of the pickle currently loaded.

This output of the script looks like this:

 $./check_mirrorlist_pkl_age.py 02 3000 5000
 WARN: mirrorlist data on proxy02 older than 3000s (3894s)

 $ ./check_mirrorlist_pkl_age.py 14 5000 7000
 OK: up-to-date mirrorlist data on proxy14 (3921s)

 $ ./check_mirrorlist_pkl_age.py 08 2000 3000
 CRIT: mirrorlist data on proxy08 older than 3000s (3938s)

Signed-off-by: Adrian Reber 
---
 .../nagios/plugins/check_mirrorlist_pkl_age.py | 96 ++
 1 file changed, 96 insertions(+)
 create mode 100755 
roles/nagios_server/files/nagios/plugins/check_mirrorlist_pkl_age.py

diff --git 
a/roles/nagios_server/files/nagios/plugins/check_mirrorlist_pkl_age.py 
b/roles/nagios_server/files/nagios/plugins/check_mirrorlist_pkl_age.py
new file mode 100755
index 0..2aa5bb2ca
--- /dev/null
+++ b/roles/nagios_server/files/nagios/plugins/check_mirrorlist_pkl_age.py
@@ -0,0 +1,96 @@
+#!/usr/bin/python3
+#
+# Script to check the age of the data used by the mirrorlist servers.
+#
+# Fedora's mirrorlist server are using a python pkl which has the creation
+# date embedded. Querying the mirrorlist interface with '' returns
+# that timestamp:
+#
+#  '# database creation time: '
+#
+# This script connects to the specified proxy and reads out that value
+# and compares it with a warning and critical threshold.
+#
+# Tested with python2 and python3
+#
+# Requires python[2,3]-requests
+#
+# Usage:
+#  check_mirrorlist_pkl_age.py   
+#  check_mirrorlist_pkl_age.py 12 3600 7200
+#
+# Author: Adrian Reber 
+
+import requests
+import sys
+import time
+import datetime
+
+if len(sys.argv) != 4:
+print("Usage:")
+print(" %s needs 3 parameters\n" % sys.argv[0])
+print(" %s   " % sys.argv[0])
+print(" %s 12 3600 7200" % sys.argv[0])
+sys.exit(3)
+
+proxy = sys.argv[1]
+warn = int(sys.argv[2])
+crit = int(sys.argv[3])
+
+check_url = 'http://proxy%s.fedoraproject.org/' % proxy
+check_url += 'mirrorlist?repo=fedora-rawhide=x86_64'
+
+headers = {'Host': 'mirrors.fedoraproject.org'}
+
+try:
+r = requests.get(check_url, headers=headers)
+except:
+print('CRIT: getting data from proxy%s failed' % proxy)
+sys.exit(2)
+
+
+if r.status_code != 200:
+print('CRIT: unexpected response (not 200) from proxy%s' % proxy)
+sys.exit(2)
+
+for line in r.iter_lines():
+if b'database creation time' not in line:
+continue
+
+ts = 0
+now = 0
+try:
+time_from_proxy = line.decode().split(': ')[1:][0]
+ts = datetime.datetime.strptime(
+time_from_proxy,
+'%Y-%m-%d %H:%M:%S.%f'
+)
+ts = int(time.mktime(ts.timetuple()))
+
+now = datetime.datetime.utcnow()
+now = int(time.mktime(now.timetuple()))
+
+except:
+print('CRIT: failure parsing result from proxy%s' % proxy)
+sys.exit(2)
+
+if (len == 0) or (now == 0):
+print('CRIT: failure parsing result from proxy%s' % proxy)
+sys.exit(2)
+
+age = int(now - ts)
+
+if age > crit:
+print(
+'CRIT: mirrorlist data on proxy%s older than %ds (%ds)' %
+(proxy, crit, age))
+sys.exit(2)
+
+if age > warn:
+print(
+'WARN: mirrorlist data on proxy%s older than %ds (%ds)' %
+(proxy, warn, age))
+sys.exit(1)
+
+print('OK: up-to-date mirrorlist data on proxy%s (%ds)' % (proxy, age))
+sys.exit(0)
-- 
2.14.3
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Script to check mirrorlist age

[Adrian Reber]

The following patch would add a script to check for the age of the
mirrorlist servers. If somebody could add the necessary nagios
configuration to let this run against the proxies I would add this
script to the repository.

The script makes it easy to verify that the mirrorlist containers have
up-to-date data:

 $ ./check_mirrorlist_pkl_age.py 02 3000 5000
 WARN: mirrorlist data on proxy02 older than 3000s (3894s)

Is this a check that would make sense for the current Nagios setup? We
probably need to figure out the right values for warning and critical.

Adrian
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org