Re: FBR bodhi-pungi: Add AtomicHost variant

[Dusty Mabe]

On 04/19/2018 10:27 PM, Mohan Boddu wrote:
> LGTM +1

pushed.. Thanks!
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR bodhi-pungi: Add AtomicHost variant

[Mohan Boddu]

LGTM +1
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

[Fedocal] Reminder meeting : Fedora Tagger - future development meeting

[jflory7]

Dear all,

You are kindly invited to the meeting:
   Fedora Tagger - future development meeting on 2018-04-20 from 14:00:00 to 
15:00:00 UTC
   At fedora-meetin...@irc.freenode.net

The meeting will be about:
This meeting is for stakeholders and interested developers in the future of the 
Fedora Tagger application. This meeting will focus on figuring out milestones 
to update the Tagger application and how to support it in Fedora's 
infrastructure going forward.

Anyone is welcome to participate. Please see the 
[fedora-infra/fedora-tagger#200](https://github.com/fedora-infra/fedora-tagger/issues/200)
 for more information.


Source: https://apps.fedoraproject.org/calendar/meeting/9217/

___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR bodhi-pungi: Add AtomicHost variant

[Kevin Fenzi]

+1

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

[PATCH] bodhi-pungi: Add AtomicHost variant

[Dusty Mabe]

Otherwise we get messages like this when trying to create
Atomic Host artifacts:

```
[WARNING ] [OSTREE_INSTALLER] Patterns in config do not match any variant: 
^AtomicHost$
```
---
 roles/bodhi2/backend/templates/variants.rpm.xml.j2 | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/roles/bodhi2/backend/templates/variants.rpm.xml.j2 
b/roles/bodhi2/backend/templates/variants.rpm.xml.j2
index 4b6625086..d36d31437 100644
--- a/roles/bodhi2/backend/templates/variants.rpm.xml.j2
+++ b/roles/bodhi2/backend/templates/variants.rpm.xml.j2
@@ -22,4 +22,11 @@
[% endif %]
 
 
+
+
+aarch64
+x86_64
+ppc64le
+
+
 
-- 
2.14.3
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

FBR bodhi-pungi: Add AtomicHost variant

[Dusty Mabe]

Hopefully one last change to get the AH artifacts
generating in bodhi/pungi.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR fixes to bodhi-pungi configs for atomichost artifacts

[Dusty Mabe]

On 04/19/2018 03:11 PM, Patrick  マルタインアンドレアス  Uiterwijk wrote:
>> This is an attempt to fix https://pagure.io/dusty/failed-composes/issue/176
>>
>> I'd like to get patrick and one other person to review these changes.
> 
> Sure, the changes look fine and easily revertable.
> Also, all these artifacts are failable.
> So +1.

Deployed and hit an issue. small fixup patch that was just applied
and approved by patrick/kevin in IRC:

```
$ git diff --ignore-all-space   
diff --git a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 
b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2  
index b7c4bd529..ea4fe8476 100644  
--- a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 
+++ b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 
@@ -195,8 +195,8 @@ volume_id_substitutions = {
 # Other ostree artifacts  
 image_build = {   
 '^AtomicHost$': [ 
-{ 
 [% for arch in ['x86_64', 'aarch64', 'ppc64le'] %] 
  
+{ 
 'image-build': {  
 'format': [('qcow2', 'qcow2'), ('raw-xz', 'raw.xz')],  
  
 'name': 'Fedora-Atomic',  
@@ -208,9 +208,9 @@ image_build = {
 'install_tree_from': "https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/branched/latest-Fedora-[[ release.version_int 
]]/compose/Everything/[[arch]]/os/" 

 'subvariant': 'AtomicHost',   
 'failable': ['*'],
+} 
 },
 [% endfor %]  
-} 
 ] 
 }
```
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: Fix the metadata handling for Rawhide messages

[Stephen John Smoogen]

+1 for the "we need to make this work since it b0rked now"

On 19 April 2018 at 16:51, Dusty Mabe  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
>
>
> On 04/19/2018 12:42 PM, Sayan Chowdhury wrote:
>> Hi,
>>
>> Here is a hotfix for fedimg. Right now Rawhide messages are not
>> getting processed. This hotfix will fix the issue.
>>
>> PR for the same[1]
>>
>> [1] https://github.com/fedora-infra/fedimg/pull/103
>>
>> +1s
>>
>
> reviewed the PR - +1, though I would like to include
> https://github.com/fedora-infra/fedimg/pull/104 as well
>
> Dusty
> -BEGIN PGP SIGNATURE-
>
> iQIzBAEBCAAdFiEEPb6zG5c6sV89tRYPMwLb1zlS5nEFAlrZAV0ACgkQMwLb1zlS
> 5nEscg//QoBhV8d9Iaq1DtEx5FdCEdFXnmba73/zwVXLfNQTLizFWV/FZONabVzb
> R1uH+zVzuPZpBLr68HyoGQLgViC+EHEYyxkJJc4JABk/MeL2kJH8SoABwnKWmAIe
> a2u9JBU+uq2SJOYdaOwHjQFBm2J2OG44vIxosWcR2LO3jqE0pyXtfnWtP8evJZgg
> 3QOtREhdFolWzUdsiX6Y7MPoH2kg3TNwrtHAEnD/92OGZi6W1WXbXPxBWhupo1SG
> NBnPGQ0LhJC0ypPzzCo/oa5gd7bfaf9ntdm+cxiMAV9u6s1zYhGc/NzGU1fZsV38
> OIZFU3uZWu80wnK67QNI/JSXG3/1rzne9rSXZYfinT1Jin/1iVk+Bh+I+t4+pX7a
> h0tRB+qGMRh7hBkGqO9odKWZdjjmLMol0+dkN5W4pHM60kyoI4Gj4MhiLa3qTTOV
> UQ1vhkuW6+e6u33UORXxeO4IyAJwuSPgjcSnA6cDVBS/cgEyir5d+lS5j/g0KYp4
> eTwK9KPxdrZwPPwT5ILL+fDsalbNqYYAc3x9nba3hHzc+kOA4W3ubog8IYB8/eJP
> 8ZIuIuxMYzpzPfwoWHFBmvkjdXLSnJq7jkCQ9VhntCDZAJAe0Mrq46iMRr7DaDO2
> F8dnaREU/6R4PH9W/f9vqNtxixIB8M08v8ZrdBd4txW4jkvIOWs=
> =cZTD
> -END PGP SIGNATURE-
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: Change to category based crawling

[Stephen John Smoogen]

+1

On 19 April 2018 at 02:54, Adrian Reber  wrote:
> As discussed previously I would like to change the crawler to crawl each
> category separately. The goal is to reduce the load on the database by
> distributing the crawling better over the whole day and to reduce the
> chance of mirrors being disabled because of the high database load.
>
> This should also remove the need for mirror administrators to create
> multiple hosts in MirrorManager to work around the 4 hours timeout per
> host.
>
> Attached is my patch. Please +1. This affects mm-crawler01 and
> mm-crawler02.
>
> Adrian
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
>



-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Freeze Break Request: re-enable fedmsg hook on src

[Stephen John Smoogen]

OK this makes sense now why it is not there. +!

On 19 April 2018 at 04:18, Pierre-Yves Chibon  wrote:
> On Thu, Apr 19, 2018 at 02:17:00AM +0200, Michal Novotny wrote:
>>This only removes the commented out line. We should actually fully revert
>>
>> https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=dc86b22f9054bdcefd51e46db90433c03d54410c,
>
> +1 to this, it'll be just as easy to revert this commit now that the 
> permission
> issue got fixed (Thanks Patrick for that!)
>
> Sorry about this commit breaking grok btw, I didn't realize commenting out one
> line in pee would also impact the next line, although it does make sense now
> that it's pointed out.
>
>
>
> Pierre
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: Fix the metadata handling for Rawhide messages

[Dusty Mabe]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256



On 04/19/2018 12:42 PM, Sayan Chowdhury wrote:
> Hi,
> 
> Here is a hotfix for fedimg. Right now Rawhide messages are not
> getting processed. This hotfix will fix the issue.
> 
> PR for the same[1]
> 
> [1] https://github.com/fedora-infra/fedimg/pull/103
> 
> +1s
> 

reviewed the PR - +1, though I would like to include 
https://github.com/fedora-infra/fedimg/pull/104 as well

Dusty 
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEPb6zG5c6sV89tRYPMwLb1zlS5nEFAlrZAV0ACgkQMwLb1zlS
5nEscg//QoBhV8d9Iaq1DtEx5FdCEdFXnmba73/zwVXLfNQTLizFWV/FZONabVzb
R1uH+zVzuPZpBLr68HyoGQLgViC+EHEYyxkJJc4JABk/MeL2kJH8SoABwnKWmAIe
a2u9JBU+uq2SJOYdaOwHjQFBm2J2OG44vIxosWcR2LO3jqE0pyXtfnWtP8evJZgg
3QOtREhdFolWzUdsiX6Y7MPoH2kg3TNwrtHAEnD/92OGZi6W1WXbXPxBWhupo1SG
NBnPGQ0LhJC0ypPzzCo/oa5gd7bfaf9ntdm+cxiMAV9u6s1zYhGc/NzGU1fZsV38
OIZFU3uZWu80wnK67QNI/JSXG3/1rzne9rSXZYfinT1Jin/1iVk+Bh+I+t4+pX7a
h0tRB+qGMRh7hBkGqO9odKWZdjjmLMol0+dkN5W4pHM60kyoI4Gj4MhiLa3qTTOV
UQ1vhkuW6+e6u33UORXxeO4IyAJwuSPgjcSnA6cDVBS/cgEyir5d+lS5j/g0KYp4
eTwK9KPxdrZwPPwT5ILL+fDsalbNqYYAc3x9nba3hHzc+kOA4W3ubog8IYB8/eJP
8ZIuIuxMYzpzPfwoWHFBmvkjdXLSnJq7jkCQ9VhntCDZAJAe0Mrq46iMRr7DaDO2
F8dnaREU/6R4PH9W/f9vqNtxixIB8M08v8ZrdBd4txW4jkvIOWs=
=cZTD
-END PGP SIGNATURE-
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Moving meeting time?

[Kevin Fenzi]

On 04/19/2018 05:32 AM, Pierre-Yves Chibon wrote:
> On Wed, Apr 18, 2018 at 02:31:31PM -0700, Kevin Fenzi wrote:
>> Greetings.
>>
>> It was noted recently that our current meeting time (thursdays at 18UTC)
>> is a bit late/poorly timed for several of our European colleagues.
>>
>> Additionally, we sometimes have the go/no-go meeting at the same time
>> which can be inconvenient.
>>
>> What would everyone think of moving the meeting to 16UTC?
>> Or is there a better time for anyone?
> 
> To be truthful, I think 14UTC would be best for me, it would basically allow 
> me
> to attend the meeting before the end of my working hours and without impact on
> the family time.
> But that does move the meeting up into your family time, so not ideal there :(

Yeah, thats 7am for me... I can do it, but I can't promise I will be
awake. ;)

Anyhow, we decided in the meeting today to do a poll.

Please vote for the time (In UTC) that you like best:

https://framadate.org/VnfFPCae5wRKZuc3

Thanks!

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR fixes to bodhi-pungi configs for atomichost artifacts

[Patrick マルタインアンドレアス Uiterwijk]

> This is an attempt to fix https://pagure.io/dusty/failed-composes/issue/176
> 
> I'd like to get patrick and one other person to review these changes.

Sure, the changes look fine and easily revertable.
Also, all these artifacts are failable.
So +1.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR fixes to bodhi-pungi configs for atomichost artifacts

[Dusty Mabe]

On 04/19/2018 08:11 AM, Mohan Boddu wrote:
> I agree with Kevin, but also it seems that patch should be working
> 
> So, +1

Thanks Kevin/Mohan. I'll merge this after today's bodhi runs are done
and then ask Mohan to kick off another 28-u-t run.

Dusty 
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

FBR: Fix the metadata handling for Rawhide messages

[Sayan Chowdhury]

Hi,

Here is a hotfix for fedimg. Right now Rawhide messages are not
getting processed. This hotfix will fix the issue.

PR for the same[1]

[1] https://github.com/fedora-infra/fedimg/pull/103

+1s

From b52a1dd1eda25bfbade026a1ebde93839fa3379b Mon Sep 17 00:00:00 2001
From: Sayan Chowdhury 
Date: Thu, 19 Apr 2018 22:05:27 +0530
Subject: [PATCH 1/2] fedimg: Add the intial files for PR#103

Signed-off-by: Sayan Chowdhury 
---
 files/hotfix/fedimg/consumers.py | 160 +++
 roles/fedimg/tasks/main.yml  |   8 ++
 2 files changed, 168 insertions(+)
 create mode 100644 files/hotfix/fedimg/consumers.py

diff --git a/files/hotfix/fedimg/consumers.py b/files/hotfix/fedimg/consumers.py
new file mode 100644
index 000..ce4d662
--- /dev/null
+++ b/files/hotfix/fedimg/consumers.py
@@ -0,0 +1,160 @@
+# -*- coding: utf-8 -*-
+# This file is part of fedimg.
+# Copyright (C) 2014-2017 Red Hat, Inc.
+#
+# fedimg is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# fedimg is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with fedimg; if not, see http://www.gnu.org/licenses,
+# or write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Authors:  David Gay 
+#   Sayan Chowdhury 
+"""
+This is the `fedmsg consumer`_ that subscribes to the topic emitted after the
+completion of the nightly and production compose. The consumer on receving the
+message uploads the image using the API of the cloud providers.
+"""
+
+import logging
+import multiprocessing.pool
+
+import fedmsg.consumers
+import fedmsg.encoding
+import fedfind.release
+import fedfind.exceptions
+
+import fedimg.uploader
+
+from fedimg.config import PROCESS_COUNT, STATUS_FILTER
+from fedimg.utils import get_rawxz_urls, get_value_from_dict
+
+_log = logging.getLogger(__name__)
+
+
+class FedimgConsumer(fedmsg.consumers.FedmsgConsumer):
+"""
+A `fedmsg consumer`_ that listens to the pungi compose topics and kicks
+of the process to upload the images to various cloud providers.
+
+Attributes:
+topic (str): The topics this consumer is subscribed to. Set to
+``org.fedoraproject.prod.pungi.compose.status.change``.
+config_key (str): The key to set to ``True`` in the fedmsg config to
+enable this consumer. The key is ``fedimgconsumer.prod.enabled``.
+"""
+topic = ['org.fedoraproject.prod.pungi.compose.status.change']
+config_key = "fedimgconsumer.prod.enabled"
+
+def __init__(self, *args, **kwargs):
+_log.info("FedimgConsumer initializing")
+super(FedimgConsumer, self).__init__(*args, **kwargs)
+
+# Threadpool for upload jobs
+_log.info("Creating thread pool of %s process", PROCESS_COUNT)
+self.upload_pool = multiprocessing.pool.ThreadPool(
+processes=PROCESS_COUNT
+)
+_log.info("FedimgConsumer initialized")
+
+def consume(self, msg):
+"""
+This is called when we receive a message matching our topics.
+
+Args:
+msg (dict): The raw message from fedmsg.
+"""
+_log.info('Received %r %r', msg['topic'], msg['body']['msg_id'])
+
+msg_info = msg['body']['msg']
+if msg_info['status'] not in STATUS_FILTER:
+_log.debug('%s is not valid status' % msg_info['status'])
+return
+
+location = msg_info['location']
+compose_id = msg_info['compose_id']
+try:
+compose_metadata =
fedfind.release.get_release(cid=compose_id).metadata
+except fedfind.exceptions.UnsupportedComposeError:
+LOG.debug("%r is unsupported compose" % compose_id)
+return
+
+
+# Till F27, both cloud-base and atomic images were available
+# under variant CloudImages. With F28 and onward releases,
+# cloud-base image compose moved to cloud variant and atomic images
+# moved under atomic variant.
+prev_rel = ['26', '27']
+if msg_info['release_version'] in prev_rel:
+images_meta = get_value_from_dict(
+compose_metadata, 'images', 'payload', 'images', 'CloudImages',
+'x86_64')
+else:
+images_meta = get_value_from_dict(
+compose_metadata, 'images', 'payload', 'images',
+'Cloud', 'x86_64')
+

Fedora Tagger - Future development meeting

[Elorm Buertey]

Hi everyone,

You are kindly invited to the Fedora tagger meeting 
Starting from Fri, April 20, 2018 - 14:00 UTC  and ending on Fri, April 20, 
2018 - 15:00:00 UTC 
At fedora-meetin...@irc.freenode.net

This meeting is for stakeholders and interested developers in the future of the 
Fedora Tagger application.
This meeting will focus on figuring out milestones to update the Tagger 
application and how to support it in Fedora's infrastructure going forward.

Anyone is welcome to participate. Please see the link below for more 
information:

https://github.com/fedora-infra/fedora-tagger/issues/200

Regards,
Elorm Buertey
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: Change to category based crawling

[Kevin Fenzi]

+1

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Moving meeting time?

[Miroslav Suchý]

Dne 18.4.2018 v 23:31 Kevin Fenzi napsal(a):
> What would everyone think of moving the meeting to 16UTC?
> Or is there a better time for anyone?

+1
I cannot attend at 18UTC, but 16UTC is much better for me.

Miroslav



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Moving meeting time?

[Pierre-Yves Chibon]

On Wed, Apr 18, 2018 at 02:31:31PM -0700, Kevin Fenzi wrote:
> Greetings.
> 
> It was noted recently that our current meeting time (thursdays at 18UTC)
> is a bit late/poorly timed for several of our European colleagues.
> 
> Additionally, we sometimes have the go/no-go meeting at the same time
> which can be inconvenient.
> 
> What would everyone think of moving the meeting to 16UTC?
> Or is there a better time for anyone?

To be truthful, I think 14UTC would be best for me, it would basically allow me
to attend the meeting before the end of my working hours and without impact on
the family time.
But that does move the meeting up into your family time, so not ideal there :(


Pierre


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Moving meeting time?

[Michal Novotny]

On Thu, Apr 19, 2018 at 2:29 PM, Michal Novotny  wrote:

> On Wed, Apr 18, 2018 at 11:31 PM, Kevin Fenzi  wrote:
>
>> Greetings.
>>
>> It was noted recently that our current meeting time (thursdays at 18UTC)
>> is a bit late/poorly timed for several of our European colleagues.
>>
>> Additionally, we sometimes have the go/no-go meeting at the same time
>> which can be inconvenient.
>>
>> What would everyone think of moving the meeting to 16UTC?
>> Or is there a better time for anyone?
>>
>
> Personally, I couldn't really attend at 16 UTC on Thursdays because that's
> the time I am travelling from work to home (I could attend but not really
> properly
> - only from cell-phone in a public transport :(). But that's just me (and
> I wasn't
> really pinpoint with the meetings lately but wanted to improve it).
>

I am located at UTC+1/UTC+2 (based on daylight savings).


>
>
>>
>> This weeks meeting we will keep the same, but we can discuss and see if
>> we want to move it starting next week.
>>
>> Thanks,
>>
>> kevin
>>
>>
>> ___
>> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
>> To unsubscribe send an email to infrastructure-le...@lists.fed
>> oraproject.org
>>
>>
>
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Moving meeting time?

[Michal Novotny]

On Wed, Apr 18, 2018 at 11:31 PM, Kevin Fenzi  wrote:

> Greetings.
>
> It was noted recently that our current meeting time (thursdays at 18UTC)
> is a bit late/poorly timed for several of our European colleagues.
>
> Additionally, we sometimes have the go/no-go meeting at the same time
> which can be inconvenient.
>
> What would everyone think of moving the meeting to 16UTC?
> Or is there a better time for anyone?
>

Personally, I couldn't really attend at 16 UTC on Thursdays because that's
the time I am travelling from work to home (I could attend but not really
properly
- only from cell-phone in a public transport :(). But that's just me (and I
wasn't
really pinpoint with the meetings lately but wanted to improve it).


>
> This weeks meeting we will keep the same, but we can discuss and see if
> we want to move it starting next week.
>
> Thanks,
>
> kevin
>
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.
> fedoraproject.org
>
>
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR fixes to bodhi-pungi configs for atomichost artifacts

[Mohan Boddu]

I agree with Kevin, but also it seems that patch should be working

So, +1
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Freeze Break Request: re-enable fedmsg hook on src

[Pierre-Yves Chibon]

On Thu, Apr 19, 2018 at 02:17:00AM +0200, Michal Novotny wrote:
>This only removes the commented out line. We should actually fully revert
>
> https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=dc86b22f9054bdcefd51e46db90433c03d54410c,

+1 to this, it'll be just as easy to revert this commit now that the permission
issue got fixed (Thanks Patrick for that!)

Sorry about this commit breaking grok btw, I didn't realize commenting out one
line in pee would also impact the next line, although it does make sense now
that it's pointed out.



Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

FBR: Change to category based crawling

[Adrian Reber]

As discussed previously I would like to change the crawler to crawl each
category separately. The goal is to reduce the load on the database by
distributing the crawling better over the whole day and to reduce the
chance of mirrors being disabled because of the high database load.

This should also remove the need for mirror administrators to create
multiple hosts in MirrorManager to work around the 4 hours timeout per
host.

Attached is my patch. Please +1. This affects mm-crawler01 and
mm-crawler02.

Adrian
From b10d5ffa7e48e934da3350186eaf8dd4fb0cebf3 Mon Sep 17 00:00:00 2001
From: Adrian Reber 
Date: Tue, 17 Apr 2018 19:42:16 +0200
Subject: [PATCH] mirror crawler: crawl each category separately

This is the first try to split up the mirror crawling by category. One
of the goals is to better distribute the load on the database. If this
actually works the effects of this change have to be monitored.

Another result could be that mirrors do not get auto-deactivated that
fast. Previously there was a crawl timeout of 4 hours for all categories
together. Now it is 4 hours per category.
---
 .../mirrormanager/crawler/files/crawler.cron  | 27 ---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/roles/mirrormanager/crawler/files/crawler.cron 
b/roles/mirrormanager/crawler/files/crawler.cron
index 33f3967f3..24d141574 100644
--- a/roles/mirrormanager/crawler/files/crawler.cron
+++ b/roles/mirrormanager/crawler/files/crawler.cron
@@ -1,4 +1,4 @@
-# run the crawler twice a day
+# run the crawler for each MirrorManager category
 # logs sent to /var/log/mirrormanager/crawler.log and crawl/* by default
 #
 # [ "`hostname -s`" == "mm-crawler02" ] && sleep 6h is used to start the crawl
@@ -10,5 +10,26 @@
 # gracefully shutdown if it gets the signal SIGALRM(14).  After the signal we
 # wait for 5 minutes to give the crawler a chance to shutdown. After that the
 # crawler is killed.  To make sure we only end the cron started crawler we look
-# for the following process "/usr/bin/python /usr/bin/mm2_crawler --threads 
25".
-0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 6h; 
pkill -14 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --threads 20"; sleep 
5m; pkill -9 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --threads 20"; 
/usr/bin/mm2_crawler --threads 20 --timeout-minutes 240 
`/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
+# for the following process "/usr/bin/python /usr/bin/mm2_crawler 
--category=25".
+
+# The number of threads is based on the possible number of existing mirrors. 
More
+# threads for categories with more mirrors.
+
+# The goal is to distribute the crawling of all categories over the whole day.
+
+# The timeout is 4 hours, but for each category.
+
+# Category: 'Fedora Linux'; twice a day, 20 threads
+0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 6h; 
pkill -14 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --category=Fedora 
Linux"; sleep 5m; pkill -9 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler 
--category=Fedora Linux"; /usr/bin/mm2_crawler --category="Fedora Linux" 
--threads 20 --timeout-minutes 240 `/usr/local/bin/run_crawler.sh 2` > 
/dev/null 2>&1
+
+# Category: 'Fedora Secondary Arches'; twice a day, 10 threads
+0 3,9 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 1h; 
pkill -14 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --category=Fedora 
Secondary Arches"; sleep 5m; pkill -9 -f "^/usr/bin/python2 -s 
/usr/bin/mm2_crawler --category=Fedora Secondary Arches"; /usr/bin/mm2_crawler 
--category="Fedora Secondary Arches" --threads 10 --timeout-minutes 240 
`/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
+
+# Category: 'Fedora EPEL'; four times a day, 20 threads
+45 */6 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 1h; 
pkill -14 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --category=Fedora 
EPEL"; sleep 5m; pkill -9 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler 
--category=Fedora EPEL"; /usr/bin/mm2_crawler --category="Fedora EPEL" 
--threads 20 --timeout-minutes 240 `/usr/local/bin/run_crawler.sh 2` > 
/dev/null 2>&1
+
+# Category: 'Fedora Archive'; once a day, 10 threads
+0 2 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 6h; 
pkill -14 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --category=Fedora 
Archive"; sleep 5m; pkill -9 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler 
--category=Fedora Archive"; /usr/bin/mm2_crawler --category="Fedora Archive" 
--threads 10 --timeout-minutes 240 `/usr/local/bin/run_crawler.sh 2` > 
/dev/null 2>&1
+
+# Category: 'Fedora Other'; once a day, 10 threads
+0 14 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 6h; 
pkill -14 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler --category=Fedora 
Other"; sleep 5m; pkill -9 -f "^/usr/bin/python2 -s /usr/bin/mm2_crawler 
--category=Fedora Other"; /usr/bin/mm2_crawler --category="Fedora Other" 
--threads 10