Re: FBR: add sysadmin-analysis to bastion

2020-04-20 Thread Kevin Fenzi
On Mon, Apr 20, 2020 at 06:50:40PM -0400, Stephen John Smoogen wrote:
> The sysadmin-analysis group is used on data-analysis01 for people to log in
> and work there. Because everyone in that group was in an existing group, I
> forgot to add it to bastion. However a new person joined and they don't
> have access.
> 
> [smooge@batcave01 ansible (master)]$ git diff
> diff --git a/inventory/group_vars/bastion b/inventory/group_vars/bastion
> index 8e63d1a..160f1d1 100644
> --- a/inventory/group_vars/bastion
> +++ b/inventory/group_vars/bastion
> @@ -23,7 +23,7 @@ custom_rules: [
> 
>  # TODO - remove modularity-wg membership here once it is not longer needed:
>  # https://fedorahosted.org/fedora-infrastructure/ticket/5363
> -fas_client_groups:
> sysadmin-ask,sysadmin-atomic,sysadmin-web,sysadmin-main,sysadmin-cvs,sysadmin-noc,sysadmin-releng,sysadmin-dba,sysadmin-hosted,sysadmin-tools,sysadmin-spin,sysadmin-cloud,fi-apprentice,sysadmin-badges,sysadmin-troubleshoot,sysadmin-qa,sysadmin-centos,sysadmin-ppc,sysadmin-koschei,sysadmin-secondary,sysadmin-fedimg,sysadmin-veteran,sysadmin-mbs,modularity-wg,pungi-devel,sysadmin-upstreamfirst,sysadmin-releasemonitoring,sysadmin-gnome,sysadmin-copr,sysadmin-coreos,sysadmin-dbgserver,sysadmin-osbs,sysadmin-odcs
> +fas_client_groups: sysadmin-analysis,
> sysadmin-ask,sysadmin-atomic,sysadmin-web,sysadmin-main,sysadmin-cvs,sysadmin-noc,sysadmin-releng,sysadmin-dba,sysadmin-hosted,sysadmin-tools,sysadmin-spin,sysadmin-cloud,fi-apprentice,sysadmin-badges,sysadmin-troubleshoot,sysadmin-qa,sysadmin-centos,sysadmin-ppc,sysadmin-koschei,sysadmin-secondary,sysadmin-fedimg,sysadmin-veteran,sysadmin-mbs,modularity-wg,pungi-devel,sysadmin-upstreamfirst,sysadmin-releasemonitoring,sysadmin-gnome,sysadmin-copr,sysadmin-coreos,sysadmin-dbgserver,sysadmin-osbs,sysadmin-odcs

+1

kevin


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


FBR: add sysadmin-analysis to bastion

2020-04-20 Thread Stephen John Smoogen
The sysadmin-analysis group is used on data-analysis01 for people to log in
and work there. Because everyone in that group was in an existing group, I
forgot to add it to bastion. However a new person joined and they don't
have access.

[smooge@batcave01 ansible (master)]$ git diff
diff --git a/inventory/group_vars/bastion b/inventory/group_vars/bastion
index 8e63d1a..160f1d1 100644
--- a/inventory/group_vars/bastion
+++ b/inventory/group_vars/bastion
@@ -23,7 +23,7 @@ custom_rules: [

 # TODO - remove modularity-wg membership here once it is not longer needed:
 # https://fedorahosted.org/fedora-infrastructure/ticket/5363
-fas_client_groups:
sysadmin-ask,sysadmin-atomic,sysadmin-web,sysadmin-main,sysadmin-cvs,sysadmin-noc,sysadmin-releng,sysadmin-dba,sysadmin-hosted,sysadmin-tools,sysadmin-spin,sysadmin-cloud,fi-apprentice,sysadmin-badges,sysadmin-troubleshoot,sysadmin-qa,sysadmin-centos,sysadmin-ppc,sysadmin-koschei,sysadmin-secondary,sysadmin-fedimg,sysadmin-veteran,sysadmin-mbs,modularity-wg,pungi-devel,sysadmin-upstreamfirst,sysadmin-releasemonitoring,sysadmin-gnome,sysadmin-copr,sysadmin-coreos,sysadmin-dbgserver,sysadmin-osbs,sysadmin-odcs
+fas_client_groups: sysadmin-analysis,
sysadmin-ask,sysadmin-atomic,sysadmin-web,sysadmin-main,sysadmin-cvs,sysadmin-noc,sysadmin-releng,sysadmin-dba,sysadmin-hosted,sysadmin-tools,sysadmin-spin,sysadmin-cloud,fi-apprentice,sysadmin-badges,sysadmin-troubleshoot,sysadmin-qa,sysadmin-centos,sysadmin-ppc,sysadmin-koschei,sysadmin-secondary,sysadmin-fedimg,sysadmin-veteran,sysadmin-mbs,modularity-wg,pungi-devel,sysadmin-upstreamfirst,sysadmin-releasemonitoring,sysadmin-gnome,sysadmin-copr,sysadmin-coreos,sysadmin-dbgserver,sysadmin-osbs,sysadmin-odcs

 #
 # This is a postfix gateway. This will pick up gateway postfix config in
base
-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


Re: FBR: To update ostree on bodhi-backend01, branched-composer, rawhide-composer

2020-04-20 Thread Kevin Fenzi
On Mon, Apr 20, 2020 at 04:23:39PM -0400, Mohan Boddu wrote:
> Hi,
> 
> Randomly we hit an issue in "ostree pull-local" command that it wont
> sync and throws out a traceback. The new ostree build adds more
> logging and should help us with concurrently pulling.
> 
> Both rawhide-composer.phx2.fp.o and branched-composer.phx2.fp.o are on
> F31 and there is a build for it and submitted as an update
> https://bodhi.fedoraproject.org/updates/FEDORA-2020-f265f98566
> 
> For bodhi-backend01.phx2.fp.o is still on F30, for which there isn't a
> build for f30. So, I backported the patches to F30 and built an infra
> build for F30 https://koji.fedoraproject.org/koji/taskinfo?taskID=43574522.
> Its not tagged because I dont have infra permissions, but I tagged it
> manually and it is signed now and tagged into f30-infra-stg. I will
> move it to f30-infra once this FBR is approved and will update the
> box.
> 
> If you think we need to be careful, then I can apply the patch to
> rawhide-composer.phx2.fp.o and based on how that goes, we can update
> the others.

+1 

kevin


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


FBR: To update ostree on bodhi-backend01, branched-composer, rawhide-composer

2020-04-20 Thread Mohan Boddu
Hi,

Randomly we hit an issue in "ostree pull-local" command that it wont
sync and throws out a traceback. The new ostree build adds more
logging and should help us with concurrently pulling.

Both rawhide-composer.phx2.fp.o and branched-composer.phx2.fp.o are on
F31 and there is a build for it and submitted as an update
https://bodhi.fedoraproject.org/updates/FEDORA-2020-f265f98566

For bodhi-backend01.phx2.fp.o is still on F30, for which there isn't a
build for f30. So, I backported the patches to F30 and built an infra
build for F30 https://koji.fedoraproject.org/koji/taskinfo?taskID=43574522.
Its not tagged because I dont have infra permissions, but I tagged it
manually and it is signed now and tagged into f30-infra-stg. I will
move it to f30-infra once this FBR is approved and will update the
box.

If you think we need to be careful, then I can apply the patch to
rawhide-composer.phx2.fp.o and based on how that goes, we can update
the others.

Thanks.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


Re: [PATCH] fas_client: fix template to correctly apply on pkgs02 and add people02

2020-04-20 Thread Stephen John Smoogen
On Mon, 20 Apr 2020 at 11:13, Pierre-Yves Chibon 
wrote:

> On Mon, Apr 20, 2020 at 02:41:51PM +, Kevin Fenzi wrote:
> > The ansible_hostname variable is actually the short name of the host,
> > not the fqdn, so this conditional didn't match before. Switch it to use
> > startswith and also add people02 as thats the other host people try and
> > login to often after changing ssh keys.
> >
> > With this, pkgs02 and people02 should hopefully update ssh keys from fas
> > every 15min and avoid manual sync requests to the team.
> >
> > Signed-off-by: Kevin Fenzi 
> > ---
> >  roles/fas_client/templates/fas-client.cron.j2 | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/roles/fas_client/templates/fas-client.cron.j2
> b/roles/fas_client/templates/fas-client.cron.j2
> > index c0de939..8dd0a78 100644
> > --- a/roles/fas_client/templates/fas-client.cron.j2
> > +++ b/roles/fas_client/templates/fas-client.cron.j2
> > @@ -1,4 +1,4 @@
> > -{% if ansible_hostname == 'pkgs02.phx2.fedoraproject.org' %}
> > +{% if ansible_hostname.startswith(('pkgs02', 'people02')) %}
> >  */15 * * * * root /usr/local/bin/lock-wrapper fasClient
> "/usr/bin/fasClient -i |& grep -vi deprecation | /usr/local/bin/nag-once
> fassync 1d 2>&1"
> >  {% else %}
> >  00 20 * * * root /usr/local/bin/lock-wrapper fasClient "/bin/sleep
> $(($RANDOM \% 3600)); /usr/bin/fasClient -i |& grep -vi deprecation |
> /usr/local/bin/nag-once fassync 1d 2>&1"
>
> +1 for me, should we include pkgs01 for stg?
>

+1 with the same caveat?



>
> Pierre
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to
> infrastructure-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
>


-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


Re: [PATCH] fas_client: fix template to correctly apply on pkgs02 and add people02

2020-04-20 Thread Pierre-Yves Chibon
On Mon, Apr 20, 2020 at 02:41:51PM +, Kevin Fenzi wrote:
> The ansible_hostname variable is actually the short name of the host,
> not the fqdn, so this conditional didn't match before. Switch it to use
> startswith and also add people02 as thats the other host people try and
> login to often after changing ssh keys.
> 
> With this, pkgs02 and people02 should hopefully update ssh keys from fas
> every 15min and avoid manual sync requests to the team.
> 
> Signed-off-by: Kevin Fenzi 
> ---
>  roles/fas_client/templates/fas-client.cron.j2 | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/roles/fas_client/templates/fas-client.cron.j2 
> b/roles/fas_client/templates/fas-client.cron.j2
> index c0de939..8dd0a78 100644
> --- a/roles/fas_client/templates/fas-client.cron.j2
> +++ b/roles/fas_client/templates/fas-client.cron.j2
> @@ -1,4 +1,4 @@
> -{% if ansible_hostname == 'pkgs02.phx2.fedoraproject.org' %}
> +{% if ansible_hostname.startswith(('pkgs02', 'people02')) %}
>  */15 * * * * root /usr/local/bin/lock-wrapper fasClient "/usr/bin/fasClient 
> -i |& grep -vi deprecation | /usr/local/bin/nag-once fassync 1d 2>&1"
>  {% else %}
>  00 20 * * * root /usr/local/bin/lock-wrapper fasClient "/bin/sleep 
> $(($RANDOM \% 3600)); /usr/bin/fasClient -i |& grep -vi deprecation | 
> /usr/local/bin/nag-once fassync 1d 2>&1"

+1 for me, should we include pkgs01 for stg?


Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


[PATCH] fas_client: fix template to correctly apply on pkgs02 and add people02

2020-04-20 Thread Kevin Fenzi
The ansible_hostname variable is actually the short name of the host,
not the fqdn, so this conditional didn't match before. Switch it to use
startswith and also add people02 as thats the other host people try and
login to often after changing ssh keys.

With this, pkgs02 and people02 should hopefully update ssh keys from fas
every 15min and avoid manual sync requests to the team.

Signed-off-by: Kevin Fenzi 
---
 roles/fas_client/templates/fas-client.cron.j2 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/roles/fas_client/templates/fas-client.cron.j2 
b/roles/fas_client/templates/fas-client.cron.j2
index c0de939..8dd0a78 100644
--- a/roles/fas_client/templates/fas-client.cron.j2
+++ b/roles/fas_client/templates/fas-client.cron.j2
@@ -1,4 +1,4 @@
-{% if ansible_hostname == 'pkgs02.phx2.fedoraproject.org' %}
+{% if ansible_hostname.startswith(('pkgs02', 'people02')) %}
 */15 * * * * root /usr/local/bin/lock-wrapper fasClient "/usr/bin/fasClient -i 
|& grep -vi deprecation | /usr/local/bin/nag-once fassync 1d 2>&1"
 {% else %}
 00 20 * * * root /usr/local/bin/lock-wrapper fasClient "/bin/sleep $(($RANDOM 
\% 3600)); /usr/bin/fasClient -i |& grep -vi deprecation | 
/usr/local/bin/nag-once fassync 1d 2>&1"
-- 
1.8.3.1
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


Re: FBR: Update all systems to nrpe-4.0.4

2020-04-20 Thread Stephen John Smoogen
On Sun, 19 Apr 2020 at 14:46, Kevin Fenzi  wrote:

> On Sun, Apr 19, 2020 at 01:55:56PM -0400, Stephen John Smoogen wrote:
> > NRPE for Fedora was updated to 4.0.4 which got auto-updated on many of
> the
> > infrastructure systems. However, the noc servers are still running a much
> > older version of nrpe which is causing some issues with monitoring.
> >
> > Plan: run on batcave01
> >
> > sudo ansible -i noc:batcave:bastion:people -m shell -a 'yum
> > --enablerepo=epel-testing update nagios* nrpe*'
> > sudo ansible -i noc:batcave:bastion:people -m shell -a 'rkhunter
> --propupd'
>
> +1, but note the problem is all rhel7 hosts. Fedora got a stable update
> and we applied it, but rhel7 still doesn't have it (still in testing).
>
> So, how about:
>
> ansible -m shell -a 'yum -y --enablerepo=epel-testing update nagios*
> nrpe*' distro_RedHat
>
>
I implemented this final version  and ran it. This FBR is complete.


> That will get only all the rhel7/8 machines. :)
>
> kevin
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to
> infrastructure-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
>


-- 
Stephen J Smoogen.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


Re: Backlog prioritization

2020-04-20 Thread Pierre-Yves Chibon
On Mon, Apr 20, 2020 at 09:48:45AM +0100, Mark O'Brien wrote:
>I would like to work on this today if possible. Obviously I'm not very
>familiar with the system so may slow you down a little but it would be
>good to get hands on with it.

Let's go this this afternoon then, we can sync on #fedora-admin if you're ok
with this :)

The agenda should be something like:
- Backup: DB, git repos, releases
- Move backups to safe place (the batcave)
- Destroy current instance
- Rebuild it as RHEL8 (adjust playbook where needed)
- Reload backups
- Figure out remaining fires

Pretty straight forward :)

Pierre
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org


Re: Backlog prioritization

2020-04-20 Thread Mark O'Brien
I would like to work on this today if possible. Obviously I'm not very
familiar with the system so may slow you down a little but it would be good
to get hands on with it.

Mark

On Sun, Apr 19, 2020 at 7:20 PM Pierre-Yves Chibon 
wrote:

> On Fri, Apr 17, 2020 at 02:13:36PM -0700, Kevin Fenzi wrote:
> > On Fri, Apr 17, 2020 at 11:40:25AM +0200, Pierre-Yves Chibon wrote:
> > > On Fri, Apr 17, 2020 at 09:03:02AM +0200, Clement Verna wrote:
> > > >  > * Migrate [6]stg.pagure.io and [7]src.stg.fedoraproject.org
> to RHEL8.
> > > >  >   While we're in freeze, I figure this is a good time to do
> this. We
> > > >  could do
> > > >  >   [8]pagure.io post-freeze and wait to do src.fp.o when it
> gets
> > > >  reinstalled in the
> > > >  >   new data-center.
> > > >
> > > >  Yeah, good to do. I was going to ask you about this the other
> day.
> > > >  Perhaps I could reinstall stg pagure with rhel8 some day my
> night and
> > > >  you could take over your next morning with reloading the old
> data/etc?
> > > >
> > > >  I agree it's a good time to do it.
> > > >
> > > >  Perhaps monday night I could try and do it and you could work
> on it
> > > >  tuesday morning?
> > > >
> > > >That could be also a good first issue for Mark to work on with
> pingou on
> > > >it since timezone will make that easier :-)
> > > >What do you think ?
> > >
> > > That sounds good to me :)
> >
> > ok, so monday night I would:
> >
> > * take down pagure-stg
> > * spin up a new rhel8 based one.
> > * run the playbook as far as it will easily go
> > * copy a dump of the database and the old data to the new vm?
> > * let pingou and mark look at bringing it up the rest of the way
> >
> > Do we want to save the old data? I guess it would be good to make sure
> > it all works before we do it in prod? Or do we want to just start out
> > with a new clean one?
> >
> > Or do you want me to just step back here and you two can do the entire
> > thing? (Thats just fine with me too).
>
> I'll let Mark answer there, but I'm fine with doing the entire procedure
> (which
> mean, we could start on this on directly Monday).
>
> Mark, any preferences?
>
>
> Pierre
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to
> infrastructure-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
>
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org