date:20180420

On 04/20/2018 10:34 AM, Pierre-Yves Chibon wrote:
> Good Morning Everyone,
> 
> We have been informed thst week at the upstream devs working on the
> product-definition-center (PDC) are moving away from the project and are going
> to leave it without a maintainer. Since we adopted PDC for a variety of data
> flows, this puts us in an awkward position. :(
...snip..
> 
> I propose we start the discussion on the list and plan for a meeting sometime
> late next week to discuss it further with the interested parties (please 
> signal
> yourself)
> 
> 
> What do you think?

So, some generic thoughts:

* I kinda don't like the tree of files in git. It would work, but it
seems poor, like we don't know how to use a database. Along with just
strange to work with.

* Right now we have (I think) 3 Django apps... mailman3, pdc and ask. Of
course the people doing the work should choose, but it seems like with
Django there's always a treadmill of keeping it on the new version thats
supported, etc. We ran into this with pdc recently where the version
upstream was using wouldn't work with rhel or fedora django without work.

* If we are going to split out just the things we need, perhaps this
would be a good time to try our hand at microservices? ie, small as we
can make them, run in openshift, deploy quickly/all the time, etc If we
do that we should make sure to gather requirements and only make those
things we need to.

Definitely include me in on the planning... thanks.

kevin

signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

2018-04-20 Thread Clement Verna

On 20 April 2018 at 20:15, Randy Barlow  wrote:
> Bodhi also uses PDC to determine which packages are critpath per release.
>
> I'm interested in this topic, so please do invite me to discuss next week!
>
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
>

fedora-packages get the list of packages from PDC during the indexing
of the xapian database. You can add me in the discussion too.

I am not very familiar with django but could we keep the current PDC
setup and just disable the django apps we are not using ? That way we
would not have to touch the other apps depending on PDC.
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

2018-04-20 Thread Randy Barlow

Bodhi also uses PDC to determine which packages are critpath per release.

I'm interested in this topic, so please do invite me to discuss next week!



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: Fedora and PDC

2018-04-20 Thread Michal Novotny

* Stream branches, branch ownership, retirement dates?
- SLA/EOL are currently stored in PDC.
- Queried by releng scripts for retirement, fedrepo-req for new
branches,
  etc..
option #1
  git repo full of yaml file similar to the override repo
  compiled into a single JSON blob
  Single place for all retired packages
  This feels like the lowest tech option.
  git gives us change control for free... but people easily get lost in
the
  "UX" of navigating a gigantic git repo full of plaintext files.

I am all for this option because it makes everything public and it is
consistent
with the /tests/ and modules/ namespace approach that has been more recently
introduced. I think that option is actually even attractive.

Just noting...I am not a core developer in this subject but this comes to
mind
as a good way to implement the "branch meta-data storage app".

clime

On Fri, Apr 20, 2018 at 7:34 PM, Pierre-Yves Chibon 
wrote:

> Good Morning Everyone,
>
> We have been informed thst week at the upstream devs working on the
> product-definition-center (PDC) are moving away from the project and are
> going
> to leave it without a maintainer. Since we adopted PDC for a variety of
> data
> flows, this puts us in an awkward position. :(
>
> Ralph and I met up on Tuesday to brainstorm the list of things we actively
> use
> PDC for today and to come up with a contingency plan for how to handle
> them. One
> overarching option is for us (fedora-infra) to take ownership of the PDC
> codebase as a whole. We didn't fully explore this option, figuring that the
> codebase is large and contains lots of tables, endpoints, and codepaths
> that we
> didn't use nor which we plan to use.
>
> Instead, below we've got the four things we use PDC for and some options
> for
> what to do with each.
>
> With the exception of /modules/, one common pattern that we like is to
> investigate splitting out the "django apps" that make up PDC into their own
> projects.  We're calling these "pdc-lite", for fun. See more below.
>
> * Modules
> The data in the /modules/ PDC endpoint is *also* in the MBS db.
> Ralph's
> team is going to just use that and stop using pdc anything for modules.
> We're going to need to patch pungi, mbs for local builds, and a few
> other
> places.  This should be a relatively low-pain transition.
>
> * Stream branches, branch ownership, retirement dates?
> - SLA/EOL are currently stored in PDC.
> - Queried by releng scripts for retirement, fedrepo-req for new
> branches,
>   etc..
> option #1
>   git repo full of yaml file similar to the override repo
>   compiled into a single JSON blob
>   Single place for all retired packages
>   This feels like the lowest tech option.
>   git gives us change control for free... but people easily get lost
> in the
>   "UX" of navigating a gigantic git repo full of plaintext files.
> option #2
>   pagure's DB/API
>   pagure knows nothing about branches currently, so that would be
> bigger
>   lift
> option #3
>   PDC internally is composed of ~20 "django apps"
>   https://github.com/product-definition-center/product-
> definition-center/tree/master/pdc/apps
>   We could pick the 2 or 3 that comprise the branches feature, copy
> them
>   out, and turn them into their own service: the "branch definition
> center":
>   BDC.
>   That would be the "pdc-lite" approach mentionned above, ie: PDC with
> only
>   the "app" of interest
>
> * release/life-circle tracking?
> option #1:
>   PDC lite with just that app of interest
> option #2:
>   JSON/yaml file on the proxies
> option #3:
>   pkgdb-lite
> option #4:
>   ???
>   compose tracking?
> impacted: fedfind
> option #1:
>   PDC-lite with just that app of interest
> option #2:
>   Drop this entirely?
>   Adam probably really wants to keep the record of composes.
> option #3:
>   ???
>
> The "pdc-lite" options are attractive, across the board.  One thing we get
> from
> this is greater clarity when discussing things formerly in PDC.  If
> something is
> in the branch-definition-center, the compose-definition-center, or the
> release-definition-center.. you know what you're talking about.  Today,
> when
> talking about whether or not something should be or is in "PDC", it is
> easy to
> get confused.
>
> I propose we start the discussion on the list and plan for a meeting
> sometime
> late next week to discuss it further with the interested parties (please
> signal
> yourself)
>
>
> What do you think?
>
> Pierre and Ralph
>
>
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-leave@lists.
> fedoraproject.org
>
>
___
infrastructure mailing list --

Fedora and PDC

Good Morning Everyone,

We have been informed thst week at the upstream devs working on the
product-definition-center (PDC) are moving away from the project and are going
to leave it without a maintainer. Since we adopted PDC for a variety of data
flows, this puts us in an awkward position. :(

Ralph and I met up on Tuesday to brainstorm the list of things we actively use
PDC for today and to come up with a contingency plan for how to handle them. One
overarching option is for us (fedora-infra) to take ownership of the PDC
codebase as a whole. We didn't fully explore this option, figuring that the
codebase is large and contains lots of tables, endpoints, and codepaths that we
didn't use nor which we plan to use.

Instead, below we've got the four things we use PDC for and some options for
what to do with each.

With the exception of /modules/, one common pattern that we like is to
investigate splitting out the "django apps" that make up PDC into their own
projects. We're calling these "pdc-lite", for fun. See more below.

* Modules
The data in the /modules/ PDC endpoint is *also* in the MBS db. Ralph's
team is going to just use that and stop using pdc anything for modules.
We're going to need to patch pungi, mbs for local builds, and a few other
places. This should be a relatively low-pain transition.

* Stream branches, branch ownership, retirement dates?
- SLA/EOL are currently stored in PDC.
- Queried by releng scripts for retirement, fedrepo-req for new branches,
etc..
option #1
git repo full of yaml file similar to the override repo
compiled into a single JSON blob
Single place for all retired packages
This feels like the lowest tech option.
git gives us change control for free... but people easily get lost in the
"UX" of navigating a gigantic git repo full of plaintext files.
option #2
pagure's DB/API
pagure knows nothing about branches currently, so that would be bigger
lift
option #3
PDC internally is composed of ~20 "django apps"

https://github.com/product-definition-center/product-definition-center/tree/master/pdc/apps
We could pick the 2 or 3 that comprise the branches feature, copy them
out, and turn them into their own service: the "branch definition center":
BDC.
That would be the "pdc-lite" approach mentionned above, ie: PDC with only
the "app" of interest

* release/life-circle tracking?
option #1:
PDC lite with just that app of interest
option #2:
JSON/yaml file on the proxies
option #3:
pkgdb-lite
option #4:
???
compose tracking?
impacted: fedfind
option #1:
PDC-lite with just that app of interest
option #2:
Drop this entirely?
Adam probably really wants to keep the record of composes.
option #3:
???

The "pdc-lite" options are attractive, across the board. One thing we get from
this is greater clarity when discussing things formerly in PDC. If something is
in the branch-definition-center, the compose-definition-center, or the
release-definition-center.. you know what you're talking about. Today, when
talking about whether or not something should be or is in "PDC", it is easy to
get confused.

I propose we start the discussion on the list and plan for a meeting sometime
late next week to discuss it further with the interested parties (please signal
yourself)

What do you think?

Pierre and Ralph

signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR $arch vs $basearch fixups for bodhi AH artifacts



On 04/20/2018 01:14 PM, Kevin Fenzi wrote:
> On 04/20/2018 09:16 AM, Dusty Mabe wrote:
>>
>> The git commit messages help explain this. It turns out the $basearch
>> issue we had a few days ago [1] could have just been fixed by using
>> $arch instead.
>>
>> Dusty
>>
>> [1] https://pagure.io/dusty/failed-composes/issue/176#comment-506967
>> ___
>> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
>> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
>>
> 
> ok. +1 to another try.

pushed! thanks
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR $arch vs $basearch fixups for bodhi AH artifacts

2018-04-20 Thread Mohan Boddu

LGTM +1
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR $arch vs $basearch fixups for bodhi AH artifacts

On 04/20/2018 09:16 AM, Dusty Mabe wrote:
> 
> The git commit messages help explain this. It turns out the $basearch
> issue we had a few days ago [1] could have just been fixed by using
> $arch instead.
> 
> Dusty
> 
> [1] https://pagure.io/dusty/failed-composes/issue/176#comment-506967
> ___
> infrastructure mailing list -- infrastructure@lists.fedoraproject.org
> To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
> 

ok. +1 to another try.

kevin




signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: no CI requirements for modules

2018-04-20 Thread Ralph Bean

Thanks!  This is done now:

https://greenwave-web-greenwave.app.os.fedoraproject.org/api/v1.0/policies

I fixed two other little things while I was in there:

- 
https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=01d70107fca595715931861e80a7e642f9df191b
- 
https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=a0314660d8e09990b3c5f23dc419e83cdaf48358


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR $arch vs $basearch fixups for bodhi AH artifacts



On 04/20/2018 12:16 PM, Dusty Mabe wrote:
> 
> The git commit messages help explain this. It turns out the $basearch
> issue we had a few days ago [1] could have just been fixed by using
> $arch instead.
> 
> Dusty
> 
> [1] https://pagure.io/dusty/failed-composes/issue/176#comment-506967

Also see: https://pagure.io/dusty/failed-composes/issue/181#comment-507421
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

[PATCH 2/2] bodhi-pugni: ostree installer, subs arch into repo urls

Since we are using a for loop for ostree_installer let's go
ahead and substitute [[arch]] in the urls there.
---
 roles/bodhi2/backend/templates/pungi.rpm.conf.j2 | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 
b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2
index 7c611340c..d92ad4f16 100644
--- a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2
+++ b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2
@@ -220,13 +220,13 @@ ostree_installer = [
 "Everything",
 [% if request.name == 'testing' %]
 # In the case of testing, also inject the last stable 
updates
-"https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/updates/f[[ release.version_int 
]]-updates/compose/Everything/$basearch/os/",
+"https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/updates/f[[ release.version_int 
]]-updates/compose/Everything/[[arch]]/os/",
 [% endif %]
 # For f28 the compose location is under /compose/branched/
 [% if release.version_int == 28 %]
-"https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/branched/latest-Fedora-[[ release.version_int 
]]/compose/Everything/$basearch/os/"
+"https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/branched/latest-Fedora-[[ release.version_int 
]]/compose/Everything/[[arch]]/os/"
 [% else %]
-"https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/[[ release.version_int ]]/latest-Fedora-[[ 
release.version_int ]]/compose/Everything/$basearch/os/"
+"https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/[[ release.version_int ]]/latest-Fedora-[[ 
release.version_int ]]/compose/Everything/[[arch]]/os/"
 [% endif %]
 ],
 'release': None,
-- 
2.14.3
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

[PATCH 1/2] bodhi-pungi: don't use for loop for arch, use $arch

It turns out we don't need a for loop for this, we just needed to
use $arch instead of $basearch in the URL. Also, the for loop doesn't
work because of koji unique NVR enforcement on image builds.
---
 roles/bodhi2/backend/templates/pungi.rpm.conf.j2 | 30 +++-
 1 file changed, 14 insertions(+), 16 deletions(-)

diff --git a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2 
b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2
index ea4fe8476..7c611340c 100644
--- a/roles/bodhi2/backend/templates/pungi.rpm.conf.j2
+++ b/roles/bodhi2/backend/templates/pungi.rpm.conf.j2
@@ -195,22 +195,20 @@ volume_id_substitutions = {
 # Other ostree artifacts
 image_build = {
 '^AtomicHost$': [
-[% for arch in ['x86_64', 'aarch64', 'ppc64le'] %]
-{
-'image-build': {
-'format': [('qcow2', 'qcow2'), ('raw-xz', 'raw.xz')],
-'name': 'Fedora-Atomic',
-'kickstart': 'fedora-atomic.ks',
-'distro': 'Fedora-22',
-'disk_size': 6,
-'target': 'f[[ release.version_int ]]',
-'arches': ['[[arch]]'],
-'install_tree_from': "https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/branched/latest-Fedora-[[ release.version_int 
]]/compose/Everything/[[arch]]/os/"
-'subvariant': 'AtomicHost',
-'failable': ['*'],
-}
-},
-[% endfor %]
+{
+'image-build': {
+'format': [('qcow2', 'qcow2'), ('raw-xz', 'raw.xz')],
+'name': 'Fedora-Atomic',
+'kickstart': 'fedora-atomic.ks',
+'distro': 'Fedora-22',
+'disk_size': 6,
+'target': 'f[[ release.version_int ]]',
+'arches': ['x86_64', 'aarch64', 'ppc64le'],
+'install_tree_from': "https://kojipkgs{{ env_suffix 
}}.fedoraproject.org/compose/branched/latest-Fedora-[[ release.version_int 
]]/compose/Everything/$arch/os/"
+'subvariant': 'AtomicHost',
+'failable': ['*'],
+}
+}
 ]
 }
 
-- 
2.14.3
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

FBR $arch vs $basearch fixups for bodhi AH artifacts


The git commit messages help explain this. It turns out the $basearch
issue we had a few days ago [1] could have just been fixed by using
$arch instead.

Dusty

[1] https://pagure.io/dusty/failed-composes/issue/176#comment-506967
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: no CI requirements for modules

+1

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: no CI requirements for modules

On Fri, Apr 20, 2018 at 11:03:15AM -0400, Ralph Bean wrote:
> Modules are configure to require some test cases that never actually
> get run for them.  This patch should treat them instead like we do
> epel: no CI requirements.
> 
> Any +1s?
> 
> diff --git a/roles/openshift-apps/greenwave/templates/configmap.yml 
> b/roles/openshift-apps/greenwave/templates/configmap.yml
> index 44eb013..c88bb9d 100644
> --- a/roles/openshift-apps/greenwave/templates/configmap.yml
> +++ b/roles/openshift-apps/greenwave/templates/configmap.yml
> @@ -91,7 +91,6 @@ data:
>  --- !Policy
>  id: "taskotron_release_critical_tasks_for_testing"
>  product_versions:
> -  - fedora-28-modular
>- fedora-28
>- fedora-27
>- fedora-26
> @@ -103,7 +102,6 @@ data:
>  --- !Policy
>  id: "taskotron_release_critical_tasks_for_stable"
>  product_versions:
> -  - fedora-28-modular
>- fedora-28
>- fedora-27
>- fedora-26
> @@ -113,8 +111,9 @@ data:
>  rules:
>- !PassingTestCaseRule {test_case_name: dist.rpmdeplint}
>  --- !Policy
> -id: "no_requirements_for_epel_testing"
> +id: "no_requirements_testing"
>  product_versions:
> +  - fedora-28-modular
>- fedora-epel-7
>- fedora-epel-6
>  decision_context: bodhi_update_push_testing
> @@ -122,8 +121,9 @@ data:
>  relevance_value: koji_build
>  rules: []
>  --- !Policy
> -id: "no_requirements_for_epel_stable"
> +id: "no_requirements_for_stable"
>  product_versions:
> +  - fedora-28-modular
>- fedora-epel-7
>- fedora-epel-6
>  decision_context: bodhi_update_push_stable

+1 for me


Pierre


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR - Adding myself to nagios permission

On Fri, Apr 20, 2018 at 08:33:28AM +0200, Clement Verna wrote:
> I would like to add myself to the nagios permission, so that I can
> acknowledge problems (mostly packages03 and OSBS).
> 
> I pushed this commit -->
> https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=2f212ac9a8c5f39bd6026b407aa0453fee5446ab
> 
> +1 to run the playbook ?

+1 for me as well


Pierre


signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

FBR: no CI requirements for modules

2018-04-20 Thread Ralph Bean

Discussed in #fedora-admin just now.

Modules are configure to require some test cases that never actually
get run for them.  This patch should treat them instead like we do
epel: no CI requirements.

Any +1s?

diff --git a/roles/openshift-apps/greenwave/templates/configmap.yml 
b/roles/openshift-apps/greenwave/templates/configmap.yml
index 44eb013..c88bb9d 100644
--- a/roles/openshift-apps/greenwave/templates/configmap.yml
+++ b/roles/openshift-apps/greenwave/templates/configmap.yml
@@ -91,7 +91,6 @@ data:
 --- !Policy
 id: "taskotron_release_critical_tasks_for_testing"
 product_versions:
-  - fedora-28-modular
   - fedora-28
   - fedora-27
   - fedora-26
@@ -103,7 +102,6 @@ data:
 --- !Policy
 id: "taskotron_release_critical_tasks_for_stable"
 product_versions:
-  - fedora-28-modular
   - fedora-28
   - fedora-27
   - fedora-26
@@ -113,8 +111,9 @@ data:
 rules:
   - !PassingTestCaseRule {test_case_name: dist.rpmdeplint}
 --- !Policy
-id: "no_requirements_for_epel_testing"
+id: "no_requirements_testing"
 product_versions:
+  - fedora-28-modular
   - fedora-epel-7
   - fedora-epel-6
 decision_context: bodhi_update_push_testing
@@ -122,8 +121,9 @@ data:
 relevance_value: koji_build
 rules: []
 --- !Policy
-id: "no_requirements_for_epel_stable"
+id: "no_requirements_for_stable"
 product_versions:
+  - fedora-28-modular
   - fedora-epel-7
   - fedora-epel-6
 decision_context: bodhi_update_push_stable



signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR - Adding myself to nagios permission

+1

simple change.

kevin



signature.asc
Description: OpenPGP digital signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org

Re: FBR: Fix the metadata handling for Rawhide messages