Hi guys,

Some time ago we started managing FreeIPA Copr repos (mkosek/freeipa) with a
target to have the latest greatest FreeIPA available for older arches (read -
RHEL/CentOS) and to allow people using older stable Fedoras (read - Fedora 20)
try FreeIPA 4.0+ releases which brought in several dependencies.

So far this was a more ad hoc approach, I think a more firm plan and tools are
due. I see several questions that needs to be decided:

1) What Copr repos do we want to maintain and what should be the expectations?
My take:

a) mkosek/freeipa: latest and greatest *released* FreeIPA. Built for F20+,
EPEL-7.0. Jan, this is the one you use in the FreeIPA CentOS container, right?
Does it fit your needs?

b) Branch repos: as mkosek/freeipa Copr repo would contain only the latest and
greatest release, would it make sense to have a Copr repo with *releases* per
supported branch to give users a choice? I.e.
* mkosek/freeipa-4.1
* mkosek/freeipa-4.0
These repos are there already, but not used consistently. I do not think we
should build all the dependency chain (too much overhead) for older systems
(F20/EPEL). But I assume we could at least build the freeipa SRPM itself for
these systems if it uses "mkosek/freeipa" as additional build root in Copr.

c) Daily repos
Should we deprecate old John's repos
(http://www.freeipa.org/page/Downloads#Bleeding_Edge) which is difficult to
maintain and replace them with Copr ones? I.e. to have common repo (e.g.
mkosek/freeipa-daily) built for the supported Fedoras (F20, F21, rawhide ATM)
including dependencies?

Should it contain daily master builds for all tracked projects (FreeIPA, SSSD,
389 DS, bind-dyndb-ldap)? Or do we simply want to let distribute it across
projects "mkosek/freeipa-master", "someone/sssd-master",
"someone/389-ds-base-master? Second option may scale better better, the list of
such repos may be maintained somewhere (freeipa.org wiki).


2) We will need to have some tool chain and Jenkins CI jobs watching these
repos to make sure the build & run deps are OK. So far I used the attached 2
clumsy bash scripts to handle the repos build and one for analysis. But we will
need something better.


3) Scalability of the approach
Some dependencies are more difficult to maintain than the others. Especially
the PKI ones often required custom Java packaging (resteasy-base) or a
complicated dependency chain (the latest jackson-jaxrs-json-provider). It would
be great if PKI team helps with this effort, as Lukas proposed. Downside is
that mkosek/freeipa installation would require 2 Copr repos. But maybe we could
have a job syncing the PKI build/runtime dependencies directly to FreeIPA copr.
Whatever works and scale.

-- 
Martin Kosek <mko...@redhat.com>
Supervisor, Software Engineering - Identity Management Team
Red Hat Inc.

Attachment: build-srpms.sh
Description: Bourne shell script

#!/usr/bin/python

from bs4 import BeautifulSoup
from optparse import OptionParser
import requests
import re

class Build(object):
    def __init__(self):
        self.build_status = build_status
        self.files = files
        self.missing_packages = missing_packages
        self.p

def get_build_status(package):
    s = requests.Session()
    url = "http://copr-be.cloud.fedoraproject.org/results/mkosek/freeipa/epel-7-x86_64/%s"; % package
    r = s.get(url)
    soup = BeautifulSoup(r.text)

    files = []
    build_result = False
    for tr in soup.find("tbody").findAll("tr"):
        tds = tr.findAll("td")
        f = tds[0].find(text=True)
        if f == 'Parent Directory':
            continue
        if f.endswith('.rpm'):
            build_result = True
        files.append(f)

    missing_packages = []
    if not build_result:
        url = "http://copr-be.cloud.fedoraproject.org/results/mkosek/freeipa/epel-7-x86_64/%s/root.log"; % package
        r = s.get(url)
        root_log = r.text

        m = re.search(r'Error: No Package found for (\S+)', root_log)
        if m is not None:
            missing_packages.append(m.groups(1)[0])

    return build_result, files, missing_packages

def main():
    # parse arguments
    parser = OptionParser()

    # parse options
    (options, args) = parser.parse_args()

    url = 'http://copr-be.cloud.fedoraproject.org/results/mkosek/freeipa/epel-7-x86_64/'
    s = requests.Session()
    r = s.get(url, data={})
    soup = BeautifulSoup(r.text)

    built_packages = []
    broken_packages = []
    missing_dependencies = {}

    for tr in soup.find("tbody").findAll("tr"):
        tds = tr.findAll("td")
        package = tds[0].find(text=True)
        package_name = package.rsplit("-", 2)[0]
        if package.endswith(".log") or package in ('Parent Directory', 'repodata'):
            continue
        build_result, files, missing_packages = get_build_status(package)

        print package_name, "OK" if build_result else "FAIL", missing_packages
        if build_result:
            built_packages.append(package_name)
        else:
            missing_dependencies[package_name] = missing_packages
            broken_packages.append(package_name)

    print
    print "BUILT:"
    for p in built_packages:
        print p
    print

    rebuild = []
    print "BROKEN:"
    for p in broken_packages:
        print p
    print

    print "TODO: add new package:"
    for p in broken_packages:
        deps = missing_dependencies[p]
        if (deps and deps[0] not in built_packages and
                     deps[0] not in broken_packages):
            print "%s (required by %s)" % (deps[0], p)
    print

    print "TODO: rebuild:"
    for p in broken_packages:
        deps = missing_dependencies[p]
        if deps and deps[0] in built_packages:
            print "%s (%s ready)" % (p, deps[0])
    print

    print "TODO: investigate:"
    for p in broken_packages:
        deps = missing_dependencies[p]
        if not deps:
            print p

if __name__ == "__main__":
    main()

Attachment: prep-srpms.sh
Description: Bourne shell script

_______________________________________________
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel

Reply via email to