Hi guys, Some time ago we started managing FreeIPA Copr repos (mkosek/freeipa) with a target to have the latest greatest FreeIPA available for older arches (read - RHEL/CentOS) and to allow people using older stable Fedoras (read - Fedora 20) try FreeIPA 4.0+ releases which brought in several dependencies.
So far this was a more ad hoc approach, I think a more firm plan and tools are due. I see several questions that needs to be decided: 1) What Copr repos do we want to maintain and what should be the expectations? My take: a) mkosek/freeipa: latest and greatest *released* FreeIPA. Built for F20+, EPEL-7.0. Jan, this is the one you use in the FreeIPA CentOS container, right? Does it fit your needs? b) Branch repos: as mkosek/freeipa Copr repo would contain only the latest and greatest release, would it make sense to have a Copr repo with *releases* per supported branch to give users a choice? I.e. * mkosek/freeipa-4.1 * mkosek/freeipa-4.0 These repos are there already, but not used consistently. I do not think we should build all the dependency chain (too much overhead) for older systems (F20/EPEL). But I assume we could at least build the freeipa SRPM itself for these systems if it uses "mkosek/freeipa" as additional build root in Copr. c) Daily repos Should we deprecate old John's repos (http://www.freeipa.org/page/Downloads#Bleeding_Edge) which is difficult to maintain and replace them with Copr ones? I.e. to have common repo (e.g. mkosek/freeipa-daily) built for the supported Fedoras (F20, F21, rawhide ATM) including dependencies? Should it contain daily master builds for all tracked projects (FreeIPA, SSSD, 389 DS, bind-dyndb-ldap)? Or do we simply want to let distribute it across projects "mkosek/freeipa-master", "someone/sssd-master", "someone/389-ds-base-master? Second option may scale better better, the list of such repos may be maintained somewhere (freeipa.org wiki). 2) We will need to have some tool chain and Jenkins CI jobs watching these repos to make sure the build & run deps are OK. So far I used the attached 2 clumsy bash scripts to handle the repos build and one for analysis. But we will need something better. 3) Scalability of the approach Some dependencies are more difficult to maintain than the others. Especially the PKI ones often required custom Java packaging (resteasy-base) or a complicated dependency chain (the latest jackson-jaxrs-json-provider). It would be great if PKI team helps with this effort, as Lukas proposed. Downside is that mkosek/freeipa installation would require 2 Copr repos. But maybe we could have a job syncing the PKI build/runtime dependencies directly to FreeIPA copr. Whatever works and scale. -- Martin Kosek <[email protected]> Supervisor, Software Engineering - Identity Management Team Red Hat Inc.
build-srpms.sh
Description: Bourne shell script
#!/usr/bin/python
from bs4 import BeautifulSoup
from optparse import OptionParser
import requests
import re
class Build(object):
def __init__(self):
self.build_status = build_status
self.files = files
self.missing_packages = missing_packages
self.p
def get_build_status(package):
s = requests.Session()
url = "http://copr-be.cloud.fedoraproject.org/results/mkosek/freeipa/epel-7-x86_64/%s"; % package
r = s.get(url)
soup = BeautifulSoup(r.text)
files = []
build_result = False
for tr in soup.find("tbody").findAll("tr"):
tds = tr.findAll("td")
f = tds[0].find(text=True)
if f == 'Parent Directory':
continue
if f.endswith('.rpm'):
build_result = True
files.append(f)
missing_packages = []
if not build_result:
url = "http://copr-be.cloud.fedoraproject.org/results/mkosek/freeipa/epel-7-x86_64/%s/root.log"; % package
r = s.get(url)
root_log = r.text
m = re.search(r'Error: No Package found for (\S+)', root_log)
if m is not None:
missing_packages.append(m.groups(1)[0])
return build_result, files, missing_packages
def main():
# parse arguments
parser = OptionParser()
# parse options
(options, args) = parser.parse_args()
url = 'http://copr-be.cloud.fedoraproject.org/results/mkosek/freeipa/epel-7-x86_64/'
s = requests.Session()
r = s.get(url, data={})
soup = BeautifulSoup(r.text)
built_packages = []
broken_packages = []
missing_dependencies = {}
for tr in soup.find("tbody").findAll("tr"):
tds = tr.findAll("td")
package = tds[0].find(text=True)
package_name = package.rsplit("-", 2)[0]
if package.endswith(".log") or package in ('Parent Directory', 'repodata'):
continue
build_result, files, missing_packages = get_build_status(package)
print package_name, "OK" if build_result else "FAIL", missing_packages
if build_result:
built_packages.append(package_name)
else:
missing_dependencies[package_name] = missing_packages
broken_packages.append(package_name)
print
print "BUILT:"
for p in built_packages:
print p
print
rebuild = []
print "BROKEN:"
for p in broken_packages:
print p
print
print "TODO: add new package:"
for p in broken_packages:
deps = missing_dependencies[p]
if (deps and deps[0] not in built_packages and
deps[0] not in broken_packages):
print "%s (required by %s)" % (deps[0], p)
print
print "TODO: rebuild:"
for p in broken_packages:
deps = missing_dependencies[p]
if deps and deps[0] in built_packages:
print "%s (%s ready)" % (p, deps[0])
print
print "TODO: investigate:"
for p in broken_packages:
deps = missing_dependencies[p]
if not deps:
print p
if __name__ == "__main__":
main()
prep-srpms.sh
Description: Bourne shell script
_______________________________________________ Freeipa-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/freeipa-devel
