Hi,
On 09/12/16 05:00 AM, Fabien Boucher wrote:
Hi,
I'm thinking about it since yesterday so and I wanted to share my thoughts here.
The idea is to have a sf-master target in Koji which is a "pot commun" where all
packages built during the CI will land. We need to have a magic tool that know
how to manage rpm(s) build/runtime dependencies according to the ZUUL_CHANGES
variables passed by Zuul.
Each build of a rpm need to have the Software commit hash (for the ones we host
and dev
on Gerrit) and the tag (for the other ones like Gerrit, Storyboard). Also I'm
thinking of a meta package called sf-<version>.rpm that contains all the
mandatories dependencies.
For example:
* sf-3.0.0-1.rpm - depends on:
** managesf-<hash>-1.rpm
** managesf-ansible-<hash>-1.rpm
** sf-ansible-<hash>-1.rpm
** gerrit-2.X.X-1.rpm
** gerrit-ansible-0.1-1.rpm
** sf-docs-3.0.0-1.rpm
I worked on xivo project doing the packaging. At the beginning, we want
package with the form package-<hash>.rpm, but there is an issue with
this system, you can't be sure than a hash will be greater than the
next one, for example:
version 2.0: 3038520
version 3.0: 1383cda
It's a problem, because you will never can install version 3.0 as the
version for 2.0 is greater.
So we decide to use the build date to name the package:
package-20161205.rpm for example (at the beginning, we used
package-date-number_of_build_today-hash.rpm, but it's horrible =))
We could also use for the stable version, the three commit ahead, but
not for branches (two branches could have the same three commit ahead
values)
[x1:software-factory (master-)]git describe HEAD
2.2.6-*70*-g9d1d7e9 (70 here)
This meta package is then the entry point to install SF and its mandatory
dependencies.
(Maybe it should also include a file with the list of extra component (such
repoxplorer, ...)
at the exact versions supported by this release of SF). We can even imagine it
contains
our reno notes. In this version of "dreamland" the SF Git repository should only
contains the "(template) ?" .spec of this meta package.
In the CI the meta package .spec file is modified according the build context.
For
example managesf is in the ZUUL_CHANGES then this meta package will be rebuilt
to pin
the freshly built version of managesf.
But doing that at this meta package level is not enough. For instance pysflib is
modified then the managesf's build/runtime rpm deps need to changed to pin the
pysflib version.
Here could be the resulting workflow of the CI to test an incoming change in
the SF CI:
We bump Gerrit:
1 -> A change in the gerrit-distgit repo is proposed
2 -> First Gerrit is built on koji (non-scratch) build and land in the "pot
commun"
3 -> The meta package is rebuild and pin the new version of Gerrit
4 -> The NVR of the meta package can maybe an epoch or the ZUUL_UUID ?
5 -> The SF image is built/or updated (if a previous on exist on the
slave - Then having the epoch make sense) using the "pot commun" koji repo
in /etc/yum.repo.d.
6 -> Test the SF image as usual
7 -> If a success (in the gate) the gerrit-distgit proposed changed lands in the
Git repo.
It could be useful to have a dev mirror (with packages builder on master
branch) and only build the packages needed for the review. We have to
investigate on centos mirror solution. If we can have a tool like
reprepro for debian, it will be easy to manage mirrors, duplicate them
for testing our changes ...
Building a master SF for our test env could be then : only install the "pot
commun"
repo in yum.repo.d and yum install sf[-<epoch>]. Note that as the meta package
sf use the epoch
as NVR so theoretically you could just yum install sf from the "pot commum" but
as
in the flow I described the meta package sf is built for each patchset then we
won't
have the guarantie the latest sf-<epoch>.rpm is a working version of SF :(.
Working on a dev env (let's say on managesf again) could be then as follow:
-> Make you change locally in managesf source repo
-> Build the SRPM
-> Send it to koji (in scratch or non scratch) whatever
-> Fetch the artifacts
-> and install them in the devel SF
-> When your are satisfied with your changes -> propose them
on Gerrit and the CI will re-test your changes as usual.
Additional note : We want to be sure at any time that all master branch
of each repo[-distgit] that are part of the SF distribution are working
together (pass the SF tests).
What about the SF release ? Let's say now we want to release sf-3.0.0.
Then we will tag (in the koji terminology)
https://fedoraproject.org/wiki/Koji#Package_Organization
a specific list of built packages. We tag them with the tag name sf-3.0.0. We do
the same when we want to release sf-3.0.1 and so on.
So each stable release of the SF (Distribution) will have its own RPM repo.
-> I'm not sure about that and need to be experimented ...
I'm not sure we have to care about tag on packaging. A release could be
a mirror with all packages and only build sf master package with the tag
(with depends on packages on the mirror).
Let's discuss about that and raise questions and issues.
I propose also to setup a semi-official Koji for our needs and we could start
using it
to experiment.
Great idea, we need to groom how we will manage the transition =)
Nico
Cheers,
Fabien
_______________________________________________
Softwarefactory-dev mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/softwarefactory-dev
_______________________________________________
Softwarefactory-dev mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/softwarefactory-dev
