On Tue, May 02, 2017 at 07:59:35PM +0200, André Boddenberg wrote: > Hi all, > > here's a (late) recap on the "dependency-artifacts" topic discussed > with Holger and Neels at OsmoDevCon. > > > The "mv $artifact"-issue, which occurs when run within a docker > container has been resolved. Now artifacts are created inside a > temporary directory within the ARTIFACT_STORE and not inside the > Docker container, so the "mv" command won't take as long as a "cp" > anymore. > > Moreover, artifacts are now stored per jenkins job and not per git > project. A JOB_NAME-directory inside the ARTIFACT_STORE is created for > each jenkins job to distinguish between them. This means artifacts > aren't shared across jobs which build the same git project. This might > change in the future, but is safe for now as some jenkins jobs might > need dependencies built with different configurations. > > Note: the necessary disk space for openBSC artifacts of one > matrix-configuration job is ~400 MB (~50 MB per axis). > > Furthermore, each jenkins job takes care about garbage collection on > its own by simply deleting the entire content of its > JOB_NAME-directory inside the ARTIFACT_STORE.
Which is, for the record, the reason to have it per-job: we always know
which artifacts need to be / we are allowed to clean up. With multiple
jobs accessing the same artifact store, GC is not trivial.
> The two scripts osmo-{build,artifacts}.sh have been tested [1][2],
> shellcheck'ed and pushed to gerrit [3].
nice!
> The following actions are pending to enable the use of artifacts
> inside contrib/jenkins.sh within each git repository, but can not be
> applied by me:
>
> - setting ARTIFACT_STORE environment variable for each jenkins slave
> - copying osmo-artifact.sh and osmo-build.sh to ~/bin of each slave user
Add to osmo-ci. Could you submit a patch for it?
git clone https://gerrit.osmocom.org/osmo-ci
> - adding a new docker volume for the ARTIFACT_STORE to the image
We're talking about the docker invocation in the OpenBSC and
OpenBSC-gerrit jobs.
> But, let's review first. =)
I notice there are hashes of various dependent builds in the tar names:
libosmocore.master.33e0306_libosmo-abis.master.bf7976c_libosmo-netif.master.56add1e_libosmo-sccp.master.b354652_libsmpp34.master.cc0bcd6_openggsn.master.19e19e3.tar.gz
I remember that we said to instead have each on its own -- but not saying
we have to, just curious.
So the semantics are: figure out all of the hashes of all dependencies, as
soon as one of them mismatches rebuild all and make a new single tar with
all included. Right?
Also curious about how a matrix build would do it: would every matrix
rebuild the same dependency? (We said that that's fine IIRC)
~N
signature.asc
Description: Digital signature
