Hi Fatih,
From: Fatih Degirmenci <[email protected]> Date: Monday, July 10, 2017 at 2:04 PM To: "Alec Hothan (ahothan)" <[email protected]>, "Beierl, Mark" <[email protected]> Cc: "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Subject: Re: [test-wg] docker container versioning Hi Alec, Your understanding about the docker image tags seem correct to me (latest vs stable) but I let someone from test projects answer to that. [Alec] I’m still a bit confused after reading Jose’s email When it comes to artifact versioning in general; you are asking really good questions. Let me go back in time and summarize what plans we had (ie what we haven't been able to implement fully) with regards to it. The questions you ask about tagging docker images is not limited to them. We have similar issues with other artifacts we produce (rpms, isos , etc.), maybe not on the same level as the docker images but we have them. [Alec] Containers generally have a much faster cycle than rpms and isos, it is not unreasonable to see multiple versions of a container being created for any given opnfv release. But you’re right that tracking the version is needed for all artifacts. In order to achieve some level of traceability and reproducibility, we record the metadata for the artifacts (rpms, isos, etc.) we build so we can go back to source and find out exact version (commit) that was used for building the artifact in question. [1] We also had plans to tag corresponding commits in our git repos but we haven't managed to fix that. [2] This includes docker images as well. [Alec] It is good that it is at least tracked. I’m a bit surprised that the community has not up voted this issue because it is a pretty serious one. Apart from our own (OPNFV) repos, some of the artifacts we build include stuff from other sources, making it tricky to achieve full traceability and making the traceability even more important. We had many discussions about how to capture this information in order to ensure we can go back to a specific commit in any upstream project we consume. (locking/pinning versions etc.) But since we have different ways of doing things and different practices employed by different projects, this hasn't happened either. (I can talk about this for hours...) [Alec] It is indeed a pretty complex problem but can we perhaps limit the discussion to containers first? That will limit the scope and hopefully allow us to get containers in a good shape and perhaps we could leverage that work for the rest of the artifacts. By the way, I am not saying we totally failed as some projects take care of this themselves but as OPNFV, we do not have common practice, except metadata files for ISOs and the docker tags that do not help at all. Long story short, this can be achieved in different ways like you exemplified; if a tag is applied to a repo, we trigger a build automatically and store & tag produced artifact in artifact repo and/or if we are building periodically, we apply the tag to git repo once the artifact is built successfully. [Alec] Can we work on a proposal and get every project that deals with containers involved? Do you usually work on a text file reviewed in gerrit? We would need to address the following: * Opnfv containers versioning strategy (how to version, workflows and relationship with releases) * How to produce and publish new container images * Support for multiple images per project (although this can be addressed separately in the short term) Thanks /Alec No matter which way we go, we need to fix this so thank you for questioning things, hopefully resulting in improvements starting with test projects. [1] http://artifacts.opnfv.org/apex/opnfv-2017-07-05.properties [2] https://jira.opnfv.org/browse/RELENG-77 /Fatih From: "Alec Hothan (ahothan)" <[email protected]> Date: Monday, 10 July 2017 at 21:45 To: Fatih Degirmenci <[email protected]>, "Beierl, Mark" <[email protected]> Cc: "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Subject: [test-wg] docker container versioning [ cc test-wg - was [opnfv-tech-discuss] Multiple docker containers from one project) ] Hi Fatih It is generally not easy to deal with container tags that do not include any information that links easily to a git repo commit (e.g. a “version” number increased by 1 for each build does not tell which git commit was used – might be one reason why this was removed) For example, if we look at the published yardstick containers as of today: “latest” is generally used to refer to the latest on master at the time of the build (so whoever does not care about the exact version and just wants the bleeding edge will pick “latest”) – apparently this container is not linked to any particular OPNFV release? Or is it implicitly linked to the current latest release (Euphrates)? “stable” is supposed to be the latest stable version (presumably more stable than latest), in the current script it is not clear in what conditions a build is triggered with BRANCH not master and RELEASE_VERSION is not set (does the project owner controls that?) Apparently unrelated to any particular OPNFV release as well although would have thought a “danube.3.0-stable” would make sense. “danube.3.0”: related to Danube 3.0 but does not indicate what yardstick repo tag it was built from. The git repo has a git tag with the same name “danube.3.0”, how are those 2 tags correlated? For example, there is no matching git tag for container “colorado.0.15” and there is no matching container for git tag “colorado.3.0”. Also not clear what yardstick project will do to publish a newer version of the yardstick container for Danube 3.0? Project owners should be able to publish finer grain versions of containers in a faster pace than the overall OPNFV release (e.g. as frequent as more than once a day) and these need to be tracked properly. The best practice – as seen by most popular container images – is to tag the container using a version string that reflects the container source code version. Translating to a project workflow, what is typical: · The project repo uses git tags to version the source code (e.g. “3.2.5”) independently of the OPNFV release versioning (e.g. “Danube 3.0”). Such versioning should be left at the discretion of the project owners (e.g. Many OpenStack projects use the pbr library to take care of component version) · Optionally the project repo can have 1 branch per OPNFV release if desired (e.g. “danube”, …) – noting that some projects will not require such branches and support every OPNFV release (or a good subset of it) from a single master branch (simpler) · Simplest (and this is how the dockerhub automated build works) is to trigger a new build either on demand (by project owners) or automatically whenever a new git tag is published (by who has permission to do so on that project) I am not familiar with the OPNFV release packaging process (for example how are containers tied to a particular release), could someone explain or point to the relevant documentation? If you look at the OpenStack model, each release (e.g. Newton, Ocata,…) is made of a large number of separate git repos that are all versioned independently (i.e. neutron or tempest don’t have version tags that contain the openstack release). And each release has a list of all projects versions that come with it. Example for Ocata: https://releases.openstack.org/ocata/ Is there a similar scheme in OPNFV? Mark: · Container versioning is orthogonal to the support for multiple containers per project (that a project has more than 1 container makes the versioning a bit more relevant). · Not being able to rebuild a container from a tag is problematic and I agree it needs to be supported Gabriel: · Using the dockerhub automated build is fine as long as the versioning of the built containers is aligned with the OPNFV versioning scheme
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
