*WHAT IS A BASE IMAGE?* Container images are built as committed transactions, a bit like how git works. This means that every image is a "delta" on top of another image. Your container runtime (CRI-O, containerd, Docker, etc.) will download and store every commit only once, so if you have 100 images all based on the same image, which we'll call the "base image," then you only ever download that same base image once. All other images would thus just be the "delta", which ideally contains just the application and a *minimal* set of dependencies.
It's important to remember that the base images can already contain some *essential* application dependencies, such as a Java runtime, a Python implementation, encryption libraries, database drivers, etc. Moreover they surely already contain operating system staples, such as glib, OpenSSL, GNU userland, etc. All these dependencies are used *instead* of those libraries on the host systems, meaning that the choice of base image is also a choice for the extent of your *vendored support*. For example, if your host operating system is from vendor X and your container base operating system is from vendor Y then vendor X cannot provide support for what your application is actually using and the same is true if your Java runtime is from vendor Z. *OUR CHALLENGES* Our community's CIA group is working on a good set of guidelines <https://wiki.onap.org/display/DW/Best+Practices> that can help minimize that "delta". But ONAP base images are still a big mess: 1) We are using *many different base images*, so we're downloading and storing quite a lot. Minimizing the "delta" helps, but obviously the operating system underneath is also large, sometimes larger. We must minimize the number of base images we use if we want to see significant gains in reducing size. 2) It's hard, if not practically impossible, to *change base images* across ONAP. The reason is that we do not have a unified methodology for building images. Some projects use Dockerfiles with hardcoded base images (the "FROM" field). Others use some templating to create a final Dockerfile. And yet others use build system plugins (Maven) to generate the image without directly writing a Dockerfile. And none of these are located in any standard location or are documented. The only way to change base images in ONAP right now would be a huge, unmaintainable patch that covers all projects. *TECHNICAL SOLUTION* First, a long-term suggestion: The ideal solution would be to have all ONAP projects use the same image building methodology and indeed the same mechanism, preferably one that is decoupled from the application, or rather handed off to it after all artifacts are created. That way users could easily control image building across the project in one place: setting the base image, setting the destination repository (staging, debugging), integration with auditing systems, etc. The ability to uniformly build images could be considered as an important element for "ONAP as a product" rather than an umbrella project of somewhat-matched components. But that's jumping ahead. In the short term there's a much easier solution that would provide much benefit: the base image as an alias. Imagine that the "FROM" field of all ONAP images were one of these: onap/base-java-8 onap/base-java-11 onap/base-python3 Each of these images could in turn derive from the same minimal base, " onap/base", just with the additionally installed dependencies. What would be the content of these images? For our reference implementation we will decide on a specific distribution. But consumers would be able the change or tweak these to compatible alternatives. Let's consider an example derivation path for one image: onap/base → onap/base-java-8 → onap/so Let's say that the reference implementation of "onap/base" is HappyLinux (I just made that one up). "onap/base-java-8" would derive from that, but add an OpenJDK 8 conveniently packaged by the HappyLinux volunteer maintainers. Then "onap/so" would use "onap/base-java-8" in its Dockerfile "FROM" field. All Java 8 components in ONAP would be doing the same, so any consumer would only ever download that JDK once. This also means that it would be relatively easy to swap out that JDK. For example, a consumer might want to use a security-patched, custom-built version of OpenJDK and not wait for ONAP to make a new release. So, they would first rebuild the " onap/base-java-8" image, and then trigger a rebuild of all derived ONAP packages. Assuming integration tests are successful, they are good to go. Other than creating production-ready images, there are benefits to development in being able to easily replace the base image. For example, it's possible to create a debuggable version of the base image, with default configuration enabling more logging, instrumentation, network capturing services, etc. It also makes it more easy to test the impact of an upgrade of a library across all of ONAP, for example moving from JDK 11 to JDK 12. In the most ideal situation *none* of our images would be using DNF, APT, YUM, APK, or any other distribution-specific way to install additional packages. By avoiding this we would guarantee an easy, centralized way to manage dependencies. I believe that reaching 100% of this goal will not be so easy, as there will always be a container image somewhere with a quirky requirement. However, if we at least require that these "exceptions" be documented in one central location then we've already gone a long way towards reaching our goals. One potential challenge would be where to put dependencies: there might be a concern that various little dependencies would be pushed into "onap/base" and that it would become bloated. There are two potential answers to this: 1) So what if it becomes a bit bloated? It is only ever downloaded once. And there's a big advantage to seeing our dependencies in one place. 2) Our set of base images can evolve over time to accommodate this. For example, let's say that a specific container requires a bunch of large PDF and PostScript manipulation libraries. It might make sense to create a new "onap/base-graphical" derivative, even if it's just for one project. Implementing this change would be relatively easy to do across ONAP. For the most part, we already know what the set of base images would look like. The current quirky build processes for each project can remain the same and it would just be changing the "FROM" field. *PROCESS AND COMMUNITY* The technical solution is, I think, straightforward. The impact on the community process requires more discussion. An immediate concern would be: Who manages the base images and decides what should go in and what shouldn't? Should it be a special team (CIA?) or committee? Should there be TSC involvement? Can we make sure that individual projects are not blocked from using dependencies they need? My thought is that it should be simple and open, and that no additional team is necessary. There can be a "base-image" git repository and all ONAP committers, or at least all PTLs, should be committers there, too, so that no project feels that another project is dominating and making decisions for other projects. In cases of conflict the issue can/should be brought to the community at large to help resolve. The responsibility for building these base images should be within the existing build and release process. Generally there should not be a lot of fiddling with this repository. There will be some work early only to create the base images, but after the rest of ONAP is aligned with it, I imagine the only major periods of activity would be when the distribution of choice gets upgraded, or a major and potentially breaking upgrade happens to a component such as to Java EE. It is important the underscore that the impact of base image decisions, e.g. which version of a specific package to include, is limited to the world of development and testing only. We are only ever building a reference implementation, not production-ready images. Of course we want ONAP to be as close as production-ready as possible, to make is as easy possible for consumers to put ONAP to work. But I believe that our #1 priority in the decision making process is to ensure a smooth and swift development, to make sure that no developers within ONAP and the community at large are blocked from contributing. I cannot speak to what process would work best for our community, as it us to all of us to decide. However, I'm confident that it's a conversation worth having when the end-goal is to make ONAP lighter, more buildable, more vendor-supportable, and easier (in some ways) to develop. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16317): https://lists.onap.org/g/onap-discuss/message/16317 Mute This Topic: https://lists.onap.org/mt/30776257/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
