On 11/22/23 2:04 PM, Michael Opdenacker via lists.openembedded.org wrote:
Greetings,
The Yocto Project has received funding for developing the processes and
tooling
for enabling binary distributions. As part of that, the project would
need some
form of prototype test binary distro to develop and test this. The main
goal of
this would be to develop the tools, and documentation needed to build a
binary
distribution and allow application and system developers to progress
through a
hierarchy of different uses of the resulting environment:
- Beginner use case: begin by fetching a binary image, ready to boot on a
target machine. Extend the system by installing extra applications
through
package feeds.
- Intermediate use case: tweak and rebuild existing packages, compile new
applications and modify images by using an eSDK and other binary
artifacts.
- Advanced use case: completely reorganize and optimize the system for a
specific target, switching to building from source
The way I always look at this and present is is a continuum of behavior for a
user from your beginner to advanced usecase.
- People will want to start with a binary only distribution. Then as they need
different behavior they will adjust configuration parameters (on the device)
like an admin in a traditional system. So test case wise, a key test is: can I
change settings and do package upgrades and do the setting remain?
Additional key tests would include can I update packages, can I add packages.
Part of the testing here needs to be on-going 'official' builds (that YP does)
and shows that there can be a continuous update cycle. In the past I had spec'd
testing that was:
1) Initial Image to 'arbitrary version' to 'latest version'
2) Initial Image to 'previous version' to 'latest version'
3) Initial Image to 'latest version'
#1 is hardest to test, so I always made the assumption arbitrary really was
"randomly pick an available version, go there first, then go the latest"
#2 is the "most common" upgrade for an existing user
#3 is the "most common" I have a new image and I'm just getting started.
So importance to me would be testing #3, then #2, and maybe #1 (given enough
resources). Note, the way this is tested could be execute ptests, but it also
could be #3 as a baseline, then compare the actual installed files in #2 and #1.
- As this moves into the intermediate use case, adjusting (runtime)
configuration is no longer enough. The user will need to make compile time
changes. Likely starting with a recipe configuration, then a patch, and finally
to something bigger, like a different SRCREV, pointing to a new fork (different
upstream), etc.
The key here is how can the user (with the eSDK) build an updated package, that
shows up AS an updated package but doesn't collide with the name space of
existing builds or future official builds/updates?
The name space I am the most concerend about is 'PR'. Our current PR is in the
format of <recipe>.<auto_distro>, i.e. r0.0 or r0.1. So if a user does make a
local change, and builds.. should they get 'r0.2'? This will clash with our
next build.. They really should have 'r0.1.1' (a new third digit) but there is
currently no mechanism to say 'I'm based on ....' or 'r0.1 is the latest so we
append to that'. I don't even know how we would implement that at this time.
The other issue here is if the version scheme changes, doesn't have a defined
incrementing format. I.e. using a 'hash' has the PV, this means that the user
would be need to update the PE... (Which ALSO could be the solution to the
above, some sort of check.. if this is an unofficial local change, it requires a
PE -- but the problem is then, when the next official update comes in it will be
'older' then the temporary custom version.. so we're back into a what is the
right answer here space.)
- Eventually this gets to the advanced case where the user is only going to use
the Yocto Project as a 'best practices' and probably an layer and sstate-cache
repository. So they want to be able to build everything (initially only local
custom stuff, eventually really everything.) But how do we sync the PR and
hash-equivalency data? OR do we NOT do this, but then it's not really the same
distribution. How do we make it clear it's no longer the same distribution?
Somewhere along this (between the intermediate and advanced) I could also see
the desire for multiple providers to be desired. YP provides the core
foundation. Someone else provides packages based on their set of layers, then
someone adds their own and so forth. Certainly not an immediate problem to
solve, but also something to consider. (I'd be more then happy to say, this is
not in the current scope. If you change the layers, it's now a variant and you
are into the intermediate or advanced scopes and it's no different then making
other changes.)
The first step in this effort is to define the scope of the prototype we
will
develop. We are looking for feedback and comments about issues we may
not have
considered.
Bear in mind that a prototype has to be limited in scope. However, the
developed tools, procedures and documentation will open a pathway for
the Yocto
Project and OpenEmbedded community to experiment with more architectures,
features, recipes and images.
Also bear in mind that the effort is not planning to adopt, replace or
repeat
any existing distributions such as Angstrom or Yoe.
Target architectures and machines
---------------------------------
The first prototype would only target the below architectures:
- "genericx86-64" architecture through the "genericx86-64" machine
(https://git.yoctoproject.org/poky/tree/meta-yocto-bsp/conf/machine/genericx86-64.conf)
- "arm64" architecture through the "qemuarm64" (and later "genericarm64"
when
hready) machine
(https://git.yoctoproject.org/poky/tree/meta/conf/machine/qemuarm64.conf)
Testing would happen through QEMU system emulation. The emulated
architectures
will be sufficient to test the outputs of this effort, but the developed
tooling and documentation should allow the community to experiment with
other
target architectures and machines.
I agree here. Set some reasonable goes and go for it. I would say no multilib
as well. So no i386 or arm support. Focus on just the machine/architecture.
One question that will come up. Is there one starter image for each
"architecture"? One per "machine"? If the former how do I add machine
specific components to make the image more 'optimized'? (This is the embrace
and extend...) If it's one per machine, we need to be able to answer why you
can't have ONE common image and then customize it. (Note when I say image here
I'm referring to userspace filesystem image. It very well may be that different
systems have different firmware (u-boot), and device-trees (and maybe even a
kernel). But for end users there is a huge advantage of a single starting
filesystem image that then gets tailored for their environment.
Target tunes
------------
The binary images and other artifacts should be generated for the below
tunes:
- armv8a
(https://git.yoctoproject.org/poky/tree/meta/conf/machine/include/arm/arch-armv8a.inc)
- core2-64
(https://git.yoctoproject.org/poky/tree/meta/conf/machine/include/x86/tune-core2.inc)
See https://docs.yoctoproject.org/ref-manual/variables.html#term-DEFAULTTUNE
for details.
Distribution, Init Manager, C library
-------------------------------------
Images and package feeds will be generated for the "poky-altcfg"
distribution
(https://git.yoctoproject.org/poky/tree/meta-poky/conf/distro/poky-altcfg.conf),
using the systemd init manager.
Choosing systemd makes the binary reference distribution as generic as
possible, enabling the widest set of features, and being more familiar
to more
users. Of course, switching to other init managers and more restricted
features
will be possible by later compiling the system from source (advanced use
case).
This also implies the choice of the GNU C library (glibc). Selecting
glibc also
makes sense to enable the widest possible selection of packages, and some
packages have limited or no support for the Musl C library. That's typically
the case of systemd, which upstream only supports glibc.
Last but not least, people who want to use the Musl C library probably
do that
to optimize the size of their system, which is a use case not well
adapted to
using ready-made binary images. Such an advanced use case will be
addressed by
re-compiling from source, after testing initial prototypes with the binary
image based on glibc.
Binary images
-------------
The prototype will provide as a reference, for each targeted architecture:
- The root filesystem binary image, generated by building the
"core-image-full-cmdline" target.
- An image containing on-target development tools (e.g compiler, debugger)
- The corresponding SD card images to be booted through QEMU. Other boot
methods will be possible, but will be outside of the scope of the
reference
distribution.
Package feeds
-------------
For each targeted architecture, several package feeds will be available to
allow for bugfix or security updates or to extend the image with additional
packages:
- Architecture independent feed
- Architecture dependent feed
- Machine dependent feed
See
https://docs.yoctoproject.org/dev-manual/packages.html#using-runtime-package-management
for details.
The feeds for the first prototype will contain all the packages which are
supported by the "poky-altcfg" distribution as produced by the "bitbake
world"
command for the set of layers supplied by Poky.
Available reference package formats
-----------------------------------
To keep the first prototype simple, we will only release images and package
feeds based on the "ipk" format supported by "opkg" package manager.
Of course, the OpenEmbedded build system also supports the "deb" and "rpm"
package formats, so users will be able to generate their own images and
feeds
based on these other formats.
Another reason for not providing "deb" and "rpm" packages is that we do not
want inexperienced users to expect to be able to grab and use such packages
produced for regular Linux distributions (Debian, Fedora, etc.). Being built
with different C library versions and toolchains, the corresponding binaries
are most likely to be broken anyway.
Q: eSDK -- how do we make sure that the SDK matches the _current_ package
repository? If you build updated packages somehow the user will need to be sure
it matches the repository they're going to be updating.. (and matching is
probably not the right word, since the act of adjusting something really means
it's not directly matching.)
Available reference binary artifacts
------------------------------------
The first prototype will make the below binary artifacts publicly available:
- Root filesystem images (as specified above)
- Binary package feeds (as specified above)
- PR (https://docs.yoctoproject.org/ref-manual/variables.html#term-PR)
database, so that people can manage updates to packages, by importing the
database in a local PR server.
- Prebuilt object data ("Shared State cache) through a public server
(see
https://docs.yoctoproject.org/ref-manual/variables.html#term-SSTATE_MIRRORS),
to reuse binary output already built by the Yocto Project
autobuilders, to
reduce the compile time of additional packages.
- Hash Equivalence data through a public server
(see
https://docs.yoctoproject.org/bitbake/2.4/bitbake-user-manual/bitbake-user-manual-ref-variables.html#term-BB_HASHSERVE_UPSTREAM),
to increase the reusability of prebuilt objects.
Share State and Hash Equivalence data would be only useful to "Intermediate"
and "Advanced" use cases, to rebuild parts or the whole system from source.
PR service and hash equivalence is not separable from the sstate-cache. As such
there needs to be a way to keep these objects in sync with the binary packages.
Additionally, users have limitations on access those resources, so direct
protocol is probably going to be a HUGE problem for a lot of users due to
corporate firewalls. https is one of the only protocols many companies will
permit (which some want full off-line access, so they want a way to download
everything.)
My point in this is this could very well point to a need to adjust the way the
data for the PR service and hash equivalence is presented to the user. LOCALLY
I agree they should be using the databased and services we have. But fetching
from remote, doing it as part of the sstate-cache fetching would solve a lot of
these transfer problems (and enable use of the CDN below!)
Such artifacts will be served through a Content Delivery Network (CDN),
to make
sure that they can be downloaded fast enough from any location in the
world, in
a way that remains much faster than re-compiling such artifacts from
sources.
Defining policies and processes
-------------------------------
This reference binary distribution prototype will also drive the
documentation
of policies and process to address future work such as:
- What criteria should a new CPU architecture or platform tuning meet to be
added / tested?
- What criteria should other layers meet to be tested, in addition to being
already tested by the Autobuilder? Should these criteria only apply to
specific recipes or to the entire layers containing them?
Yocto Project compliance should be a key requirement here. If that can NOT be
done for some reason, this is important information to capture as it can help
direct revisions to YP compliance and/or documentation.
Testing plans
-------------
Here are the tests we plan to implement:
- Producing and testing binary images through the autobuilder
- Making nightly builds available to testers
- Testing that package updates apply successfully:
- Within the current stable branch (nanbield): from one release
update to the
next, and from the latest release update to the tip of the branch
- Between release branches: from the latest update to our latest LTS
(kirkstone), to the latest update to the current stable release
(nanbield).
- Testing that it's also possible to build and deploy packages from the
meta-openembedded and meta-virtualization layers, as a way to assess the
relevance of our policies and process for additional layers or
recipes. This
would also be a way to demonstrate the ability to build container images,
which would be a great way to ship binary distributions.
We will work with recipe maintainers to publish fixes for update issues that
testing will find in recipes. Each issue will be reported in Bugzilla.
Thank you in advance for your review and feedback.
My suggestion for builds:
- At tagged releases, a new 'base image' is generated.
- Daily build through the autobuilder of _stable_ (checked in content) is built,
locally tested and if that passes published as updates to the base image.
Package update testing:
I would focus on current release updates/tip of branch and NOT between release
branches. Trying to update from kirkstone to nanbield (or similar) is something
we should factor into this, BUT I fully expect it to not work. What will allow
us to get there at some point is regular daily builds and update testing from
'last stable release to current in development' so when failures are introduced
people can be notified -- but general 'big bang' version updates (outside of our
control) very well could mean there is no easy way for us to deal with it short
of 'Hey user, you need to start with a new base image'. (The types of things
expected to break -- major kernel version change, out of tree modules are likely
to break. Toolchain version change.. libgcc, libstdc++ can break in unexpected
ways more or less breaking _everything_ during the upgrade... other major
components, like libc, wayland/X11, mesa, etc....) All valuable information to
know, but I expect problems so this is a 'future goal' vs something "right now"
in my mind.
--Mark
Cheers
Michael.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1847):
https://lists.openembedded.org/g/openembedded-architecture/message/1847
Mute This Topic: https://lists.openembedded.org/mt/102755562/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
