On Mon, Dec 2, 2019 at 9:37 PM Mark Hatle
<mark.ha...@kernel.crashing.org> wrote:
>
> Problem Statement
> -----------------
> In the current world there is an increasing number of heterogeneous
> systems being developed.  Currently these components can be built
> independently of each other, and then combined later.  For ease of use,
> it would be nice to be able to build these systems with a single build,
> including generating a final bootable image.  A heterogeneous system may
> simply be different configurations for different components, different
> operating systems for different components, or a system made up of
> diverse processor architectures.  Recently the Yocto Project has added
> multiconfig in order to enable these types of configurations, but
> suggested workflows for various configurations are needed to avoid
> confusion and to avoid developers implementing their own schemes.
>
> Proposal
> --------
> During the course of discussing this with other people a number of terms
> have been involved, and after numerous discussions it is clear that
> people have a different definition for the various components.  The
> purpose of this document is to explain the various pieces of this
> proposal, so that we can all be using the same items for the various
> components necessary to build a heterogeneous system.
>
> For a homogeneous build, the traditional Open Embedded/Yocto Project
> components that are used include:
>
> * Build Configuration (local.conf)
> * MACHINE
>    * Specifies target device information, including hardware
>      capabilities, console settings, boot image configurations, etc.
>      These settings are used by MACHINE packages, as well as image
>      generation.
>    * Define MACHINE in the conf/local.conf file in the Build Directory.
> * SOC_FAMILY (optional, but implied by the machine)
>    * Way to group together machines based upon common System On Chip
>      components.
>    * A SOC_FAMILY by itself is not a fully configured and bootable
>      machine, but may be used by a series of machines to specify the
>      common components.
> * TUNE (typically specified implicitly by the machine)
>    * Specifies the CPU (instruction set), and ABI settings available to
>      the user
>    * DEFAULTTUNE then selects which of the available tunes is to be used.
>      This is usually set by the machine.
> * DISTRO (Distribution Configuration)
>    * Specifies cross recipe configurations that together result in an
>      overall distribution configuration.
>    * Define DISTRO in the conf/local.conf file in the Build Directory.
>      If not specified, a default "nodistro" distribution is used.
> * Recipes (.bb files)
>    * Specify how to generically build individual items.  These recipes
>      will inherit the system wide settings from the distribution
>      configuration, and tune.  For MACHINE specific packages they can
>      also inherit machine specific settings as well.
> * Image Recipes
>    * Specify a series of dependencies that cause recipes to be built and
>      a list of resulting packages to be installed into a target image.
>    * Image recipes are responsible for constructing a filesystem image.
>      Further the system can extend these into a bootable disc image
>      format.
>
> In any sort of heterogeneous configuration we want to use and build upon
> the existing homogeneous components.  A heterogeneous solution is really
> comprised of a number of homogeneous configurations that when deployed
> together result in a fully functional device.  In other words, each of
> individual parts of the heterogeneous build are standalone and not tied
> to the assembly of a specific system.
>
> Based on this, we want to avoid any changes that complicate the existing
> homogeneous build components or even adding additional levels of
> configuration as this will complicate existing and future uses.  For
> example, in the past there have been suggestions for SUBMACHINE or other
> levels of hierarchy between SOC_FAMILY and MACHINE.  Adding this level
> of indirection can make it more difficult to combine different
> configurations into new heterogeneous solutions.  For example, if
> someone has already defined a heterogeneous solution using a MACHINE,
> SUBMACHINE, and SOC_FAMILY heirachy and you wish to extend it the
> existing MACHINE / SUBMACHINE may interfear with your own systems unique
> configuration.
>
> There are a few types of heterogenous systems that I have seen.  Each of
> them can be constructed by combining the output of homogeneous
> configurations.  The most basic heterogenous systems I have seen include
> either a collection of containers or different OSes, a primary CPU +
> co-processors, or all of the CPUs are independent of each other but
> share resources.  It is also possible to combine these heterogenous
> configurations such as a multiple CPU system, with some CPUs having
> co-processors with one or more CPU running containers.
>
> In the case of the container based system, you really want a master
> homogeneous machine configuration along with a few additional
> configurations that can be incorporated into that image.  Using a
> multilib configuration you would have something like:
>
> build (build directory)
>   conf
>    local.conf:
>     MACHINE = "genericx86_64"
>     DISTRO = "poky"
>     BBMULTICONFIG = "container1 containter2"
>    multiconfig
>     container1.conf:
>      MACHINE = "genericx86_64"
>      DISTRO = "mydistro1"
>      TMPDIR = "${TOPDIR}/tmp/multi/container1"
>     container2.conf:
>      MACHINE = "genericx86_64"
>      DISTRO = "mydistro2"
>      TMPDIR = "${TOPDIR}/tmp/multi/container2"
> layers
>   meta-<custom_layer>
>     conf
>      distro
>       mydistro1.conf
>       mydistro2.conf
>     recipes-images
>      microservices
>       service-image-1.bb
>       service-image-2.bb
>      other
>       my-custom-image-recipe.bb:
>        do_image[mcdepends] = "mc:container1:service-image1:do_rootfs \
>                        mc:container2:service-image2:do_rootfs"
>        do_image() { ... instructions for combining stuff ... }
>
>
> This configuration allows for individual configurations for each
> container and changes to the multiconfigs, but general re-use of much of
> the system.  Especially if the distros in each one are same or similar.
>
> Similar to the above, you could use a multiconfig system to combine
> different operating systems.  For instance, instead of building a
> standard-alone bare-metal style bootloader as part of the OS
> configuration, you could think of it as an external non-OS application.
> A configuration might look like:
>
> build
>   conf
>    local.conf:
>     MACHINE = "genericx86_64"
>     DISTRO = "poky"
>     BBMULTICONFIG = "bootloader"
>    multiconfig
>     bootloader.conf:
>      MACHINE = "genericx86_64"
>      DISTRO = "baremetal"
>      TMPDIR = "${TOPDIR}/tmp/multi/bootloader"
> layers
>   meta-mybootloader
>    conf
>     distro
>      baremetal.conf
>    recipes
>     newlib
>      newlib_ver.bb
>     first_stage
>      first_stage.bb
>     second_stage
>      second_stage.bb
>     bootloader
>      bootloader.bb
>       DEPENDS = "newlib first_stage second_stage"
>   meta-<custom_layer>
>    recipes
>     bootloader
>      mybootloader.bb
>       depend on mc:bootloader:second_stage:....
>
>
> For a simple heterogeneous solution, where the main CPU may need to load
> software for co-processors, the configuration would be similar to the
> above bare-metal example.  The difference would be that the MACHINE and
> DISTRO settings for a DSP would need to have the necessary
> configurations to properly build applications for the DSP.  Then the
> Linux side of things can take this baremetal software and package it up
> independently or in conjunction with software to actually configure and
> load that software:
>
> build
>   conf
>    local.conf:
>     MACHINE = "myarmccpu"
>     DISTRO = "poky"
>     BBMULTICONFIG = "dsp"
>    multiconfig
>     dsp.conf:
>      MACHINE = "magic-dsp"
>      DISTRO = "baremetal"
>      TMPDIR = "${TOPDIR}/tmp/multi/dsp"
> layers
>   meta-dsp
>    conf
>     distro
>      baremetal.conf
>     machine
>      dsp.conf
>    recipes-dsp
>     newlib
>      newlib_ver.bb
>     library
>      library.bb
>     application
>      application.bb
>       DEPENDS = "newlib library application"
>    recipes-linux
>     dsp-application
>      ...depends on application...
>
> Note: in the above, separating the distro, machine and application
> components into individual layers may be needed for Yocto Project
> compliance.
>
>
> In all of the above examples, the user has manually configured the
> multiconfig within their project.  There is a simple way to move that
> configuration to a layer, simply place it in a conf/multiconfig
> directory within that layer.
>
> This ability suggests to be that there should be a standard way to
> specify a layer above that of machine, which defines the overall
> characteristics of the system.
>
> I'm proposing calling this new layer type as a system layer and it's
> configuration variable "SYSTEM".  It will be used instead of MACHINE
> when there is no single machine to quantify the contents of the produced
> system image.  When implementing a system, we do not want to make major
> changes to any other components.  Due to the existing implementation
> requiring MACHINE and certain TUNE parameters, this will require us to
> provide a special MACHINE value that can be used for a heterogeneous
> system.  I suggest we create a new 'nomachine' system that only
> defines/uses an equivalent noarch style tune.  This will instruct the
> system that this configuration can be used to noarch software and create
> images (including with wic), but it is not able to compile specific
> applications.  Each of these applications or images must come from a
> defined MACHINE.
>
> The SYSTEM level multiconfig could be used to combine any homogeneous or
> heterogeneous configuration.  For example:
>
> build
>   conf
>    local.conf:
>     SYSTEM = "mysystem"
> layers
>   meta-<system>
>    conf
>     system
>      mysystem.conf
>       MACHINE = "nomachine"
>       BBMULTICONFIG = "bootloader fpga linux"
>      mysystem.wks
>     multiconfig
>      bootloader.conf:
>       MACHINE = "zcu102_microblaze"
>       DISTRO = "baremetal"
>       TMPDIR = "${TOPDIR}/tmp/multi/bootloader"
>      fpga.conf:
>       MACHINE = "zcu_fpga"
>       DISTRO = "baremetal"
>       TMPDIR = "${TOPDIR}/tmp/multi/fpga"
>      linux.conf:
>       MACHINE = "zcu_cortex-a72"
>       DISTRO = "poky"
>       TMPDIR = "${TOPDIR}/tmp/multi/linux"
>    recipes
>     images
>      system-images.bb
>       do_image[mcdepends] = "mc:bootloader:application:do_deploy \
>                              mc:fpga:application:do_deploy \
>                              mc:linux:core-image-minimal:do_rootfs"
>       do_image() { ... instructions for combining stuff ... }
>   meta-<machine>
>    conf
>     machine
>      zcu_microblaze.conf
>      zcu_fpga.conf
>      zcu_cortext-a72.conf
>   meta-mybootloader
>    conf
>     distro
>      baremetal.conf
>    recipes
>     newlib
>      newlib_ver.bb
>     first_stage
>      first_stage.bb
>     second_stage
>      second_stage.bb
>     bootloader
>      bootloader.bb
>       DEPENDS = "newlib first_stage second_stage"
>   meta-fpga
>    conf
>     distro
>      baremetal.conf
>    recipes-baremetal
>     newlib
>      newlib_ver.bb
>     library
>      library.bb
>     application
>      application.bb
>       DEPENDS = "newlib library application"

A lot of what you're talking about here we've been doing for a while
in meta-oryx with SYSTEM_PROFILE and APPLICATION_PROFILE.

https://gitlab.com/oryx/meta-oryx/tree/master

├── conf
│   ├── application-profiles
│   │   ├── full-cmdline.conf
│   │   ├── host.conf
│   │   ├── host-mender-update-modules.conf
│   │   ├── host-test.conf
│   │   └── minimal.conf
│   ├── distro
│   │   └── oryx.conf
│   ├── layer.conf
│   └── system-profiles
│       ├── guest-mender-update-module.conf
│       ├── native.conf
│       └── native-mender.conf

In what we're doing, we're mainly focusing on native hosts and
containerised guests, but there is no reason this couldn't be expanded
to fit your use case.

 -b

>
> So proposal:
>
> Introduce a new OPTIONAL "SYSTEM" variable.  Which, like machine, would
> automatically "include conf/${SYSTEM}.conf", if defined.
>
> Introduce nomachine and notune, and adjust system components to know how
> to deal with these properly.  I.e. recipes that are NOT noarch would be
> be rejected in this configuration.
>
> Introduce system layers to the yocto check script, and layer index.
>
> Add documentation explaining the new SYSTEM variable and it's usage for
> building complex heterogeneous systems.
>
> Option:
>
> If SYSTEM (as described above) does not make sense, then we could re-use
> MACHINE instead, but I would still like a notune option for a 'machine'
> that shouldn't have software compiled for it.
>
> --Mark
> This email and any attachments are intended for the sole use of the named
> recipient(s) and contain(s) confidential information that may be proprietary,
> privileged or copyrighted under applicable law. If you are not the intended
> recipient, do not read, copy, or forward this email message or any 
> attachments.
> Delete this email message and any attachments immediately.
> _______________________________________________
> Openembedded-architecture mailing list
> Openembedded-architecture@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture

Reply via email to