Re: [Openembedded-architecture] Heterogeneous System Proposal
Taking just this part of the discussion for now (since it is a small familiar aspect to me) On Tuesday, 3 December 2019 1:08:54 PM NZDT Mark Hatle wrote: > > You may want to consider this now. Adding system with the "nomachine" > > mode you mention is a nightmare scenario for parsing. Bitbake has a > > problem with knowing when to parse or not to parse since to 'skip' > > parsing a recipe, it needs to parse enough to know it should skip it. > > COMPATIBLE_MACHINE is implemented with the skip mechanism for example. > > If parsing is a concern (and I agree it is), you could probably get > > much further with a BBMASK approach to just the system-images recipe as > > then bitbake would know what it needs to parse without needing to read > > the recipes. > > The problem I have with BBMASK is that when a user tries to do something with > it, they don't get a reasonable error message. > > For instance if I BBMASK out bash, and the user does bitbake bash they get an > error. Instead of a 'bash is unavailable because ...'. The only practical way for that to work is the existing skip mechanism, since PN isn't always completely defined by the .bb filename, we need to actually parse to determine its value. gcc-cross and other parts of the toolchain are examples. BBMASK is such a usability pain - both in your scenario but also in the reverse where you have to hack it until it only masks exactly what you do want masked - that I think we would be best advised not to make any more use of it than we currently do, especially if it's only to save a bit of time parsing. Cheers Paul -- Paul Eggleton Intel System Software Products ___ Openembedded-architecture mailing list Openembedded-architecture@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
Re: [Openembedded-architecture] Heterogeneous System Proposal
On 12/2/19 5:53 PM, Richard Purdie wrote: > On Mon, 2019-12-02 at 17:19 -0600, Mark Hatle wrote: >> >> On 12/2/19 3:58 PM, Richard Purdie wrote: >>> On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote: In all of the above examples, the user has manually configured the multiconfig within their project. There is a simple way to move that configuration to a layer, simply place it in a conf/multiconfig directory within that layer. This ability suggests to be that there should be a standard way to specify a layer above that of machine, which defines the overall characteristics of the system. I'm proposing calling this new layer type as a system layer and it's configuration variable "SYSTEM". It will be used instead of MACHINE when there is no single machine to quantify the contents of the produced system image. When implementing a system, we do not want to make major changes to any other components. Due to the existing implementation requiring MACHINE and certain TUNE parameters, this will require us to provide a special MACHINE value that can be used for a heterogeneous system. I suggest we create a new 'nomachine' system that only defines/uses an equivalent noarch style tune. This will instruct the system that this configuration can be used to noarch software and create images (including with wic), but it is not able to compile specific applications. Each of these applications or images must come from a defined MACHINE. >>> >>> I'm not sure I 'buy' this explanation and I think a "nomachine" >>> target >>> will be horrible in practice. In your example below, why can't >>> mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other >>> bits of >>> linux.conf and BBMULTICONFIG = "bootloader fpga" ? >> >> This assumes the linux side of things will load the others. In the >> systems I am thinking of Linux may not be a primary system. >> >> I.e. a baremetal system based on a microblaze may be the core >> system, which then bootstraps and launches the linux system. The >> core 'machine' should it be defined as baremetal and >> zcu102_microblaze then? > > Sure, why not? I picked on 'linux' but I could have equally chosen any > one of the targets, there wasn't any specific reason. > >> Problem is baremetal isn't really an operating system, so how do we >> handle that? > > I'm not sure it matters, MACHINE doesn't define an operating system. Correct distro does, I was confusing my statements. Both MACHINE and DISTRO have to work together to define the system. My assumption was that a system would define reasonable defaults, like a multiconfig does. (Maybe this is an error on my part though.) >> Introduce nomachine and notune, and adjust system components to know how to deal with these properly. I.e. recipes that are NOT noarch would be be rejected in this configuration. >>> >>> I definitely do not like this at all. I'm not sure why its needed. >> >> I want a way to say that this configuration is a sum of it's parts, >> but by itself can't do anything. It's not arm, it's not ppc, it's >> not x86, etc.. > > The more I read and think about this, the more I'm concluding it still > *is* a MACHINE. A MACHINE represents a target piece of hardware, not an > OS, or an architecture. Those are configurations of that hardware > combined with software. > >> This is common on FPGA systems that may have to load from special >> configurations, the hardware itself may be loading a bitstream (not a >> cpu instruction set) that in turn may be a first stage loader for one >> or multiple on system CPUs, which may in turn instruct other aspects >> of the system to load. >> >> I don't want the proposal to be FPGA specific, as I expect it will be >> used in other heterogeneous cases. Primarily ones that have a single >> host CPU (that boot) and multiple plugin cards. In those cases, it >> -CAN- have both a SYSTEM and a MACHINE defined, or just simply a >> MACHINE with multiconfigs that the MACHINE has a way of >> understanding. > > I think you need a convention for MACHINE that works for these systems. > I'm not sure we need a "new" level specially for this. Agreed. The only reason to add a SYSTEM would be to distinguish it in the local.conf, but from a practical reason it and MACHINE are equivalent in hierarchy in my mind. SYSTEM is 1..n multiconfigs, while MACHINE requires a DISTRO to provide a working system. >>> I'd also put the idea out there that in many ways what you're >>> describing is actually a configuration or system configuration. Its >>> configurations like this we actually test on the project >>> autobuilder >>> and its what the code in yocto-autobuilder-helper generates from >>> json >>> files. >> >> I would expect in my particular use-case that I do need to generate >> configurations based on certain
Re: [Openembedded-architecture] Heterogeneous System Proposal
On 12/2/19 5:47 PM, Richard Purdie wrote:
> On Mon, 2019-12-02 at 15:27 -0800, Alejandro Enedino Hernandez
> Samaniego wrote:
>> So I have tried something similar to this, and while it does work,
>> due to some parsing issue (I still have not figured out exactly what)
>> the multiconfig dependencies between multiconfigs are not being taken
>> into account, which defeats the whole purpose.
>
> It likely makes sense to understand why its breaking since wouldn't
> those issues affect this regardless of which direction things go in?
>
>> While I understand that introducing a new variable might create some
>> confusion, at the same time I do believe that it would be
>> better if we do it that way, and in some way separate workflows when
>> we are building a "full" heterogeneous system vs a single
>> architecture system, the difference lies in the fact that we dont set
>> what the "master" or core MACHINE is, they are all equal.
>>
>> I have been working on the implementation of this proposal and here
>> is what I currently have for which of course I am open to change.
>>
>> Like I said, I had issues with defining a placeholder machine for the
>> multiconfig dependencies, so I did go with defining
>>
>> SYSTEM = "dummy-system"
>>
>> on local.conf (also not a huge fan of the name since to me its a bit
>> too generic,but as RP said, the name is probably the least of our
>> problems)
>>
>> then
>> include conf/system/${SYSTEM}.conf
>>
>> and I have the following set up on a layer:
>>
>> meta-heterogeneous
>> conf/layer.conf
>> conf/system/dummy-system.conf
>> conf/multiconfig/dummy-arm64.conf
>> conf/multiconfig/dummy-x86-64.conf
>> recipes-heterogeneous/heterogeneous-example_1.0.bb
>>
>> Explanation:
>> layer.conf - normal layer.conf, nothing out of the ordinary
>> dummy-system.conf - Defines BBMULTICONFIG as dummy-arm64 and dummy-
>> x86-64, AND defines the multiconfig dependencies between them
>> dummy-*64.conf - Define each of the multiconfigs, in them there are
>> only 3 variables, DISTRO, MACHINE (qemuarm64 and qemux86-64
>> respectively), and TMPDIR which is simpy defined as tmp-${MACHINE}
>> heterogeneous-example_1.0.bb : This was a placeholder recipe, since
>> to my knowledge bitbake required a target, and it simply contained a
>> line that set up a multiconfig dependency that would allow:
>>
>> $ bitbake heterogeneous-example
>
> I'm going to try and make a concise reply to both you and Mark.
>
> What worries me is that in this "dummy-system" context there is only
> one valid recipe which is the heterogeneous-example, everything else is
> meaningless.
My assumption was that, like a baremetal system, recipes would be limited to a
small number of things -- primarily different image recipes. But this is out of
convention rather then actually modifying things to say 'only images can be
used'.
> If we go this route we'll start having to add special corner casing for
> "nomachine" and "notune" and everywhere, then special case those values
> since there is code they will need to pass through. I don't think these
> concepts make sense to the majority of users and I think the damage
> we'd do to the metadata trying to retrofit them will be significant and
> not in the interests of the project.
(I choose the names nomachine and notune to match the existing nodistro option.
Provide reasonable defaults that do the 'right thing' within that context.)
I was expecting the opposite here. We wouldn't modify any existing code (I
think), so hashes, metadata etc wouldn't need to be modified. Since nomachine
would presumably load notune.. Then the notune could define a blank tune that
doesn't "do antyhing".
The only place we may need some code is in an anon python chunk that if notune
is defined and the recipe isn't noarch/all.. it would raise the skipRecipe
exception with an appropriate message.
> Looking at things a different way, the proposal makes little difference
> to adding a new machine above called "dummy-system" which sets up the
> multiconfigs, it just gives it a different name, SYSTEM instead of
> MACHINE. I think you can achieve much of (all?) the same things with a
> naming convention rather than needing a new namespace. If multiconfig
> has bugs with that, we should fix them?
Agreed. SYSTEM, MACHINE, MACHINE w/ a convention.. it's all fine, as long as we
have a standard way to refer to the item and define it.
--Mark
> Cheers,
>
> Richard
>
>
>
___
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
Re: [Openembedded-architecture] Heterogeneous System Proposal
On Mon, 2019-12-02 at 17:19 -0600, Mark Hatle wrote: > > On 12/2/19 3:58 PM, Richard Purdie wrote: > > On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote: > > > In all of the above examples, the user has manually configured > > > the > > > multiconfig within their project. There is a simple way to move > > > that > > > configuration to a layer, simply place it in a conf/multiconfig > > > directory within that layer. > > > > > > This ability suggests to be that there should be a standard way > > > to > > > specify a layer above that of machine, which defines the overall > > > characteristics of the system. > > > > > > I'm proposing calling this new layer type as a system layer and > > > it's > > > configuration variable "SYSTEM". It will be used instead of > > > MACHINE > > > when there is no single machine to quantify the contents of the > > > produced system image. When implementing a system, we do not > > > want to > > > make major changes to any other components. Due to the existing > > > implementation requiring MACHINE and certain TUNE parameters, > > > this > > > will require us to provide a special MACHINE value that can be > > > used > > > for a heterogeneous system. I suggest we create a new > > > 'nomachine' > > > system that only defines/uses an equivalent noarch style > > > tune. This > > > will instruct the system that this configuration can be used to > > > noarch software and create images (including with wic), but it is > > > not > > > able to compile specific applications. Each of these > > > applications or > > > images must come from a defined MACHINE. > > > > I'm not sure I 'buy' this explanation and I think a "nomachine" > > target > > will be horrible in practice. In your example below, why can't > > mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other > > bits of > > linux.conf and BBMULTICONFIG = "bootloader fpga" ? > > This assumes the linux side of things will load the others. In the > systems I am thinking of Linux may not be a primary system. > > I.e. a baremetal system based on a microblaze may be the core > system, which then bootstraps and launches the linux system. The > core 'machine' should it be defined as baremetal and > zcu102_microblaze then? Sure, why not? I picked on 'linux' but I could have equally chosen any one of the targets, there wasn't any specific reason. > Problem is baremetal isn't really an operating system, so how do we > handle that? I'm not sure it matters, MACHINE doesn't define an operating system. > Introduce nomachine and notune, and adjust system components to > > > know > > > how to deal with these properly. I.e. recipes that are NOT > > > noarch > > > would be be rejected in this configuration. > > > > I definitely do not like this at all. I'm not sure why its needed. > > I want a way to say that this configuration is a sum of it's parts, > but by itself can't do anything. It's not arm, it's not ppc, it's > not x86, etc.. The more I read and think about this, the more I'm concluding it still *is* a MACHINE. A MACHINE represents a target piece of hardware, not an OS, or an architecture. Those are configurations of that hardware combined with software. > This is common on FPGA systems that may have to load from special > configurations, the hardware itself may be loading a bitstream (not a > cpu instruction set) that in turn may be a first stage loader for one > or multiple on system CPUs, which may in turn instruct other aspects > of the system to load. > > I don't want the proposal to be FPGA specific, as I expect it will be > used in other heterogeneous cases. Primarily ones that have a single > host CPU (that boot) and multiple plugin cards. In those cases, it > -CAN- have both a SYSTEM and a MACHINE defined, or just simply a > MACHINE with multiconfigs that the MACHINE has a way of > understanding. I think you need a convention for MACHINE that works for these systems. I'm not sure we need a "new" level specially for this. > > I'd also put the idea out there that in many ways what you're > > describing is actually a configuration or system configuration. Its > > configurations like this we actually test on the project > > autobuilder > > and its what the code in yocto-autobuilder-helper generates from > > json > > files. > > I would expect in my particular use-case that I do need to generate > configurations based on certain parameters (specifically a system > device tree). However, other configurations are going to be very > static and then predefined 'systems' can be provided by layer that > give the user the ability to modify (as they would any other part of > the system today.) > > > There is a little known/used piece of bitbake which is BBTARGETS. > > If > > your myssystem.conf sets: > > > > BBTARGETS = "system-images:do_image" > > I wasn't aware this was a thing, but it could very well be used by a > system configuration to define the right combination of things to > build. I didn't
Re: [Openembedded-architecture] Heterogeneous System Proposal
On Mon, 2019-12-02 at 15:27 -0800, Alejandro Enedino Hernandez
Samaniego wrote:
> So I have tried something similar to this, and while it does work,
> due to some parsing issue (I still have not figured out exactly what)
> the multiconfig dependencies between multiconfigs are not being taken
> into account, which defeats the whole purpose.
It likely makes sense to understand why its breaking since wouldn't
those issues affect this regardless of which direction things go in?
> While I understand that introducing a new variable might create some
> confusion, at the same time I do believe that it would be
> better if we do it that way, and in some way separate workflows when
> we are building a "full" heterogeneous system vs a single
> architecture system, the difference lies in the fact that we dont set
> what the "master" or core MACHINE is, they are all equal.
>
> I have been working on the implementation of this proposal and here
> is what I currently have for which of course I am open to change.
>
> Like I said, I had issues with defining a placeholder machine for the
> multiconfig dependencies, so I did go with defining
>
> SYSTEM = "dummy-system"
>
> on local.conf (also not a huge fan of the name since to me its a bit
> too generic,but as RP said, the name is probably the least of our
> problems)
>
> then
> include conf/system/${SYSTEM}.conf
>
> and I have the following set up on a layer:
>
> meta-heterogeneous
> conf/layer.conf
> conf/system/dummy-system.conf
> conf/multiconfig/dummy-arm64.conf
> conf/multiconfig/dummy-x86-64.conf
> recipes-heterogeneous/heterogeneous-example_1.0.bb
>
> Explanation:
> layer.conf - normal layer.conf, nothing out of the ordinary
> dummy-system.conf - Defines BBMULTICONFIG as dummy-arm64 and dummy-
> x86-64, AND defines the multiconfig dependencies between them
> dummy-*64.conf - Define each of the multiconfigs, in them there are
> only 3 variables, DISTRO, MACHINE (qemuarm64 and qemux86-64
> respectively), and TMPDIR which is simpy defined as tmp-${MACHINE}
> heterogeneous-example_1.0.bb : This was a placeholder recipe, since
> to my knowledge bitbake required a target, and it simply contained a
> line that set up a multiconfig dependency that would allow:
>
> $ bitbake heterogeneous-example
I'm going to try and make a concise reply to both you and Mark.
What worries me is that in this "dummy-system" context there is only
one valid recipe which is the heterogeneous-example, everything else is
meaningless.
If we go this route we'll start having to add special corner casing for
"nomachine" and "notune" and everywhere, then special case those values
since there is code they will need to pass through. I don't think these
concepts make sense to the majority of users and I think the damage
we'd do to the metadata trying to retrofit them will be significant and
not in the interests of the project.
Looking at things a different way, the proposal makes little difference
to adding a new machine above called "dummy-system" which sets up the
multiconfigs, it just gives it a different name, SYSTEM instead of
MACHINE. I think you can achieve much of (all?) the same things with a
naming convention rather than needing a new namespace. If multiconfig
has bugs with that, we should fix them?
Cheers,
Richard
___
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
Re: [Openembedded-architecture] Heterogeneous System Proposal
Hey guys,
On Mon, Dec 2, 2019 at 3:24 PM Mark Hatle
wrote:
>
>
> On 12/2/19 4:09 PM, Richard Purdie wrote:
> > On Mon, 2019-12-02 at 21:58 +, Richard Purdie wrote:
> >> On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote:
> >>> In all of the above examples, the user has manually configured the
> >>> multiconfig within their project. There is a simple way to move
> >>> that
> >>> configuration to a layer, simply place it in a conf/multiconfig
> >>> directory within that layer.
> >>>
> >>> This ability suggests to be that there should be a standard way to
> >>> specify a layer above that of machine, which defines the overall
> >>> characteristics of the system.
> >>>
> >>> I'm proposing calling this new layer type as a system layer and
> >>> it's
> >>> configuration variable "SYSTEM". It will be used instead of
> >>> MACHINE
> >>> when there is no single machine to quantify the contents of the
> >>> produced system image. When implementing a system, we do not want
> >>> to
> >>> make major changes to any other components. Due to the existing
> >>> implementation requiring MACHINE and certain TUNE parameters, this
> >>> will require us to provide a special MACHINE value that can be used
> >>> for a heterogeneous system. I suggest we create a new 'nomachine'
> >>> system that only defines/uses an equivalent noarch style
> >>> tune. This
> >>> will instruct the system that this configuration can be used to
> >>> noarch software and create images (including with wic), but it is
> >>> not
> >>> able to compile specific applications. Each of these applications
> >>> or
> >>> images must come from a defined MACHINE.
> >>
> >> I'm not sure I 'buy' this explanation and I think a "nomachine"
> >> target will be horrible in practice. In your example below, why can't
> >> mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other bits
> >> of linux.conf and BBMULTICONFIG = "bootloader fpga" ?
> >
> > In fact, if you create a conf/machine/system-zcu_cortex-a72.conf and
> > have it set something like:
> >
> > BBMULTICONFIG = "bootloader fpga"
> > MACHINE = "zcu_cortex-a72"
> > require conf/machine/${MACHINE}.conf
> >
> > I think what I/you describe may even work today without a new variable
> > and simply a new convention of a special set of new machine name
> > targets in layers...
>
So I have tried something similar to this, and while it does work, due to
some parsing issue (I still have not figured out exactly what)
the multiconfig dependencies between multiconfigs are not being taken into
account, which defeats the whole purpose.
While I understand that introducing a new variable might create some
confusion, at the same time I do believe that it would be
better if we do it that way, and in some way separate workflows when we are
building a "full" heterogeneous system vs a single
architecture system, the difference lies in the fact that we dont set what
the "master" or core MACHINE is, they are all equal.
I have been working on the implementation of this proposal and here is what
I currently have for which of course I am open to change.
Like I said, I had issues with defining a placeholder machine for the
multiconfig dependencies, so I did go with defining
SYSTEM = "dummy-system"
on local.conf (also not a huge fan of the name since to me its a bit too
generic,but as RP said, the name is probably the least of our problems)
then
include conf/system/${SYSTEM}.conf
and I have the following set up on a layer:
meta-heterogeneous
conf/layer.conf
conf/system/dummy-system.conf
conf/multiconfig/dummy-arm64.conf
conf/multiconfig/dummy-x86-64.conf
recipes-heterogeneous/heterogeneous-example_1.0.bb
Explanation:
layer.conf - normal layer.conf, nothing out of the ordinary
dummy-system.conf - Defines BBMULTICONFIG as dummy-arm64 and dummy-x86-64,
AND defines the multiconfig dependencies between them
dummy-*64.conf - Define each of the multiconfigs, in them there are only 3
variables, DISTRO, MACHINE (qemuarm64 and qemux86-64 respectively), and
TMPDIR which is simpy defined as tmp-${MACHINE}
heterogeneous-example_1.0.bb : This was a placeholder recipe, since to my
knowledge bitbake required a target, and it simply contained a line that
set up a multiconfig dependency that would allow:
$ bitbake heterogeneous-example
to start a buld for core-image-minimal (for example) for all archs in the
SYSTEM.
Now that I read RP's email about BBTARGETS I believe this can be improved
and at the very least delete the example recipe, let me know what yo guys
think
Cheers,
Alejandro
>
> Yes, SYSTEM could be implemented as something like:
>
> MACHINE ??= ${@'system-${SYSTEM}' if d.getVar('SYSTEM')}
>
> (I'm not really sure if that will work on not...)
>
> The key though, I want a way to specify a system that by itself can't build
> binaries -- but can only include binaries from other architectures. (It
> -CAN-
> however build noarch/all [text file] components to be included in the
> image.)
>
> So you
Re: [Openembedded-architecture] Heterogeneous System Proposal
On 12/2/19 4:09 PM, Richard Purdie wrote:
> On Mon, 2019-12-02 at 21:58 +, Richard Purdie wrote:
>> On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote:
>>> In all of the above examples, the user has manually configured the
>>> multiconfig within their project. There is a simple way to move
>>> that
>>> configuration to a layer, simply place it in a conf/multiconfig
>>> directory within that layer.
>>>
>>> This ability suggests to be that there should be a standard way to
>>> specify a layer above that of machine, which defines the overall
>>> characteristics of the system.
>>>
>>> I'm proposing calling this new layer type as a system layer and
>>> it's
>>> configuration variable "SYSTEM". It will be used instead of
>>> MACHINE
>>> when there is no single machine to quantify the contents of the
>>> produced system image. When implementing a system, we do not want
>>> to
>>> make major changes to any other components. Due to the existing
>>> implementation requiring MACHINE and certain TUNE parameters, this
>>> will require us to provide a special MACHINE value that can be used
>>> for a heterogeneous system. I suggest we create a new 'nomachine'
>>> system that only defines/uses an equivalent noarch style
>>> tune. This
>>> will instruct the system that this configuration can be used to
>>> noarch software and create images (including with wic), but it is
>>> not
>>> able to compile specific applications. Each of these applications
>>> or
>>> images must come from a defined MACHINE.
>>
>> I'm not sure I 'buy' this explanation and I think a "nomachine"
>> target will be horrible in practice. In your example below, why can't
>> mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other bits
>> of linux.conf and BBMULTICONFIG = "bootloader fpga" ?
>
> In fact, if you create a conf/machine/system-zcu_cortex-a72.conf and
> have it set something like:
>
> BBMULTICONFIG = "bootloader fpga"
> MACHINE = "zcu_cortex-a72"
> require conf/machine/${MACHINE}.conf
>
> I think what I/you describe may even work today without a new variable
> and simply a new convention of a special set of new machine name
> targets in layers...
Yes, SYSTEM could be implemented as something like:
MACHINE ??= ${@'system-${SYSTEM}' if d.getVar('SYSTEM')}
(I'm not really sure if that will work on not...)
The key though, I want a way to specify a system that by itself can't build
binaries -- but can only include binaries from other architectures. (It -CAN-
however build noarch/all [text file] components to be included in the image.)
So you end up with:
system-zcu.conf
zcu-cortex-a72.conf
zcu-r5.conf
zcu-microblaze.conf
In the end the users should be able to build each of the components
independently WITHOUT ever using system, if they choose the 'old' way of
combining parts and pieces post build outside of the build system.
If they want a pre-defined multiconfig build, then they could use the system-zcu
to select all of the defaults for them.
--Mark
> Cheers,
>
> Richard
>
___
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
Re: [Openembedded-architecture] Heterogeneous System Proposal
On 12/2/19 3:58 PM, Richard Purdie wrote:
> On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote:
>> In all of the above examples, the user has manually configured the
>> multiconfig within their project. There is a simple way to move that
>> configuration to a layer, simply place it in a conf/multiconfig
>> directory within that layer.
>>
>> This ability suggests to be that there should be a standard way to
>> specify a layer above that of machine, which defines the overall
>> characteristics of the system.
>>
>> I'm proposing calling this new layer type as a system layer and it's
>> configuration variable "SYSTEM". It will be used instead of MACHINE
>> when there is no single machine to quantify the contents of the
>> produced system image. When implementing a system, we do not want to
>> make major changes to any other components. Due to the existing
>> implementation requiring MACHINE and certain TUNE parameters, this
>> will require us to provide a special MACHINE value that can be used
>> for a heterogeneous system. I suggest we create a new 'nomachine'
>> system that only defines/uses an equivalent noarch style tune. This
>> will instruct the system that this configuration can be used to
>> noarch software and create images (including with wic), but it is not
>> able to compile specific applications. Each of these applications or
>> images must come from a defined MACHINE.
>
> I'm not sure I 'buy' this explanation and I think a "nomachine" target
> will be horrible in practice. In your example below, why can't
> mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other bits of
> linux.conf and BBMULTICONFIG = "bootloader fpga" ?
This assumes the linux side of things will load the others. In the systems I am
thinking of Linux may not be a primary system.
I.e. a baremetal system based on a microblaze may be the core system, which
then bootstraps and launches the linux system. The core 'machine' should it be
defined as baremetal and zcu102_microblaze then?
Problem is baremetal isn't really an operating system, so how do we handle that?
>> The SYSTEM level multiconfig could be used to combine any homogeneous
>> or heterogeneous configuration. For example:
>>
>> build
>> conf
>>local.conf:
>> SYSTEM = "mysystem"
>> layers
>> meta-
>>conf
>> system
>> mysystem.conf
>> MACHINE = "nomachine"
>> BBMULTICONFIG = "bootloader fpga linux"
>> mysystem.wks
>> multiconfig
>> bootloader.conf:
>> MACHINE = "zcu102_microblaze"
>> DISTRO = "baremetal"
>> TMPDIR = "${TOPDIR}/tmp/multi/bootloader"
>> fpga.conf:
>> MACHINE = "zcu_fpga"
>> DISTRO = "baremetal"
>> TMPDIR = "${TOPDIR}/tmp/multi/fpga"
>> linux.conf:
>> MACHINE = "zcu_cortex-a72"
>> DISTRO = "poky"
>> TMPDIR = "${TOPDIR}/tmp/multi/linux"
>>recipes
>> images
>> system-images.bb
>> do_image[mcdepends] = "mc:bootloader:application:do_deploy \
>> mc:fpga:application:do_deploy \
>> mc:linux:core-image-minimal:do_rootfs"
>> do_image() { ... instructions for combining stuff ... }
>> meta-
>>conf
>> machine
>> zcu_microblaze.conf
>> zcu_fpga.conf
>> zcu_cortext-a72.conf
>> meta-mybootloader
>>conf
>> distro
>> baremetal.conf
>>recipes
>> newlib
>> newlib_ver.bb
>> first_stage
>> first_stage.bb
>> second_stage
>> second_stage.bb
>> bootloader
>> bootloader.bb
>> DEPENDS = "newlib first_stage second_stage"
>> meta-fpga
>>conf
>> distro
>> baremetal.conf
>>recipes-baremetal
>> newlib
>> newlib_ver.bb
>> library
>> library.bb
>> application
>> application.bb
>> DEPENDS = "newlib library application"
>>
>> So proposal:
>>
>> Introduce a new OPTIONAL "SYSTEM" variable. Which, like machine,
>> would automatically "include conf/${SYSTEM}.conf", if defined.
>
> I know we talked and I think I mentioned SYSTEM. I'm still not entirely
> sold on that name, however the name is only one piece of the puzzle so
> I'll defer that for now.
>
>> Introduce nomachine and notune, and adjust system components to know
>> how to deal with these properly. I.e. recipes that are NOT noarch
>> would be be rejected in this configuration.
>
> I definitely do not like this at all. I'm not sure why its needed.
I want a way to say that this configuration is a sum of it's parts, but by
itself can't do anything. It's not arm, it's not ppc, it's not x86, etc..
This is common on FPGA systems that may have to load from special
configurations, the hardware itself may be loading a bitstream (not a cpu
instruction set) that in turn may be a first stage loader for one or multiple on
system CPUs, which may in turn instruct other aspects of the system to load.
I don't want the proposal to be FPGA specific, as I expect it will be used in
other
Re: [Openembedded-architecture] Heterogeneous System Proposal
On Mon, 2019-12-02 at 21:58 +, Richard Purdie wrote:
> On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote:
> > In all of the above examples, the user has manually configured the
> > multiconfig within their project. There is a simple way to move
> > that
> > configuration to a layer, simply place it in a conf/multiconfig
> > directory within that layer.
> >
> > This ability suggests to be that there should be a standard way to
> > specify a layer above that of machine, which defines the overall
> > characteristics of the system.
> >
> > I'm proposing calling this new layer type as a system layer and
> > it's
> > configuration variable "SYSTEM". It will be used instead of
> > MACHINE
> > when there is no single machine to quantify the contents of the
> > produced system image. When implementing a system, we do not want
> > to
> > make major changes to any other components. Due to the existing
> > implementation requiring MACHINE and certain TUNE parameters, this
> > will require us to provide a special MACHINE value that can be used
> > for a heterogeneous system. I suggest we create a new 'nomachine'
> > system that only defines/uses an equivalent noarch style
> > tune. This
> > will instruct the system that this configuration can be used to
> > noarch software and create images (including with wic), but it is
> > not
> > able to compile specific applications. Each of these applications
> > or
> > images must come from a defined MACHINE.
>
> I'm not sure I 'buy' this explanation and I think a "nomachine"
> target will be horrible in practice. In your example below, why can't
> mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other bits
> of linux.conf and BBMULTICONFIG = "bootloader fpga" ?
In fact, if you create a conf/machine/system-zcu_cortex-a72.conf and
have it set something like:
BBMULTICONFIG = "bootloader fpga"
MACHINE = "zcu_cortex-a72"
require conf/machine/${MACHINE}.conf
I think what I/you describe may even work today without a new variable
and simply a new convention of a special set of new machine name
targets in layers...
Cheers,
Richard
___
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
