I’ve had success with this. Thanks to all of you for providing information.
Here is the solution that worked for me (I’m using ‘honister’):
On my Ubuntu-18.04-LTS host I have the following in /usr/bin:
|qemu-aarch64-static qemu-mips64el-static qemu-ppc64-static qemu-alpha-static
qemu-mips64-static qemu-ppc-static qemu-armeb-static qemu-mipsel-static
qemu-s390x-static qemu-arm-static qemu-mipsn32el-static qemu-sh4eb-static
qemu-cris-static qemu-mipsn32-static qemu-sh4-static qemu-hppa-static
qemu-mips-static qemu-sparc32plus-static qemu-i386-static qemu-nios2-static
qemu-sparc64-static qemu-m68k-static qemu-or1k-static qemu-sparc-static
qemu-microblazeel-static qemu-ppc64abi32-static qemu-tilegx-static
qemu-microblaze-static qemu-ppc64le-static qemu-x86_64-static |
In my distro ‘local.conf’, I added the following:
|ASSUME_PROVIDED += " \ qemu-native \ qemu-helper-native \ qemu-system-native
\ qemu-xilinx \ qemu-xilinx-multiarch-helper-native \ qemu-xilinx-native \
qemu-xilinx-system-native \ " INHERIT+="blacklist"
PNBLACKLIST[qemu]="blacklist" PNBLACKLIST[qemu-xilinx]="blacklist"
PNBLACKLIST[qemu-xilinx-multiarch-helper-native]="blacklist"
PNBLACKLIST[qemu-xilinx-native]="blacklist"
PNBLACKLIST[qemu-xilinx-system-native]="blacklist" |
NOTE: I had to add the ‘qemu-xilinx-*’ to the ASSUME_PROVIDED to prevent yocto
from trying to build them. Without those the build would complain that they
were dependencies and that they were skipped due to ‘blacklist’.
In my 'image-common.inc' which is required by all 4 of the image variants that
I am building, I added this line:
|IMAGE_FSTYPES:remove = " wic.qemu-sd" |
This prevents the meta-xilinx layer from attempting to generate a qemu-sd
image during the regular image packaging step; it still produces the regular
SD image. After that it all works nicely. I think that I can use this same
technique for some of the other slow-build dependencies such as
‘binutils-native’.
Anyway for me this is a valuable technique for a somewhat unique use-case.
FYI, the issues that I have with sstate-cache and/or needing to start from
scratch are really more related to the fact that sometimes I need to force the
yocto flow to go back to the earlier stages of the build/parsing process or it
will miss some changes in the parsing process. It is not related directly to
issues with meta-xilinx other than meta-xilinx’s default behavior. Also, this
build flow is eventually going into CI/CD so I often want to check to see how
it builds from scratch. I’ve seen numerous times where I get a successful
build after developing for a while but if I do a clean build from scratch (no
sstate, downloads or tmp) I get build failures. This is usually due to a
previously built or old dependency that is cached but has been removed from
the recipes. It yields ‘false successes’. I think all of what I’m trying to
achieve here is useful when you are specifically developing distro, image, or
machine features, or getting a build-flow ready for CI/CD.
Thank you all again for the help!
-David
Met vriendelijke groet / kind regards,
*Mike Looijmans*
System Expert
////
//
thumb_TOPIC logo email.gif
*TOPIC Embedded Products B.V.*
Materiaalweg 4
5681 RJ Best T: +31 (0) 499 33 69 69
The Netherlands E: [email protected]
<mailto:[email protected]>
W: www.topic.nl <http://www.topic.nl>
L: LinkedIn
<https://www.linkedin.com/company/topic-embedded-systems>
Please consider the environment before printing this e-mail
On Mon, Jan 23, 2023 at 10:55 AM Philip Balister <[email protected]> wrote:
On 1/23/23 11:34, David Babich wrote:
> Yes that works for me much of the time, however I still often need to
> delete everything to start from scratch. I won't be in this mode of
> development forever, but for now it's been very painful.
Any idea what the failure mode is? I realize you are trying to get stuff
done, but we really are interested in failure modes people find. (Of
course if you are on an older release, the issue may already be fixed)
Philip
> Thanks
>
> On Mon, Jan 23, 2023 at 8:28 AM Philip Balister <[email protected]
> <mailto:[email protected]>> wrote:
>
> Removing the tmp directory and keeping the shared state directory
might
> help recover from build issues without requiring a complete rebuild of
> qemu and other packages.
>
> Philip
>
> On 1/23/23 10:54, David Babich wrote:
> > I had a similar reply from Ulf. Here is my response:
> >
> > "Yes, I'm in a heavy development flow where I am designing the
build
> > system to support 4 images and 3 machines (2 dev-kits and custom
> > hardware). There are a lot of common features shared between all
> of the
> > configurations, but there are also numerous differences. As I'm
> > developing it often the state of my build system gets messed up
> to the
> > point where I need to completely clean and build from scratch. I
> also
> > often have to build from scratch to test that changes I made work
> from
> > start to finish on the various configurations. The qemu components
> > extend the build time significantly, and I have no need for
> them. So it
> > would help me greatly in meeting the design schedule if I can
remove
> > them from the build."
> >
> > So I basically don't want qemu /now/ because it is severely
> slowing down
> > my development cycles. We should have replied to the whole list,
> Ulf
> > has provided me the following information which I will try today:
> >
> > "Here is a discussion on how to do this in Gentoo:
> >
> > https://wiki.gentoo.org/wiki/Yocto
> <https://wiki.gentoo.org/wiki/Yocto>
> <https://wiki.gentoo.org/wiki/Yocto
> <https://wiki.gentoo.org/wiki/Yocto>>
> >
> > They use the native qemu installation provied by Gentoo, so they
> don't
> > even use the Yocto build.
> >
> > They simply add
> >
> > ASSUME_PROVIDED += "qemu-native libsdl-native"
> >
> > to local.conf and are done.
> >
> > According to this Wiki, qemu-native is used when generating the
> rootfs,
> > so *removing* qemu-native is not possible.
> >
> > =========================
> > On my Ubuntu-18.04 system, I have /usr/bin/qemu-* files, so it
> might be
> > possible in my system as well.
> >
> > =========================
> >
> > If You want to use the Yocto builts stuff:
> >
> > After you bitbaked qemu-native you will have a
> >
> > TOPDIR/build/tmp/work/x86_64-linux/qemu-native/6.2.0-r0/image/
> directory.
> >
> > Step down in this until you reach recipe-sysroot-native/
> >
> > In this directory, you find what Yocto will run.
> >
> > .
> > └── usr
> > ├── bin
> > │ ├── qemu-aarch64
> > │ ├── qemu-arm
> > │ ├── qemu-i386
> > │ ├── qemu-mips
> > │ ├── qemu-mips64
> > │ ├── qemu-mips64el
> > │ ├── qemu-mipsel
> > │ ├── qemu-mips.real
> > │ ├── qemu-ppc
> > │ ├── qemu-ppc64
> > │ ├── qemu-ppc64le
> > │ ├── qemu-riscv32
> > │ ├── qemu-riscv64
> > │ ├── qemu-sh4
> > │ └── qemu-x86_64
> > ├── include
> > │ └── qemu-plugin.h
> > └── share
> > └── qemu
> > ├── keymaps
> > │ ├── ar
> > │ ├── bepo
> > │ ├── cz
> > │ ├── da
> > │ ├── de
> > │ ├── de-ch
> > │ ├── en-gb
> > │ ├── en-us
> > │ ├── es
> > │ ├── et
> > │ ├── fi
> > │ ├── fo
> > │ ├── fr
> > │ ├── fr-be
> > │ ├── fr-ca
> > │ ├── fr-ch
> > │ ├── hr
> > │ ├── hu
> > │ ├── is
> > │ ├── it
> > │ ├── ja
> > │ ├── lt
> > │ ├── lv
> > │ ├── mk
> > │ ├── nl
> > │ ├── no
> > │ ├── pl
> > │ ├── pt
> > │ ├── pt-br
> > │ ├── ru
> > │ ├── sl
> > │ ├── sv
> > │ ├── th
> > │ └── tr
> > └── trace-events-all
> >
> > One way would be to copy it all to your host.
> > Be careful, since if I check my own Ubuntu 18.04 I have:
> >
> > ls -1 /usr/bin/qemu*
> > /usr/bin/qemu-aarch64-static
> > /usr/bin/qemu-alpha-static
> > /usr/bin/qemu-armeb-static
> > /usr/bin/qemu-arm-static
> > /usr/bin/qemu-cris-static
> > /usr/bin/qemu-hppa-static
> > /usr/bin/qemu-i386-static
> > /usr/bin/qemu-img
> > /usr/bin/qemu-io
> > /usr/bin/qemu-m68k-static
> > /usr/bin/qemu-microblazeel-static
> > /usr/bin/qemu-microblaze-static
> > /usr/bin/qemu-mips64el-static
> > /usr/bin/qemu-mips64-static
> > /usr/bin/qemu-mipsel-static
> > /usr/bin/qemu-mipsn32el-static
> > /usr/bin/qemu-mipsn32-static
> > /usr/bin/qemu-mips-static
> > /usr/bin/qemu-nbd
> > /usr/bin/qemu-nios2-static
> > /usr/bin/qemu-or1k-static
> > /usr/bin/qemu-ppc64abi32-static
> > /usr/bin/qemu-ppc64le-static
> > /usr/bin/qemu-ppc64-static
> > /usr/bin/qemu-ppc-static
> > /usr/bin/qemu-s390x-static
> > /usr/bin/qemu-sh4eb-static
> > /usr/bin/qemu-sh4-static
> > /usr/bin/qemu-sparc32plus-static
> > /usr/bin/qemu-sparc64-static
> > /usr/bin/qemu-sparc-static
> > /usr/bin/qemu-system-i386
> > /usr/bin/qemu-system-x86_64
> > /usr/bin/qemu-system-x86_64-spice
> > /usr/bin/qemu-tilegx-static
> > /usr/bin/qemu-x86_64-static
> >
> > I do not have a name collision, but that is not necessary the
> case for
> > all systems.
> > You might overwrite something.
> >
> > You might also consider manually symlinking the QEMU applications
> if a
> > simple ASSUME_PROVIDED does not work.
> >
> > I.E:
> >
> > cd /usr/bin ; sudo ln -sf qemu-aarch64-static qemu-aarch64
> >
> >
> > I have not tested this, but it seems worth a try.
> >
> >
> > Best Regards
> >
> > Ulf Samuelsson"
> >
> > I'll post status on my success or failure. Thanks
> > -David
> >
> > On Sun, Jan 22, 2023 at 10:08 AM Mark Hatle
> > <[email protected]
> <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>>
> > wrote:
> >
> > What are you trying to accomplish, why are you trying to get
> rid of
> > qemu and
> > which version of the Yocto Project are you using?
> >
> > You need some qemu support for parts of the system to function
> > properly. They
> > expect on arm/aarch64 hardware to be able to run test scripts
> and/or
> > post-install scripts as the target architecture during the root
> > filesystem
> > generation.
> >
> > The dependencies for qemu based items are ingrained into the
base
> > system
> > configuration.
> >
> > For instance, if the filesystem type of 'qemu-sd' is enabled
> (and it
> > is in most
> > configurations), then a dependency on
> qemu-xilinx-system-native will
> > be introduced.
> >
> > On 1/19/23 2:56 PM, David Babich wrote:
> > > I've been trying in vain to remove all qemu support from my
> > custom build. I've
> > > add the following to my distro.conf file:
> > >
> > > PNBLACKLIST[qemu]="blacklist"
> > > PNBLACKLIST[qemu-native]="blacklist"
> > > PNBLACKLIST[qemu-helper-native]="blacklist"
> > > PNBLACKLIST[qemu-helper-native-dev]="blacklist"
> > > PNBLACKLIST[qemu-system-native]="blacklist"
> > > #PNBLACKLIST[qemu-xilinx]="blacklist"
> > > #PNBLACKLIST[qemu-xilinx-native]="blacklist"
> > > #PNBLACKLIST[qemu-xilinx-system-native]="blacklist"
> > > PNBLACKLIST[nativesdk-qemu]="blacklist"
> > > PNBLACKLIST[nativesdk-qemu-helper]="blacklist"
> > > PNBLACKLIST[nativesdk-packagegroup-sdk-host]="blacklist"
> > > MACHINE_FEATURES_BACKFILL_CONSIDERED = " \
> > > qemu-usermode \
> > > "
> > >
> > > However when I uncomment the qemu-xilinx,
> qemu-xilinx-native and
> > > qemu-xilinx-system-native I get errors indicating
unbuildable
> > dependencies for
> > > those missing packages. I've also tried adding them to
> > PACKAGE_EXCLUDE,
> > > IMAGE_FEATURES:remove, DISTRO_FEATURES:remove, and
> > IMAGE_INSTALL:remove with no
> > > luck. There must be some way to completely disable this
> without
> > forking the
> > > meta-xilinx layers and manually removing them.
> >
> > If I could understand your use-case and reason to disable
them, I
> > might be able
> > to suggest something. But generally, no you can't just
> completely
> > disable qemu
> > without significantly changing aspects of the system as a
whole.
> >
> > At a minimum you will want to remove (and resolve dangling
> > references to):
> >
> > meta-xilinx-core/conf/machine/include/machine-xilinx-qemu.inc
> > meta-xilinx-core/classes/image-types-xilinx-qemu.bbclass
> >
> > PNBLACKLIST or SKIP_RECIPE
> >
> > If you are simply trying to ensure that qemu never goes to the
> > target image,
> > then you only need to skip qemu/qemu-xilinx.
> >
> > The qemu-*native and nativesdk-qemu* packages are helpers
during
> > compilation
> > time and do not get deployed into the target image.
> >
> > --Mark
> >
> > > Thanks in advance for any help.
> > > -David
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > --
> > *David Babich*
> > Principal Embedded Software Engineer/Founder
> > Bootseeds Engineering LLC.
> > 303-913-4863
> > [email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>
> > www.bootseeds.com <http://www.bootseeds.com>
<http://www.bootseeds.com>
> <http://www.bootseeds.com <http://www.bootseeds.com>>
> >
> >
> >
> >
>
>
>
> --
> *David Babich*
> Principal Embedded Software Engineer/Founder
> Bootseeds Engineering LLC.
> 303-913-4863
> [email protected] <mailto:[email protected]>
> www.bootseeds.com <http://www.bootseeds.com> <http://www.bootseeds.com>
>
>
>
>
--
*David Babich*
Principal Embedded Software Engineer/Founder
Bootseeds Engineering LLC.
303-913-4863
[email protected]
www.bootseeds.com <http://www.bootseeds.com>
