I’ve had success with this. Thanks to all of you for providing information.
Here is the solution that worked for me (I’m using ‘honister’):
On my Ubuntu-18.04-LTS host I have the following in /usr/bin:
qemu-aarch64-static qemu-mips64el-static qemu-ppc64-static
qemu-alpha-static qemu-mips64-static qemu-ppc-static
qemu-armeb-static qemu-mipsel-static qemu-s390x-static
qemu-arm-static qemu-mipsn32el-static qemu-sh4eb-static
qemu-cris-static qemu-mipsn32-static qemu-sh4-static
qemu-hppa-static qemu-mips-static qemu-sparc32plus-static
qemu-i386-static qemu-nios2-static qemu-sparc64-static
qemu-m68k-static qemu-or1k-static qemu-sparc-static
qemu-microblazeel-static qemu-ppc64abi32-static qemu-tilegx-static
qemu-microblaze-static qemu-ppc64le-static qemu-x86_64-static
In my distro ‘local.conf’, I added the following:
ASSUME_PROVIDED += " \
qemu-native \
qemu-helper-native \
qemu-system-native \
qemu-xilinx \
qemu-xilinx-multiarch-helper-native \
qemu-xilinx-native \
qemu-xilinx-system-native \
"
INHERIT+="blacklist"
PNBLACKLIST[qemu]="blacklist"
PNBLACKLIST[qemu-xilinx]="blacklist"
PNBLACKLIST[qemu-xilinx-multiarch-helper-native]="blacklist"
PNBLACKLIST[qemu-xilinx-native]="blacklist"
PNBLACKLIST[qemu-xilinx-system-native]="blacklist"
NOTE: I had to add the ‘qemu-xilinx-*’ to the ASSUME_PROVIDED to prevent
yocto from trying to build them. Without those the build would complain
that they were dependencies and that they were skipped due to ‘blacklist’.
In my 'image-common.inc' which is required by all 4 of the image variants
that I am building, I added this line:
IMAGE_FSTYPES:remove = " wic.qemu-sd"
This prevents the meta-xilinx layer from attempting to generate a qemu-sd
image during the regular image packaging step; it still produces the
regular SD image. After that it all works nicely. I think that I can use
this same technique for some of the other slow-build dependencies such as
‘binutils-native’.
Anyway for me this is a valuable technique for a somewhat unique use-case.
FYI, the issues that I have with sstate-cache and/or needing to start from
scratch are really more related to the fact that sometimes I need to force
the yocto flow to go back to the earlier stages of the build/parsing
process or it will miss some changes in the parsing process. It is not
related directly to issues with meta-xilinx other than meta-xilinx’s
default behavior. Also, this build flow is eventually going into CI/CD so I
often want to check to see how it builds from scratch. I’ve seen numerous
times where I get a successful build after developing for a while but if I
do a clean build from scratch (no sstate, downloads or tmp) I get build
failures. This is usually due to a previously built or old dependency that
is cached but has been removed from the recipes. It yields ‘false
successes’. I think all of what I’m trying to achieve here is useful when
you are specifically developing distro, image, or machine features, or
getting a build-flow ready for CI/CD.
Thank you all again for the help!
-David
On Mon, Jan 23, 2023 at 10:55 AM Philip Balister <[email protected]>
wrote:
> On 1/23/23 11:34, David Babich wrote:
> > Yes that works for me much of the time, however I still often need to
> > delete everything to start from scratch. I won't be in this mode of
> > development forever, but for now it's been very painful.
>
> Any idea what the failure mode is? I realize you are trying to get stuff
> done, but we really are interested in failure modes people find. (Of
> course if you are on an older release, the issue may already be fixed)
>
> Philip
>
> > Thanks
> >
> > On Mon, Jan 23, 2023 at 8:28 AM Philip Balister <[email protected]
> > <mailto:[email protected]>> wrote:
> >
> > Removing the tmp directory and keeping the shared state directory
> might
> > help recover from build issues without requiring a complete rebuild
> of
> > qemu and other packages.
> >
> > Philip
> >
> > On 1/23/23 10:54, David Babich wrote:
> > > I had a similar reply from Ulf. Here is my response:
> > >
> > > "Yes, I'm in a heavy development flow where I am designing the
> build
> > > system to support 4 images and 3 machines (2 dev-kits and custom
> > > hardware). There are a lot of common features shared between all
> > of the
> > > configurations, but there are also numerous differences. As I'm
> > > developing it often the state of my build system gets messed up
> > to the
> > > point where I need to completely clean and build from scratch. I
> > also
> > > often have to build from scratch to test that changes I made work
> > from
> > > start to finish on the various configurations. The qemu
> components
> > > extend the build time significantly, and I have no need for
> > them. So it
> > > would help me greatly in meeting the design schedule if I can
> remove
> > > them from the build."
> > >
> > > So I basically don't want qemu /now/ because it is severely
> > slowing down
> > > my development cycles. We should have replied to the whole list,
> > Ulf
> > > has provided me the following information which I will try today:
> > >
> > > "Here is a discussion on how to do this in Gentoo:
> > >
> > > https://wiki.gentoo.org/wiki/Yocto
> > <https://wiki.gentoo.org/wiki/Yocto>
> > <https://wiki.gentoo.org/wiki/Yocto
> > <https://wiki.gentoo.org/wiki/Yocto>>
> > >
> > > They use the native qemu installation provied by Gentoo, so they
> > don't
> > > even use the Yocto build.
> > >
> > > They simply add
> > >
> > > ASSUME_PROVIDED += "qemu-native libsdl-native"
> > >
> > > to local.conf and are done.
> > >
> > > According to this Wiki, qemu-native is used when generating the
> > rootfs,
> > > so *removing* qemu-native is not possible.
> > >
> > > =========================
> > > On my Ubuntu-18.04 system, I have /usr/bin/qemu-* files, so it
> > might be
> > > possible in my system as well.
> > >
> > > =========================
> > >
> > > If You want to use the Yocto builts stuff:
> > >
> > > After you bitbaked qemu-native you will have a
> > >
> > > TOPDIR/build/tmp/work/x86_64-linux/qemu-native/6.2.0-r0/image/
> > directory.
> > >
> > > Step down in this until you reach recipe-sysroot-native/
> > >
> > > In this directory, you find what Yocto will run.
> > >
> > > .
> > > └── usr
> > > ├── bin
> > > │ ├── qemu-aarch64
> > > │ ├── qemu-arm
> > > │ ├── qemu-i386
> > > │ ├── qemu-mips
> > > │ ├── qemu-mips64
> > > │ ├── qemu-mips64el
> > > │ ├── qemu-mipsel
> > > │ ├── qemu-mips.real
> > > │ ├── qemu-ppc
> > > │ ├── qemu-ppc64
> > > │ ├── qemu-ppc64le
> > > │ ├── qemu-riscv32
> > > │ ├── qemu-riscv64
> > > │ ├── qemu-sh4
> > > │ └── qemu-x86_64
> > > ├── include
> > > │ └── qemu-plugin.h
> > > └── share
> > > └── qemu
> > > ├── keymaps
> > > │ ├── ar
> > > │ ├── bepo
> > > │ ├── cz
> > > │ ├── da
> > > │ ├── de
> > > │ ├── de-ch
> > > │ ├── en-gb
> > > │ ├── en-us
> > > │ ├── es
> > > │ ├── et
> > > │ ├── fi
> > > │ ├── fo
> > > │ ├── fr
> > > │ ├── fr-be
> > > │ ├── fr-ca
> > > │ ├── fr-ch
> > > │ ├── hr
> > > │ ├── hu
> > > │ ├── is
> > > │ ├── it
> > > │ ├── ja
> > > │ ├── lt
> > > │ ├── lv
> > > │ ├── mk
> > > │ ├── nl
> > > │ ├── no
> > > │ ├── pl
> > > │ ├── pt
> > > │ ├── pt-br
> > > │ ├── ru
> > > │ ├── sl
> > > │ ├── sv
> > > │ ├── th
> > > │ └── tr
> > > └── trace-events-all
> > >
> > > One way would be to copy it all to your host.
> > > Be careful, since if I check my own Ubuntu 18.04 I have:
> > >
> > > ls -1 /usr/bin/qemu*
> > > /usr/bin/qemu-aarch64-static
> > > /usr/bin/qemu-alpha-static
> > > /usr/bin/qemu-armeb-static
> > > /usr/bin/qemu-arm-static
> > > /usr/bin/qemu-cris-static
> > > /usr/bin/qemu-hppa-static
> > > /usr/bin/qemu-i386-static
> > > /usr/bin/qemu-img
> > > /usr/bin/qemu-io
> > > /usr/bin/qemu-m68k-static
> > > /usr/bin/qemu-microblazeel-static
> > > /usr/bin/qemu-microblaze-static
> > > /usr/bin/qemu-mips64el-static
> > > /usr/bin/qemu-mips64-static
> > > /usr/bin/qemu-mipsel-static
> > > /usr/bin/qemu-mipsn32el-static
> > > /usr/bin/qemu-mipsn32-static
> > > /usr/bin/qemu-mips-static
> > > /usr/bin/qemu-nbd
> > > /usr/bin/qemu-nios2-static
> > > /usr/bin/qemu-or1k-static
> > > /usr/bin/qemu-ppc64abi32-static
> > > /usr/bin/qemu-ppc64le-static
> > > /usr/bin/qemu-ppc64-static
> > > /usr/bin/qemu-ppc-static
> > > /usr/bin/qemu-s390x-static
> > > /usr/bin/qemu-sh4eb-static
> > > /usr/bin/qemu-sh4-static
> > > /usr/bin/qemu-sparc32plus-static
> > > /usr/bin/qemu-sparc64-static
> > > /usr/bin/qemu-sparc-static
> > > /usr/bin/qemu-system-i386
> > > /usr/bin/qemu-system-x86_64
> > > /usr/bin/qemu-system-x86_64-spice
> > > /usr/bin/qemu-tilegx-static
> > > /usr/bin/qemu-x86_64-static
> > >
> > > I do not have a name collision, but that is not necessary the
> > case for
> > > all systems.
> > > You might overwrite something.
> > >
> > > You might also consider manually symlinking the QEMU applications
> > if a
> > > simple ASSUME_PROVIDED does not work.
> > >
> > > I.E:
> > >
> > > cd /usr/bin ; sudo ln -sf qemu-aarch64-static qemu-aarch64
> > >
> > >
> > > I have not tested this, but it seems worth a try.
> > >
> > >
> > > Best Regards
> > >
> > > Ulf Samuelsson"
> > >
> > > I'll post status on my success or failure. Thanks
> > > -David
> > >
> > > On Sun, Jan 22, 2023 at 10:08 AM Mark Hatle
> > > <[email protected]
> > <mailto:[email protected]>
> > <mailto:[email protected]
> > <mailto:[email protected]>>>
> > > wrote:
> > >
> > > What are you trying to accomplish, why are you trying to get
> > rid of
> > > qemu and
> > > which version of the Yocto Project are you using?
> > >
> > > You need some qemu support for parts of the system to function
> > > properly. They
> > > expect on arm/aarch64 hardware to be able to run test scripts
> > and/or
> > > post-install scripts as the target architecture during the
> root
> > > filesystem
> > > generation.
> > >
> > > The dependencies for qemu based items are ingrained into the
> base
> > > system
> > > configuration.
> > >
> > > For instance, if the filesystem type of 'qemu-sd' is enabled
> > (and it
> > > is in most
> > > configurations), then a dependency on
> > qemu-xilinx-system-native will
> > > be introduced.
> > >
> > > On 1/19/23 2:56 PM, David Babich wrote:
> > > > I've been trying in vain to remove all qemu support from my
> > > custom build. I've
> > > > add the following to my distro.conf file:
> > > >
> > > > PNBLACKLIST[qemu]="blacklist"
> > > > PNBLACKLIST[qemu-native]="blacklist"
> > > > PNBLACKLIST[qemu-helper-native]="blacklist"
> > > > PNBLACKLIST[qemu-helper-native-dev]="blacklist"
> > > > PNBLACKLIST[qemu-system-native]="blacklist"
> > > > #PNBLACKLIST[qemu-xilinx]="blacklist"
> > > > #PNBLACKLIST[qemu-xilinx-native]="blacklist"
> > > > #PNBLACKLIST[qemu-xilinx-system-native]="blacklist"
> > > > PNBLACKLIST[nativesdk-qemu]="blacklist"
> > > > PNBLACKLIST[nativesdk-qemu-helper]="blacklist"
> > > > PNBLACKLIST[nativesdk-packagegroup-sdk-host]="blacklist"
> > > > MACHINE_FEATURES_BACKFILL_CONSIDERED = " \
> > > > qemu-usermode \
> > > > "
> > > >
> > > > However when I uncomment the qemu-xilinx,
> > qemu-xilinx-native and
> > > > qemu-xilinx-system-native I get errors indicating
> unbuildable
> > > dependencies for
> > > > those missing packages. I've also tried adding them to
> > > PACKAGE_EXCLUDE,
> > > > IMAGE_FEATURES:remove, DISTRO_FEATURES:remove, and
> > > IMAGE_INSTALL:remove with no
> > > > luck. There must be some way to completely disable this
> > without
> > > forking the
> > > > meta-xilinx layers and manually removing them.
> > >
> > > If I could understand your use-case and reason to disable
> them, I
> > > might be able
> > > to suggest something. But generally, no you can't just
> > completely
> > > disable qemu
> > > without significantly changing aspects of the system as a
> whole.
> > >
> > > At a minimum you will want to remove (and resolve dangling
> > > references to):
> > >
> > > meta-xilinx-core/conf/machine/include/machine-xilinx-qemu.inc
> > > meta-xilinx-core/classes/image-types-xilinx-qemu.bbclass
> > >
> > > PNBLACKLIST or SKIP_RECIPE
> > >
> > > If you are simply trying to ensure that qemu never goes to the
> > > target image,
> > > then you only need to skip qemu/qemu-xilinx.
> > >
> > > The qemu-*native and nativesdk-qemu* packages are helpers
> during
> > > compilation
> > > time and do not get deployed into the target image.
> > >
> > > --Mark
> > >
> > > > Thanks in advance for any help.
> > > > -David
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > --
> > > *David Babich*
> > > Principal Embedded Software Engineer/Founder
> > > Bootseeds Engineering LLC.
> > > 303-913-4863
> > > [email protected] <mailto:[email protected]>
> > <mailto:[email protected] <mailto:[email protected]>>
> > > www.bootseeds.com <http://www.bootseeds.com>
> > <http://www.bootseeds.com <http://www.bootseeds.com>>
> > >
> > >
> > >
> > >
> >
> >
> >
> > --
> > *David Babich*
> > Principal Embedded Software Engineer/Founder
> > Bootseeds Engineering LLC.
> > 303-913-4863
> > [email protected] <mailto:[email protected]>
> > www.bootseeds.com <http://www.bootseeds.com>
> >
> >
> >
> >
>
--
*David Babich*
Principal Embedded Software Engineer/Founder
Bootseeds Engineering LLC.
303-913-4863
[email protected]
www.bootseeds.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5139):
https://lists.yoctoproject.org/g/meta-xilinx/message/5139
Mute This Topic: https://lists.yoctoproject.org/mt/96386630/21656
Group Owner: [email protected]
Unsubscribe: https://lists.yoctoproject.org/g/meta-xilinx/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
