On Mon, 2023-10-30 at 14:50 +0100, Alexander Kanavin wrote: > On Thu, 14 Sept 2023 at 14:56, Richard Purdie > <richard.pur...@linuxfoundation.org> wrote: > > There are design elements to this work. We need to work out how we can > > make eSDK and "normal" builds more similar and less of an overhead to > > switch between one and the other. A "bblock all" command does partly > > get you to an eSDK effectively so part of this may be switching eSDK to > > use the new lock command. What other differences are there? What other > > differences are necessary or make sense for the use cases eSDK was > > designed for? How would you turn an existing build into an eSDK like > > one? Could you provide a copy of a local build to someone else easily > > using something like eSDK's tooling? What does the eSDK look like at > > the end of this. One section we don't have good answers to yet is setup > > and configuration although I know you've started on some of that. > > So I see the following differences between esdk and normal modes: > > 1. Environment and tooling availability. > > a) esdk sets a number of variables from its initialization script that > aid with cross-compiling components directly (e.g. the core use case > of SDKs). Normal mode doesn't do that, but recently added > meta-ide-support will generate a similar initialization script that > will set up the same environment from the normal mode. There are tests > and documentation for it.
In that case, this one is something we can document as how to make the functionality available in the normal build. > b) PATH. eSDK has a number of items in PATH that point to various > locations inside tmp/sysroots/, collectively they provide the > cross-toolchain. > > eSDK also puts a selection of yocto tools into path - wic, devtool but > not bitbake: > > ============================ > alex@Zen2:~/poky_sdk$ ls -l sysroots/x86_64-pokysdk-linux/usr/bin/ > total 48 > lrwxrwxrwx 1 alex alex 39 Oct 30 12:52 devtool -> > ../../../../layers/poky/scripts/devtool > lrwxrwxrwx 1 alex alex 54 Oct 30 12:52 oe-find-native-sysroot -> > ../../../../layers/poky/scripts/oe-find-native-sysroot > lrwxrwxrwx 1 alex alex 42 Oct 30 12:52 recipetool -> > ../../../../layers/poky/scripts/recipetool > lrwxrwxrwx 1 alex alex 39 Oct 30 12:52 runqemu -> > ../../../../layers/poky/scripts/runqemu > lrwxrwxrwx 1 alex alex 55 Oct 30 12:52 runqemu-addptable2image -> > ../../../../layers/poky/scripts/runqemu-addptable2image > lrwxrwxrwx 1 alex alex 53 Oct 30 12:52 runqemu-export-rootfs -> > ../../../../layers/poky/scripts/runqemu-export-rootfs > lrwxrwxrwx 1 alex alex 51 Oct 30 12:52 runqemu-extract-sdk -> > ../../../../layers/poky/scripts/runqemu-extract-sdk > lrwxrwxrwx 1 alex alex 51 Oct 30 12:52 runqemu-gen-tapdevs -> > ../../../../layers/poky/scripts/runqemu-gen-tapdevs > lrwxrwxrwx 1 alex alex 46 Oct 30 12:52 runqemu-ifdown -> > ../../../../layers/poky/scripts/runqemu-ifdown > lrwxrwxrwx 1 alex alex 44 Oct 30 12:52 runqemu-ifup -> > ../../../../layers/poky/scripts/runqemu-ifup > lrwxrwxrwx 1 alex alex 100 Oct 30 12:52 unfsd -> > ../../../../tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/recipe-sysroot-native/usr/bin/unfsd > lrwxrwxrwx 1 alex alex 35 Oct 30 12:52 wic -> > ../../../../layers/poky/scripts/wic > ============================== > > 'normal mode' puts bitbake/bin/ and oe-core/scripts in PATH. > Cross-toolchain can be added by the same environment script made by > meta-ide-support as mentioned in 1a. Right, so in theory we can change PATH and change this which can also easily be documented. > 2. Configuration (e.g. local.conf). > > eSDK local.conf is local.conf from the normal mode that was used to > produce eSDK, stripped of all comments, and with a bunch of extra > settings: > > ============================ > INHERIT:remove = "buildhistory icecc" > CONNECTIVITY_CHECK_URIS = "" > > SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK = "none" > > SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "warn" > > BB_HASHCONFIG_IGNORE_VARS:append = " SIGGEN_UNLOCKED_RECIPES" > > BB_SETSCENE_ENFORCE_IGNORE_TASKS = "%:* *:do_shared_workdir > *:do_rm_work wic-tools:* *:do_addto_recipe_sysroot" > > BUILDCFG_HEADER = "" > > METADATA_REVISION:poky = "4a1e0b9625729e422fcf24e632ee2a3c79f986d5" > > # Provide a flag to indicate we are in the EXT_SDK Context > WITHIN_EXT_SDK = "1" > > SSTATE_MIRRORS += " file://universal/(.*) file://universal-4.9/\1 > file://universal-4.9/(.*) file://universal-4.8/\1" > Perhaps some of this should become a generic include file which is included which would then make it easy to document as how to lock things down to match it? That would take the config out of the class file too which is probably nicer. > require conf/locked-sigs.inc > require conf/unlocked-sigs.inc > =============================== > > Devtool also has a special configuration: > ================= > alex@Zen2:~/poky_sdk$ cat conf/devtool.conf > [General] > bitbake_subdir = layers/poky/bitbake > init_path = layers/poky/oe-init-build-env > core_meta_subdir = layers/poky/meta These are likely there to allow devtool to work with the modified files layout. We should probably document it but I'm going to propose we "ignore" layout for the purposes of this so we can just ignore things which are for layout only. > [SDK] > sdk_targets = core-image-minimal Not quite sure about this one. Maybe something to do with the target used to generate the eSDK? Not sure what uses it. > ================== > There is currently no tooling to add/remove these extras in either > esdk mode or normal mode as far as I understand. Their individual > purposes and effects are also not exactly clear to me, and need to be > investigated one by one. > > 3. Setting up a normal mode in a eSDK installation. > > This is actually pretty easy: rather than sourcing the sdk environment > script, source the poky/oe-init-build-env: > ============== > alex@Zen2:~/poky_sdk$ . layers/poky/oe-init-build-env . > > ### Shell environment set up for builds. ### > ... > =============== > Bitbake will then simply run, albeit with all those extra > esdk-specific items in local.conf (which can be easily taken out to > achieve a pristine normal mode). There will be some warnings but > builds will proceed: > =============== > WARNING: You are using a local hash equivalence server but have > configured an sstate mirror. This will likely mean no sstate will > match from the mirror. You may wish to disable the hash equivalence > use (BB_HASHSERVE), or use a hash equivalence server alongside the > sstate mirror. > ... > WARNING: spirv-headers-native-1_1.3.261.1-r0 do_fetch: Failed to fetch > URL git://github.com/KhronosGroup/SPIRV-Headers;protocol=https;branch=main, > attempting MIRRORS if available > WARNING: libepoxy-1.5.10-r0 do_fetch: Failed to fetch URL > git://github.com/anholt/libepoxy;branch=master;protocol=https, > attempting MIRRORS if available > WARNING: spirv-tools-native-1_1.3.261.1-r0 do_fetch: Failed to fetch > URL git://github.com/KhronosGroup/SPIRV-Tools.git;branch=main;protocol=https, > attempting MIRRORS if available > WARNING: shaderc-native-2023.6-r0 do_fetch: Failed to fetch URL > git://github.com/google/shaderc.git;protocol=https;branch=main, > attempting MIRRORS if available > =============== > 4. Setting up esdk mode in a normal yocto installation. > > This is on one hand easy: cross-toolchain and unified sysroots are > made available for direct component builds by meta-ide-support and > build-sysroots. > > On the other hand, setting up the exact same environment as eSDK sets > up isn't currently possible: tooling to replicate all the local.conf > tweaks (that includes locked-sigs items) would be necessary, at a > minimum. A more pedantic approach would also ensure that > - only the 'allowed' set of tools is in PATH, and not everything from > bitbake/bin/ and scripts/ > - environment variables and toolchain sysroots match between what eSDK > sets up, and what meta-ide-support sets up > > 5. In conclusion: to be honest, right now I'm not sure which direction > should be taken to unify and simplify things. Hopefully the above > sparks some ideas; they might pop into my head later as well :-) The biggest 'gap' appears to be the config to go along with the locked sigs. If we move that to an include, that probably gets us quite close to them being similar ignoring layout changes for the files and the differences to the environment? Thinking out loud, I guess we could have a "filtered" scripts directory too as part of the normal build, then the eSDK switches PATH from one to the other? Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61520): https://lists.yoctoproject.org/g/yocto/message/61520 Mute This Topic: https://lists.yoctoproject.org/mt/101356418/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
