On Mon, May 30, 2022 at 3:51 AM Richard Purdie <
[email protected]> wrote:
> I've gone around in circles on this for a long time. The background
> isn't obvious at first glance.
>
> "S" represents where our source is. There is code like do_unpack which
> wants to clean S before rerunning the task. It can't just clean
> everything in WORKDIR as there is T, recipe-sysroot* and more that need
> to be left alone (e.g. recipe-sysroot-native may have
> do_fetch/do_unpack dependencies).
>
> S is usually defined as ${BPN}-${PV} via ${BP}.
Maybe disallow workdir to be used as S it’s a rare case anyway. Add a QA
check and that
Should help recipes to migrate away
>
> Many SRC_URIs are tarballs and tarballs usually extract into a
> directory, which often happens to be BPN-PV. The tarball itself
> therefore creates the ${S} directory.
>
> For git, the git fetcher extracts into git/ so git recipes end up
> setting:
>
> S = "${WORKDIR}/git"
>
> This was done historically since using ${PV} for the source dir for a
> git recipe was likely going to complicate the paths needlessly.
>
> The problem comes with recipes like keymaps which only have local files
> referenced in SRC_URI. These do:
>
> S = "${WORKDIR}"
>
> so if do_unpack cleans S, it cleans WORKDIR which is very bad. We have
> horrible code to guard against that. This is the first thing it would
> be nice to fix.
>
> We also have a second problem which is that if do_fetch/do_unpack
> rerun, a file installed by a SRC_URI in the first run but removed in
> the second, will remain in ${WORKDIR}. This means you can change a
> recipe and the rebuild might not be deterministic. Again, this is
> rather horrible.
>
> It is very hard for the code to know which files it should be removing
> when it reruns do_unpack. This is the second issue I'd ideally like to
> fix.
>
> Ideas:
>
> The first issue can be improved with code like:
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index bdb3ac33c67..0da22b34e8a 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -161,8 +161,10 @@ python base_do_fetch() {
> bb.fatal("Bitbake Fetcher Error: " + repr(e))
> }
>
> +UNPACK_DIR = "${WORKDIR}"
> +
> addtask unpack after do_fetch
> -do_unpack[dirs] = "${WORKDIR}"
> +do_unpack[dirs] = "${UNPACK_DIR}"
>
> do_unpack[cleandirs] = "${@d.getVar('S') if
> os.path.normpath(d.getVar('S')) != os.path.normpath(d.getVar('WORKDIR'))
> else os.path.join('${S}', 'patches')}"
>
> @@ -173,7 +175,7 @@ python base_do_unpack() {
>
> try:
> fetcher = bb.fetch2.Fetch(src_uri, d)
> - fetcher.unpack(d.getVar('WORKDIR'))
> + fetcher.unpack(d.getVar('UNPACK_DIR'))
> except bb.fetch2.BBFetchException as e:
> bb.fatal("Bitbake Fetcher Error: " + repr(e))
> }
>
>
> i.e. we make UNPACK_DIR configurable, then a recipe like keymaps can
> do:
>
> diff --git a/meta/recipes-bsp/keymaps/keymaps_1.0.bb
> b/meta/recipes-bsp/keymaps/keymaps_1.0.bb
> index 84d09cb965d..2927e7f1392 100644
> --- a/meta/recipes-bsp/keymaps/keymaps_1.0.bb
> +++ b/meta/recipes-bsp/keymaps/keymaps_1.0.bb
> @@ -25,14 +25,14 @@ SRC_URI = "file://keymap.sh"
> INITSCRIPT_NAME = "keymap.sh"
> INITSCRIPT_PARAMS = "start 01 S ."
>
> -S = "${WORKDIR}"
> +UNPACK_DIR = "${S}"
>
> do_install () {
> # Only install the script if 'sysvinit' is in DISTRO_FEATURES
> # THe ulitity this script provides could be achieved by
> systemd-vconsole-setup.service
> if
> ${@bb.utils.contains('DISTRO_FEATURES','sysvinit','true','false',d)}; then
> install -d ${D}${sysconfdir}/init.d/
> - install -m 0755 ${WORKDIR}/keymap.sh ${D}${sysconfdir}/init.d/
> + install -m 0755 ${S}/keymap.sh ${D}${sysconfdir}/init.d/
> fi
> }
>
>
> which I think then has the recipe make more logical sense.
>
> For the second issue, the best idea I could come up with was to make
> do_unpack extract to a subdirectory or WORKDIR, then symlink the files
> into place in WORKDIR. do_unpack can then remove any symlink it finds
> in WORKDIR when it reruns.
>
> Whether either or both of these improvements are worthwhile and whether
> there is the communuity demand and support to implement them and
> migrate metadata to match, I don't know but I wanted to put the ideas
> down here.
>
> Cheers,
>
> Richard
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1556):
https://lists.openembedded.org/g/openembedded-architecture/message/1556
Mute This Topic: https://lists.openembedded.org/mt/91428199/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
