I've read over the proposal. I don't have any concerns with the change.
We need to make sure people understand the results could end up being incompatible with prior releases which means people WILL need to branch, and not just rely on adding 'yet another' compatible version to their layer.conf.
--Mark On 4/25/24 4:38 PM, Richard Purdie wrote:
As people probably realise, WORKDIR is a bit of a mess at the moment. do_unpack writes things directly into there and we don't keep track of what it writes there very easily. I know there are source tracer solutions but the whole situation is messy. I'd like to try and improve things somehow but it isn't anywhere near as easy as people think. The key hidden issues are that: * release tarballs create the subdir under WORKDIR (BPN-PV) * the git fetcher fetches into a "git" subdir by default * plain file:// referenced files end up in WORKDIR directly. This means: * git recipes set S = "${WORKDIR}/git" * tarball release recipes don't usually set S as the default works * do_compile/do_install often reference file from SRC_URI in WORKDIR Why is any of this a problem? If you have a recipe with SRC_URI = "file://myfile1 file://myfile2", run a build, then remove myfile2 from SRC_URI and rebuild, myfile2 will still be in WORKDIR. There are situations that breaks things. Put another way, the fetcher can't undo the result of do_unpack, there is no directory it can clean unless S != WORKDIR. It also results in lots of files in WORKDIR which can be very confusing to people. I think it is time we tried to start and fix this. It is however going to be a little painful. My initial proposed steps are: a) Add UNPACKDIR as a variable, defaulting to WORKDIR and switch do_unpack to unpack there instead of WORKDIR directly. b) Change recipes that use S = "${WORKDIR}" to S = "${WORKDIR}/sources" UNPACKDIR = "${S}"instead. Their do_compile/do_install tasks would need to change${WORKDIR} -> ${S} if they don't have that already. c) Make S = "${WORKDIR}" a fatal error d) Audit do_compile/do_install usage of ${WORKDIR} and change to ${UNPACKDIR}. e) Add sanity checks to make ${WORKDIR} references in compile/install fatal f) make do_unpack clean ${UNPACKDIR} before it reruns if it isn't equal to WORKDIR g) We could consider passing config into the fetcher to change the "git" default to match BPN-PV to simplify things. This in theory then sets the stage for us to change UNPACKDIR to be something like ${WORKDIR}/unpack-sources and we could put links in for "${BPN}-${PV}" and 'git' into WORKDIR. We could then always clean UNPACKDIR. Every time I think about all this, my head hurts, people complain about having breaking changes and I end up trying to ignore it a bit more. I don't think it is possible to have a completely scripted transition where the end result is actually nice looking, to get better looking layout and recipes, we're going to have to inject a variable like UNPACKDIR. Thoughts? Any better ideas? I do have some patches doing a-c for OE-Core. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1994): https://lists.openembedded.org/g/openembedded-architecture/message/1994 Mute This Topic: https://lists.openembedded.org/mt/105739556/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
