On Tue, 2024-01-16 at 09:23 -0500, Robert P. J. Day wrote:
> a very weird question, to be sure, but i literally just ran across a
> recipe that does the following (i will paraphrase some stuff to
> protect the innocent).
>
> the recipe processes a bunch of XML source files, and generates from
> them JSON files, so any change in any of the XML source files should
> (obviously) trigger a rebuild of the recipe. and here's the problem.
>
> the XML files are not copied into the recipe source directory, and
> are not even referenced by SRC_URI. rather, the files live in a
> well-known location on the build host (placed there by a monstrous
> "repo sync"), and to save time and space, the recipe just *symlinks*
> from inside ${S} to the XML source files outside of ${S}, then
> proceeds to process them to generate the subsequent JSON files.
>
> the complaint is that, when the build pipeline started taking
> advantage of sstate-cache, even after some of the XML files were
> changed, the recipe would not rebuild.
>
> once i understood what was happening, my immediate reaction was
> along the lines, "well, since what is under ${S} is simply symlinks to
> files *outside* of ${S} and (worse) those symlink names will never
> change (all that will change is the XML files at the other end), how
> is bitbake supposed to have any idea that those XML files have
> changed to kick off a rebuild?"
>
> so my initial response was, "this does not surprise me at all," but
> does my reasoning hold up? obviously, this should probably use a
> combination of bin_package and externalsrc, but as it is now, is it
> moderately accurate to suggest that it is those symlinks that are
> preventing bitbake from understanding that anything has changed, so
> that it continues to use the old sstate-cache?
Inputs to a recipe need to come through SRC_URI. That way bitbake can
know when they change. If you point at some random place via a symlink
how is bitbake supposed to know if/when they change?
Yes, pointing at a large tree of files will take time to hash and yes,
that can make parsing slow. Either you care when it changes or you
don't.
The inputs are used to generate a hash which makes up the hash
representing the task. When the hash changes, we know that sstate is
invalid and we need to rebuild.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193875):
https://lists.openembedded.org/g/openembedded-core/message/193875
Mute This Topic: https://lists.openembedded.org/mt/103762271/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
