On Thu, 2026-06-18 at 09:19 +0200, Anders Heimer wrote: > Replace the sort/grep/sed parts of copydebugsources with Python > filtering on the NUL-separated source list. Keep cpio since it is > faster than doing it in python. > > Use an explicit prefix + "/" match before stripping the prefix so source > selection is limited to files in the mapped debug source directory. > > Replace the find/sed symlink fixup pipeline with os.walk() plus cpio, > use an argv-list mv for the externalsrc relocation, and pass the > empty-directory find command as an argv list. > > The first cpio copy pass continues to ignore failures as before since > some inputs are expected to fail. The symlink fixup copy still reports > cpio failures. > > Signed-off-by: Anders Heimer <[email protected]> > --- > meta/lib/oe/package.py | 70 +++++++++++++++++++++++++++++------------- > 1 file changed, 48 insertions(+), 22 deletions(-) > > diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py > index c375acc124..c4ad364b64 100644 > --- a/meta/lib/oe/package.py > +++ b/meta/lib/oe/package.py > @@ -1017,26 +1017,48 @@ def copydebugsources(debugsrcdir, sources, d): > bb.utils.mkdirhier(basepath) > cpath.updatecache(basepath) > > - for pmap in prefixmap: > - # Ignore files from the recipe sysroots (target and native) > - cmd = "LC_ALL=C ; sort -z -u '%s' | egrep -v -z > '((<internal>|<built-in>)$|/.*recipe-sysroot.*/)' | " % sourcefile > - # We need to ignore files that are not actually ours > - # we do this by only paying attention to items from this package > - cmd += "fgrep -zw '%s' | " % prefixmap[pmap] > - # Remove prefix in the source paths > - cmd += "sed 's#%s/##g' | " % (prefixmap[pmap]) > - cmd += "(cd '%s' ; cpio -pd0mlLu --no-preserve-owner '%s%s' > 2>/dev/null)" % (pmap, dvar, prefixmap[pmap]) > + # Ignore files from the recipe sysroots (target and native), and > + # compiler internal entries. > + with open(sourcefile, "rb") as f: > + sourcepaths = sorted({path for path in f.read().split(b"\0") > + if path > + and not path.endswith((b"<internal>", > b"<built-in>")) > + and b"recipe-sysroot" not in > os.path.dirname(path)}) > + > + for pmap, prefix in prefixmap.items(): > + dstroot = dvar + prefix > + prefix_slash = os.fsencode(prefix) + b"/" > + relpaths = [path[len(prefix_slash):] > + for path in sourcepaths > + if path.startswith(prefix_slash)] > + > + if relpaths: > + subprocess.run(["cpio", "-pd0mlLu", "--no-preserve-owner", > dstroot], > + input=b"\0".join(relpaths) + b"\0", > + cwd=pmap, stdout=subprocess.DEVNULL, > + stderr=subprocess.DEVNULL, check=False)
I feel like there are a couple of cases of doing a bit too much at once
here. It took me a few looks at it to realise that a set comprehension
is used to populate sourcepaths, so it's removing duplicate entries.
I would find the following more readable:
with open(sourcefile, "rb") as f:
rawpaths = f.read().split(b"\0")
# Ignore files from the recipe sysroots (target and native), and
# compiler internal entries. Use a set comprehension to prevent
# duplicate entries.
sourcepaths = {path for path in rawpaths
if path
and not path.endswith((b"<internal>", b"<built-in>"))
and b"recipe-sysroot" not in os.path.dirname(path)}
for pmap, prefix in prefixmap.items():
dstroot = dvar + prefix
prefix_slash = os.fsencode(prefix) + b"/"
relpaths = [path.removeprefix(prefix_slash) for path in sourcepaths
if path.startswith(prefix_slash)]
if relpaths:
subprocess.run(["cpio", "-pd0mlLu", "--no-preserve-owner", dstroot],
input=b"\0".join(sorted(relpaths)) + b"\0",
cwd=pmap, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, check=False)
Our minimum Python version is 3.9 so we can use removeprefix(). And
placing the sorted() call in the cpio invocation matches what we do
again later in the function.
I assume that path.removeprefix(prefix_slash) can never result in a
zero-length string because we'll never see an entry in sourcefile
exactly matching the prefix with the trailing '/'.
The rest of the patch LGTM!
Best regards,
--
Paul Barker
signature.asc
Description: This is a digitally signed message part
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#239084): https://lists.openembedded.org/g/openembedded-core/message/239084 Mute This Topic: https://lists.openembedded.org/mt/119863717/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
