As people have probably gathered we have some concerns about rare file
mode corruption with pseudo. This is really really rare but its not
right and we need to do something about it.
As discussed on list recently, we need to make pseudo abort() when it
encounters such an issue rather than hacking around it.
Patches to make pseudo abort in such a case are in master-next, along
with a set of other changes. These changes make pseudo ignore a set of
directories from a permissions/ownership perspective.
The current setting is (line wrapped to make it more readable):
+PSEUDO_IGNORE_PATHS = " \
/usr/,
/etc/,
/lib,
/dev/,
${T},
${WORKDIR}/recipe-sysroot,
${SSTATE_DIR},
${STAMPS_DIR},
${WORKDIR}/pkgdata-sysroot,
${TMPDIR}/sstate-control,
${DEPLOY_DIR},
${WORKDIR}/deploy-,
${TMPDIR}/buildstats,
${WORKDIR}/sstate-build-package_,
${WORKDIR}/sstate-install-package_,
${TMPDIR}/sysroots-components"
which seems to give us a good start. In particular, this makes
packaging during do_package_write_* mostly happen under pseudo but the
final files are not written out in pseudo active paths. This is
deliberate and necessary but also slightly complicates things.
Also note there is conditional code to add ${S} and ${B} to this list
if they're not ${WORKDIR}. Path matching happens by the start of the
path matching one of these fragments.
If you run with these patches you'll probably end up with one of a
number of issues:
ERROR: Task
(/home/pokybuild/yocto-worker/non-gpl3/build/meta/recipes-core/glibc/glibc_2.32.bb:do_package)
failed with exit code '134'
means the task aborted. To debug, take a look at the pseudo log file:
pokybuild@tumbleweed-ty-3:~/yocto-worker/non-gpl3/build/build-renamed/tmp/work/core2-32-poky-linux/glibc/2.32-r0/pseudo>
cat pseudo.log
debug_logfile: fd 2
pid 1997 [parent 1996], doing new pid setup and server start
Setup complete, sending SIGUSR1 to pid 1996.
path mismatch [2 links]: ino 236066483 db
'/home/pokybuild/yocto-worker/non-gpl3/build/build/tmp/work/core2-32-poky-linux/glibc/2.32-r0/image/usr/share/locale/cs'
req
'/home/pokybuild/yocto-worker/non-gpl3/build/build/tmp/work/core2-32-poky-linux/glibc/2.32-r0/sstate-build-package/package'.
db cleanup for server shutdown, 14:15:45.384
memory-to-file backup complete, 14:15:45.385.
db cleanup finished, 14:15:45.385
The abort is from the path mismatch.
It means that a lookup was seen for:
/home/pokybuild/yocto-worker/non-gpl3/build/build/tmp/work/core2-32-poky-linux/glibc/2.32-r0/sstate-build-package/package
but it wasn't in the database. There was a matching inode entry in the database
of:
/home/pokybuild/yocto-worker/non-gpl3/build/build/tmp/work/core2-32-poky-linux/glibc/2.32-r0/image/usr/share/locale/cs
This is puzzling as files in WIORKDIR/image/ would come from
do_install, so something has removed a file added during do_install but
has done so outside of pseudo's "sight" (a task without fakeroot
context?).
At this point the work is in figuring out what happened to that file
and ensuring pseudo does see what happens to anything in WORKDIR/image.
You may also see user/group number lookup issues, usually where its
tried to resolve the build user ID. Usually something like:
Exception: KeyError: 'getpwuid(): uid not found: 1000'
If this is from the sstatesig code, the file being accessed should be
listed before the traceback. Its then a question of debugging why that
file isn't being seen by pseudo (and whether it should be or not).
We need to start to look into the failing cases and figure out what the
patterns are, why they're not being seen by pseudo and work through
them.
For one case for locale-archive in do_rootfs, I had to add new
functionality to pseudo.
In that case it was an access pattern of:
fd = fopen("A")
link("A", "B")
unlink("A")
fchown(fd)
pseudo maintains a list of paths for the open fds and this breaks with
the above scenario.
I've put the patches in master-next on the autobuilder and will start
working through the issues it highlights but help would be appreciated.
Its hard to tell how big this problem even is right now.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142937):
https://lists.openembedded.org/g/openembedded-core/message/142937
Mute This Topic: https://lists.openembedded.org/mt/77218866/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
