On Thu, 2021-09-16 at 08:45 +0100, Richard Purdie via lists.openembedded.org wrote: > On Wed, 2021-09-15 at 04:46 -0700, Pgowda wrote: > > The race issue resulted in some empty directories not being deleted. > > During staging_copyfile, it results in "file already exists error" on > > creating link for directories during do_prepare_recipe_sysroot. > > The following patch checks whether the directory and its sub directories > > have no files in it and removes them. > > > > The issue was observed on running rust-hello-world for glibc and musl > > one after the other. > > TCLIBC=glibc MACHINE=qemuarm bitbake rust-hello-world > > TCLIBC=musl MACHINE=qemuarm bitbake rust-hello-world > > > > On fixing the above issue, it was found that > > arm-poky-linux-musleabi.json files are not being generated. > > $/del/build/tmp/work/x86_64-linux/rust-cross-arm/1.54.0-r0/targets/ > > arm-poky-linux-musleabi.json': No such file or directory > > > > It is generated and builds successfully when musl lib is built standalone. > > However, when musl is built after glibc or viceversa results in the > > latter json file not being generated. > > > > Signed-off-by: Pgowda <[email protected]> > > --- > > meta/classes/staging.bbclass | 10 ++++++++++ > > 1 file changed, 10 insertions(+) > > > > diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass > > index af3397bab6..4b751af4d3 100644 > > --- a/meta/classes/staging.bbclass > > +++ b/meta/classes/staging.bbclass > > @@ -145,6 +145,16 @@ def staging_copyfile(c, target, dest, postinsts, > > seendirs): > > if os.path.islink(c): > > linkto = os.readlink(c) > > if os.path.lexists(dest): > > + # Check if directory is not deleted for race issue > > + if os.path.isdir(dest): > > + count = 0 > > + for root, dirs, files in os.walk(dest): > > + for Files in files: > > + count += 1 > > + if count == 0: > > + import shutil > > + shutil.rmtree(dest) > > + os.symlink(linkto, dest) > > if not os.path.islink(dest): > > raise OSError(errno.EEXIST, "Link %s already exists as a > > file" % dest, dest) > > if os.readlink(dest) == linkto: > > This change looks like a good fix but it can be improved a little. > > Firstly, rather than the count and os.walk, we can just use > "if not os.listdir(dest):". > > Secondly, if we know the directory is empty and is a directory, we don' need > rmtree, we can just use os.rmdir() which should be much faster. > > I'd also clarify the comment to something like: > > # Empty directories are not removed due to race issues but if we > # find a conflict, we can remove empty directories here.
Looking further at this and your description, I think you might be saying there are multiple levels of empty directories. If that is the case my simplification may not work but I am more worried about why we have multiple levels of this issue. I would also like to understand why these files overlap and are symlinks in one case but directories in another. Can you show some specific examples of the files in question? The missing json file sounds like there could be some file not being handled by sstate but it is hard to know without looking at the code. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#156103): https://lists.openembedded.org/g/openembedded-core/message/156103 Mute This Topic: https://lists.openembedded.org/mt/85625195/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
