On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote:
> The archiver doesn't support using multiconfig sharing the same TMPDIR.
> When in the multiconfig and using the same TMPDIR: the two machines can make
> the same copy at the same time to the same destination, which will not work.
>
> This issiue can be replicated with meta-ti layer:
> > # use a multiconfig machine
> > MACHINE = "am64xx-evm"
> > # archive sources for target recipes in conf/local.conf
> > INHERIT += "archiver"
> > COPYLEFT_RECIPE_TYPES = "target"
> > ARCHIVER_MODE[src] = "original"
> > ARCHIVER_MODE[diff] = "1"
> > # and run bitbake with:
> > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch
> > -f
>
> Fixes the following:
> > ERROR: mc:k3r5:gcc-source-11.2.0-11.2.0-r0 do_unpack_and_patch: Error
> > executing a python function in exec_func_python() autogenerated:
> >
> > The stack trace of python calls that resulted in this exception/failure was:
> > File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
> > 0001:
> > *** 0002:do_unpack_and_patch(d)
> > 0003:
> > File:
> > '/build/conf/../../layers/openembedded-core/meta/classes/archiver.bbclass',
> > lineno: 489, function: do_unpack_and_patch
> > 0485: # Save the original source for creating the patches
> > 0486: if d.getVarFlag('ARCHIVER_MODE', 'diff') == '1':
> > 0487: src = d.getVar('S').rstrip('/')
> > 0488: src_orig = '%s.orig' % src
> > *** 0489: oe.path.copytree(src, src_orig)
> > 0490:
> > 0491: if bb.data.inherits_class('dos2unix', d):
> > 0492: bb.build.exec_func('do_convert_crlf_to_lf', d)
> > 0493:
> > File: '/build/conf/../../layers/openembedded-core/meta/lib/oe/path.py',
> > lineno: 94, function: copytree
> > 0090: # This way we also preserve hardlinks between files in the
> > tree.
> > 0091:
> > 0092: bb.utils.mkdirhier(dst)
> > 0093: cmd = "tar --xattrs --xattrs-include='*' -cf - -S -C %s -p . |
> > tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dst)
> > *** 0094: subprocess.check_output(cmd, shell=True,
> > stderr=subprocess.STDOUT)
> > 0095:
> > 0096:def copyhardlinktree(src, dst):
> > 0097: """Make a tree of hard links when possible, otherwise copy."""
> > 0098: bb.utils.mkdirhier(dst)
> > File: '/usr/lib/python3.10/subprocess.py', lineno: 420, function:
> > check_output
> > 0416: else:
> > 0417: empty = b''
> > 0418: kwargs['input'] = empty
> > 0419:
> > *** 0420: return run(*popenargs, stdout=PIPE, timeout=timeout,
> > check=True,
> > 0421: **kwargs).stdout
> > 0422:
> > 0423:
> > 0424:class CompletedProcess(object):
> > File: '/usr/lib/python3.10/subprocess.py', lineno: 524, function: run
> > 0520: # We don't call process.wait() as .__exit__ does that
> > for us.
> > 0521: raise
> > 0522: retcode = process.poll()
> > 0523: if check and retcode:
> > *** 0524: raise CalledProcessError(retcode, process.args,
> > 0525: output=stdout, stderr=stderr)
> > 0526: return CompletedProcess(process.args, retcode, stdout, stderr)
> > 0527:
> > 0528:
> > Exception: subprocess.CalledProcessError: Command 'tar --xattrs
> > --xattrs-include='*' -cf - -S -C
> > /build/tmp-lmp/work-shared/gcc-11.2.0-r0/gcc-11.2.0 -p . | tar --xattrs
> > --xattrs-include='*' |-xf - -C
> > /build/tmp-lmp/work-shared/gcc-11.2.0-r0/gcc-11.2.0.orig' returned non-zero
> > exit status 2.
> >
> > Subprocess output:
> > tar: ./gcc/config/moxie/predicates.md: file changed as we read it
> > tar: ./patches: Cannot stat: No such file or directory
> > tar: Exiting with failure status due to previous errors
> >
> > ERROR: Logfile of failure stored in:
> > /build/tmp-lmp/work-shared/gcc-11.2.0-r0/temp/log.do_unpack_and_patch.291906
> > ERROR: Task
> > (mc:k3r5:/build/conf/../../layers/openembedded-core/meta/recipes-devtools/gcc/gcc-source_11.2.bb:do_unpack_and_patch)
> > failed with exit code '1'
>
> Signed-off-by: Jose Quaresma <[email protected]>
> ---
>
> v2: replace "bb.error" with "bb.fatal"
> v3: instead of failing make it more resilient
>
> meta/classes/archiver.bbclass | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
> index c19c770d11..b1d7aa0247 100644
> --- a/meta/classes/archiver.bbclass
> +++ b/meta/classes/archiver.bbclass
> @@ -485,7 +485,8 @@ python do_unpack_and_patch() {
> # Save the original source for creating the patches
> if d.getVarFlag('ARCHIVER_MODE', 'diff') == '1':
> src = d.getVar('S').rstrip('/')
> - src_orig = '%s.orig' % src
> + # append machine name to avoid races in multiconfig when using the
> same TMPDIR
> + src_orig = '%s.orig.%s' % (src, d.getVar('MACHINE'))
> oe.path.copytree(src, src_orig)
>
> if bb.data.inherits_class('dos2unix', d):
I'm not convinced this is correct. I did have a quick look at that
multiconfig (a pointer to the config helps a lot!) and it looks correct
in that it is just two different MACHINE settings being built within
the same TMPDIR. That should mean there is only one gcc-source and that
gcc-source should only run once. Obviously that isn't true so something
is broken. This patch just works around it rather than identifying what
is broken and fixing that. We need to find the real problem.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166794):
https://lists.openembedded.org/g/openembedded-core/message/166794
Mute This Topic: https://lists.openembedded.org/mt/91650171/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
