Le ven. 6 oct. 2023 à 12:38, Richard Purdie
<richard.pur...@linuxfoundation.org> a écrit :
>
> On Fri, 2023-10-06 at 12:34 +0200, Julien Stephan wrote:
> > Le sam. 30 sept. 2023 à 12:01, Julien Stephan via
> > lists.openembedded.org <jstephan=baylibre....@lists.openembedded.org>
> > a écrit :
> > >
> > > Le mer. 27 sept. 2023 à 22:36, Richard Purdie
> > > <richard.pur...@linuxfoundation.org> a écrit :
> > > >
> > > > On Wed, 2023-09-27 at 18:50 +0200, Julien Stephan wrote:
> > > > > [YOCTO #15162]
> > > > >
> > > > > When recipes such as bzip2 have several sources in SRC_URI, all the
> > > > > sources are correctly extracted into devtool temporary workdir, but
> > > > > currently, we only get back the primary source and all files declared
> > > > > using "files://", then the devtool temporary workdir is deleted
> > > > >
> > > > > Fix this by moving all remaining files from devtool temporary workdir
> > > > > into actual WORKDIR (except "source-date-epoch", "recipe-sysroot",
> > > > > and "recipe-sysroot-native") before deleting it
> > > > >
> > > > > Here is the stack trace from devtool modify/build bzip2:
> > > > >
> > > > > NOTE: bzip2: compiling from external source tree
> > > > > <...>/build/workspace/sources/bzip2
> > > > > ERROR: bzip2-1.0.8-r0 do_install_ptest_base:
> > > > > ExecutionError('<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368',
> > > > > 1, None, None)
> > > > > ERROR: Logfile of failure stored in:
> > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/log.do_install_ptest_base.3368
> > > > > Log data follows:
> > > > > > DEBUG: Executing shell function do_install_ptest_base
> > > > > > NOTE: make -j 16
> > > > > > DESTDIR=<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest
> > > > > > install-ptest
> > > > > > sed -n '/^runtest:/,/^install-ptest:/{/^install-ptest:/!p}' \
> > > > > > ../../../../../../workspace/sources/bzip2/Makefile.am
> > > > > > >
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/Makefile
> > > > > > cp ../../../../../../workspace/sources/bzip2/sample1.ref
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
> > > > > > cp ../../../../../../workspace/sources/bzip2/sample2.ref
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
> > > > > > cp ../../../../../../workspace/sources/bzip2/sample3.ref
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
> > > > > > cp ../../../../../../workspace/sources/bzip2/sample1.bz2
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
> > > > > > cp ../../../../../../workspace/sources/bzip2/sample2.bz2
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
> > > > > > cp ../../../../../../workspace/sources/bzip2/sample3.bz2
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
> > > > > > ln -s /usr/bin/bzip2
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2
> > > > > > cp: cannot stat
> > > > > > '<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/git/commons-compress':
> > > > > > No such file or directory
> > > > > > WARNING:
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368:189
> > > > > > exit 1 from 'cp -r <...>/build/tmp/work/core2-64-poky-linux/bzip2/
> > > > > 1.0.8/git/commons-compress
> > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2-tests/commons-compress'
> > > > > > WARNING: Backtrace (BB generated script):
> > > > > > #1: do_install_ptest,
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368,
> > > > > > line 189
> > > > > > #2: do_install_ptest_base,
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368,
> > > > > > line 158
> > > > > > #3: main,
> > > > > > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368,
> > > > > > line 226
> > > > > ERROR: Task
> > > > > (<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base)
> > > > > failed with exit code '1'
> > > > > NOTE: Tasks Summary: Attempted 776 tasks of which 765 didn't need to
> > > > > be rerun and 1 failed.
> > > > >
> > > > > Summary: 1 task failed:
> > > > >
> > > > > <...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base
> > > > >
> > > > > Signed-off-by: Julien Stephan <jstep...@baylibre.com>
> > > > > ---
> > > > > scripts/lib/devtool/standard.py | 8 ++++++++
> > > > > 1 file changed, 8 insertions(+)
> > > > >
> > > > > diff --git a/scripts/lib/devtool/standard.py
> > > > > b/scripts/lib/devtool/standard.py
> > > > > index d53fb810071..6c8ad7a6c8c 100644
> > > > > --- a/scripts/lib/devtool/standard.py
> > > > > +++ b/scripts/lib/devtool/standard.py
> > > > > @@ -684,6 +684,14 @@ def _extract_source(srctree, keep_temp,
> > > > > devbranch, sync, config, basepath, works
> > > > > shutil.move(srcsubdir, srctree)
> > > > > symlink_oelocal_files_srctree(d,srctree)
> > > > >
> > > > > + # In case SRC_URI contains multiple sources
> > > > > + # they are extracted into the devtool temporary workdir
> > > > > + # We need to move them into WORKDIR, otherwise they will
> > > > > be lost
> > > > > + for path in os.listdir(os.path.join(tempdir, 'workdir')):
> > > > > + if not path in ["source-date-epoch",
> > > > > "recipe-sysroot", "recipe-sysroot-native"]:
> > > > > + workdir = d.getVar('WORKDIR')
> > > > > + shutil.move(os.path.join(tempdir, 'workdir',
> > > > > path), workdir)
> > > > > +
> > > > > if is_kernel_yocto:
> > > > > logger.info('Copying kernel config to srctree')
> > > > > shutil.copy2(os.path.join(tempdir, '.config'), srctree)
> > > >
> > > > Looks like a good start although I worry a bit about the hardcoded list
> > > > of exceptions.
> > >
> > >
> > > Hi Richard,
> > > Yes, I am not very happy with listing files but I don't know if I can
> > > easily get the list of secondary sources?
> > >
> > > Here are the alternatives I can see:
> > > * encapsulate the move into a try/catch block and ignore exceptions if
> > > the destination file/directory already exists
> > > * retrieve SRC_URI and manually build the list of secondary sources to
> > > move. This looks a little bit overkill here...
> > > What do you think?
> >
> > I need to rework this patch. As is it doesn't work: if the recipe was
> > already built then the sources I am trying to move during devtool
> > modify from the devtool temporary workdir to WORKDIR already exist in
> > WORKDIR.
> > This makes several tests from devtool selftest to fail.
> >
> > Any suggestions on how I should handle this?
>
> There is some work on the bitbake list from Alberto which adds unpack
> tracing. I'm wondering if by default we should track some really basic
> form of unpack accesses which would then give you a list of target
> locations to merge?
>
Looks very interesting, let me check this :)
Cheers
Julien
> Cheers,
>
> Richard
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188765):
https://lists.openembedded.org/g/openembedded-core/message/188765
Mute This Topic: https://lists.openembedded.org/mt/101621289/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
