Hi Alexander Thanks for the review and suggestion. Will add and resend v2.
Thanks, Jamin > -----Original Message----- > From: Alexander Kanavin <[email protected]> > Sent: Wednesday, May 20, 2026 5:24 PM > To: Jamin Lin <[email protected]> > Cc: [email protected]; Troy Lee > <[email protected]>; Vince Chang <[email protected]>; > [email protected] > Subject: Re: [OE-core] [PATCH v1] scripts/scriptutils: Convert nested git > repos to > standalone clones in devtool workspace > > There have been patches fixing similar issues, for which oe-core has no tests > or > ways to reproduce. Rather than ask reviewers to set up zephyr builds so they > can observe the issue, can you add a testcase for this to devtool's > oe-selftest? > > Alex > > On Wed, 20 May 2026 at 07:18, Jamin Lin via lists.openembedded.org > <[email protected]> wrote: > > > > When a recipe uses multiple git SRC_URI entries with different > > destsuffix values (e.g. Zephyr-based recipes with separate > > repositories for the kernel, modules and application), do_unpack > > clones each source tree with 'git clone -n -s'. > > > > The -s flag uses git's shared-object mechanism: > > instead of copying objects locally it writes a > > .git/objects/info/alternates file pointing back to the bare repository under > the downloads directory (DL_DIR/git2/). > > > > git_convert_standalone_clone() is called by devtool_post_unpack to > > make the workspace standalone: it runs 'git repack -a' to copy all > > objects into the local object store and then removes the alternates file. > > > > However it only processes the top-level source directory. Each nested > > git repo created by a separate SRC_URI entry retains its own > > alternates file still pointing into downloads/. > > > > Steps to reproduce: > > 1. devtool modify <recipe-with-multiple-git-SRC_URI> > > 2. bitbake -c cleanall <recipe> > > 3. bitbake <recipe> > > > > At step 2, 'bitbake -c cleanall' calls fetcher.clean() which deletes > > the bare repositories from downloads/git2/. The top-level workspace > > repo is standalone (alternates already removed by the original code), > > but the nested repos still hold alternates pointing to the now-deleted > > paths. > > > > At step 3, srctree_hash_files() runs 'git add -A .' with a custom > > GIT_INDEX_FILE. Git internally calls 'git status --porcelain=2' on > > each nested repo to check for changes; this fails with exit 128 > > because the nested alternates are broken: > > error: unable to normalize alternate object path: > > .../downloads/git2/github.com.zephyrproject-rtos.acpica//objects > > fatal: bad object HEAD > > fatal: 'git status --porcelain=2' failed in submodule > > modules/lib/acpica > > > > This halts the BitBake parse phase with a CalledProcessError and > > leaves the workspace in an unrecoverable state without manual intervention. > > > > Fix by extending git_convert_standalone_clone() to walk the source > > tree and apply the same 'git repack -a' + remove-alternates treatment > > to every nested git repository found, making the entire workspace > > fully standalone at devtool modify time. The walk uses dirs[:] = [] > > to stop at each git boundary so it never descends into already-converted > repos. > > > > Signed-off-by: Jamin Lin <[email protected]> > > --- > > scripts/lib/scriptutils.py | 28 ++++++++++++++++++++++++---- > > 1 file changed, 24 insertions(+), 4 deletions(-) > > > > diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py > > index 32e749dbb1..0a83470373 100644 > > --- a/scripts/lib/scriptutils.py > > +++ b/scripts/lib/scriptutils.py > > @@ -100,16 +100,36 @@ def load_plugins(logger, plugins, pluginpath): > > > > > > def git_convert_standalone_clone(repodir): > > - """If specified directory is a git repository, ensure it's a standalone > clone""" > > + """ > > + If specified directory is a git repository, ensure it's a standalone > > clone. > > + Also converts any nested git repositories (created by multiple SRC_URI > git > > + entries with different destsuffix values) so that none of their > > contents > > + depend on the shared downloads directory via alternates. > > + """ > > + > > import bb.process > > - if os.path.exists(os.path.join(repodir, '.git')): > > - alternatesfile = os.path.join(repodir, '.git', 'objects', 'info', > 'alternates') > > + > > + def _convert(gitdir, workdir): > > + alternatesfile = os.path.join(gitdir, 'objects', 'info', > > + 'alternates') > > if os.path.exists(alternatesfile): > > # This will have been cloned with -s, so we need to convert it > so none > > # of the contents is shared > > - bb.process.run('git repack -a', cwd=repodir) > > + bb.process.run('git repack -a', cwd=workdir) > > os.remove(alternatesfile) > > > > + if os.path.exists(os.path.join(repodir, '.git')): > > + _convert(os.path.join(repodir, '.git'), repodir) > > + > > + # Also handle nested git repos created by multiple SRC_URI git entries > > + # with different destsuffix values. Each nested repo is cloned with -s > > + # and has its own alternates pointing to the downloads directory. > > + for root, dirs, files in os.walk(repodir): > > + if root == repodir: > > + continue > > + if '.git' in dirs: > > + _convert(os.path.join(root, '.git'), root) > > + dirs[:] = [] # don't recurse into nested repos > > + > > def _get_temp_recipe_dir(d): > > # This is a little bit hacky but we need to find a place where we can > put > > # the recipe so that bitbake can find it. We're going to delete > > it at the > > -- > > 2.43.0 > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#237480): https://lists.openembedded.org/g/openembedded-core/message/237480 Mute This Topic: https://lists.openembedded.org/mt/119403462/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
