When a recipe uses multiple git SRC_URI entries with different destsuffix
values (e.g. Zephyr-based recipes with separate repositories for the
kernel, modules and application), do_unpack clones each source tree with
'git clone -n -s'.
The -s flag uses git's shared-object mechanism:
instead of copying objects locally it writes a .git/objects/info/alternates file
pointing back to the bare repository under the downloads directory
(DL_DIR/git2/).
git_convert_standalone_clone() is called by devtool_post_unpack to make
the workspace standalone: it runs 'git repack -a' to copy all objects
into the local object store and then removes the alternates file.
However it only processes the top-level source directory. Each nested
git repo created by a separate SRC_URI entry retains its own alternates
file still pointing into downloads/.
Steps to reproduce:
1. devtool modify <recipe-with-multiple-git-SRC_URI>
2. bitbake -c cleanall <recipe>
3. bitbake <recipe>
At step 2, 'bitbake -c cleanall' calls fetcher.clean() which deletes
the bare repositories from downloads/git2/. The top-level workspace
repo is standalone (alternates already removed by the original code),
but the nested repos still hold alternates pointing to the now-deleted
paths.
At step 3, srctree_hash_files() runs 'git add -A .' with a custom
GIT_INDEX_FILE. Git internally calls 'git status --porcelain=2' on
each nested repo to check for changes; this fails with exit 128 because
the nested alternates are broken:
error: unable to normalize alternate object path:
.../downloads/git2/github.com.zephyrproject-rtos.acpica//objects
fatal: bad object HEAD
fatal: 'git status --porcelain=2' failed in submodule modules/lib/acpica
This halts the BitBake parse phase with a CalledProcessError and leaves
the workspace in an unrecoverable state without manual intervention.
Fix by extending git_convert_standalone_clone() to walk the source tree
and apply the same 'git repack -a' + remove-alternates treatment to
every nested git repository found, making the entire workspace fully
standalone at devtool modify time. The walk uses dirs[:] = [] to stop
at each git boundary so it never descends into already-converted repos.
Signed-off-by: Jamin Lin <[email protected]>
---
scripts/lib/scriptutils.py | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py
index 32e749dbb1..0a83470373 100644
--- a/scripts/lib/scriptutils.py
+++ b/scripts/lib/scriptutils.py
@@ -100,16 +100,36 @@ def load_plugins(logger, plugins, pluginpath):
def git_convert_standalone_clone(repodir):
- """If specified directory is a git repository, ensure it's a standalone
clone"""
+ """
+ If specified directory is a git repository, ensure it's a standalone clone.
+ Also converts any nested git repositories (created by multiple SRC_URI git
+ entries with different destsuffix values) so that none of their contents
+ depend on the shared downloads directory via alternates.
+ """
+
import bb.process
- if os.path.exists(os.path.join(repodir, '.git')):
- alternatesfile = os.path.join(repodir, '.git', 'objects', 'info',
'alternates')
+
+ def _convert(gitdir, workdir):
+ alternatesfile = os.path.join(gitdir, 'objects', 'info', 'alternates')
if os.path.exists(alternatesfile):
# This will have been cloned with -s, so we need to convert it so
none
# of the contents is shared
- bb.process.run('git repack -a', cwd=repodir)
+ bb.process.run('git repack -a', cwd=workdir)
os.remove(alternatesfile)
+ if os.path.exists(os.path.join(repodir, '.git')):
+ _convert(os.path.join(repodir, '.git'), repodir)
+
+ # Also handle nested git repos created by multiple SRC_URI git entries
+ # with different destsuffix values. Each nested repo is cloned with -s
+ # and has its own alternates pointing to the downloads directory.
+ for root, dirs, files in os.walk(repodir):
+ if root == repodir:
+ continue
+ if '.git' in dirs:
+ _convert(os.path.join(root, '.git'), root)
+ dirs[:] = [] # don't recurse into nested repos
+
def _get_temp_recipe_dir(d):
# This is a little bit hacky but we need to find a place where we can put
# the recipe so that bitbake can find it. We're going to delete it at the
--
2.43.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#237382):
https://lists.openembedded.org/g/openembedded-core/message/237382
Mute This Topic: https://lists.openembedded.org/mt/119403462/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
