Hi,
While investigating this undocumented "fullclone" feature, I realized
that what the git fetcher attempts to do is not obvious at all, and
that most probably things as they are could be done in a much more
efficient way, and things could be done in a much more useful way.
Frow what I gather:
* advertised parameter "tag" does not seem to be used, but there is an
undocumented "branch" one which seems to replace it ?
* another parameter, "subpath", is not documented either - I have not
yet tried to understand what it does, the whole thing would be
easier to grasp with a couple of comments...
* in default (non-fullclone) mode:
- a full "git clone --no-checkout" is done (or a cached version of
that repo clone is unpacked)
- some magic is done under the "Remove all but the .git directory",
(maybe to fetch the requested "tag" if it changed since cached
clone ?), and things are removed first (for backward compat
from before --no-checkout was used ? anyway "rm *" will miss
dotfiles, "git clean -fdx" would be more efficient and safer)
Anyway, what is intended here is (from the use done from it) is a
bare repo; so "clone --bare" would be even better for new clones.
Migrating to bare repos is not that trivial, but can still be done
with not so much work.
- source is checked out with plumbing commands and avoids getting a
.git dir. Is that to save space ?
=> when I saw that, I thought that "fullclone" would be something
avoid this non-standard behaviour (eg. for programs getting their
version info from history) - if it was ever the case, it obviously
does not work so simply.
- that checked-out source is cached as source tarball
Note that this latter source most likely never existed on upstream's
dev box. People either work from a git tree, or from a dist tarball
usually built with something more complicated than "rm -rf .git".
* in fullclone mode:
- the tarball with just .git is told to contain the whole source
- it is thus unpacked as "tmp/work/<arch>/<package>/.git/", just
besides directory "temp"
=> no wonder the build step does not find the source
Is it just a regression which broke a "fullclone" that should have
worked as I expected, or am I just mistaken on what it's supposed to
be useful for ?
As a PoC I tried to implement what I was missing, as a "realgit" param
(see attached patch), and it basically does what I need, but I got
into other problems (which may or not be on bitbake side - see
http://trac.shr-project.org/trac/ticket/1169), which prevent me to
check that there are no nasty side-effects.
* as far as disk space is a concern for git history, modern versions
of git support a --depth flag which would allow not to download all
of history. Probably it is not a good idea to activate it by
default, but for some heavy repos "--depth=1" could be a win.
Best regards,
--
Yann
diff --git a/lib/bb/fetch/git.py b/lib/bb/fetch/git.py
index 41ebc5b..0f6b478 100644
--- a/lib/bb/fetch/git.py
+++ b/lib/bb/fetch/git.py
@@ -114,7 +114,7 @@ class Git(Fetch):
os.chdir(ud.clonedir)
mirror_tarballs = data.getVar("BB_GENERATE_MIRROR_TARBALLS", d, True)
- if mirror_tarballs != "0" or 'fullclone' in ud.parm:
+ if mirror_tarballs != "0" or 'fullclone' in ud.parm or 'realgit' in ud.parm:
bb.msg.note(1, bb.msg.domain.Fetcher, "Creating tarball of git repository")
runfetchcmd("tar -czf %s %s" % (repofile, os.path.join(".", ".git", "*") ), d)
@@ -141,12 +141,17 @@ class Git(Fetch):
readpathspec = ""
coprefix = os.path.join(codir, "git", "")
- bb.mkdirhier(codir)
- os.chdir(ud.clonedir)
- runfetchcmd("%s read-tree %s%s" % (ud.basecmd, ud.tag, readpathspec), d)
- runfetchcmd("%s checkout-index -q -f --prefix=%s -a" % (ud.basecmd, coprefix), d)
+ if 'realgit' in ud.parm:
+ runfetchcmd("%s clone %s %s" % (ud.basecmd, ud.clonedir, coprefix), d)
+ os.chdir(codir)
+ runfetchcmd("%s checkout %s" % (ud.basecmd, ud.tag), d)
+ else:
+ bb.mkdirhier(codir)
+ os.chdir(ud.clonedir)
+ runfetchcmd("%s read-tree %s%s" % (ud.basecmd, ud.tag, readpathspec), d)
+ runfetchcmd("%s checkout-index -q -f --prefix=%s -a" % (ud.basecmd, coprefix), d)
- os.chdir(codir)
+ os.chdir(codir)
bb.msg.note(1, bb.msg.domain.Fetcher, "Creating tarball of git checkout")
runfetchcmd("tar -czf %s %s" % (ud.localpath, os.path.join(".", "*") ), d)
_______________________________________________
Bitbake-dev mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/bitbake-dev
