If we know a repository content has what we need, we can flag it as static and then optimize accordingly for the future, by not trying to do any fetch operations against it. In a way, this can be thought of as the special case of GITFETCHREFS = None.
There are multiple advantages to having this. We can skip trying any updates and speed things up for a project we know is EOL. We can avoid doing "ls-remote" operations against servers, easing use for mirror/no-network users. We can open the door to accepting tags in SRC_URI more often as absolute references, instead of the less humanly readable SHA. Archives that change upstream in incompatible ways, or have simply vanished, can exist as a snapshot in time after initial creation via mirror tarball or manually. Finally, we use the flexibility of git-config to store that static information in the cloned repo itself, so we can tell at any time, independent of the SRC_URI, that the repo has static content. This will be used for reference purposes, but might also be used in the download dir itself outside of BB recipes -- for example a script could easily spot any static content and hence prioritize it as a possible mirroring candidate. Signed-off-by: Paul Gortmaker <[email protected]> --- .../bitbake-user-manual-fetching.rst | 5 ++++ bitbake/lib/bb/fetch2/git.py | 27 +++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst index 0f12eaa3a81f..94f4cf3363f6 100644 --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst @@ -426,6 +426,11 @@ This fetcher supports the following parameters: - *"rev":* The revision to use for the checkout. The default is "master". +- *"static":* The repository is a "one and done" - clone/untar and the + content is static and unchanging. This allows optimizations like + skipping fetch/pull and doing "ls-remote" on servers. Typically used + for repositories that will act as a commit library for other repos. + - *"tag":* Specifies a tag to use for the checkout. To correctly resolve tags, BitBake must access the network. For that reason, tags are often not used. As far as Git is concerned, the "tag" parameter diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py index de698a3d9f24..7cba990c3be7 100644 --- a/bitbake/lib/bb/fetch2/git.py +++ b/bitbake/lib/bb/fetch2/git.py @@ -54,6 +54,11 @@ Supported SRC_URI options are: referring to commit which is valid in tag instead of branch. The default is "0", set nobranch=1 if needed. +- static + Repository is of fixed/static content with no active/new development. + Once cloned, nothing will change, so optimize accordingly. The default + is "0", set static=1 when appropriate. + - usehead For local git:// urls to use the current branch HEAD as the revision for use with AUTOREV. Implies nobranch. @@ -158,6 +163,8 @@ class Git(FetchMethod): ud.nobranch = ud.parm.get("nobranch","0") == "1" + ud.static = ud.parm.get("static","0") == "1" + ud.dlname = ud.parm.get("dlname","") # usehead implies nobranch @@ -167,6 +174,10 @@ class Git(FetchMethod): raise bb.fetch2.ParameterError("The usehead option is only for use with local ('protocol=file') git repositories", ud.url) ud.nobranch = 1 + if ud.static: + if ud.rebaseable: + raise bb.fetch2.ParameterError("The rebaseable option is incompatible with the static option", ud.url) + # bareclone implies nocheckout ud.bareclone = ud.parm.get("bareclone","0") == "1" if ud.bareclone: @@ -311,6 +322,8 @@ class Git(FetchMethod): def clonedir_need_update(self, ud, d): if not os.path.exists(ud.clonedir): return True + if ud.static: + return False if ud.shallow and ud.write_shallow_tarballs and self.clonedir_need_shallow_revs(ud, d): return True for name in ud.names: @@ -341,6 +354,17 @@ class Git(FetchMethod): return False return True + def get_git_config(self, ud, d, repo, cfgvar): + try: + output = runfetchcmd("%s config --get %s" % (ud.basecmd, cfgvar), d, workdir=repo) + except (bb.fetch2.FetchError,ValueError): + return "" + return output.split()[0] + + def repo_is_static(self, ud, d, repo): + static = self.get_git_config(ud, d, repo, "bitbake.static") + return (static == "true") + def download(self, ud, d): """Fetch url""" @@ -368,6 +392,9 @@ class Git(FetchMethod): progresshandler = GitProgressHandler(d) runfetchcmd(clone_cmd, d, log=progresshandler) + if ud.static: + runfetchcmd("%s config --bool --add bitbake.static 1" % ud.basecmd, d, workdir=ud.clonedir) + # Update the checkout if needed if self.clonedir_need_update(ud, d): output = runfetchcmd("%s remote" % ud.basecmd, d, quiet=True, workdir=ud.clonedir) -- 2.25.1
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#9639): https://lists.yoctoproject.org/g/linux-yocto/message/9639 Mute This Topic: https://lists.yoctoproject.org/mt/81808157/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/linux-yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
