On Tue, 2023-08-08 at 10:10 +0100, Richard Purdie via lists.openembedded.org
wrote:
> There is a recent discussion about AUTOREV which has got me thinking a
> bit about the big picture and incremental development.
>
> Currently AUTOREV injects into PV. The reasons for that are historical,
> when OE was not taskhash based we had to do that to trigger updates.
> There are now different ways it could be done with the revision just
> needing injection into PKGPV for packaging purposes.
>
> I've also been looking at externalsrc bugs.
>
> This got me thinking, if we did make AUTOREV work without messing with
> PV, we could probably go a step further and allow the fetcher to
> incrementally update an existing git checkout rather than remove and
> start again from scratch. Patches may or may not be an issue, we could
> mandate a clean single git:// srcuri to start with although the patch
> code can remove the patch stack already. This would need to hook in to
> disable some of the cleanup/removal code.
>
> The usecase would be to allow the developer to pull updates from a
> remote repo, or even commit changes locally and then have the system
> "flow" the changes through from there. This would remove the annoying
> clean and re-checkout process which drives some developers crazy. It
> might also allow externalsrc to change to a new model with less
> deviation from the normal workflow.
I tried some experiments and this doesn't actually turn out to be so
difficult to do. I've included a patch below which basically drops the
SRCPV pieces into PKGV, the conditional can probably be simplified
further.
This patch isn't quite right in that we need to split get_srcrev into
two versions, one which returns the SRCREV_FORMAT string and one which
returns the full hashes for taskhash purposes.
The nice thing about this approach is that SRCPV can simply be removed
over time, it no longer does anything. That means compatibility is
retained with existing recipes.
I've not tested this extensively yet with AUTOREV but normal builds
appear to work ok with it and the WORKDIR directory naming
simplification is probably a win.
Cheers,
Richard
From: Richard Purdie <richard.pur...@linuxfoundation.org>
Subject: fetch: Move SRCPV from PV to PKGV
Signed-off-by: Richard Purdie <richard.pur...@linuxfoundation.org>
---
bitbake/lib/bb/fetch2/__init__.py | 5 ++++-
meta/classes-global/base.bbclass | 26 +-------------------------
meta/classes-global/package.bbclass | 6 +++++-
meta/conf/bitbake.conf | 5 +----
4 files changed, 11 insertions(+), 31 deletions(-)
diff --git a/bitbake/lib/bb/fetch2/__init__.py
b/bitbake/lib/bb/fetch2/__init__.py
index ad9d8a31a7e..ea51a009e1d 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -753,7 +753,7 @@ def get_autorev(d):
d.setVar("__BBAUTOREV_SEEN", True)
return "AUTOINC"
-def get_srcrev(d, method_name='sortable_revision'):
+def get_srcrev(d, method_name='sortable_revision', allow_none=False):
"""
Return the revision string, usually for use in the version string (PV) of
the current package
Most packages usually only have one SCM so we just pass on the call.
@@ -781,6 +781,9 @@ def get_srcrev(d, method_name='sortable_revision'):
scms.append(u)
if not scms:
+ if allow_none:
+ d.delVar("__BBINSRCREV")
+ return ""
raise FetchError("SRCREV was used yet no valid SCM was found in
SRC_URI")
if len(scms) == 1 and len(urldata[scms[0]].names) == 1:
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index cbda8d12f09..2567b7d24be 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -130,7 +130,7 @@ addtask fetch
do_fetch[dirs] = "${DL_DIR}"
do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
do_fetch[file-checksums] += " ${@get_lic_checksum_file_list(d)}"
-do_fetch[vardeps] += "SRCREV"
+do_fetch[vardepvalue] += "${@bb.fetch.get_srcrev(d, allow_none=True)}"
do_fetch[network] = "1"
python base_do_fetch() {
@@ -606,7 +606,6 @@ python () {
bb.debug(1, "Skipping recipe %s because of incompatible
license(s): %s" % (pn, ' '.join(incompatible_lic)))
raise bb.parse.SkipRecipe("it has incompatible license(s):
%s" % ' '.join(incompatible_lic))
- needsrcrev = False
srcuri = d.getVar('SRC_URI')
for uri_string in srcuri.split():
uri = bb.fetch.URI(uri_string)
@@ -619,24 +618,17 @@ python () {
# Svn packages should DEPEND on subversion-native
if uri.scheme == "svn":
- needsrcrev = True
d.appendVarFlag('do_fetch', 'depends', '
subversion-native:do_populate_sysroot')
# Git packages should DEPEND on git-native
elif uri.scheme in ("git", "gitsm"):
- needsrcrev = True
d.appendVarFlag('do_fetch', 'depends', '
git-native:do_populate_sysroot')
# Mercurial packages should DEPEND on mercurial-native
elif uri.scheme == "hg":
- needsrcrev = True
d.appendVar("EXTRANATIVEPATH", ' python3-native ')
d.appendVarFlag('do_fetch', 'depends', '
mercurial-native:do_populate_sysroot')
- # Perforce packages support SRCREV = "${AUTOREV}"
- elif uri.scheme == "p4":
- needsrcrev = True
-
# OSC packages should DEPEND on osc-native
elif uri.scheme == "osc":
d.appendVarFlag('do_fetch', 'depends', '
osc-native:do_populate_sysroot')
@@ -645,7 +637,6 @@ python () {
d.appendVarFlag('do_fetch', 'depends', '
nodejs-native:do_populate_sysroot')
elif uri.scheme == "repo":
- needsrcrev = True
d.appendVarFlag('do_fetch', 'depends', '
repo-native:do_populate_sysroot')
# *.lz4 should DEPEND on lz4-native for unpacking
@@ -676,21 +667,6 @@ python () {
elif path.endswith('.deb'):
d.appendVarFlag('do_unpack', 'depends', '
xz-native:do_populate_sysroot')
- if needsrcrev:
- d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
-
- # Gather all named SRCREVs to add to the sstate hash calculation
- # This anonymous python snippet is called multiple times so we
- # need to be careful to not double up the appends here and cause
- # the base hash to mismatch the task hash
- for uri in srcuri.split():
- parm = bb.fetch.decodeurl(uri)[5]
- uri_names = parm.get("name", "").split(",")
- for uri_name in filter(None, uri_names):
- srcrev_name = "SRCREV_{}".format(uri_name)
- if srcrev_name not in (d.getVarFlag("do_fetch", "vardeps") or
"").split():
- d.appendVarFlag("do_fetch", "vardeps", "
{}".format(srcrev_name))
-
set_packagetriplet(d)
# 'multimachine' handling
diff --git a/meta/classes-global/package.bbclass
b/meta/classes-global/package.bbclass
index e8055a9cdc5..004973f4b0e 100644
--- a/meta/classes-global/package.bbclass
+++ b/meta/classes-global/package.bbclass
@@ -316,8 +316,12 @@ python package_get_auto_pr() {
#
python package_convert_pr_autoinc() {
- pkgv = d.getVar("PKGV")
+ # Expand SRCPV into PKGV if not present
+ srcpv = bb.fetch.get_srcrev(d, allow_none=True)
+ if srcpv and srcpv not in d.getVar("PKGV"):
+ d.appendVar("PKGV", srcpv)
+ pkgv = d.getVar("PKGV")
# Adjust pkgv as necessary...
if 'AUTOINC' in pkgv:
d.setVar("PKGV", pkgv.replace("AUTOINC", "${PRSERV_PV_AUTOINC}"))
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 475d6523bb8..51b4ef8483e 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -735,10 +735,7 @@ SRC_URI[vardepsexclude] += "\
SRCDATE = "${DATE}"
SRCREV ??= "INVALID"
AUTOREV = "${@bb.fetch2.get_autorev(d)}"
-AUTOREV[vardepvalue] = "${SRCPV}"
-# Set Dynamically in base.bbclass
-# SRCPV = "${@bb.fetch2.get_srcrev(d)}"
-SRCPV[vardepvalue] = "${SRCPV}"
+SRCPV = ""
SRC_URI = ""
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1731):
https://lists.openembedded.org/g/openembedded-architecture/message/1731
Mute This Topic: https://lists.openembedded.org/mt/100618420/21656
Group Owner: openembedded-architecture+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
