> -----Original Message-----
> From: [email protected]
> <[email protected]> On Behalf Of Richard Purdie
> Sent: den 10 augusti 2023 18:53
> To: openembedded-architecture
> <[email protected]>
> Subject: Re: [Openembedded-architecture] Workflow improvements - incremental
> source updates?
>
> On Tue, 2023-08-08 at 10:10 +0100, Richard Purdie via lists.openembedded.org
> wrote:
> > There is a recent discussion about AUTOREV which has got me thinking a
> > bit about the big picture and incremental development.
> >
> > Currently AUTOREV injects into PV. The reasons for that are historical,
> > when OE was not taskhash based we had to do that to trigger updates.
> > There are now different ways it could be done with the revision just
> > needing injection into PKGPV for packaging purposes.
> >
> > I've also been looking at externalsrc bugs.
> >
> > This got me thinking, if we did make AUTOREV work without messing with
> > PV, we could probably go a step further and allow the fetcher to
> > incrementally update an existing git checkout rather than remove and
> > start again from scratch. Patches may or may not be an issue, we could
> > mandate a clean single git:// srcuri to start with although the patch
> > code can remove the patch stack already. This would need to hook in to
> > disable some of the cleanup/removal code.
> >
> > The usecase would be to allow the developer to pull updates from a
> > remote repo, or even commit changes locally and then have the system
> > "flow" the changes through from there. This would remove the annoying
> > clean and re-checkout process which drives some developers crazy. It
> > might also allow externalsrc to change to a new model with less
> > deviation from the normal workflow.
>
> I tried some experiments and this doesn't actually turn out to be so
> difficult to do. I've included a patch below which basically drops the
> SRCPV pieces into PKGV, the conditional can probably be simplified
> further.
>
> This patch isn't quite right in that we need to split get_srcrev into
> two versions, one which returns the SRCREV_FORMAT string and one which
> returns the full hashes for taskhash purposes.
>
> The nice thing about this approach is that SRCPV can simply be removed
> over time, it no longer does anything. That means compatibility is
> retained with existing recipes.
>
> I've not tested this extensively yet with AUTOREV but normal builds
> appear to work ok with it and the WORKDIR directory naming
> simplification is probably a win.
>
> Cheers,
>
> Richard
I decided to give the related changes you have in master-next a try. I
cherry-picked the following three commits:
bitbake: fetch2: Add new srcrev fetcher API
recipes/classes/scripts: Drop SRCPV usage in OE-Core
base/package: Move source revision information from PV to PKGV
I am of course aware that the above changes are not final, but I have
some concerns.
I tested building "iso-codes" and "xcursor-transparent-theme". They are
both allarch recipes with few dependencies. The former has a SRCREV that
matches PV, and the latter does not (and thus uses PV = "0.1.1+git${SRCPV}").
Here are the names of the resulting RPM packages before the changes are
applied:
iso-codes-4.15.0-r0.4.noarch.rpm
xcursor-transparent-theme-0.1.1+git0+23c8af5ba4-r0.0.noarch.rpm
And here they are afterwards:
iso-codes-4.15.0AUTOINC+69ba16daef-r0.0.noarch.rpm
xcursor-transparent-theme-0.1.1+gitAUTOINC+23c8af5ba4-r0.0.noarch.rpm
The first problem is of course that AUTOINC is not replaced as intended
when a PR server is active. This is due to the code in package_get_auto_pr()
that should update PRSERV_PV_AUTOINC not being triggered and thus it
retains its default value "AUTOINC". The code isn't triggered because it
looks for "AUTOINC" in PKGV, but it isn't there (yet) since ${SRCPV} is
no longer used. The following patch solves it:
diff --git a/meta/classes-global/package.bbclass
b/meta/classes-global/package.bbclass
index 7f55b123c4..bf824b0b83 100644
--- a/meta/classes-global/package.bbclass
+++ b/meta/classes-global/package.bbclass
@@ -266,6 +266,9 @@ python package_get_auto_pr() {
d.setVar("PRSERV_HOST", host)
pkgv = d.getVar("PKGV")
+ srcpv = bb.fetch.get_pkgv_string(d)
+ if srcpv:
+ pkgv += srcpv
# PR Server not active, handle AUTOINC
if not d.getVar('PRSERV_HOST'):
The second problem is that the value from bb.fetch.get_pkgv_string() is
concatenated to the version without any separator. So if the AUTOINC
part had been replaced as expected, the RPM name for the iso-codes
package would have become iso-codes-4.15.00+69ba16daef-r0.0.noarch.rpm,
which I doubt was intended... The easiest solution should be to make
get_pkgv_string() return "+AUTOINC+<ID>" instead of the current
"AUTOINC+<ID>".
Third, I am not sure if it is a good idea to always include the SHA-1
in the package name. Before it was typically only done when the SRCREV
did not match PV. For most packages it is redundant. It also means that
the PR server needs to keep track of the version for all Git recipes,
not just the ones where ${SRCPV} is used today. However, I understand
that it is probably not easily solvable to automatically detect when
it is needed and when it is not. So it is either do it manually as
today and risk missing to do it when it is needed, or always do it and
have it be redundant in most cases.
One thing that I would prefer is if there is a variable, similar to
the old SRCPV, that contains "${@bb.fetch.get_pkgv_string(d)}" by
default and is used from package_convert_pr_autoinc() instead of
calling the method directly. The reason for that is to make it
possible to use a different method to determine the value. E.g., in
our case we have a variable called MAINTPV, which is similar to
SRCPV. The reason for it is that it handles Gerrit references and
expands them in a more suitable way than just a SHA-1. Thus I would
like to be able to use its underlying function instead of
bb.fetch.get_pkgv_string(d) to generate the string for PKGV.
//Peter
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1734):
https://lists.openembedded.org/g/openembedded-architecture/message/1734
Mute This Topic: https://lists.openembedded.org/mt/100618420/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
