Hi Richard,
> FWIW I do want to see the system used to do things like this. I've kind
> of envisaged you'd do it "up front" though. For example it should be
> trivial for a script to use tinfoil, iterate through and generate an
> include file of the revisions you want, using the fetcher calls.
>
> Yes, its slightly more annoying to generate the config, then build it
> but if it stops the core becoming unmaintainable with corner cases,
> that could be the right decision.
>
> If you do something like that it would be great to share it so others
> can see what you did and be able to build off it...
This is actually something I've been working on and thinking about a lot for
the past year and a half. Our application layer has probably 30 or so recipes
that use AUTOREV. For the purposes of reproducible builds, we need to generate
similar output to what buildhistory-collect-srcrevs generates, e.g.
SRCREV_pn-my_daemon = "githash"
I've gone back and forth and re-implemented the code to do so a half dozen
times. Chronologically, these are the fundamental strategies I've attempted:
1. Simply in terms of buildhistory-collect-srcrevs.
Pros: dead simple
Cons: requires that you actually build stuff first; requires
buildhistory to be enabled
2. As a bbclass, injecting a task between do_fetch and do_unpack
("do_capture_srcrevs")
Pros: can be invoked without actually building stuff, e.g. bitbake
my-image --runall=do_capture_srcrevs
Cons: adds yet another task to the recipe; a little ugly since you
either need to INHERIT the class, or individually inherit it in recipes
3. As a separate tinfoil script (as you suggest) which I run before the actual
bitbake.
Pros: decoupled from the actual build
Cons: since it's decoupled, it's slower - each recipe needs to be parsed
4. Hacking the srcrev cache (inside fetch2) to log all information necessary to
generate the SRCREV overrides, paired with a script that reads the cache and
does it
Pros: Equally suitable for the "config-then-build" workflow ("bitbake
--parse-only && generate_overrides.py >> local.conf && bitbake image") or the
"build-then-config" workflow ("bitbake image && generate_overrides.py >
overrides.conf")
Cons: Invasive, required large changes to fetch2/__init__.py and some
smaller changes to fetch2/git.py
In the current incarnation (#4), I also go a step further and, in cases where a
SRC_URI has a parameterized branch, I generate BRANCH override lines as well.
For example, if I see:
MY_BRANCH ??= "master"
SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1;
branch=${MY_BRANCH}"
I generate this line along with the corresponding SRCREV override:
MY_BRANCH_pn-recipe = "master"
The algorithm for detecting the branch variable in the SRC_URI entry is as a
little complicated, and not super relevant at this moment, but I want to get it
off my chest :). The code first tries to find a parameterized branch by using a
regex to find stuff like "branch=${VAR}". If that fails, it could be because
the branch is expressed as something more complicated like this:
BRANCH_RELEASE_NUMBER ??= "1.0"
SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1;
branch=release-${BRANCH_RELEASE_NUMBER}"
In that case, I fall back on a breadth-first search on all the variables that
SRC_URI references (recursively) to find the "deepest" one that controls the
branch. I wrote this in terms of d.expandWithRefs, and can detail it further if
people are interested. Basically, I didn't want to enforce a particular
convention on the name of the branch variable. You'd think BRANCH would cover
it, but the kernel recipes also have KMETA and stuff like that.
I realize the branch stuff as currently implemented is maybe too opinionated
(and a bit dangerous) for inclusion in bitbake. But I digress.
I'm very interested in discussing the SRCREV (and perhaps BRANCH) stuff further
with anyone interested. I would love to see some strategy for doing this make
its way into bitbake or poky, so I can stop reimplementing it and second
guessing myself :).
Thanks,
Chris
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
