I'm a bit worried about some of the mulitlib issues we've seen
recently. In particular we have a bug where license exclusion doesn't
work properly in multilib builds, which is fairly serious.
To dive into the details of what I found, as things stand today, the
code sequence is:
a) bb.event.RecipePreFinalise
b) bb.data.expandKeys()
c) base.bbclass anonymous python
d) multilib.bbclass anonymous python
The mutlilib code does half the work in a) but delays half to d) since
it has to happen after python key expansion, there is no way to avoid
that.
The issue is that there is code in base.bbclass's anonymous python
(step c) that assumes the package information in the datastore is
correct, yet it isn't under after d).
For example if you set:
PACKAGES += "X-bar"
LICENSE_${PN}-bar = "Y"
(and PN is X)
then base.bbclass can't see the LICENSE value by iterating PACKAGES.
Yes, you could argue PACKAGES should match the keys used but that is
hard to enforce and really shouldn't be necessary.
Having debugged this I decided an "easy" fix would be to add a new
event, bb.event.RecipePostKeyExpansion which happens after key
expansion and before anonymous python. This updates the flow to:
a) bb.event.RecipePreFinalise
b) bb.data.expandKeys()
c) multilib bb.event.RecipePostKeyExpansion
d) base.bbclass anonymous python
However things break :(.
There is anonymous python that assumes it runs before multilib, e.g.
mesa so we have to smatter in MLPREFIX references to make the code work
again as after this change, all anonymous python runs after multilib so
must use MLPREFIX in changes. Ok, fine, this at least gives us a
consistent API.
Sadly, things still break. For example:
DEPENDS += "${@bb.utils.contains('GI_DATA_ENABLED', 'True', 'libxslt-native',
'', d)}"
since this is expanded by the multilib RecipePostKeyExpansion event,
this happens before DISTRO_FEATURES is manipulated by base.bbclass and
the value is lost.
PACKAGES += "${@bb.utils.contains('PACKAGECONFIG', 'pulseaudio',
'alsa-plugins-pulseaudio-conf', '', d)}"
which also suffered from effectively the same issue.
There were also problems from:
RECOMMENDS_${PN} += "${BOOST_PACKAGES}"
Since BOOST_PACKAGES is unset until after anonymous python runs and the
new multilib code tried to create packages like:
"lib32-${BOOST_PACKAGES}" which translated to "lib32-lib32-boost-A
lib32-boost-B".
I'm struggling to decide how best we can fix much of this. One approach
is to avoid expanding PACKAGES and DEPENDS, instead we can wrap them,
saving the unexpanded value to some hidden variable and creating a
function which dynamically expands them on the fly. There are patches
to do this in master-next. Its rather ugly magic but does appear to
work, I'm testing on the autobuilder atm just to see if it does fix
issues.
Its possible we could create a new bitbake API for this with variables
having "filter" functions, called when the value is returned, I just
worry about performance overheads.
Does anyone have any other ideas or opinions on this?
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1052):
https://lists.openembedded.org/g/openembedded-architecture/message/1052
Mute This Topic: https://lists.openembedded.org/mt/73131657/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
