Just for your information there are more complaints about that issue [1] of some bad side effects of sharing the TMP folder in multi configs.
[1] https://lists.openembedded.org/g/bitbake-devel/message/14860 Jose Ricardo Salveti <[email protected]> escreveu no dia segunda, 30/01/2023 à(s) 18:27: > On Mon, Jan 30, 2023 at 2:57 PM Jose Quaresma <[email protected]> > wrote: > > Denys Dmytriyenko <[email protected]> escreveu no dia segunda, 30/01/2023 > à(s) 17:18: > >> > >> On Mon, Jan 30, 2023 at 11:02:25AM -0600, Ryan Eatmon via > lists.yoctoproject.org wrote: > >> > > >> > > >> > On 1/30/2023 9:20, Jose Quaresma wrote: > >> > >This patch adds the possibility to change some bitbake variable > >> > >that cannot be changed anywhere else, one of then is the TMPDIR. > >> > > > >> > >Using the same TMPDIR it is very sensitive and prone to several > errors > >> > >when used in more complex situations. > >> > >This configuration forces that all native packages have to be the > same between all > >> > >machines and this requirement is very easy to break. > >> > >Suppose you use a macinhe override somewhere on a native recipe > >> > >> Even w/o multiconfig, this will break for multi-machine builds. Native > >> packages are for the build host and have no knowledge about the target. > >> You should probably look into cross packages instead of native ones for > >> this use case. > >> > >> Altering native packages between different targets will cause them to > >> rebuild, which they shouldn't. Moreover, it will mess up your sstate > >> and lead to weird issues down the road when trying to re-use your > >> sstate mirror/cache. > > > > Is that the issue of rebuilding some native packages that I am trying to > avoid. > > But it is very hard to fix this in OE-core and everything else layers > what happens. > > > > For your information I have some other issues with multiconfig and > native packages > > that have the source files provided on the layer. this two recipes don't > work with the > > rm_work bbclass because the in one of the machines > > texinfo-dummy-native gettext-minimal-native > > > > Another issue is that sometimes one of the machines uses the > SOURCE_DATE_EPOCH_FALLBACK > > and the other uses SOURCE_DATE_EPOCH and this also triggers a rebuild > of one of the machines. > > > > Recently a new issue with the rm_work and the spdx > > comes in > https://lists.openembedded.org/g/openembedded-core/message/174276 > > where I have the steps to reproduce. > > > > All of these issues can be solved easily with TMDIR for each machine, > > but because I understand it can be invasive I submit this patch that > doesn't change anything > > in meta-ti but can add the possibility to use a different TMPDIR for > each machine. > > > > A different TMPDIR for each machine is also referenced in the official > documentation > > "Minimally, each configuration file must define the machine and the > temporary directory BitBake uses for the build. Suggested practice dictates > that you do not overlap the temporary directories used during the builds." > > > https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-intro.html#executing-a-multiple-configuration-build > > > >> > >and this requirement is no longer met. > >> > >Many of these anomalies can be verified with the use of the rm_work > and > >> > >create-spdx bitbake classes. > >> > > > >> > >A previous attempt [1] had already been made but was refused > >> > >but this way it can be done without side effects for other users. > >> > > > >> > >[1] https://lists.yoctoproject.org/g/meta-ti/message/14767 > >> > > >> > > >> > So the difference between this patch and the previous one, is that > >> > you are now included a file that is not there normally so that you > >> > can put the contents of the previous patch in the new include file > >> > to get around the issue in the archiver. > > > > The archiver issue is fixed but it will be very easy to break again. > > Myself recently broke it again so I added a test to help there > > > https://git.yoctoproject.org/poky/commit/?id=b5baa7dc8bd1c0d2d78c532d97a44120346b5edd > > > >> > > >> > I'm not saying I'm opposed to this approach (yet), I'm just trying > >> > to understand the entire story. > >> > > >> > Did you find anything in looking at fixing the archiver to better > >> > support the multi-config issue? > > > > The issue now is different and is related with spdx and rm_work > > https://lists.openembedded.org/g/openembedded-core/message/174276 > > > > With this patch I can set a new TMPDIR folder for each machine and > > can build successfully. > > If you don't have spdx enabled then these sort of issues happen less > often, but after the above patch was merged in oe-core (and also > backported to kirkstone), we cannot build multiconfig targets from > meta-ti anymore with spdx enabled. > > This is special to spdx because of the way the task dependencies are > done, which requires it to be executed before rm_work. This is OK when > a machine specific TMPDIR is used and explodes quite quickly (due > races) when sharing TMPDIR because of the parallel rm_work tasks that > end up removing stuff that is still required by other parallel tasks. > > Cheers, > -- > Ricardo Salveti > -- Best regards, José Quaresma
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16811): https://lists.yoctoproject.org/g/meta-ti/message/16811 Mute This Topic: https://lists.yoctoproject.org/mt/96629864/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/meta-ti/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
