> -----Original Message----- > From: Richard Purdie <[email protected]> > Sent: Monday, 1 June 2026 09:18 > To: Daniel Turull <[email protected]>; [email protected]; > Mark Hatle <[email protected]>; openembedded- > [email protected] > Subject: Re: [Openembedded-architecture] Stable version upgrades on OE > stable branches > > On Mon, 2026-06-01 at 07:06 +0000, Daniel Turull wrote: > > > -----Original Message----- > > > From: [email protected] > > > <[email protected]> On Behalf Of > > > Richard Purdie via lists.openembedded.org > > > Sent: Sunday, 31 May 2026 10:50 > > > To: [email protected]; Mark Hatle > > > <[email protected]>; openembedded- > > > [email protected] > > > Subject: Re: [Openembedded-architecture] Stable version upgrades on > > > OE > > > stable branches > > > > > Also remember that most ptests are functional tests for specific > > > things. What OE needs is integration tests, which are quite > > > different. > > > I'd therefore argue that ptests aren't as helpful as people think > > > as > > > functional tests can pass but the overall integration can be > > > totally > > > broken. If I had to prioritise, I'd prefer more integration tests > > > rather than ptests, we have very few of them and they are much more > > > important IMO. ptests do have a place but I think people miss this > > > difference. > > > > Richard, could you good examples of integration tests that can be use > > as a reference to improve the situation? By integration test, do you > > mean the oe-core selftests? > > I can point our team to start looking at them > > Let me first give a couple of illustrations. > > The biggest win we ever had was the image boot test. If you can confirm > the image boots, that is a huge improvement and the act of booting > tests a large and important part of the system. > > Another key example was the on target compiler being able to compile > and run a "Hello World" binary. That confirms the compiler, linker, > headers and so on all are correct and working together. You can see the > gcc ones here: > > https://git.ope/ > nembedded.org%2Fopenembedded- > core%2Ftree%2Fmeta%2Flib%2Foeqa%2Fruntime%2Fcases%2Fgcc.py&data=0 > 5%7C02%7Cdaniel.turull%40ericsson.com%7C8a7cb322c98a4599f81608debfa > ddabe%7C92e84cebfbfd47abbe52080c6b87953f%7C0%7C0%7C63915895059 > 2960044%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiO > iIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C > 0%7C%7C%7C&sdata=%2FHwNknvTShZzX42%2Fee7Ew0CXyXTiiaUIzeeNFKUl9 > Rs%3D&reserved=0 > > They're very simple but effective. I appreciate many won't ship such a > thing in their production images, but for OE-Core, it is a key data > point that keeps our toolchains functional. > > The key question then becomes, what other simple things can we run/do > in the target image that would show the system is functional and all > working together correctly? > > One idea is to check binaries can be executed successfully (e.g. even > just show their help output). That means they're at least valid > binaries with their libraries present. > > Another thought would be to import python modules and check there > aren't import errors. > > There are probably many other things we could do with a bit of thought, > the aim being, to see the pieces on the target are functional and all > work together. We'd ideally focus on things with larger dependency > chains, where the test tests the dependencies as well as the item under > test. > > Does that help describe the kinds of things I'm thinking of?
Yes, it does. I will discuss it internally and look where we can help. Having this kind of direction is very good to know where to focus. Cheers, Daniel > > There is obviously overlap with some areas of ptests and we don't want > to duplicate that but I believe there is a lot of scope to improve in > this area. > > Cheers, > > Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#2372): https://lists.openembedded.org/g/openembedded-architecture/message/2372 Mute This Topic: https://lists.openembedded.org/mt/119437109/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
