On Mon, 2026-06-01 at 07:06 +0000, Daniel Turull wrote: > > -----Original Message----- > > From: [email protected] > > <[email protected]> On Behalf Of > > Richard Purdie via lists.openembedded.org > > Sent: Sunday, 31 May 2026 10:50 > > To: [email protected]; Mark Hatle > > <[email protected]>; openembedded- > > [email protected] > > Subject: Re: [Openembedded-architecture] Stable version upgrades on > > OE > > stable branches > > > Also remember that most ptests are functional tests for specific > > things. What OE needs is integration tests, which are quite > > different. > > I'd therefore argue that ptests aren't as helpful as people think > > as > > functional tests can pass but the overall integration can be > > totally > > broken. If I had to prioritise, I'd prefer more integration tests > > rather than ptests, we have very few of them and they are much more > > important IMO. ptests do have a place but I think people miss this > > difference. > > Richard, could you good examples of integration tests that can be use > as a reference to improve the situation? By integration test, do you > mean the oe-core selftests? > I can point our team to start looking at them
Let me first give a couple of illustrations. The biggest win we ever had was the image boot test. If you can confirm the image boots, that is a huge improvement and the act of booting tests a large and important part of the system. Another key example was the on target compiler being able to compile and run a "Hello World" binary. That confirms the compiler, linker, headers and so on all are correct and working together. You can see the gcc ones here: https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/runtime/cases/gcc.py They're very simple but effective. I appreciate many won't ship such a thing in their production images, but for OE-Core, it is a key data point that keeps our toolchains functional. The key question then becomes, what other simple things can we run/do in the target image that would show the system is functional and all working together correctly? One idea is to check binaries can be executed successfully (e.g. even just show their help output). That means they're at least valid binaries with their libraries present. Another thought would be to import python modules and check there aren't import errors. There are probably many other things we could do with a bit of thought, the aim being, to see the pieces on the target are functional and all work together. We'd ideally focus on things with larger dependency chains, where the test tests the dependencies as well as the item under test. Does that help describe the kinds of things I'm thinking of? There is obviously overlap with some areas of ptests and we don't want to duplicate that but I believe there is a lot of scope to improve in this area. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#2371): https://lists.openembedded.org/g/openembedded-architecture/message/2371 Mute This Topic: https://lists.openembedded.org/mt/119437109/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
