On Sat, 2026-05-30 at 20:30 +0000, Peter Marko via lists.openembedded.org wrote: > > On 5/27/26 11:42 AM, Richard Purdie via lists.openembedded.org wrote: > > > On Wed, 2026-05-27 at 14:16 +0000, Daniel Turull wrote: > > > > To help with the updates, allowing ptest to be backported could also > > > > help on finding issues in the recipe updates or CVE backporting. > > > > > > ptests are not good candidates for backports. We have huge amounts of > > > problems with them in master, let alone the stable backports. Very few > > > actually dive in and try and fix the issues in master. The new patches > > > often seem to be AI generated and contain questionable cod. All my > > > experience says backporting them is a really bad idea. > > > > > > Sorry, but definitely no from me. > > > > Sorry for the late reply. I had missed this email. > > > > I agree with Richard. Test cases need to actually test "something". Just > > backporting ptests for the sake of more tests doesn't help anyone. It > > doesn't > > test regressions, it doesn't test functionality, and just adds noise to the > > reports. (More positive, more negatives and more skips suddenly appear.) > > > > I'm all in favor of adding ptests when something else has been added to a > > stable > > version. I.e. a CVE is fixed in a recipe. If the CVE behavior can be > > checked > > for in a ptest, I would be in favor of adding that ptest and backporting it > > where appropriate. (But just the ONE test.) > > I must disagree here. > > The main problem with backporting CVEs to recipes without ptest is fact that > it's impossible to verify the patch sideeffects. > Especially for a 3-4 year old component version, where there are several > cherry-pick conflicts and some code paths are rewritten meanwhile. > Testing that CVE patch fixes a CVE is nice (assuming that exploit is at > publicly available), however testing that nothing else got broken is crucial > here. > > I certainly understand that tests can be unstable, however that can be solved > be selectively skipping single testcases. > Personally, I'd love to see all ptests being backported as the confidence on > CVE patches would be significantly boosted.
You're missing my point. Yes, ptests can help validate the CVEs, I absolutely agree with that. However they also have a development and maintenance cost and there is not enough people helping with either of those areas. If people want to see ptests being backported we need: a) ptest patch submissions which aren't poorly reviewed AI generated code b) people other than Ross and a handful of others willing to review them and help get them into a state where they can be maintained long term c) people joining the triage meetings to help review test failures d) people actually working on reducing the ptest failures counts I have a pile of ptests sitting in master-next as I can't get anyone to help review the things. Ross is nearly at the point of refusing to look at ptest patches as the AI code quality is poor and people aren't listening/acting on the feedback. So whilst you might disagree for your personal priority which is CVEs, from the overall project perspective the situation is different and I'm not going to push this problem to the LTS maintainer as well. Also remember that most ptests are functional tests for specific things. What OE needs is integration tests, which are quite different. I'd therefore argue that ptests aren't as helpful as people think as functional tests can pass but the overall integration can be totally broken. If I had to prioritise, I'd prefer more integration tests rather than ptests, we have very few of them and they are much more important IMO. ptests do have a place but I think people miss this difference. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#2368): https://lists.openembedded.org/g/openembedded-architecture/message/2368 Mute This Topic: https://lists.openembedded.org/mt/119437109/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
