On 6/7/2022 6:00 pm, Sebastian Huber wrote: > Yes, if tests go wrong the tester can kill a test execution after the > specified > timeout.
Killing should be taken as a sign something in the test equipment is broken. > Why do we need this arbitrary SIS -tlim of 400 s? There was a few values and I select this one based on those. If you have a better value please say. I do not recommend removing the time limit option from testing. If you want to operate that way create a user config with: [erc32-sis] sis_time_limit = > Which problem does this solve? Repeatable test results across wide ranging hosts and host operating systems. > Why can't we let the tests run in SIS without a limit just like we do it for > Qemu? Qemu is painful to make work in a consistent and reliable way. > Normally, if a test is done, it terminates the SIS execution. The time outs are for the cases that end abnormally. > The new performance tests can be used to catch performance regressions. We need to consider the existing benchmarks. > In the > long run I think we need a more modular approach. For example, one component > which runs tests and reports the test output. Another component which analyses > the test output. The test outputs can be archived. The rtems-test command is required to report regressions and as simply as possible. A user wants to build, test then know what they have matches what we released. Chris _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel