Epic created by Gedare Bloom: https://gitlab.rtems.org/groups/rtems/-/work_items/39
<!-- Use this template for project descriptions that are intended to be shown at https://projects.rtems.org/ Set the project::<size> label to make it appear in the list. Set the gsoc and the gsoc::<size> label to make it appear in the GSoC Ideas List --> ## Summary Our `testsuites` has grown overly cumbersome and contributes to excessive build and test run times. The goal of this epic is to reduce and consolidate tests without losing test coverage so that RTEMS development work can be simpler and faster. As an example of the current state and problem, building a (rv64imafdc) BSP with `BUILD_TESTS = true` compiles 4638 files and on a workstation took 1 minute 30 seconds, while building with `BUILD_TESTS = false` compiles 1483 files and took 27 seconds on the same workstation. Although there are several options to reduce the test suites at a coarse-grained level, this puts the burden on a developer to know and configure which tests are relevant for a specific change. The validation tests trade off building time for running. They run really long, but build many fewer tests. They also brute-force API limits, that's a big part of the time overhead. Since their purpose is to support qualification efforts, their suitability for general-purpose or even coverage testing of RTEMS is not clear. <!-- Provide background information about the project idea. --> ## Approach ### Unit Testing Improving the ability to write unit tests for RTEMS has been attempted in various ways in the past. There is an obsolete port of `rtems-gtest` that has been archived on our FTP server. The current direction that has been taken is the [RTEMS Test Framework](https://docs.rtems.org/docs/main/eng/test-framework.html) but adoption and use of this framework has been slow going. Converting existing tests into the Test Framework could be a good direction for making improvements. Improving the parsing ability of `rtems-test` to understand the RTEMS Testing Framework is integral to the long-term success of such an effort, along with improving the documentation "how to" write tests with it. ### Requirements and Documentation Part of this effort would need to establish the requirements/expectations around testsuites. That will give us something concrete for making any decisions. There are brief sections in the [User Manual](https://docs.rtems.org/docs/main/user/testing/index.html) and the [Software Engineering Manual](https://docs.rtems.org/docs/main/eng/test-plan.html) that might serve as a starting point for documenting the course of action. Some initial requirements that come to mind for any work done in this direction follows: * Merge requests that refactor and remove existing tests needs to [run coverage](https://docs.rtems.org/docs/main/user/testing/coverage.html) before and after major changes to ensure nothing is lost. * Test sizes need to accommodate the variety of targets supported by RTEMS. * Tests that exercise tight resource allocations need to preserve those constraints. -- View it on GitLab: https://gitlab.rtems.org/groups/rtems/-/work_items/39 You're receiving this email because of your account on gitlab.rtems.org. Unsubscribe from this thread: https://gitlab.rtems.org/-/sent_notifications/4-a5hqwpjp16vrcpsyzw8pihqxs-3/unsubscribe | Manage all notifications: https://gitlab.rtems.org/-/profile/notifications | Help: https://gitlab.rtems.org/help
_______________________________________________ bugs mailing list [email protected] http://lists.rtems.org/mailman/listinfo/bugs
