chengkai15 commented on issue #21478: URL: https://github.com/apache/nuttx/issues/21478#issuecomment-6660708943
Thanks @LingaoM. Agreed that per-commit regression has to run on a simulated PHY, and AutoPTS is for periodic qualification runs. For reference, the Bluetooth testing in Zephyr is done in four ways: | | Where | How it runs | What it covers | Per PR in CI | |---|---|---|---|---| | 1. Twister + ztest | `tests/bluetooth/` | on `native_sim` / `qemu_x86`; hardware boards are build-only in CI | unit and small integration tests, e.g. host internals with fff mocks (`tests/bluetooth/host/`), `bt_crypto`, `buf`, `uuid` | yes (`twister.yaml`) | | 2. BabbleSim | `tests/bsim/bluetooth/` | images for `nrf52_bsim` built by Twister, run by shell scripts with `bs_2G4_phy_v1` | multi-device end-to-end tests of host, audio and mesh; EDTT (gap/gatt/hci/ll lists); the BTP tester (`tests/bsim/bluetooth/tester`) | yes (`bsim-tests.yaml`, on changes to `subsys/bluetooth/**`) | | 3. BTP tester + AutoPTS | `tests/bluetooth/tester` | tester on a board, driven by AutoPTS with PTS and a PTS dongle | qualification test cases from the ICS (`tests/bluetooth/qualification/`) | no, daily ([bluetooth-qualification](https://github.com/zephyrproject-rtos/bluetooth-qualification)) | | 4. Hardware | as 1, with `twister --device-testing` | on real boards | board-specific behaviour | no, vendor labs | How this maps to zblue: - **1**: the host unit tests depend on ztest and on mocking Zephyr internals, so they do not carry over directly. Tests of pure functions can be reused; the `bt_crypto` vectors already pass on `sim:zblue`. - **2**: this is the per-commit regression in Zephyr. What ties it to Zephyr is the `nrf52_bsim` board and the nRF HW models rather than Twister, which only builds the images. BSWorld avoids that: the host connects over H4 to a virtual controller, and `sim:zblue` already uses H4 through `/dev/ttyHCIx`, so it should plug in without changes to the host. The EDTT LL lists test the controller; for a host only the gap/gatt lists apply. - **3**: PR 7 of the roadmap ports the BTP tester. As Zephyr does in 2, the same application can be driven by BSWorld scenarios in CI and by AutoPTS on hardware. - **4**: a hardware lab for regular regression is useful, and can be tracked separately from this roadmap. I have updated PR 7 in the roadmap above to: BTP tester for AutoPTS, plus BSWorld scenarios with `sim:zblue` in CI. Could you confirm that BSWorld can drive a NuttX sim host over H4 today, and which controller you would use for it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
