Hello,

last couple of months I have been working on automated test suite for OpenIndiana. You can find it's dashboard at

  https://openqa.oi.mnowak.cz

It makes use of openQA (http://open.qa/), an open source automated test tool that makes possible to test the whole installation process of operating system, OS configuration, regression and sanity testing of individual components. openQA is currently used by Fedora (https://openqa.fedoraproject.org/) and openSUSE (https://openqa.opensuse.org/) for package updates and snapshot release testing, respectively.

Test suite sources and guidance on how to setup your local openQA instance are on GitHub

  https://github.com/Mno-hime/os-autoinst-distri-openindiana/

Currently Live, Text, and Minimal variants of both ISO and USB images are tested as VM guests on KVM, VirtualBox, and Xen.

Some of the test scenarios implemented:

* General installation via text- as well as GUI-installer to disk, reboot to the installed system, OS setup, shutdown. - installation_mate@64bit: https://openqa.oi.mnowak.cz/tests/1820 (on KVM host) - installation_textmode_raidz@64bit: https://openqa.oi.mnowak.cz/tests/1841 (on KVM host) - installation_textmode_mbr@vbox64_ide_piix3: https://openqa.oi.mnowak.cz/tests/1805 (on VirtualBox host) - installation_mate@xen-hvm: https://openqa.oi.mnowak.cz/tests/1846 (on Xen host; note: illumos guest Xen drivers are in bad shape atm)

* Upgrade scenarios for latest OpenIndiana snapshots to the latest incorporation, e.g. upgrade_textmode_20161030: https://openqa.oi.mnowak.cz/tests/1808

* illumos KVM virtualization test: https://openqa.oi.mnowak.cz/tests/1796

* Extended `pkg` console tests: https://openqa.oi.mnowak.cz/tests/1816

* Toolchain (pkg_src & oi-userland build) tests: https://openqa.oi.mnowak.cz/tests/1815

* Build of OpenIndiana snapshot images from latest published packages: https://openqa.oi.mnowak.cz/group_overview/2

* When things go wrong: installation_textmode@vbox64_sas_LSILogicSAS: https://openqa.oi.mnowak.cz/tests/1795

My test-host/worker - a 5 year old 4 core Ivy Bridge laptop with 12 GB of RAM, and SSD - is able to run 3 jobs in parallel reliably. Full build validation takes about 8 hours. Bare metal server is certainly desired.

I believe that this test suite could help us produce OpenIndiana hipster snapshots more validated, with higher level of confidence to particular updates, periodically run package's upstream testsuite in real-life environment as opposed to a bare-bone build environment, or release snapshots more often (if desired).

How to get involved: Deploy your local openQA instance in accordance to the documentation at http://open.qa on a Linux host (sic; I never deployed openQA on illumos as I don't have illumos capable hardware), follow OpenIndiana-specific guide at https://github.com/Mno-hime/os-autoinst-distri-openindiana/blob/master/README.md, clone particular job from openqa.oi.mnowak.cz, see how your changes affect the test run, and contribute the code back as a pull request.

Similar test suite can be implemented for other illumos distributions like OmniOS CE and SmartOS.

Feel free to contact me (mnowak_) on #illumos or #oi-dev.

Michal

_______________________________________________
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev

Reply via email to