last couple of months I have been working on automated test suite for
OpenIndiana. You can find it's dashboard at
It makes use of openQA (http://open.qa/), an open source automated test
tool that makes possible to test the whole installation process of
operating system, OS configuration, regression and sanity testing of
individual components. openQA is currently used by Fedora
(https://openqa.fedoraproject.org/) and openSUSE
(https://openqa.opensuse.org/) for package updates and snapshot release
Test suite sources and guidance on how to setup your local openQA
instance are on GitHub
Currently Live, Text, and Minimal variants of both ISO and USB images
are tested as VM guests on KVM, VirtualBox, and Xen.
Some of the test scenarios implemented:
* General installation via text- as well as GUI-installer to disk,
reboot to the installed system, OS setup, shutdown.
- installation_mate@64bit: https://openqa.oi.mnowak.cz/tests/1820 (on
https://openqa.oi.mnowak.cz/tests/1841 (on KVM host)
https://openqa.oi.mnowak.cz/tests/1805 (on VirtualBox host)
- installation_mate@xen-hvm: https://openqa.oi.mnowak.cz/tests/1846
(on Xen host; note: illumos guest Xen drivers are in bad shape atm)
* Upgrade scenarios for latest OpenIndiana snapshots to the latest
incorporation, e.g. upgrade_textmode_20161030:
* illumos KVM virtualization test: https://openqa.oi.mnowak.cz/tests/1796
* Extended `pkg` console tests: https://openqa.oi.mnowak.cz/tests/1816
* Toolchain (pkg_src & oi-userland build) tests:
* Build of OpenIndiana snapshot images from latest published packages:
* When things go wrong: installation_textmode@vbox64_sas_LSILogicSAS:
My test-host/worker - a 5 year old 4 core Ivy Bridge laptop with 12 GB
of RAM, and SSD - is able to run 3 jobs in parallel reliably. Full build
validation takes about 8 hours. Bare metal server is certainly desired.
I believe that this test suite could help us produce OpenIndiana hipster
snapshots more validated, with higher level of confidence to particular
updates, periodically run package's upstream testsuite in real-life
environment as opposed to a bare-bone build environment, or release
snapshots more often (if desired).
How to get involved: Deploy your local openQA instance in accordance to
the documentation at http://open.qa on a Linux host (sic; I never
deployed openQA on illumos as I don't have illumos capable hardware),
follow OpenIndiana-specific guide at
clone particular job from openqa.oi.mnowak.cz, see how your changes
affect the test run, and contribute the code back as a pull request.
Similar test suite can be implemented for other illumos distributions
like OmniOS CE and SmartOS.
Feel free to contact me (mnowak_) on #illumos or #oi-dev.
oi-dev mailing list