We discussed the topics with Adam over IRC so just a short summary:
Ad (1): It should be doable, first I need to understand how Vagrant does
stuff, will have a look at oi-packer.
Ad (2):
The dashboard, scheduler, and DB parts of openQA can run in a VM
instance (and they do now), however worker (the part where the
system-under-test aka VM actually runs) needs hypervizor to spawn VMs.
My laptop is the current KVM, VirtualBox, and Xen host. First two
hypervizors can be switched on-the-fly, Xen environment is accessible
only via reboot.
For jobs which require KVM the hypervizor host is openQA worker as well.
For jobs which require VirtualBox or Xen hypervizor, the openQA worker
and the hypervizor host may be different systems which can talk to one
another via HTTP and SSH, but for HW reasons they are united in the
current setup.
openQA has a good CLI which leverages REST-like API: e.g.
https://openqa.oi.mnowak.cz/xyz.
Michal
On 11/25/2017 10:16 AM, Adam Števko wrote:
Hello,
nice job!
Few questions:
1) Would you be able to incorporate vagrant box testing? 2) We could
try and get an instance for running this. Can this run inside the VM
reliably? I see KVM/Xen/VirtualBox host. Can you confirm these are
not your laptop, but dedicated hosts?
With this there is also a much more incentive to automate the
creation of images as tests can run. Is there a good CLI client/API
for managing openqa instance?
Cheers Adam
On Nov 24, 2017, at 9:28 PM, Michal Nowak <mno...@startmail.com>
wrote:
Hello,
last couple of months I have been working on automated test suite
for OpenIndiana. You can find it's dashboard at
https://openqa.oi.mnowak.cz
It makes use of openQA (http://open.qa/), an open source automated
test tool that makes possible to test the whole installation
process of operating system, OS configuration, regression and
sanity testing of individual components. openQA is currently used
by Fedora (https://openqa.fedoraproject.org/) and openSUSE
(https://openqa.opensuse.org/) for package updates and snapshot
release testing, respectively.
Test suite sources and guidance on how to setup your local openQA
instance are on GitHub
https://github.com/Mno-hime/os-autoinst-distri-openindiana/
Currently Live, Text, and Minimal variants of both ISO and USB
images are tested as VM guests on KVM, VirtualBox, and Xen.
Some of the test scenarios implemented:
* General installation via text- as well as GUI-installer to disk,
reboot to the installed system, OS setup, shutdown. -
installation_mate@64bit: https://openqa.oi.mnowak.cz/tests/1820 (on
KVM host) - installation_textmode_raidz@64bit:
https://openqa.oi.mnowak.cz/tests/1841 (on KVM host) -
installation_textmode_mbr@vbox64_ide_piix3:
https://openqa.oi.mnowak.cz/tests/1805 (on VirtualBox host) -
installation_mate@xen-hvm: https://openqa.oi.mnowak.cz/tests/1846
(on Xen host; note: illumos guest Xen drivers are in bad shape
atm)
* Upgrade scenarios for latest OpenIndiana snapshots to the latest
incorporation, e.g. upgrade_textmode_20161030:
https://openqa.oi.mnowak.cz/tests/1808
* illumos KVM virtualization test:
https://openqa.oi.mnowak.cz/tests/1796
* Extended `pkg` console tests:
https://openqa.oi.mnowak.cz/tests/1816
* Toolchain (pkg_src & oi-userland build) tests:
https://openqa.oi.mnowak.cz/tests/1815
* Build of OpenIndiana snapshot images from latest published
packages: https://openqa.oi.mnowak.cz/group_overview/2
* When things go wrong:
installation_textmode@vbox64_sas_LSILogicSAS:
https://openqa.oi.mnowak.cz/tests/1795
My test-host/worker - a 5 year old 4 core Ivy Bridge laptop with 12
GB of RAM, and SSD - is able to run 3 jobs in parallel reliably.
Full build validation takes about 8 hours. Bare metal server is
certainly desired.
I believe that this test suite could help us produce OpenIndiana
hipster snapshots more validated, with higher level of confidence
to particular updates, periodically run package's upstream
testsuite in real-life environment as opposed to a bare-bone build
environment, or release snapshots more often (if desired).
How to get involved: Deploy your local openQA instance in
accordance to the documentation at http://open.qa on a Linux host
(sic; I never deployed openQA on illumos as I don't have illumos
capable hardware), follow OpenIndiana-specific guide at
https://github.com/Mno-hime/os-autoinst-distri-openindiana/blob/master/README.md,
clone particular job from openqa.oi.mnowak.cz, see how your changes
affect the test run, and contribute the code back as a pull
request.
Similar test suite can be implemented for other illumos
distributions like OmniOS CE and SmartOS.
Feel free to contact me (mnowak_) on #illumos or #oi-dev.
Michal
_______________________________________________ oi-dev mailing
list oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev
_______________________________________________ oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev
_______________________________________________
oi-dev mailing list
oi-dev@openindiana.org
https://openindiana.org/mailman/listinfo/oi-dev
