VIRTUAL TEST FRAMEWORK
The Virtual Test Framework (VTF) establishes a completely virtual
environment for testing software. It consists of the following key
components (see Figure 1):
o host system
o internal virtual switch used to establish a virtual subnet
o virtual router used to connect virtual subnet to host network
o virtual NICs used by guests to connect to virtual subnet
o guest operating system instances
o virtual consoles
o virtual test framework software
external network (optional)
x
|
:-----------------------------x-------------------------------:
| [nic] |
| |
| Host |
| |
| [vrouter] |
| :--------------------------x------------ -------------: |
| | vswitch ... | |
| :------x------------x------------x------ ------x------: |
| | | | | |
| :----x----: :----x----: :----x----: :----x----: |
| | [vnic1] | | [vnic2] | | [vnic3] | | [vnicN] | |
| | | | | | | ... | | |
| | Guest1 | | Guest2 | | Guest3 | | GuestN | |
| | | | | | | | | |
| :----x----: :----x----: :----x----: :----x----: |
| | | | | |
| vconsol1 vconsol2 vconsol3 vconsolN |
| |
:-------------------------------------------------------------:
Figure 1. Virtual Test Framework
Using the Virtual Test Framework, a host can interact with one or more
guest operating system instances including client, server configurations
to test system, component and application level software. In addition,
the testing can be performed with internal only or external networking
enabled. Figures 2-4 show some of the possible VTF configurations.
x
|
:---------x---------:
| |
| Host |
| |
| :------x------: |
| :------x------: |
| | |
| :----x----: |
| | | |
| | Guest | |
| | | |
| :---------: |
:-------------------:
Figure 2. Single Guest Configuration
x
|
:----------------------x----------------------:
| |
| Host |
| |
| :-------------------x-------------------: |
| :------x------------x------------x------: |
| | | | |
| :----x----: :----x----: :----x----: |
| | | | | | | |
| | Install | | Install | | Install | |
| | Server | | Client | | Client | |
| :---------: :---------: :---------: |
:---------------------------------------------:
Figure 3. Install Test Configuration
:---------------------------------------------:
| |
| Host |
| |
| :-------------------x-------------------: |
| :------x------------x------------x------: |
| | | | |
| :----x----: :----x----: :----x----: |
| | | | | | | |
| |Benchmark| |Benchmark| |Benchmark| |
| | Server | | Client | | Client | |
| :---------: :---------: :---------: |
:---------------------------------------------:
Figure 4. Benchmark Test Configuration (internal only)
The Virtual Test Framework can be implemented using various virtual machine
technologies including, Sun VirtualBox, Solaris LDOMs, Solaris xVM and Solaris
Containers, VMware, etc... In addition, elements of this framework can be
applied to non-test virtual machine applications and deployments.
To focus the discussion, the rest of this paper will concentrate on the
VirtualBox implementation, or the VirtualBox Test Framework as it applies
to testing the Solaris Image Packaging System (IPS).
VIRTUALBOX TEST FRAMEWORK
The VirtualBox Test Framework[1] (VBT) consist of the following key components,
1) Crossbow[2] is used to establish the virtual subnet and router on the
host and the virutal NICs for the guests, 2) VirtualBox[3] and the VBoxManage
command is used to configure and manage the guest instances, 3) the ssh and
scp commands are used to provide the primary communication interface to the
guests, 4) a serial console interface using sockets is available for console
access, guest network configuration and console logging, 5) ZFS[4] is used to
manage the Virtual Disk Images (VDI) and provide an efficient and robust
snapshot interface, 6) the VBT software. The following list summarizes some
of the key features:
o easy-to-use software interface
o direct (sequential) and distributed (parallel) modes of operation
o auto VDI image configuration
o fully virtualized networking using crossbow
o active network detection and configuration
o network and serial console guest interfaces
o can be used on virtually any type of hardware (servers, laptops)
o zfs snapshots to preserve VDI images and manage multiple checkpoints
o quality of service customizable at the virtual switch or VNIC level
VBT HOST INTERFACE
vbtHost.setup()
This function detects the current active host network and sets up a virtual
subnet using Crossbow on the host machine consisting of a virtual switch
(etherstub) and virtual router NIC used to route IP traffic to the host
network. Virtual NICs may than be added to the virtual subnet for each
guest. In addition, NAT mapping can be established using ipfilter so guests
can access the external network as needed.
vbtHost.teardown()
This function removes the virtual router NIC and virtual subnet and returns
the host to it's initial state.
VBT GUEST INTERFACE
gobj.vbtGuest.setup()
This function creates a virtual NIC and configures a guest. New
guest VDI images are automatically registered.
gobj.vbtGuest.teardown()
This function removes a guest virtual NIC and restores a guest VDI
image to it's initial state.
gobj.vbtGuest.start()
This function takes a snapshot of a guest VDI image and starts the
guest, then it configures the network and console interfaces and tests
them to ensure they can be used. The console log is also started.
gobj.vbtGuest.stop()
This function performs an ACPI power button shutdown of a guest and
by default returns the VDI image to the state prior to guest startup
by rolling back the initial snapshot.
gobj.vbtGuest.ssh(cmd)
This function performs a single command line on a guest using the ssh command.
gobj.vbtGuest.reboot()
This function reboots a guest and ensures the network and console interfaces
are re-established.
gobj.vbtGuest.snapshot(snap)
This function puts a guest into a safe state and takes a snapshot of the VDI
image. This may require the guest to be powered down and restarted.
gobj.vbtGuest.rollback(snap)
This function puts a guest into a safe state and rolls back the VDI image to
a prior snapshot state.
gobj.vbtGuest.scp(file)
This function copies a file to a guest using the secure copy command.
gobj.vbtGuest.session_start()
This function logs into a guest and starts a ssh session.
gobj.vbtGuest.session_cmd()
This function performs a command on a guest within a ssh session.
gobj.vbtGuest.session_end()
This function ends an active guest ssh session.
gobj.vbtGuest.console()
This function performs a command on the guest using the console interface.
EXAMPLE VBT INTERFACE CALL SEQUENCE
The following shows a basic VBT interface call sequence where the host
system is setup, a guest is setup, test cases on the guest are performed,
the guest VDI image is restored to it's initial state and the host is
restored to it's initial state.
vbtHost.setup()
vbtGuest.setup()
-------------------
**TestCase1**
vbtGuest.start()
vbtGuest.ssh()
...
vbtGuest.stop()
-------------------
**TestCase2**
vbtGuest.start()
vbtGuest.ssh()
...
vbtGuest.stop()
-------------------
vbtGuest.teardown()
vbtHost.teardown()
VBT OPERATIONAL MODES
1) Direct (Sequential) Mode
In direct (sequential) mode, the host performs discrete operations on
the guests using the ssh, scp and session interfaces. This centralizes
control on the host system and may make it easier to track failures,
but is limited by it's sequential nature and the execution load on the
host.
Example:
...
osdevbuild.vbtGuest.ssh("pfexec pkg list")
osdevbuild.vbtGuest.ssh("pfexec pkg image-update")
osdevbuild.vbtGuest.reboot()
osdevbuild.vbtGuest.ssh("pfexec pkg list")
verify steps
...
2) Distributed (Parallel) Mode
In distributed (parallel) mode, the host may setup a guest then
initiate a program on the guest whereby the guest operates for
the most part independent of the host. This is especially useful
for client server testing.
Example:
..
osdevbuild.vbtGuest.scp("testsuitefiles")
osdevbuild.vbtGuest.ssh("runtestsuite")
..
VBT DATA COMPONENTS
The VirtualBox configuration files contain the global and machine specific
VirtualBox data. They are created automatically if they do not exist.
These files are user centric. Below is an example file structure for
the root user:
/root/
`-- .VirtualBox/
|-- VirtualBox.xml
|-- compreg.dat
`-- xpti.dat
The VirtualBox HardDisk directory contains the VDI images. In the
VirtualBox Test Framework each VDI image is stored in a separate
ZFS file system. This allows ZFS snapshots to be used instead of
the native VirtualBox snapshots, since ZFS provides a more robust
snapshot and file system interface. To add a new VDI image for use
in testing, a new ZFS file system is created and the VDI image is
located within it. In addition, a ZFS file system is used to maintain
the VirtualBox Machines directory which contains the VirtualBox
machine configuration file and VirtualBox log files. A standard
naming convention can be used to organize the file system and files.
The following shows an example HardDisk and Machines file system
structure. ZFS file systems are indicated using "(fs)".
/rpool/vbt (fs)
`-- VirtualBox (fs)
|-- HardDisks (fs)
| |-- osol-dev-125 (fs)
| | |-- osol-dev-125.vdi
| | |-- osol-dev-...@snap200911130924001
| | `-- osol-dev-...@tc30_200911130630001
| `-- osol-rel-0906 (fs)
| `-- osol-rel-0906.vdi
`-- Machines (fs)
|-- osol-dev-125
| |-- Logs
| | |-- VBox.log
| | `-- VBox.log.1
| `-- osol-dev-125.xml
`-- osol-rel-0906
`-- osol-rel-0906.xml
===========================================================================
Notes
[1] This is based on code and testing methods developed by Sean McGrath and
other members of the Sun Ireland Solaris Performance Pre-Integration
Testing organization, and Dave Marker, of the Sun Solaris Kernel
organization.
[2] The OpenSolaris Crossbow community and documentation is located at:
http://hub.opensolaris.org/bin/view/Project+crossbow/
[3] The Sun VirtualBox website and documentation is located:
http://www.virtualbox.org/
http://www.virtualbox.org/wiki/Documentation
[4] The OpenSolaris ZFS community and documentation is located at:
http://hub.opensolaris.org/bin/view/Community+Group+zfs/
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
