Hello Stefan,
thank you for your kind reply.
Am 05.12.23 um 15:44 schrieb Stefan Hajnoczi:
On Tue, 5 Dec 2023 at 04:53, Philipp Hahn <h...@univention.de> wrote:
>
by accident I stumbled over "VMware Instant Clone" ¹, which allows
cloning of running VMs by copy-on-write-sharing the disk images and
memory content; the network MAC address gets changed (or a different
bridge is used?).
I wonder if something similar can also be done with Qemu? My current
solution would be to:
- start and install the VM
- create a live-snapshot into the qcow2 file
- clone the disk image, e.g. put a qcow2 overlay on it per clone
- start and restore the clones from that live-snapshot
- put the clones in individual bridges and let the host do some network
address translation (NAT) to give each clone a unique external IP address.
Has someone done something similar or is there even a better alternative?
Background: our test suite currently provisions a set of multiple VMs,
which are dependent on each other. Provisioning them takes sometimes
many hours. After that the test suite runs inside these VMs and again
takes many hours.
I'd like to speed that up by parallelizing these tests, e.g.
1. setup the VM environment once
2. clone the VM environments as the resources allow
3. distribute tests over these environments to run in parallel and to
allow running flaky tests multiple times from a clean clone again
It would be simplest to use qcow2 backing files and boot each new
instance from scratch. This involves setting up a master image and
then "qemu-img create -f qcow2 -b master.img vm001.qcow2" to create
the instance image. You may be able to use systemd or your distro's
"first boot" functionality to recreate unique IDs and cryptographic
keys when the new instance boots.
Actually I do not want to modify the clones at all: While the machine ID
is probably less interesting to others, I can even live with re-using
the SSH keys as this is only for *internal* testing: I can tell `ssh` to
not check the keys as I can control all the networking, so security is
of little concern here.
If you really want to use a RAM snapshot then I suggest creating a
qcow2 master image with the savevm command and using "cp
--reflink=always master.qcow2 vm001.qcow2" to create an efficient copy
of the qcow2 file. You'll need some custom scripts to recreate unique
IDs and cryptographic keys inside the new instance after loadvm.
Is there a major difference between doing a "savevm" to an external file
and doing a live snapshot, which stores the "savevm" inside the qcow2
file itself. The later has the benefit for me, that I only have to
handle one file; I could even store it for later use if needed.
My main problem currently is cloning the MAC address: As our product is
an operating system the MAC addresses of the involved systems is stored
in some databases; while in most cases they are not required, I do not
want to hunt for these in all kind of different locations and change
them to some cloned MAC address.
I already had a look at "Virtual Routing and Forwarding"², which allows
me to resue the same MAC addresses in different network bridge
interfaces, but what I did not yet get to work is the "routing" between
them. I found some very nice articles³⁴ on how to do NAT with VRF, but
it is not yet working.
Philipp
¹<https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-853B1E2B-76CE-4240-A654-3806912820EB.html>
²<https://www.kernel.org/doc/html/latest/networking/vrf.html>
³<https://blog.oddbit.com/post/2023-02-19-vrf-and-nat/>
⁴<https://www.linuxquestions.org/questions/linux-newbie-8/iptables-nat-for-vrf-4175721876/>
