On Fri, 15 Dec 2023 at 06:01, Philipp Hahn <h...@univention.de> wrote: > > Hello Stefan, > > thank you for your kind reply. > > Am 05.12.23 um 15:44 schrieb Stefan Hajnoczi: > > On Tue, 5 Dec 2023 at 04:53, Philipp Hahn <h...@univention.de> wrote: > > > >> by accident I stumbled over "VMware Instant Clone" ¹, which allows > >> cloning of running VMs by copy-on-write-sharing the disk images and > >> memory content; the network MAC address gets changed (or a different > >> bridge is used?). > >> I wonder if something similar can also be done with Qemu? My current > >> solution would be to: > >> - start and install the VM > >> - create a live-snapshot into the qcow2 file > >> - clone the disk image, e.g. put a qcow2 overlay on it per clone > >> - start and restore the clones from that live-snapshot > >> - put the clones in individual bridges and let the host do some network > >> address translation (NAT) to give each clone a unique external IP address. > >> > >> Has someone done something similar or is there even a better alternative? > >> > >> Background: our test suite currently provisions a set of multiple VMs, > >> which are dependent on each other. Provisioning them takes sometimes > >> many hours. After that the test suite runs inside these VMs and again > >> takes many hours. > >> I'd like to speed that up by parallelizing these tests, e.g. > >> 1. setup the VM environment once > >> 2. clone the VM environments as the resources allow > >> 3. distribute tests over these environments to run in parallel and to > >> allow running flaky tests multiple times from a clean clone again > > > > It would be simplest to use qcow2 backing files and boot each new > > instance from scratch. This involves setting up a master image and > > then "qemu-img create -f qcow2 -b master.img vm001.qcow2" to create > > the instance image. You may be able to use systemd or your distro's > > "first boot" functionality to recreate unique IDs and cryptographic > > keys when the new instance boots. > > Actually I do not want to modify the clones at all: While the machine ID > is probably less interesting to others, I can even live with re-using > the SSH keys as this is only for *internal* testing: I can tell `ssh` to > not check the keys as I can control all the networking, so security is > of little concern here. > > > If you really want to use a RAM snapshot then I suggest creating a > > qcow2 master image with the savevm command and using "cp > > --reflink=always master.qcow2 vm001.qcow2" to create an efficient copy > > of the qcow2 file. You'll need some custom scripts to recreate unique > > IDs and cryptographic keys inside the new instance after loadvm. > > Is there a major difference between doing a "savevm" to an external file > and doing a live snapshot, which stores the "savevm" inside the qcow2 > file itself. The later has the benefit for me, that I only have to > handle one file; I could even store it for later use if needed.
With the reflink approach you still snapshot the VM into the original qcow2 file (with the "savevm" command), not into an external file. The reflink creates an efficient copy of the file for each instance of the VM that you with to clone. But since you've said you don't want to modify the clones at all, maybe this approach is overkill because you have to manage these new qcow2 files. Have you tried the -snapshot command-line option? It creates a temporary qcow2 overlay that is discarded when QEMU exits. That allows the guest to write to the disk but those changes won't be permanent and you can run as many guests simultaneously as you want (each has its own temporary qcow2 overlay that is managed by QEMU behind the scenes). > > > My main problem currently is cloning the MAC address: As our product is > an operating system the MAC addresses of the involved systems is stored > in some databases; while in most cases they are not required, I do not > want to hunt for these in all kind of different locations and change > them to some cloned MAC address. > I already had a look at "Virtual Routing and Forwarding"², which allows > me to resue the same MAC addresses in different network bridge > interfaces, but what I did not yet get to work is the "routing" between > them. I found some very nice articles³⁴ on how to do NAT with VRF, but > it is not yet working. I'm not knowledgeable about VRF. You could also use -netdev user,hostfwd=tcp::$VM_SSH_NAT_PORT-:22 where VM_SSH_NAT_PORT is a unique port assigned by the script that launches the guest. That way each guest can have the same MAC address and IP address but receive incoming SSH connections. Stefan
