I intend to change the NVMM API in order to reduce the data movements. The patches are available here [1].
Basically until now the API expected the user to create structures and then pass them in a libnvmm function. Typically: /* The structures. */ struct nvmm_machine mach; struct nvmm_x64_state state; struct nvmm_event event; struct nvmm_exit exit; /* Create the machine and VCPU0. */ nvmm_machine_create(&mach); nvmm_vcpu_create(&mach, 0); /* Set RAX=123. */ nvmm_vcpu_getstate(&mach, 0, &state, NVMM_X64_STATE_GPRS); state.gprs[NVMM_X64_GPR_RAX] = 123; nvmm_vcpu_setstate(&mach, 0, &state, NVMM_X64_STATE_GPRS); /* Inject an #UD exception. */ event.type = NVMM_EXCEPTION; event.vector = 6; nvmm_vcpu_inject(&mach, 0, &event); /* Run. */ nvmm_vcpu_run(&mach, 0, &exit); switch (exit.reason) { ... } This design has some problems: the structures can occupy a lot of stack (nvmm_x64_state is > 1000 bytes), and must be copied in and out of the comm page by libnvmm. We can avoid this by giving the user direct access to the comm page. A new nvmm_vcpu structure is introduced, and contains pointers to the comm page. The new usage is: /* The structures. */ struct nvmm_machine mach; struct nvmm_vcpu vcpu; /* Create the machine and VCPU0. */ nvmm_machine_create(&mach); nvmm_vcpu_create(&mach, 0, &vcpu); /* Set RAX=123. getstate populates vcpu.state->gprs[]. */ nvmm_vcpu_getstate(&mach, &vcpu, NVMM_X64_STATE_GPRS); vcpu.state->gprs[NVMM_X64_GPR_RAX] = 123; nvmm_vcpu_setstate(&mach, &vcpu, NVMM_X64_STATE_GPRS); /* Inject an #UD exception. */ vcpu.event->type = NVMM_EXCEPTION; vcpu.event->vector = 6; nvmm_vcpu_inject(&mach, &vcpu); /* Run. */ nvmm_vcpu_run(&mach, &vcpu); switch (vcpu.exit->reason) { ... } No kernel changes are required. This should be part of NetBSD 9. [1] https://m00nbsd.net/garbage/nvmm/nvmm-nocopy.zip