I wanted to let folks know the ESX provisioning module should be finished, or at least in a beta form. I have closed the JIRA ticket (VCL-29) corresponding to the creation of this module. Here is a review of the new functionality:

Manage multiple ESX or ESX 3i hypervisors
Deploys virtual machines onto these hypervisors
Supports virtual machine capture routing to allow for updating a VM image, and the creation of "new" derivative images

One last improvement will include refactoring the way our module gathers the private IP address of the VM that is being provisioned. The change include deprecating the "watching of the arp table" in favor of monitoring the dhcpd.leases file.

While VCL now supports ESX/ESX 3i netboot based virtual machines, the VCL architecture presents a lot of real challenges for making this a scalable solution, and I'd like to identify and discuss a few points/ concerns here.

VCL requires an entry in the computers table for each VM, and this entry needs to be tied to a vmhost. By hard selecting the virtual machine entries in the computer table (a VM "slot") up front, the decision about where to place the next virtual machine isn't handled effectively. Each VM "slot" gets statically assigned physical characteristics at creation time. This very quickly creates a situation where there is space in the datacenter for a particular VM on one hypervisor or another, but VCL can't figure it out because the large RAM slots got used first for other images, and now the image in question can't find a slot to meet it's meta-data requirements. VCL will incorrectly report that there is not space in the infrastructure, when there really is. This is bad.

Also, it makes the setup much more difficult since an average, modern blade can run 20 - 30 VMs, and if you manage an entire blade center you're manually creating 350 computer table entries. Each entry requires multiple, manual updates to the database. This is not a tractable solution, and needs to be addressed.

As a possible solution (or part of one), one major moving part is the placement decision for a particular reservation (tantamount to which hypervisor this VM will be reserved on). Placement today in VCL is decided in the front-end without asking the hypervisors what their capabilities are of accepting the next VM (or cluster of VMs) in question. One possible way around this is to create a ESX placement controller module which determines where to place things. This module would be part of the VCL backend (although it could be called from the frontend). The ESX provisioning module authors each VMX file on the fly based on meta-data from the database, so it is already dynamic enough.


Brian Bouterse
Secure Open Systems Initiative

Reply via email to