Ok, we have done a few things to make that process much faster. Add in a lots of memory in Compute Node. That helps because the reading speed of HDD is extremely slow. But if node has large DRMA, Linux is smart to read large trunk of file into DRAM.
In addition, we use network multicast, instead of N times p2p reading from image resource, to distribute the image to ALL nodes simultaneously. So, the provisioning time is not growing with number of node linearly anymore. We have a plan to use Software RDMA instead of TCP, later. With these approaches and 10gbe network, we see potential to boot 1000vms, with 3G+ image size, in 60s Code is not production read. We are trying them in larger cluster. BR Leon -----Original Message----- From: Turbo Fredriksson [mailto:[email protected]] Sent: Wednesday, September 28, 2016 3:58 PM To: Leon Liu Cc: [email protected] Subject: Re: [Openstack] Provisioning VMs in Openstack cluster On Sep 28, 2016, at 8:34 PM, Leon Liu wrote: > We are trying to provision a few VM images in a small Openstack > cluster. Our image is about 3GB size. It has taken six minutes to > provision just three VMs, one on each host; and it grows linearly. > Now, we are wondering it may take 200 minutes for 100 VMs to be ready. That would be slow. I have the same experience, if that helps :(. And my images is not any where near that size, only 463MB! Using Heat, creating a volume for each instance on a remote SAN, 'takes ages' (haven't clocked it, but creating eight instances takes well over half an hour)! I'm "fairly" certain that it's not the network - I have four trunked 1Gbps links directly down to my storage (and I've triple checked that that is used). I've had other things on my mind and I'm not creating a lot of instances now that most of them have been created, but I've always intended to go back to this problem so I'm going to follow this thread closely. -- Ehhhhm - The battle cry of the cronical masturbater. - Charlie Harper _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
