Re: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM
Hi, In order for the VM to be booted the image needs to be on a datastore accessible by the host. By default the data tore will not have the image. This is copied from glance tot he datastore. This is most probably where the problem is. This may take a while depending on the connectivity between the openstack setup and your backbend datastore. Once you have done this you will see a directory on the datastore called vmware_base. This will contain that image. From then on it should be smooth sailing. Please note that we are working on a number of things to improve this: 1. Image cache aging (blueprint is implemented and pending review) 2. Adding a Vmware glance datastore – which will greatly improve the copy process described above Thanks Gary From: Ray Sun xiaoq...@gmail.commailto:xiaoq...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Wednesday, January 8, 2014 4:30 AM To: OpenStack Dev openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM Stackers, I tried to create a new VM using the driver VMwareVCDriver, but I found it's very slow when I try to create a new VM, for example, 7GB Windows Image spent 3 hours. Then I tried to use curl to upload a iso to vcenter directly. curl -H Expect: -v --insecure --upload-file windows2012_server_cn_x64.iso https://administrator:root123.@200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath=dataCenterdsName=datastore2https://urldefense.proofpoint.com/v1/url?u=https://administrator:root123.%40200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath%3DdataCenter%26dsName%3Ddatastore2k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=lhA2fUha%2FtHWjSl8QcZq5lQ9MSETFSwBcyKMNtXnhx0%3D%0As=e4e18c64b88329d7c54cdaef299e186aa13f2dd63f37654a0171a70d70b42cd3 The average speed is 0.8 MB/s. Finally, I tried to use vSpere web client to upload it, it's only 250 KB/s. I am not sure if there any special configurations for web interface for vcenter. Please help. Best Regards -- Ray ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM
Gary, Thanks. Curretly, our upload speed is in the normal range? Best Regards -- Ray On Wed, Jan 8, 2014 at 4:31 PM, Gary Kotton gkot...@vmware.com wrote: Hi, In order for the VM to be booted the image needs to be on a datastore accessible by the host. By default the data tore will not have the image. This is copied from glance tot he datastore. This is most probably where the problem is. This may take a while depending on the connectivity between the openstack setup and your backbend datastore. Once you have done this you will see a directory on the datastore called vmware_base. This will contain that image. From then on it should be smooth sailing. Please note that we are working on a number of things to improve this: 1. Image cache aging (blueprint is implemented and pending review) 2. Adding a Vmware glance datastore – which will greatly improve the copy process described above Thanks Gary From: Ray Sun xiaoq...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Wednesday, January 8, 2014 4:30 AM To: OpenStack Dev openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM Stackers, I tried to create a new VM using the driver VMwareVCDriver, but I found it's very slow when I try to create a new VM, for example, 7GB Windows Image spent 3 hours. Then I tried to use curl to upload a iso to vcenter directly. curl -H Expect: -v --insecure --upload-file windows2012_server_cn_x64.iso https://administrator:root123.@200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath=dataCenterdsName=datastore2https://urldefense.proofpoint.com/v1/url?u=https://administrator:root123.%40200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath%3DdataCenter%26dsName%3Ddatastore2k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=lhA2fUha%2FtHWjSl8QcZq5lQ9MSETFSwBcyKMNtXnhx0%3D%0As=e4e18c64b88329d7c54cdaef299e186aa13f2dd63f37654a0171a70d70b42cd3 The average speed is 0.8 MB/s. Finally, I tried to use vSpere web client to upload it, it's only 250 KB/s. I am not sure if there any special configurations for web interface for vcenter. Please help. Best Regards -- Ray ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM
From: Ray Sun xiaoq...@gmail.commailto:xiaoq...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Wednesday, January 8, 2014 4:09 PM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM Gary, Thanks. Curretly, our upload speed is in the normal range? [Gary] #hrs for a 7G file is far too long. For testing I have a 1G image. This takes about 2 minutes to upload to the cache. Please note that I am running on a virtual setup so thing take far longer than they would if it was bare metal Best Regards -- Ray On Wed, Jan 8, 2014 at 4:31 PM, Gary Kotton gkot...@vmware.commailto:gkot...@vmware.com wrote: Hi, In order for the VM to be booted the image needs to be on a datastore accessible by the host. By default the data tore will not have the image. This is copied from glance tot he datastore. This is most probably where the problem is. This may take a while depending on the connectivity between the openstack setup and your backbend datastore. Once you have done this you will see a directory on the datastore called vmware_base. This will contain that image. From then on it should be smooth sailing. Please note that we are working on a number of things to improve this: 1. Image cache aging (blueprint is implemented and pending review) 2. Adding a Vmware glance datastore – which will greatly improve the copy process described above Thanks Gary From: Ray Sun xiaoq...@gmail.commailto:xiaoq...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Wednesday, January 8, 2014 4:30 AM To: OpenStack Dev openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM Stackers, I tried to create a new VM using the driver VMwareVCDriver, but I found it's very slow when I try to create a new VM, for example, 7GB Windows Image spent 3 hours. Then I tried to use curl to upload a iso to vcenter directly. curl -H Expect: -v --insecure --upload-file windows2012_server_cn_x64.iso https://administrator:root123.@200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath=dataCenterdsName=datastore2https://urldefense.proofpoint.com/v1/url?u=https://administrator:root123.%40200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath%3DdataCenter%26dsName%3Ddatastore2k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=lhA2fUha%2FtHWjSl8QcZq5lQ9MSETFSwBcyKMNtXnhx0%3D%0As=e4e18c64b88329d7c54cdaef299e186aa13f2dd63f37654a0171a70d70b42cd3 The average speed is 0.8 MB/s. Finally, I tried to use vSpere web client to upload it, it's only 250 KB/s. I am not sure if there any special configurations for web interface for vcenter. Please help. Best Regards -- Ray ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-devhttps://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-devk=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=9a6QiCBNYRRJYDjh9SisIB0JUGIlDmd5rmicgbmlEVw%3D%0As=2323d9a1eee95934422fcce0089a81480b96a8acf29badb57e47d81fda353394 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM
On 01/07/2014 06:30 PM, Ray Sun wrote: Stackers, I tried to create a new VM using the driver VMwareVCDriver, but I found it's very slow when I try to create a new VM, for example, 7GB Windows Image spent 3 hours. Then I tried to use curl to upload a iso to vcenter directly. curl -H Expect: -v --insecure --upload-file windows2012_server_cn_x64.iso https://administrator:root123.@200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath=dataCenterdsName=datastore2; The average speed is 0.8 MB/s. Finally, I tried to use vSpere web client to upload it, it's only 250 KB/s. I am not sure if there any special configurations for web interface for vcenter. Please help. I'm not fully versed in the plumbing, but while you are pushing via curl to 200.21.0.99 you might check the netstat statistics at the sending side, say once a minute, and see what the TCP retransmission rate happens to be. If 200.21.0.99 has to push the bits to somewhere else you should follow that trail back to the point of origin, checking statistics on each node as you go. You could, additionally, try running the likes of netperf (or iperf, but I have a natural inclination to suggest netperf...) between the same pairs of systems. If netperf gets significantly better performance then you (probably) have an issue at the application layer rather than in the networking. Depending on how things go with those, it may be desirable to get a packet trace of the upload via the likes of tcpdump. It will be very much desirable to start the packet trace before the upload so you can capture the TCP connection establishment packets (aka the TCP SYNchronize segments) as those contain some important pieces of information about the capabilities of the connection. rick jones ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM
Rick, Thanks for your response. I make another testing to upload my iso to my ESXi host directly, the speed is much faster now, avg is more than 40 MB/s. By the way, 200.21.0.99 is my vcenter server. I will keep you update if I have some new found. Thanks a lot. Best Regards -- Ray On Thu, Jan 9, 2014 at 1:09 AM, Rick Jones rick.jon...@hp.com wrote: On 01/07/2014 06:30 PM, Ray Sun wrote: Stackers, I tried to create a new VM using the driver VMwareVCDriver, but I found it's very slow when I try to create a new VM, for example, 7GB Windows Image spent 3 hours. Then I tried to use curl to upload a iso to vcenter directly. curl -H Expect: -v --insecure --upload-file windows2012_server_cn_x64.iso https://administrator:root123.@200.21.0.99/folder/ iso/windows2012_server_cn_x64.iso?dcPath=dataCenterdsName=datastore2 The average speed is 0.8 MB/s. Finally, I tried to use vSpere web client to upload it, it's only 250 KB/s. I am not sure if there any special configurations for web interface for vcenter. Please help. I'm not fully versed in the plumbing, but while you are pushing via curl to 200.21.0.99 you might check the netstat statistics at the sending side, say once a minute, and see what the TCP retransmission rate happens to be. If 200.21.0.99 has to push the bits to somewhere else you should follow that trail back to the point of origin, checking statistics on each node as you go. You could, additionally, try running the likes of netperf (or iperf, but I have a natural inclination to suggest netperf...) between the same pairs of systems. If netperf gets significantly better performance then you (probably) have an issue at the application layer rather than in the networking. Depending on how things go with those, it may be desirable to get a packet trace of the upload via the likes of tcpdump. It will be very much desirable to start the packet trace before the upload so you can capture the TCP connection establishment packets (aka the TCP SYNchronize segments) as those contain some important pieces of information about the capabilities of the connection. rick jones ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova][Vmware]Bad Performance when creating a new VM
Stackers, I tried to create a new VM using the driver VMwareVCDriver, but I found it's very slow when I try to create a new VM, for example, 7GB Windows Image spent 3 hours. Then I tried to use curl to upload a iso to vcenter directly. curl -H Expect: -v --insecure --upload-file windows2012_server_cn_x64.iso https://administrator:root123.@200.21.0.99/folder/iso/windows2012_server_cn_x64.iso?dcPath=dataCenterdsName=datastore2 The average speed is 0.8 MB/s. Finally, I tried to use vSpere web client to upload it, it's only 250 KB/s. I am not sure if there any special configurations for web interface for vcenter. Please help. Best Regards -- Ray ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
