Hi We have been trying to bring up a setup with devstack+opendaylight+vpp components for last few weeks without much success. Sorry for putting long e-mail here. Hope I could convey the problem that we are facing. Could anyone point us if there is any fundamental issue in our setup? Appreciate any inputs here and plz let us know if u need any specific logs.
1. For this setup, we are following the README instructions from the following devstack-vpp gerrit patch: https://git.opendaylight.org/gerrit/#/c/47512/14/demos/vpp-gbp-devstack/README.rst 2. We tried to bring up these components in different setups: a. Setup 1: i. 8 core, 24GB RAM, Ubuntu 16.04 ii. Used VirtualBox version 5.1 with Vagrant version 1.8.6 iii. Used Vagrant file (from above gerrit patch), brought up two CentOS 7.2 VMs: One for devstack-controller (7GB RAM, 2CPU) and one for devstack-compute-1(7GB RAM, 2CPU) iv. Installed VPP & HC 17.01 on both controller and compute nodes v. QEMU version on the nodes was 2.6.0 vi. Ran ODL with vpp renderer inside a 3rd VM (Ubuntu) b. Setup 2: i. Very similar to setup1, but on higher configuration servers and VMs: 40 core, 160GB RAM, Ubuntu 16.04 ii. Used VirtualBox version 5.1 with Vagrant version 1.8.6 iii. Used Vagrant file (from above gerrit patch), brought up two CentOS 7.2 VMs: One for devstack-controller (16GB RAM, 8CPU) and one for devstack-compute-1(8GB RAM, 8CPU) iv. Installed VPP & HC 17.01 on both controller and compute nodes v. QEMU version on the nodes was 2.6.0 vi. Ran ODL with vpp renderer inside a 3rd VM (Ubuntu) c. Setup 3: i. Very similar to serup 2, except Instead of using virtualized environment for devstack, installed devstack directly on the physical server. ii. Installed devstack-all-in-one configuration on the physical server with Ubuntu 16.04 iii. Installed VPP & HC 17.01 on the physical server iv. QEMU version on the physical server was 2.5.0 v. Huge pages were bumped from 512 to 8192 3. As specified in the README, we created a cerros image and tried it with different nova flavors (512MB, 1024MB, 2048MB RAM setting) 4. When we boot up nova instances, we have noticed different types of nova errors at different times in different setups. Some sample error logs from nova daemons: In all these setups, except in setup1, there is abundant amount of free RAM available on the VPP nodes. a. On Setup1 i. 512MB based flavor: ^[[01;31m2017-02-02 18:32:34.232 TRACE nova.compute.manager ^[[01;35m[instance: 343dd3bc-d4f6-471b-ba31-57460ebd79da] ^[[00mlibvirtError: Requested operation is not valid: cpu affinity is not supported ii. 1024MB based flavor: path=/dev/hugepages/libvirt/qemu,share=yes,size=1073741824,host-nodes=0,policy=bind: unable to map backing store for guest RAM: Cannot allocate memory b. On Setup2 i. 512MB based flavor: ^[[01;31m2017-02-13 19:42:06.184 TRACE nova.compute.manager ^[[01;35m[instance: df856ed7-5cdc-48b9-8392-9fd2dbeba353] ^[[00mlibvirtError: Requested operation is not valid: cpu affinity is not supported ii. 2048MB based flavor: ^[[01;31m2017-02-13 18:44:20.321 TRACE nova.compute.manager ^[[01;35m[instance: 86a38404-7abb-49ea-a6df-7696f16f479c] ^[[00m2017-02-13T18:44:19.952716Z qemu-kvm: -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu,share=yes,size=2147483648,host-nodes=0,policy=bind: unable to map backing store for guest RAM: Cannot allocate memory iii. 3072MB based flavor: 2017-02-13 19:55:58.310 ^[[01;35m[instance: b8fe63cb-72ef-410e-aad0-ac5cba955fb8] ^[[01;31mError from last host: devstack-compute-1 (node devstack-compute-1): [u'Traceback (most recent call last):\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1783,: Build of instance b8fe63cb-72ef-410e-aad0-ac5cba955fb8 was re-scheduled: Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology.\n']^[[00m c. On Setup3 (with huge pages bumped to 8192) i. 2048MB based flavor: 2017-02-14 15:17:07.270 ^[[01;35m[instance: 6653f474-2077-4422-9582-bc54d118b86a] ^[[01;31mError from last host: node.sri-pod10. (node node.sri-pod10.xos-pg0.wisc.cloudlab.us): [u'Traceback (most recent call last):\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1783,: Build of instance 6653f474-2077-4422-9582-bc54d118b86a was re-scheduled: Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology.\n']^[[00m Thanks Srikanth
_______________________________________________ vpp-dev mailing list [email protected] https://lists.fd.io/mailman/listinfo/vpp-dev
