Hi guys, I am trying to set up an openstack small cluster and I am facing a problem. I have two nodes, a controller node on which I have installed all the nova-* services and a compute node on which I have installed nova-compute and nova-network.
When I run 'nova-manage service list', I only see the nova-compute service from the compute-node, not the other services from the controller node. However, if I run 'ps aux | grep nova-' on the controller node, all the nova-* are up and running. It seems that my controller's nova-* services do not want to register to the mysql DB. My guess is that services are not able to connect to the local mysql db on the controller node may be due to iptables bad settings but I am not able to figure it out. Worth to mention it, my nova-controller node is running in a virtual machine on virtual box, both nodes are ubuntu natty server amd64. If someone have any idea on that problem, please let me know, I added some more information about my configuration in attachment below. Than you in advance. Best Regards, Ivan Frain HERE IS MY NOVA.CONF FILE: IT IS THE SAME ON BOTH MACHINES: --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --verbose --sql_connection=mysql://nova:notnova@nova-controller/nova --s3_host=nova-controller --rabbit_host=nova-controller --ec2_api=nova-controller --ec2_url=http://nova-controller:8773/services/Cloud --fixed_range=192.168.0.0/16 --network_size=8 --routing_source_ip=nova-controller THE OUTPUT OF PS AUX | GREP NOVA COMMAND: root@nova-controller:~# ps aux | grep nova rabbitmq 1118 0.9 5.3 53644 27224 ? Ssl 21:06 0:47 /usr/lib/erlang/erts-5.7.4/bin/beam -W w -K true -A30 -P 1048576 -- -root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@nova-controller -boot /var/lib/rabbitmq/mnesia/rabbit@nova-controller-plugins-expand/rabbit -kernel inet_default_listen_options [{nodelay,true}] -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -kernel error_logger {file,"/var/log/rabbitmq/[email protected]"} -sasl sasl_error_logger {file,"/var/log/rabbitmq/[email protected]"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@nova-controller" -noshell -noinput root 19856 0.1 3.2 61228 16232 ? S 22:24 0:00 /usr/bin/python /usr/bin/nova-objectstore --uid 108 --gid 65534 --pidfile /var/run/nova/nova-objectstore.pid --flagfile=/etc/nova/nova.conf --nodaemon --logfile=/var/log/nova/nova-objectstore.log nova 23074 0.0 0.2 35708 1280 ? Ss 22:26 0:00 su -c nova-compute --flagfile=/etc/nova/nova.conf nova nova 23075 30.3 8.2 145228 41740 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-compute --flagfile=/etc/nova/nova.conf nova 23132 0.0 0.2 35708 1276 ? Ss 22:26 0:00 su -c nova-api --flagfile=/etc/nova/nova.conf nova nova 23133 35.0 5.3 89264 26884 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-api --flagfile=/etc/nova/nova.conf nova 23145 0.0 0.2 35708 1280 ? Ss 22:26 0:00 su -c nova-network --flagfile=/etc/nova/nova.conf nova nova 23146 23.0 4.2 75476 21596 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-network --flagfile=/etc/nova/nova.conf nova 23152 0.0 0.2 35708 1280 ? Ss 22:26 0:00 su -c nova-scheduler --flagfile=/etc/nova/nova.conf nova nova 23153 0.0 3.7 72924 19112 ? R 22:26 0:00 /usr/bin/python /usr/bin/nova-scheduler --flagfile=/etc/nova/nova.conf root 23167 0.0 0.2 9136 1060 pts/1 S+ 22:26 0:00 grep --color=auto nova THE OUTPUT OF SERVICE-MANAGE COMMAND: root@nova-controller:~# nova-manage service list Obelix nova-compute enabled :-) 2011-08-08 20:27:26 NB: Obelix is the name of my compute node. MYSQL> SELECT HOST,USER,PASSWORD FROM USER; +-----------------+------------------+-------------------------------------------+ | Host | User | Password | +-----------------+------------------+-------------------------------------------+ | localhost | root | *0BE3B501084D35F4C66DD3AC4569EAE5EA738212 | | nova-controller | root | *0BE3B501084D35F4C66DD3AC4569EAE5EA738212 | | 127.0.0.1 | root | *0BE3B501084D35F4C66DD3AC4569EAE5EA738212 | | localhost | debian-sys-maint | *B23864E8C293F7A6E977A8CF31AC4ED99570A161 | | % | nova | *8134721C1A2B651300B26AC135712CD373E4D471 | +-----------------+------------------+-------------------------------------------+ A PROBLEM IN THE OUTPUT FROM NOVA-NETWORK LOG (IPTABLES PROBLEM): root@nova-controller:~# tail -n 50 /var/log/nova/nova-network.log 2011-08-08 19:34:11,852 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "apply"... from (pid=10073) inner /usr/lib/pymodules/python2.7/nova/utils.py:599 2011-08-08 19:34:11,854 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t filter from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:11,870 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:11,911 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t nat from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:11,929 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:11,963 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166 2011-08-08 19:34:11,963 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179 2011-08-08 19:34:12,776 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:12,802 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166 2011-08-08 19:34:12,803 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179 2011-08-08 19:34:14,616 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:14,641 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166 2011-08-08 19:34:14,642 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179 2011-08-08 19:34:15,723 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:15,742 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166 2011-08-08 19:34:15,742 DEBUG nova.utils [-] ['sudo', 'iptables-restore'] failed. Retrying. from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:179 2011-08-08 19:34:16,703 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:150 2011-08-08 19:34:16,732 DEBUG nova.utils [-] Result was 2 from (pid=10073) execute /usr/lib/pymodules/python2.7/nova/utils.py:166 2011-08-08 19:34:16,732 CRITICAL nova [-] Unexpected error while running command. Command: sudo iptables-restore Exit code: 2 Stdout: '' Stderr: 'iptables-restore v1.4.10: Bad IP address "nova"\n\nError occurred at line: 18\nTry `iptables-restore -h\' or \'iptables-restore --help\' for more information.\n' (nova): TRACE: Traceback (most recent call last): (nova): TRACE: File "/usr/bin/nova-network", line 48, in (nova): TRACE: service.serve() (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/service.py", line 290, in serve (nova): TRACE: x.start() (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/service.py", line 84, in start (nova): TRACE: self.manager.init_host() (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/manager.py", line 499, in init_host (nova): TRACE: super(VlanManager, self).init_host() (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/manager.py", line 128, in init_host (nova): TRACE: self.driver.init_host() (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/linux_net.py", line 378, in init_host (nova): TRACE: iptables_manager.apply() (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/utils.py", line 607, in inner (nova): TRACE: retval = f(*args, **kwargs) (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/linux_net.py", line 292, in apply (nova): TRACE: attempts=5) (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/network/linux_net.py", line 666, in _execute (nova): TRACE: return utils.execute(*cmd, **kwargs) (nova): TRACE: File "/usr/lib/pymodules/python2.7/nova/utils.py", line 173, in execute (nova): TRACE: cmd=' '.join(cmd)) (nova): TRACE: ProcessExecutionError: Unexpected error while running command. (nova): TRACE: Command: sudo iptables-restore (nova): TRACE: Exit code: 2 (nova): TRACE: Stdout: '' (nova): TRACE: Stderr: 'iptables-restore v1.4.10: Bad IP address "nova"\n\nError occurred at line: 18\nTry `iptables-restore -h\' or \'iptables-restore --help\' for more information.\n' (nova): TRACE: A PROBLEM OF DB CONNECTION IN NOVA COMPUTE LOG FROM THE CONTROLLER: (nova): TRACE: ProgrammingError: (ProgrammingError) (1146, "Table 'nova.instances' doesn't exist") 'SELECT instances.created_at AS instances_created_at, [......] ting_ips_1.deleted = %s \nWHERE instances.host = %s AND instances.deleted = %s' (False, False, False, False, False, 'nova-controller', False) (nova): TRACE: IFCONFIG AND IPTABLES OF THE CONTROLLER NODE: root@nova-controller:~# ifconfig br100 Link encap:Ethernet HWaddr 08:00:27:3d:e9:7f inet addr:192.168.1.4 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fe3d:e97f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:352069 errors:0 dropped:0 overruns:0 frame:0 TX packets:178694 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:28108787 (28.1 MB) TX bytes:18241764 (18.2 MB) eth0 Link encap:Ethernet HWaddr 08:00:27:3d:e9:7f inet6 addr: fe80::a00:27ff:fe3d:e97f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:352848 errors:0 dropped:0 overruns:0 frame:0 TX packets:179157 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:33117855 (33.1 MB) TX bytes:18271426 (18.2 MB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:498 errors:0 dropped:0 overruns:0 frame:0 TX packets:498 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:108091 (108.0 KB) TX bytes:108091 (108.0 KB) virbr0 Link encap:Ethernet HWaddr 22:0f:89:53:eb:82 inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) root@nova-controller:~# iptables --list Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT tcp -- anywhere anywhere tcp dpt:bootps Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere 192.168.122.0/24 state RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere ACCEPT all -- anywhere anywhere REJECT all -- anywhere anywhere reject-with icmp-port-unreachable REJECT all -- anywhere anywhere reject-with icmp-port-unreachable Chain OUTPUT (policy ACCEPT) target prot opt source destination
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
