Re: [ovirt-users] AIO setup problem with master failes due to nfs
On Thu, Aug 21, 2014 at 6:43 PM, Sandro Bonazzola sbona...@redhat.com wrote: Il 20/08/2014 09:17, Gianluca Cecchi ha scritto: On Wed, Aug 20, 2014 at 9:06 AM, Gianluca Cecchi gianluca.cec...@gmail.com mailto:gianluca.cec...@gmail.com wrote: result = vdscli.connect().getVdsCapabilities() File /usr/lib64/python2.6/xmlrpclib.py, line 1199, in __call__ return self.__send(self.__name, args) File /usr/lib64/python2.6/xmlrpclib.py, line 1489, in __request verbose=self.__verbose File /usr/lib64/python2.6/xmlrpclib.py, line 1235, in request self.send_content(h, request_body) File /usr/lib64/python2.6/xmlrpclib.py, line 1349, in send_content connection.endheaders() File /usr/lib64/python2.6/httplib.py, line 908, in endheaders self._send_output() File /usr/lib64/python2.6/httplib.py, line 780, in _send_output self.send(msg) File /usr/lib64/python2.6/httplib.py, line 739, in send self.connect() File /usr/lib64/python2.6/site-packages/vdsm/sslutils.py, line 181, in connect sock = socket.create_connection((self.host, self.port), self.timeout) File /usr/lib64/python2.6/socket.py, line 567, in create_connection raise error, msg error: [Errno 111] Connection refused 2014-08-20 08:49:48 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Closing up': [Errno 111] Connection refused Sorry, rereading better, it seems not an nfs problem but with getting host capabilities... Probably it should be better to put an echo in setup to confirm nfs part has finished and that we are in another part of setup... The server is a kvm guest on Fedora 20 system Indeed when running manually I get [root@ovirtita host-deploy]# vdsClient -s 0 getVdsCaps Connection to 0.0.0.0:54321 http://0.0.0.0:54321 refused what to check? Maybe the vdsmd daemon is not running. check if it's up with service vdsmd status. Looks like the vdsmd daemon is not responding This is in the middle of install itself What is the workflow for engine-setup in AIO install? At the beginning of engine-setup process I have: [ INFO ] Hardware supports virtualization Then in configuration preview I have: Configure VDSM on this host : True Then I see [ INFO ] Stage: Transaction setup [ INFO ] Stopping engine service [ INFO ] Stopping ovirt-fence-kdump-listener service [ INFO ] Stopping websocket-proxy service [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Initializing PostgreSQL [ INFO ] Creating PostgreSQL 'engine' database [ INFO ] Configuring PostgreSQL [ INFO ] Creating/refreshing Engine database schema [ INFO ] Creating CA [ INFO ] Configuring WebSocket Proxy [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up and [ INFO ] Starting engine service [ INFO ] Restarting httpd [ INFO ] Restarting nfs services [ ERROR ] Failed to execute stage 'Closing up': [Errno 111] Connection refused [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20140820084635-z4gm0h.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20140820084948-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed So where is the vdsmd part configuration information? DO I have to check vdsm logs or host-deploy logs for this kind of problem? Gianluca ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] AIO setup problem with master failes due to nfs
Il 22/08/2014 09:59, Gianluca Cecchi ha scritto: On Thu, Aug 21, 2014 at 6:43 PM, Sandro Bonazzola sbona...@redhat.com mailto:sbona...@redhat.com wrote: Il 20/08/2014 09:17, Gianluca Cecchi ha scritto: On Wed, Aug 20, 2014 at 9:06 AM, Gianluca Cecchi gianluca.cec...@gmail.com mailto:gianluca.cec...@gmail.com mailto:gianluca.cec...@gmail.com mailto:gianluca.cec...@gmail.com wrote: result = vdscli.connect().getVdsCapabilities() File /usr/lib64/python2.6/xmlrpclib.py, line 1199, in __call__ return self.__send(self.__name, args) File /usr/lib64/python2.6/xmlrpclib.py, line 1489, in __request verbose=self.__verbose File /usr/lib64/python2.6/xmlrpclib.py, line 1235, in request self.send_content(h, request_body) File /usr/lib64/python2.6/xmlrpclib.py, line 1349, in send_content connection.endheaders() File /usr/lib64/python2.6/httplib.py, line 908, in endheaders self._send_output() File /usr/lib64/python2.6/httplib.py, line 780, in _send_output self.send(msg) File /usr/lib64/python2.6/httplib.py, line 739, in send self.connect() File /usr/lib64/python2.6/site-packages/vdsm/sslutils.py, line 181, in connect sock = socket.create_connection((self.host, self.port), self.timeout) File /usr/lib64/python2.6/socket.py, line 567, in create_connection raise error, msg error: [Errno 111] Connection refused 2014-08-20 08:49:48 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Closing up': [Errno 111] Connection refused Sorry, rereading better, it seems not an nfs problem but with getting host capabilities... Probably it should be better to put an echo in setup to confirm nfs part has finished and that we are in another part of setup... The server is a kvm guest on Fedora 20 system Indeed when running manually I get [root@ovirtita host-deploy]# vdsClient -s 0 getVdsCaps Connection to 0.0.0.0:54321 http://0.0.0.0:54321 http://0.0.0.0:54321 refused what to check? Maybe the vdsmd daemon is not running. check if it's up with service vdsmd status. Looks like the vdsmd daemon is not responding This is in the middle of install itself What is the workflow for engine-setup in AIO install? It first configure and run ovirt-engine, then use the SDK for creating the local datacenter and add the host to it. At the beginning of engine-setup process I have: [ INFO ] Hardware supports virtualization Then in configuration preview I have: Configure VDSM on this host : True Then I see [ INFO ] Stage: Transaction setup [ INFO ] Stopping engine service [ INFO ] Stopping ovirt-fence-kdump-listener service [ INFO ] Stopping websocket-proxy service [ INFO ] Stage: Misc configuration [ INFO ] Stage: Package installation [ INFO ] Stage: Misc configuration [ INFO ] Initializing PostgreSQL [ INFO ] Creating PostgreSQL 'engine' database [ INFO ] Configuring PostgreSQL [ INFO ] Creating/refreshing Engine database schema [ INFO ] Creating CA [ INFO ] Configuring WebSocket Proxy [ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf' [ INFO ] Stage: Transaction commit [ INFO ] Stage: Closing up and [ INFO ] Starting engine service [ INFO ] Restarting httpd [ INFO ] Restarting nfs services [ ERROR ] Failed to execute stage 'Closing up': [Errno 111] Connection refused [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20140820084635-z4gm0h.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20140820084948-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed So where is the vdsmd part configuration information? DO I have to check vdsm logs or host-deploy logs for this kind of problem? Both of them I guess. Gianluca -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] AIO setup problem with master failes due to nfs
On Fri, Aug 22, 2014 at 10:24 AM, Sandro Bonazzola sbona...@redhat.com wrote: So where is the vdsmd part configuration information? DO I have to check vdsm logs or host-deploy logs for this kind of problem? Both of them I guess. Gianluca -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ok, I think I have understood the problem. I remember I successfully completed an AIO install on the same environment around 10 or11 of August so there had to be something different in my way of proceeding. In order to rerun many times engine-setup and with different options I setup an environment where on a clean CentOS 6.5 I installed the necessary oVirt packages. Then I shutdown the server and save its disk to use it as a starting point for recurring installations. It seems that the workflow - install ovirt packages - shutdown and reboot server - run engine-setup creates this kind of problem. Possibly because as part of the install I have services marked to be started at boot, even if no configuration has been made for them yet So as soon as the packages are installed the services have not been started yet, but they will do on next reboot both if you will run engine-setup before shutdown or not As soon as installation completes I have these new services configured at boot cgconfig 0:off 1:off 2:on 3:on 4:on 5:on 6:off iscsi 0:off 1:off 2:off 3:on 4:on 5:on 6:off iscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:off ksm 0:off 1:off 2:off 3:on 4:on 5:on 6:off ksmtuned 0:off 1:off 2:off 3:on 4:on 5:on 6:off libvirt-guests 0:off 1:off 2:on 3:on 4:on 5:on 6:off libvirtd 0:off 1:off 2:off 3:on 4:on 5:on 6:off sanlock 0:off 1:off 2:on 3:on 4:on 5:on 6:off supervdsmd 0:off 1:off 2:on 3:on 4:on 5:on 6:off vdsmd 0:off 1:off 2:on 3:on 4:on 5:on 6:off wdmd 0:off 1:off 2:on 3:on 4:on 5:on 6:off so after restart of the server, this creates problems. Instead if I run engine-setup right after install I don't get this error. But now I get [ INFO ] Starting engine service [ INFO ] Restarting httpd [ INFO ] Restarting nfs services [ ERROR ] Failed to execute stage 'Closing up': No permission to read file: /etc/pki/vdsm/keys/vdsmkey.pem [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20140822110116-nwyf6c.log and in fact, also executing from command line [root@ovirtita setup]# vdsClient -s 0 getVdsCapabilities Traceback (most recent call last): File /usr/share/vdsm/vdsClient.py, line 2744, in module serv.do_connect(hostPort) File /usr/share/vdsm/vdsClient.py, line 187, in do_connect self.s = vdscli.connect(hostPort, self.useSSL, self.truststore) File /usr/lib64/python2.6/site-packages/vdsm/vdscli.py, line 108, in connect raise Exception(No permission to read file: %s % f) Exception: No permission to read file: /etc/pki/vdsm/keys/vdsmkey.pem The file actually is not there, there is no problem of permissions [root@ovirtita setup]# ll /etc/pki/vdsm/keys/ total 4 -rw---. 1 vdsm kvm 10 Aug 21 22:34 libvirt_password Which part should have created the file? After failure: # service vdsmd status VDS daemon is not running no logs from vdsm (probably never started yet because of the above problem): [root@ovirtita exported-artifacts]# ll /var/log/vdsm/ total 4 drwxr-xr-x. 2 vdsm kvm 4096 Aug 21 22:34 backup -rw-r--r--. 1 vdsm kvm 0 Aug 22 12:18 connectivity.log -rw-r--r--. 1 vdsm kvm 0 Aug 22 12:18 mom.log -rw-r--r--. 1 root root0 Aug 22 12:18 supervdsm.log -rw-r--r--. 1 vdsm kvm 0 Aug 22 12:18 vdsm.log [root@ovirtita exported-artifacts]# ll /var/log/vdsm/backup/ total 0 Also, [root@ovirtita exported-artifacts]# ll /var/log/ovirt-engine/host-deploy/ total 0 engine-setup log here https://drive.google.com/file/d/0BwoPbcrMv8mvaWNqakhnS19jZTQ/edit?usp=sharing my current setup of packages is: [root@ovirtita setup]# rpm -qa|egrep engine|vdsm|ovirt vdsm-python-zombiereaper-4.16.0-231.git7f489da.el6.noarch vdsm-xmlrpc-4.16.0-231.git7f489da.el6.noarch ovirt-engine-setup-plugin-websocket-proxy-3.6.0-0.0.master.20140816103500.gitb920a2d.el6.noarch ovirt-host-deploy-java-1.3.0-0.0.master.20140821183616.git82dce3c.el6.noarch ovirt-engine-setup-3.6.0-0.0.master.20140816103500.gitb920a2d.el6.noarch ovirt-release-master-001-0.0.master.noarch ovirt-engine-sdk-python-3.6.0.0-1.20140820.gitc625968.el6.noarch ovirt-engine-userportal-3.6.0-0.0.master.20140816103500.gitb920a2d.el6.noarch ovirt-engine-backend-3.6.0-0.0.master.20140816103500.gitb920a2d.el6.noarch ovirt-host-deploy-1.3.0-0.0.master.20140821183616.git82dce3c.el6.noarch ovirt-image-uploader-3.6.0-0.0.master.20140808125503.git066ed25.el6.noarch ovirt-engine-lib-3.6.0-0.0.master.20140816103500.gitb920a2d.el6.noarch ovirt-engine-extensions-api-impl-3.6.0-0.0.master.20140816103500.gitb920a2d.el6.noarch
Re: [ovirt-users] AIO setup problem with master failes due to nfs
My previous tests were made with local rpm packages released by jenkins jobs for ita translation from master (tried with both 16/08 and 18/08 no other successfull jobs for CentOS 6.x after). Now, to verify if this could influence the problem I actually confirmed to get the same error and behavior with pure master tree. See engine log here: https://drive.google.com/file/d/0BwoPbcrMv8mvanZwYWR5X2tWRGs/edit?usp=sharing Instead with current 3.5 repo just tried on the same re-cleaned environment I have: [ INFO ] Starting engine service [ INFO ] Restarting httpd [ INFO ] Restarting nfs services [ INFO ] Waiting for VDSM host to become operational. This may take several minutes... [ INFO ] The VDSM Host is now operational [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20140822150237-13m6av.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20140822150621-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ INFO ] Execution of setup completed successfully and [root@ovirtita setup]# ll /etc/pki/vdsm/keys/ total 8 -rw---. 1 vdsm kvm 10 Aug 21 10:34 libvirt_password -r--r-. 1 vdsm kvm 1679 Aug 22 15:05 vdsmkey.pem [root@ovirtita ~]# service vdsmd status VDS daemon server is running see engine log for 3.5 here: https://drive.google.com/file/d/0BwoPbcrMv8mvQWg3ZkRTQl9GQ3c/edit?usp=sharing For both environments I have used: yum localinstall http://resources.ovirt.org/pub/yum-repo/repo-name.rpm (where repo-name is ovirt-release35 for 3.5 or ovirt-release-master.rpm for master) yum install ovirt-engine-setup-plugin-allinone engine-setup with all default to answers (except for Yes for vdsm and weak password for admin user). So I think there is some sort of regression between the current 3.5 and 3.6 master for AIO setup... I'm sure that on 13/08 I successfully used jenkins package for ita (generated on 11th) with master tree as of 13/08 without this failure problems (apart that vdsmd wasn't able to start during engine-setup; I had only 6Gb of ram on the VM and the host went into Unissigned state) so I don't now what happened in the middle... Gianluca --engine-setup output for master: [root@ovirtita ~]# engine-setup [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-aio.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf'] Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20140822133624-0qjxbv.log Version: otopi-1.3.0_master (otopi-1.3.0-0.0.master.20140728.git336a22e.el6) [ INFO ] Hardware supports virtualization [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [ INFO ] Stage: Environment customization --== PRODUCT OPTIONS ==-- Configure Engine on this host (Yes, No) [Yes]: Configure WebSocket Proxy on this host (Yes, No) [Yes]: --== PACKAGES ==-- [ INFO ] Checking for product updates... [ INFO ] No product updates found --== ALL IN ONE CONFIGURATION ==-- Configure VDSM on this host? (Yes, No) [No]: Yes Local storage domain path [/var/lib/images]: Local storage domain name [local_storage]: --== NETWORK CONFIGURATION ==-- Setup can automatically configure the firewall on this system. Note: automatic configuration of the firewall may overwrite current settings. Do you want Setup to configure the firewall? (Yes, No) [Yes]: [ INFO ] iptables will be configured as firewall manager. Host fully qualified DNS name of this server [ovirtita.localdomain.local]: [WARNING] Failed to resolve ovirtita.localdomain.local using DNS, it can be resolved only locally --== DATABASE CONFIGURATION ==-- Where is the Engine database located? (Local, Remote) [Local]: Setup can configure the local postgresql server automatically for the engine to run. This may conflict with existing applications. Would you like Setup to automatically configure postgresql and create Engine database, or prefer to perform that manually? (Automatic, Manual) [Automatic]: --== OVIRT ENGINE CONFIGURATION ==-- Engine admin password: Confirm engine admin password: [WARNING] Password is weak: it is based on a dictionary word Use weak password? (Yes, No) [No]: Yes Application mode (Both, Virt, Gluster) [Both]: Default storage type: (NFS, FC, ISCSI, POSIXFS, GLUSTERFS) [NFS]: --== PKI CONFIGURATION ==-- Organization name for certificate [localdomain.local]: --== APACHE CONFIGURATION ==-- Setup can configure the default page of the web server to present the application home page. This may conflict
Re: [ovirt-users] AIO setup problem with master failes due to nfs
Il 20/08/2014 09:17, Gianluca Cecchi ha scritto: On Wed, Aug 20, 2014 at 9:06 AM, Gianluca Cecchi gianluca.cec...@gmail.com mailto:gianluca.cec...@gmail.com wrote: result = vdscli.connect().getVdsCapabilities() File /usr/lib64/python2.6/xmlrpclib.py, line 1199, in __call__ return self.__send(self.__name, args) File /usr/lib64/python2.6/xmlrpclib.py, line 1489, in __request verbose=self.__verbose File /usr/lib64/python2.6/xmlrpclib.py, line 1235, in request self.send_content(h, request_body) File /usr/lib64/python2.6/xmlrpclib.py, line 1349, in send_content connection.endheaders() File /usr/lib64/python2.6/httplib.py, line 908, in endheaders self._send_output() File /usr/lib64/python2.6/httplib.py, line 780, in _send_output self.send(msg) File /usr/lib64/python2.6/httplib.py, line 739, in send self.connect() File /usr/lib64/python2.6/site-packages/vdsm/sslutils.py, line 181, in connect sock = socket.create_connection((self.host, self.port), self.timeout) File /usr/lib64/python2.6/socket.py, line 567, in create_connection raise error, msg error: [Errno 111] Connection refused 2014-08-20 08:49:48 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Closing up': [Errno 111] Connection refused Sorry, rereading better, it seems not an nfs problem but with getting host capabilities... Probably it should be better to put an echo in setup to confirm nfs part has finished and that we are in another part of setup... The server is a kvm guest on Fedora 20 system Indeed when running manually I get [root@ovirtita host-deploy]# vdsClient -s 0 getVdsCaps Connection to 0.0.0.0:54321 http://0.0.0.0:54321 refused what to check? Maybe the vdsmd daemon is not running. check if it's up with service vdsmd status. Looks like the vdsmd daemon is not responding [root@ovirtita host-deploy]# lsmod|grep kvm kvm_intel 54317 0 kvm 333734 1 kvm_intel Thanks, Gianluca ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] AIO setup problem with master failes due to nfs
On Wed, Aug 20, 2014 at 9:06 AM, Gianluca Cecchi gianluca.cec...@gmail.com wrote: result = vdscli.connect().getVdsCapabilities() File /usr/lib64/python2.6/xmlrpclib.py, line 1199, in __call__ return self.__send(self.__name, args) File /usr/lib64/python2.6/xmlrpclib.py, line 1489, in __request verbose=self.__verbose File /usr/lib64/python2.6/xmlrpclib.py, line 1235, in request self.send_content(h, request_body) File /usr/lib64/python2.6/xmlrpclib.py, line 1349, in send_content connection.endheaders() File /usr/lib64/python2.6/httplib.py, line 908, in endheaders self._send_output() File /usr/lib64/python2.6/httplib.py, line 780, in _send_output self.send(msg) File /usr/lib64/python2.6/httplib.py, line 739, in send self.connect() File /usr/lib64/python2.6/site-packages/vdsm/sslutils.py, line 181, in connect sock = socket.create_connection((self.host, self.port), self.timeout) File /usr/lib64/python2.6/socket.py, line 567, in create_connection raise error, msg error: [Errno 111] Connection refused 2014-08-20 08:49:48 ERROR otopi.context context._executeMethod:161 Failed to execute stage 'Closing up': [Errno 111] Connection refused Sorry, rereading better, it seems not an nfs problem but with getting host capabilities... Probably it should be better to put an echo in setup to confirm nfs part has finished and that we are in another part of setup... The server is a kvm guest on Fedora 20 system Indeed when running manually I get [root@ovirtita host-deploy]# vdsClient -s 0 getVdsCaps Connection to 0.0.0.0:54321 refused what to check? [root@ovirtita host-deploy]# lsmod|grep kvm kvm_intel 54317 0 kvm 333734 1 kvm_intel Thanks, Gianluca ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
