Re: [ovirt-users] Hosted engine install failed; vdsm upset about broker (revised)
> On Apr 20, 2017, at 9:18 AM, Simone Tiraboschi wrote: > Could you please share the output of > sudo -u vdsm sudo service sanlock status That command line prompts for vdsm’s password, which it doesn’t have. But output returned as root is below. Is that ‘operation not permitted’ related? Thanks, -j [root@sc5-ovirt-2 jlawrence]# service sanlock status Redirecting to /bin/systemctl status sanlock.service ● sanlock.service - Shared Storage Lease Manager Loaded: loaded (/usr/lib/systemd/system/sanlock.service; disabled; vendor preset: disabled) Active: active (running) since Wed 2017-04-19 16:56:40 PDT; 17h ago Process: 16764 ExecStart=/usr/sbin/sanlock daemon (code=exited, status=0/SUCCESS) Main PID: 16765 (sanlock) CGroup: /system.slice/sanlock.service ├─16765 /usr/sbin/sanlock daemon └─16766 /usr/sbin/sanlock daemon Apr 19 16:56:40 sc5-ovirt-2.squaretrade.com systemd[1]: Starting Shared Storage Lease Manager... Apr 19 16:56:40 sc5-ovirt-2.squaretrade.com systemd[1]: Started Shared Storage Lease Manager. Apr 19 16:56:40 sc5-ovirt-2.squaretrade.com sanlock[16765]: 2017-04-19 16:56:40-0700 482 [16765]: set scheduler RR|RESET_ON_FORK priority 99 failed: Operation not permitted ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted engine install failed; vdsm upset about broker (revised)
On Thu, Apr 20, 2017 at 2:14 AM, Jamie Lawrence wrote: > > So, tracing this further, I’m pretty sure this is something about sanlock. > > As best I can tell this[1] seems to be the failure that is blocking > importing the pool, creating storage domains, importing the HE, etc. > Contrary to the log, sanlock is running; I verified it starts on > system-boot and restarts just fine. > > I found one reference to someone having a similar problem in 3.6, but that > appeared to have been a permission issue I’m not afflicted with. > > How can I move past this? > Could you please share the output of sudo -u vdsm sudo service sanlock status ? > > TIA, > > -j > > > [1] agent.log: > MainThread::WARNING::2017-04-19 17:07:13,537::agent::209:: > ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, > attempt '6' > MainThread::INFO::2017-04-19 17:07:13,567::hosted_engine:: > 242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname) > Found certificate common name: sc5-ovirt-2.squaretrade.com > MainThread::INFO::2017-04-19 17:07:13,569::hosted_engine:: > 604::ovirt_hosted_engine_ha.agent.hosted_engine. > HostedEngine::(_initialize_vdsm) Initializing VDSM > MainThread::INFO::2017-04-19 17:07:16,044::hosted_engine:: > 630::ovirt_hosted_engine_ha.agent.hosted_engine. > HostedEngine::(_initialize_storage_images) Connecting the storage > MainThread::INFO::2017-04-19 17:07:16,045::storage_server:: > 219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) > Connecting storage server > MainThread::INFO::2017-04-19 17:07:20,876::storage_server:: > 226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) > Connecting storage server > MainThread::INFO::2017-04-19 17:07:20,893::storage_server:: > 233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) > Refreshing the storage domain > MainThread::INFO::2017-04-19 17:07:21,160::hosted_engine:: > 657::ovirt_hosted_engine_ha.agent.hosted_engine. > HostedEngine::(_initialize_storage_images) Preparing images > MainThread::INFO::2017-04-19 17:07:21,160::image::126:: > ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images > MainThread::INFO::2017-04-19 17:07:23,954::hosted_engine:: > 660::ovirt_hosted_engine_ha.agent.hosted_engine. > HostedEngine::(_initialize_storage_images) Refreshing vm.conf > MainThread::INFO::2017-04-19 17:07:23,955::config::485:: > ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) > Reloading vm.conf from the shared storage domain > MainThread::INFO::2017-04-19 17:07:23,955::config::412:: > ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine. > config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher > copy of vm configuration from the OVF_STORE > MainThread::WARNING::2017-04-19 17:07:26,741::ovf_store::107:: > ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find > OVF_STORE > MainThread::ERROR::2017-04-19 17:07:26,744::config::450:: > ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine. > config::(_get_vm_conf_content_from_ovf_store) Unable to identify the > OVF_STORE volume, falling back to initial vm.conf. Please ensure you > already added your first data domain for regular VMs > MainThread::INFO::2017-04-19 17:07:26,770::hosted_engine:: > 509::ovirt_hosted_engine_ha.agent.hosted_engine. > HostedEngine::(_initialize_broker) Initializing ha-broker connection > MainThread::INFO::2017-04-19 17:07:26,771::brokerlink::130: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Starting monitor ping, options {'addr': '10.181.26.1'} > MainThread::INFO::2017-04-19 17:07:26,774::brokerlink::141: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Success, id 140621269798096 > MainThread::INFO::2017-04-19 17:07:26,774::brokerlink::130: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Starting monitor mgmt-bridge, options {'use_ssl': 'true', 'bridge_name': > 'ovirtmgmt', 'address': '0'} > MainThread::INFO::2017-04-19 17:07:26,791::brokerlink::141: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Success, id 140621269798544 > MainThread::INFO::2017-04-19 17:07:26,792::brokerlink::130: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Starting monitor mem-free, options {'use_ssl': 'true', 'address': '0'} > MainThread::INFO::2017-04-19 17:07:26,793::brokerlink::141: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Success, id 140621269798224 > MainThread::INFO::2017-04-19 17:07:26,794::brokerlink::130: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) > Starting monitor cpu-load-no-engine, options {'use_ssl': 'true', 'vm_uuid': > '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'} > MainThread::INFO::2017-04-19 17:07:26,796::brokerlink::141: > :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) >
[ovirt-users] Hosted engine install failed; vdsm upset about broker (revised)
So, tracing this further, I’m pretty sure this is something about sanlock. As best I can tell this[1] seems to be the failure that is blocking importing the pool, creating storage domains, importing the HE, etc. Contrary to the log, sanlock is running; I verified it starts on system-boot and restarts just fine. I found one reference to someone having a similar problem in 3.6, but that appeared to have been a permission issue I’m not afflicted with. How can I move past this? TIA, -j [1] agent.log: MainThread::WARNING::2017-04-19 17:07:13,537::agent::209::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '6' MainThread::INFO::2017-04-19 17:07:13,567::hosted_engine::242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname) Found certificate common name: sc5-ovirt-2.squaretrade.com MainThread::INFO::2017-04-19 17:07:13,569::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm) Initializing VDSM MainThread::INFO::2017-04-19 17:07:16,044::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Connecting the storage MainThread::INFO::2017-04-19 17:07:16,045::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2017-04-19 17:07:20,876::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Connecting storage server MainThread::INFO::2017-04-19 17:07:20,893::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server) Refreshing the storage domain MainThread::INFO::2017-04-19 17:07:21,160::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Preparing images MainThread::INFO::2017-04-19 17:07:21,160::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images) Preparing images MainThread::INFO::2017-04-19 17:07:23,954::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images) Refreshing vm.conf MainThread::INFO::2017-04-19 17:07:23,955::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf) Reloading vm.conf from the shared storage domain MainThread::INFO::2017-04-19 17:07:23,955::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE MainThread::WARNING::2017-04-19 17:07:26,741::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE MainThread::ERROR::2017-04-19 17:07:26,744::config::450::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Unable to identify the OVF_STORE volume, falling back to initial vm.conf. Please ensure you already added your first data domain for regular VMs MainThread::INFO::2017-04-19 17:07:26,770::hosted_engine::509::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Initializing ha-broker connection MainThread::INFO::2017-04-19 17:07:26,771::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor ping, options {'addr': '10.181.26.1'} MainThread::INFO::2017-04-19 17:07:26,774::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140621269798096 MainThread::INFO::2017-04-19 17:07:26,774::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor mgmt-bridge, options {'use_ssl': 'true', 'bridge_name': 'ovirtmgmt', 'address': '0'} MainThread::INFO::2017-04-19 17:07:26,791::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140621269798544 MainThread::INFO::2017-04-19 17:07:26,792::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor mem-free, options {'use_ssl': 'true', 'address': '0'} MainThread::INFO::2017-04-19 17:07:26,793::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140621269798224 MainThread::INFO::2017-04-19 17:07:26,794::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor cpu-load-no-engine, options {'use_ssl': 'true', 'vm_uuid': '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'} MainThread::INFO::2017-04-19 17:07:26,796::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id 140621269796816 MainThread::INFO::2017-04-19 17:07:26,796::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor engine-health, options {'use_ssl': 'true', 'vm_uuid': '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'} MainThread