[ovirt-users] Hosted engine on iscsi storage
Hi All,
I am trying to install the hosted engine on an iscsi target but get the
following error:
[root@virt-host01 ~]# hosted-engine --deploy
...
--== STORAGE CONFIGURATION ==--
During customization use CTRL-D to abort.
Please specify the storage you would like to use (glusterfs,
iscsi, fc, nfs3, nfs4)[nfs3]: iscsi
Please specify the iSCSI portal IP address: 10.43.0.100
Please specify the iSCSI portal port [3260]:
Please specify the iSCSI portal user:
Please specify the target name
(iqn.2001-05.com.equallogic:0-8a0906-8bb896109-606000b57145-iscsi-vol-01)
[iqn.2001-05.com.equallogic:0-8a0906-8bb896109-606000b57145-iscsi-vol-01]:
[ INFO ] Discovering iSCSI node
[ INFO ] Connecting to the storage server
[ INFO ] Discovering iSCSI node
[ INFO ] Connecting to the storage server
[ ERROR ] Failed to execute stage 'Environment customization': Unable to
retrieve the list of LUN(s) please check the SELinux log and settings on
your iscsi target
The relevant excerpt from the log file is:
2016-05-05 16:05:09 DEBUG otopi.context context._executeMethod:156
method exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146,
in _executeMethod
method['method']()
File
"/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/blockd.py",
line 591, in _customization
lunGUID = self._customize_lun(self.domainType, target)
File
"/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/blockd.py",
line 209, in _customize_lun
iqn=target,
File
"/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/blockd.py",
line 374, in _iscsi_get_lun_list
raise RuntimeError("Unable to retrieve the list of LUN(s) please "
RuntimeError: Unable to retrieve the list of LUN(s) please check the
SELinux log and settings on your iscsi target
2016-05-05 16:05:09 ERROR otopi.context context._executeMethod:165
Failed to execute stage 'Environment customization': Unable to retrieve
the list of LUN(s) please check the SELinux log and settings on your
iscsi target
This is on CentOS 7 with ovirt 3.6.
selinux is disabled.
I can mount the iscsi target fine using iscsiadm or the Dell Equallogic
Host Integration Toolkit commands.
I think the first problem is the call to self.cli.getDeviceList in
blockd.py is returning an empty list, but I don't know what that
function is actually doing.
Has anyone experienced similar behaviour or has any suggestions as to
what I should check next?
Thanks,
Darran.
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Hosted engine on iscsi storage
On Thu, May 5, 2016 at 10:22 AM, Darran Carey
wrote:
> Hi All,
>
> I am trying to install the hosted engine on an iscsi target but get the
> following error:
>
> [root@virt-host01 ~]# hosted-engine --deploy
> ...
> --== STORAGE CONFIGURATION ==--
>
> During customization use CTRL-D to abort.
> Please specify the storage you would like to use (glusterfs,
> iscsi, fc, nfs3, nfs4)[nfs3]: iscsi
> Please specify the iSCSI portal IP address: 10.43.0.100
> Please specify the iSCSI portal port [3260]:
> Please specify the iSCSI portal user:
> Please specify the target name
> (iqn.2001-05.com.equallogic:0-8a0906-8bb896109-606000b57145-iscsi-vol-01)
> [iqn.2001-05.com.equallogic:0-8a0906-8bb896109-606000b57145-iscsi-vol-01]:
> [ INFO ] Discovering iSCSI node
> [ INFO ] Connecting to the storage server
> [ INFO ] Discovering iSCSI node
> [ INFO ] Connecting to the storage server
> [ ERROR ] Failed to execute stage 'Environment customization': Unable to
> retrieve the list of LUN(s) please check the SELinux log and settings on
> your iscsi target
>
>
> The relevant excerpt from the log file is:
>
> 2016-05-05 16:05:09 DEBUG otopi.context context._executeMethod:156 method
> exception
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in
> _executeMethod
> method['method']()
> File
> "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/blockd.py",
> line 591, in _customization
> lunGUID = self._customize_lun(self.domainType, target)
> File
> "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/blockd.py",
> line 209, in _customize_lun
> iqn=target,
> File
> "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/storage/blockd.py",
> line 374, in _iscsi_get_lun_list
> raise RuntimeError("Unable to retrieve the list of LUN(s) please "
> RuntimeError: Unable to retrieve the list of LUN(s) please check the SELinux
> log and settings on your iscsi target
> 2016-05-05 16:05:09 ERROR otopi.context context._executeMethod:165 Failed to
> execute stage 'Environment customization': Unable to retrieve the list of
> LUN(s) please check the SELinux log and settings on your iscsi target
Can you please attach the whole hosted-engine-setup log file and vdsm logs?
> This is on CentOS 7 with ovirt 3.6.
> selinux is disabled.
>
> I can mount the iscsi target fine using iscsiadm or the Dell Equallogic Host
> Integration Toolkit commands.
>
> I think the first problem is the call to self.cli.getDeviceList in blockd.py
> is returning an empty list, but I don't know what that function is actually
> doing.
>
> Has anyone experienced similar behaviour or has any suggestions as to what I
> should check next?
>
> Thanks,
> Darran.
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Stale VM template
Hi Tomas, I have used unlock_entity.sh script to unlock the template. The setup is out now, will send out the logs once it is up. Thanks, Bhaskarakiran. On Thu, May 5, 2016 at 12:26 PM, Tomas Jelinek wrote: > Hi Bhaskarkairan, > > how long the template has been locked? > Could you please attach the logs (engine, vdsm) from the relevant time > period? > > Thank you, > Tomas > > - Original Message - > > From: "Bhaskarakiran" > > To: users@ovirt.org > > Sent: Wednesday, May 4, 2016 6:58:26 PM > > Subject: [ovirt-users] Stale VM template > > > > Hi, > > > > I was trying to make a template of RHEL7.2 VM from GUI (hosted engine) > and > > the status shows "Image Locked" for a very long time now. Tried > restarting > > ovirt-engine and didn't help. Can someone let me know how do i cancel > that > > operation. > > > > Thanks, > > Bhaskarkairan. > > > > ___ > > Users mailing list > > Users@ovirt.org > > http://lists.ovirt.org/mailman/listinfo/users > > > ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Prepare for version 4 of oVirt API
On 05/05/2016 11:34 AM, René Koch wrote: > On 05/04/2016 04:39 PM, Juan Hernández wrote: >> Hello all, >> >> For those of you that use oVirt API, I wanted to inform you that >> version 4 of oVirt will by default use a new version of the API, >> incompatible with the previous one. The previous version will still be >> supported, but only for clients that explicitly send the "Version: 3" >> header, as described here: >> >> >> https://github.com/oVirt/ovirt-engine/tree/master/backend/manager/modules/restapi > So with version 3 everything should work in futher version of oVirt, but > it's recommended to switch to Version 4, right? Are there any plans yet > to drop support of Version 3 API in e.g. oVirt 4.1? Correct, oVirt 4.0 will support version 3 of the API, and if you explicitly request it (using the "Version: 3" header) everything should work as it worked in oVirt 3.6. The current plan is to drop version 3 of the API with oVirt 4.1, but that isn't written in stone yet, may be later (but not earlier). >> If you are aware of any piece of software that uses the oVirt API, >> please let me know, so that we can inform the authors. > Thanks for sending a patch to check_rhev3, I've merged it into master. > So monitoring oVirt 4.0 with Nagios/Icinga should work :) -- Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta 3ºD, 28016 Madrid, Spain Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Prepare for version 4 of oVirt API
On 05/04/2016 04:39 PM, Juan Hernández wrote: Hello all, For those of you that use oVirt API, I wanted to inform you that version 4 of oVirt will by default use a new version of the API, incompatible with the previous one. The previous version will still be supported, but only for clients that explicitly send the "Version: 3" header, as described here: https://github.com/oVirt/ovirt-engine/tree/master/backend/manager/modules/restapi So with version 3 everything should work in futher version of oVirt, but it's recommended to switch to Version 4, right? Are there any plans yet to drop support of Version 3 API in e.g. oVirt 4.1? If you are aware of any piece of software that uses the oVirt API, please let me know, so that we can inform the authors. Thanks for sending a patch to check_rhev3, I've merged it into master. So monitoring oVirt 4.0 with Nagios/Icinga should work :) Thanks in advance, Juan Hernandez smime.p7s Description: S/MIME Cryptographic Signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] node unresponsive after reboot
Cam,
It is really hard to understand partial logs. Please send me directly
full logs and I will analyze.
Thanks,
Piotr
On Mon, May 2, 2016 at 4:35 PM, Cam Mac wrote:
> Hi Piotr,
>
> Attached are the vdsm log, the engine log and the supervdsm log. I've
> attached them as a .tgz.
>
> I noticed it is complaining about configuring an interface in one of the
> node logs. It shows as up in the engine web GUI though (and on the command
> line).
>
> Thanks for the help.
>
> -Cam
>
> On Mon, May 2, 2016 at 1:38 PM, Piotr Kliczewski
> wrote:
>>
>> Cam,
>>
>> Please provide engine and failing vdsm logs.
>>
>> Thanks,
>> Piotr
>>
>> On Sun, May 1, 2016 at 4:05 PM, Cam Mac wrote:
>> > Hi,
>> >
>> > I have a two node + engine ovirt setup, and I was having problems
>> > doing a live migration between nodes. I looked in the vdsm logs and
>> > noticed selinux errors, so I checked the selinux config, and both the
>> > ovirt-engine host and one of the nodes had selinux disabled. So I
>> > thought I would enable it on these two hosts, as it is officially
>> > supported anyway. I started with the node, and put it into maintenance
>> > mode, which interestingly, migrated the VMs off to the other node
>> > without issue. After modifying the selinux config, I then rebooted
>> > that node, which came back up. I then tried to activate the node but
>> > it fails and marks it as unresponsive.
>> >
>> > --8<--
>> >
>> > 2016-04-28 16:34:31,326 INFO
>> > [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp
>> > Reactor) [29acb18b] Connecting to
>> > kvm-ldn-02/172.16.23.12
>> > 2016-04-28 16:34:31,327 ERROR
>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Command
>> > 'GetCapabilitiesVDSCommand(HostName = kvm-ldn-02,
>> > VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
>> > hostId='b12c0b80-d64d-42fd-8a55-94f92b9ca3aa',
>> > vds='Host[kvm-ldn-02,b12c0b80-d64d-42fd-8a55-94f92b9ca3aa]'})'
>> > execution failed:
>> > org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection
>> > failed
>> > 2016-04-28 16:34:31,327 ERROR
>> > [org.ovirt.engine.core.vdsbroker.HostMonitoring]
>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Failure to refresh Vds
>> > runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>> > Connection failed
>> > 2016-04-28 16:34:31,327 ERROR
>> > [org.ovirt.engine.core.vdsbroker.HostMonitoring]
>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Exception:
>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
>> > org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection
>> > failed
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:157)
>> > [vdsbroker.jar:]
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:120)
>> > [vdsbroker.jar:]
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
>> > [vdsbroker.jar:]
>> > at
>> > org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
>> > [dal.jar:]
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:467)
>> > [vdsbroker.jar:]
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:652)
>> > [vdsbroker.jar:]
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.HostMonitoring.refreshVdsRunTimeInfo(HostMonitoring.java:119)
>> > [vdsbroker.jar:]
>> > at
>> >
>> > org.ovirt.engine.core.vdsbroker.HostMonitoring.refresh(HostMonitoring.java:84)
>> > [vdsbroker.jar:]
>> > at
>> > org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:227)
>> > [vdsbroker.jar:]
>> > at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown
>> > Source) [:1.8.0_71]
>> > at
>> >
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > [rt.jar:1.8.0_71]
>> > at java.lang.reflect.Method.invoke(Method.java:497)
>> > [rt.jar:1.8.0_71]
>> > at
>> >
>> > org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:81)
>> > [scheduler.jar:]
>> > at
>> > org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:52)
>> > [scheduler.jar:]
>> > at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
>> > [quartz.jar:]
>> > at
>> >
>> > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
>> > [quartz.jar:]
>> > Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>> > Connection failed
>> > at
>> >
>> > org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:157)
>> > [vdsm-jsonrpc-java-client.jar:]
>> > at
>> >
>> > org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:
Re: [ovirt-users] node unresponsive after reboot
No problem Piotr, I will send you the full logs shortly.
Thanks,
Campbell
On 5 May 2016 at 11:37, Piotr Kliczewski wrote:
> Cam,
>
> It is really hard to understand partial logs. Please send me directly
> full logs and I will analyze.
>
> Thanks,
> Piotr
>
> On Mon, May 2, 2016 at 4:35 PM, Cam Mac wrote:
>> Hi Piotr,
>>
>> Attached are the vdsm log, the engine log and the supervdsm log. I've
>> attached them as a .tgz.
>>
>> I noticed it is complaining about configuring an interface in one of the
>> node logs. It shows as up in the engine web GUI though (and on the command
>> line).
>>
>> Thanks for the help.
>>
>> -Cam
>>
>> On Mon, May 2, 2016 at 1:38 PM, Piotr Kliczewski
>> wrote:
>>>
>>> Cam,
>>>
>>> Please provide engine and failing vdsm logs.
>>>
>>> Thanks,
>>> Piotr
>>>
>>> On Sun, May 1, 2016 at 4:05 PM, Cam Mac wrote:
>>> > Hi,
>>> >
>>> > I have a two node + engine ovirt setup, and I was having problems
>>> > doing a live migration between nodes. I looked in the vdsm logs and
>>> > noticed selinux errors, so I checked the selinux config, and both the
>>> > ovirt-engine host and one of the nodes had selinux disabled. So I
>>> > thought I would enable it on these two hosts, as it is officially
>>> > supported anyway. I started with the node, and put it into maintenance
>>> > mode, which interestingly, migrated the VMs off to the other node
>>> > without issue. After modifying the selinux config, I then rebooted
>>> > that node, which came back up. I then tried to activate the node but
>>> > it fails and marks it as unresponsive.
>>> >
>>> > --8<--
>>> >
>>> > 2016-04-28 16:34:31,326 INFO
>>> > [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp
>>> > Reactor) [29acb18b] Connecting to
>>> > kvm-ldn-02/172.16.23.12
>>> > 2016-04-28 16:34:31,327 ERROR
>>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
>>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Command
>>> > 'GetCapabilitiesVDSCommand(HostName = kvm-ldn-02,
>>> > VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
>>> > hostId='b12c0b80-d64d-42fd-8a55-94f92b9ca3aa',
>>> > vds='Host[kvm-ldn-02,b12c0b80-d64d-42fd-8a55-94f92b9ca3aa]'})'
>>> > execution failed:
>>> > org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection
>>> > failed
>>> > 2016-04-28 16:34:31,327 ERROR
>>> > [org.ovirt.engine.core.vdsbroker.HostMonitoring]
>>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Failure to refresh Vds
>>> > runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>>> > Connection failed
>>> > 2016-04-28 16:34:31,327 ERROR
>>> > [org.ovirt.engine.core.vdsbroker.HostMonitoring]
>>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Exception:
>>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
>>> > org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection
>>> > failed
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:157)
>>> > [vdsbroker.jar:]
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:120)
>>> > [vdsbroker.jar:]
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65)
>>> > [vdsbroker.jar:]
>>> > at
>>> > org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:33)
>>> > [dal.jar:]
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:467)
>>> > [vdsbroker.jar:]
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:652)
>>> > [vdsbroker.jar:]
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.HostMonitoring.refreshVdsRunTimeInfo(HostMonitoring.java:119)
>>> > [vdsbroker.jar:]
>>> > at
>>> >
>>> > org.ovirt.engine.core.vdsbroker.HostMonitoring.refresh(HostMonitoring.java:84)
>>> > [vdsbroker.jar:]
>>> > at
>>> > org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:227)
>>> > [vdsbroker.jar:]
>>> > at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown
>>> > Source) [:1.8.0_71]
>>> > at
>>> >
>>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> > [rt.jar:1.8.0_71]
>>> > at java.lang.reflect.Method.invoke(Method.java:497)
>>> > [rt.jar:1.8.0_71]
>>> > at
>>> >
>>> > org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:81)
>>> > [scheduler.jar:]
>>> > at
>>> > org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:52)
>>> > [scheduler.jar:]
>>> > at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
>>> > [quartz.jar:]
>>> > at
>>> >
>>> > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
>>> > [quartz.jar:]
>>> > Caused by: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>>> > Con
Re: [ovirt-users] Hosted engine on iscsi storage
On Thu, May 5, 2016 at 2:35 PM, Darran Carey wrote:
> Hi Simone,
>
> Please find the log files attached. Thank you very much for taking the time
> to look at this problem.
>
> Regards,
> Darran.
Indeed VDSM is returning an empty device list:
Thread-17518::DEBUG::2016-05-05
16:05:05,409::task::595::Storage.TaskManager.Task::(_updateState)
Task=`6e96b9df-b656-45bd-899b-c94ec9be5c52`::moving from state init ->
state preparing
Thread-17518::INFO::2016-05-05
16:05:05,410::logUtils::48::dispatcher::(wrapper) Run and protect:
getDeviceList(storageType=3, guids=(), checkStatus=True, options={})
Thread-17518::DEBUG::2016-05-05
16:05:05,410::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.sdc.refreshStorage)
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.iscsi.rescan)
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,412::iscsi::434::Storage.ISCSI::(rescan) Performing SCSI
scan, this will take up to 30 seconds
Thread-17518::DEBUG::2016-05-05
16:05:05,413::iscsiadm::97::Storage.Misc.excCmd::(_runCmd)
/usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /sbin/iscsiadm -m
session -R (cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,463::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,464::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.hba.rescan)
Thread-17518::DEBUG::2016-05-05
16:05:05,464::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,464::hba::56::Storage.HBA::(rescan) Starting scan
Thread-17518::DEBUG::2016-05-05
16:05:05,661::hba::62::Storage.HBA::(rescan) Scan finished
Thread-17518::DEBUG::2016-05-05
16:05:05,662::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,662::multipath::77::Storage.Misc.excCmd::(rescan)
/usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/multipath
(cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,747::multipath::77::Storage.Misc.excCmd::(rescan) SUCCESS:
= ''; = 0
Thread-17518::DEBUG::2016-05-05
16:05:05,748::utils::671::root::(execCmd) /usr/bin/taskset --cpu-list
0-7 /sbin/udevadm settle --timeout=5 (cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,768::utils::689::root::(execCmd) SUCCESS: = ''; =
0
Thread-17518::DEBUG::2016-05-05
16:05:05,771::lvm::497::Storage.OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,772::lvm::499::Storage.OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,772::lvm::508::Storage.OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,773::lvm::510::Storage.OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,773::lvm::528::Storage.OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,774::lvm::530::Storage.OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,774::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,775::lvm::319::Storage.OperationMutex::(_reloadpvs) Operation
'lvm reload operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,777::lvm::290::Storage.Misc.excCmd::(cmd) /usr/bin/taskset
--cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/lvm pvs --config ' devices {
preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1
write_cache_state=0 disable_after_error_count=3 filter = [
'\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1
wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50
retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|'
--ignoreskippedcluster -o
uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size
(cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,987::lvm::290::Storage.Misc.excCmd::(cmd) SUCCESS: = '
WARNING: lvmetad is running but disabled. Restart lvmetad before
enabling it!\n'; = 0
Thread-17518::DEBUG::2016-05-05
16:05:05,987::lvm::347::Storage.OperationMutex::(_reloadpvs) Operation
'lvm reload operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:06,003::lvm::290::Storage.Misc.excCmd::(cmd) /usr/bin/taskset
--cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/lvm pvcre
[ovirt-users] vdsmd and libvirtd services failed to start
Hi, I have a HCI setup running on 3 nodes and created 6 VM's. Was running IO (like dd and linux untar) on those VM's overnight. Next day i saw that for 2 of the nodes vdsmd and libvirtd services failed and if manually started, they don't come up. All the VM's state has changed to 'unknown' and failed to migrate. Can someone help looking at the logs and figure out the RCA. Let me know what all logs are needed, i can post the same. Thanks, Bhaskarakiran. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] node unresponsive after reboot
To update the thread here after some offline discussion with Piotr, the
problem was that vdsmd had stopped running and wasn't able to start. It
wasn't reported in the logs that I could see, but I noticed it after trying
to restart it at Piotr's suggestion, after running a 'systemctl'. A
'journalctl -xe' reported:
[root@kvm-ldn-02 vdsm]# journalctl -xe
-- Unit vdsmd.service has failed.
--
-- The result is failed.
May 05 15:15:15 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: vdsm:
Running check_is_configured
May 05 15:15:16 kvm-ldn-02.ldn.org sasldblistusers2[19047]: DIGEST-MD5
common mech free
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: Error:
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: One of the
modules is not configured to work with VDSM.
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: To
configure the module use the following:
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: 'vdsm-tool
configure [--module module-name]'.
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: If all
modules are not configured try to use:
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: 'vdsm-tool
configure --force'
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: (The force
flag will stop the module's service and start it
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: afterwards
automatically to load the new configuration.)
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: Current
revision of multipath.conf detected, preserving
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: libvirt is
already configured for vdsm
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: Modules
sebool are not configured
May 05 15:15:16 kvm-ldn-02.ldn.org vdsmd_init_common.sh[18999]: vdsm:
stopped during execute check_is_configured task (task returned with error
code 1).
May 05 15:15:16 kvm-ldn-02.ldn.org systemd[1]: vdsmd.service: control
process exited, code=exited status=1
May 05 15:15:16 kvm-ldn-02.ldn.org systemd[1]: Failed to start Virtual
Desktop Server Manager.
-- Subject: Unit vdsmd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
So I ran a 'vdsm-tool configure --module sebool', and then I was able to
start vdsmd and mom-vdsm. This fixed the connectivity issue and the node is
now back online.
Many thanks to Piotr for all his help.
Cheers,
Cam
On Mon, May 2, 2016 at 3:35 PM, Cam Mac wrote:
> Hi Piotr,
>
> Attached are the vdsm log, the engine log and the supervdsm log. I've
> attached them as a .tgz.
>
> I noticed it is complaining about configuring an interface in one of the
> node logs. It shows as up in the engine web GUI though (and on the command
> line).
>
> Thanks for the help.
>
> -Cam
>
> On Mon, May 2, 2016 at 1:38 PM, Piotr Kliczewski <
> piotr.kliczew...@gmail.com> wrote:
>
>> Cam,
>>
>> Please provide engine and failing vdsm logs.
>>
>> Thanks,
>> Piotr
>>
>> On Sun, May 1, 2016 at 4:05 PM, Cam Mac wrote:
>> > Hi,
>> >
>> > I have a two node + engine ovirt setup, and I was having problems
>> > doing a live migration between nodes. I looked in the vdsm logs and
>> > noticed selinux errors, so I checked the selinux config, and both the
>> > ovirt-engine host and one of the nodes had selinux disabled. So I
>> > thought I would enable it on these two hosts, as it is officially
>> > supported anyway. I started with the node, and put it into maintenance
>> > mode, which interestingly, migrated the VMs off to the other node
>> > without issue. After modifying the selinux config, I then rebooted
>> > that node, which came back up. I then tried to activate the node but
>> > it fails and marks it as unresponsive.
>> >
>> > --8<--
>> >
>> > 2016-04-28 16:34:31,326 INFO
>> > [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp
>> > Reactor) [29acb18b] Connecting to
>> > kvm-ldn-02/172.16.23.12
>> > 2016-04-28 16:34:31,327 ERROR
>> > [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Command
>> > 'GetCapabilitiesVDSCommand(HostName = kvm-ldn-02,
>> > VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
>> > hostId='b12c0b80-d64d-42fd-8a55-94f92b9ca3aa',
>> > vds='Host[kvm-ldn-02,b12c0b80-d64d-42fd-8a55-94f92b9ca3aa]'})'
>> > execution failed:
>> > org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection
>> > failed
>> > 2016-04-28 16:34:31,327 ERROR
>> > [org.ovirt.engine.core.vdsbroker.HostMonitoring]
>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Failure to refresh Vds
>> > runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>> > Connection failed
>> > 2016-04-28 16:34:31,327 ERROR
>> > [org.ovirt.engine.core.vdsbroker.HostMonitoring]
>> > (DefaultQuartzScheduler_Worker-32) [ac322cb] Exception:
>> > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException
Re: [ovirt-users] Hosted engine on iscsi storage
Hi Simone,
The output from the vdsClient command is:
[root@virt-host01 ~]# vdsClient -s 0 getVdsCaps | grep
ISCSIInitiatorName
ISCSIInitiatorName = 'iqn.2016-04.au.org.pawsey:initiator01'
The ACL on the PS4000 is as follows:
ID Initiator Ipaddress AuthMethod UserName
Apply-To
--- - --- -- --
1 10.43.0.* none
both
which should allow access from any host in the 10.43.0 subnet. I
currently don't have CHAP configured. I will configure it and see if
that helps.
Regards,
Darran.
On 2016-05-05 21:14, Simone Tiraboschi wrote:
On Thu, May 5, 2016 at 2:35 PM, Darran Carey
wrote:
Hi Simone,
Please find the log files attached. Thank you very much for taking the
time
to look at this problem.
Regards,
Darran.
Indeed VDSM is returning an empty device list:
Thread-17518::DEBUG::2016-05-05
16:05:05,409::task::595::Storage.TaskManager.Task::(_updateState)
Task=`6e96b9df-b656-45bd-899b-c94ec9be5c52`::moving from state init ->
state preparing
Thread-17518::INFO::2016-05-05
16:05:05,410::logUtils::48::dispatcher::(wrapper) Run and protect:
getDeviceList(storageType=3, guids=(), checkStatus=True, options={})
Thread-17518::DEBUG::2016-05-05
16:05:05,410::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.sdc.refreshStorage)
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.iscsi.rescan)
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,412::iscsi::434::Storage.ISCSI::(rescan) Performing SCSI
scan, this will take up to 30 seconds
Thread-17518::DEBUG::2016-05-05
16:05:05,413::iscsiadm::97::Storage.Misc.excCmd::(_runCmd)
/usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /sbin/iscsiadm -m
session -R (cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,463::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,464::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.hba.rescan)
Thread-17518::DEBUG::2016-05-05
16:05:05,464::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,464::hba::56::Storage.HBA::(rescan) Starting scan
Thread-17518::DEBUG::2016-05-05
16:05:05,661::hba::62::Storage.HBA::(rescan) Scan finished
Thread-17518::DEBUG::2016-05-05
16:05:05,662::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,662::multipath::77::Storage.Misc.excCmd::(rescan)
/usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/multipath
(cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,747::multipath::77::Storage.Misc.excCmd::(rescan) SUCCESS:
= ''; = 0
Thread-17518::DEBUG::2016-05-05
16:05:05,748::utils::671::root::(execCmd) /usr/bin/taskset --cpu-list
0-7 /sbin/udevadm settle --timeout=5 (cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,768::utils::689::root::(execCmd) SUCCESS: = ''; =
0
Thread-17518::DEBUG::2016-05-05
16:05:05,771::lvm::497::Storage.OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,772::lvm::499::Storage.OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,772::lvm::508::Storage.OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,773::lvm::510::Storage.OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,773::lvm::528::Storage.OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,774::lvm::530::Storage.OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,774::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,775::lvm::319::Storage.OperationMutex::(_reloadpvs) Operation
'lvm reload operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,777::lvm::290::Storage.Misc.excCmd::(cmd) /usr/bin/taskset
--cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/lvm pvs --config ' devices {
preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1
write_cache_state=0 disable_after_error_count=3 filter = [
'\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1
wait_for_locks=1 use_lvmetad=0 } backup { retai
Re: [ovirt-users] Hosted engine on iscsi storage
Unfortunately I get the same result when using CHAP.
Darran.
On 2016-05-05 23:06, Darran Carey wrote:
Hi Simone,
The output from the vdsClient command is:
[root@virt-host01 ~]# vdsClient -s 0 getVdsCaps | grep
ISCSIInitiatorName
ISCSIInitiatorName = 'iqn.2016-04.au.org.pawsey:initiator01'
The ACL on the PS4000 is as follows:
ID Initiator Ipaddress AuthMethod UserName
Apply-To
--- - --- -- --
1 10.43.0.* none
both
which should allow access from any host in the 10.43.0 subnet. I
currently don't have CHAP configured. I will configure it and see if
that helps.
Regards,
Darran.
On 2016-05-05 21:14, Simone Tiraboschi wrote:
On Thu, May 5, 2016 at 2:35 PM, Darran Carey
wrote:
Hi Simone,
Please find the log files attached. Thank you very much for taking
the time
to look at this problem.
Regards,
Darran.
Indeed VDSM is returning an empty device list:
Thread-17518::DEBUG::2016-05-05
16:05:05,409::task::595::Storage.TaskManager.Task::(_updateState)
Task=`6e96b9df-b656-45bd-899b-c94ec9be5c52`::moving from state init ->
state preparing
Thread-17518::INFO::2016-05-05
16:05:05,410::logUtils::48::dispatcher::(wrapper) Run and protect:
getDeviceList(storageType=3, guids=(), checkStatus=True, options={})
Thread-17518::DEBUG::2016-05-05
16:05:05,410::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.sdc.refreshStorage)
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.iscsi.rescan)
Thread-17518::DEBUG::2016-05-05
16:05:05,411::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,412::iscsi::434::Storage.ISCSI::(rescan) Performing SCSI
scan, this will take up to 30 seconds
Thread-17518::DEBUG::2016-05-05
16:05:05,413::iscsiadm::97::Storage.Misc.excCmd::(_runCmd)
/usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /sbin/iscsiadm -m
session -R (cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,463::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,464::misc::750::Storage.SamplingMethod::(__call__) Trying to
enter sampling method (storage.hba.rescan)
Thread-17518::DEBUG::2016-05-05
16:05:05,464::misc::753::Storage.SamplingMethod::(__call__) Got in to
sampling method
Thread-17518::DEBUG::2016-05-05
16:05:05,464::hba::56::Storage.HBA::(rescan) Starting scan
Thread-17518::DEBUG::2016-05-05
16:05:05,661::hba::62::Storage.HBA::(rescan) Scan finished
Thread-17518::DEBUG::2016-05-05
16:05:05,662::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,662::multipath::77::Storage.Misc.excCmd::(rescan)
/usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/multipath
(cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,747::multipath::77::Storage.Misc.excCmd::(rescan) SUCCESS:
= ''; = 0
Thread-17518::DEBUG::2016-05-05
16:05:05,748::utils::671::root::(execCmd) /usr/bin/taskset --cpu-list
0-7 /sbin/udevadm settle --timeout=5 (cwd None)
Thread-17518::DEBUG::2016-05-05
16:05:05,768::utils::689::root::(execCmd) SUCCESS: = ''; =
0
Thread-17518::DEBUG::2016-05-05
16:05:05,771::lvm::497::Storage.OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,772::lvm::499::Storage.OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,772::lvm::508::Storage.OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,773::lvm::510::Storage.OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,773::lvm::528::Storage.OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,774::lvm::530::Storage.OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,774::misc::760::Storage.SamplingMethod::(__call__) Returning
last result
Thread-17518::DEBUG::2016-05-05
16:05:05,775::lvm::319::Storage.OperationMutex::(_reloadpvs) Operation
'lvm reload operation' got the operation mutex
Thread-17518::DEBUG::2016-05-05
16:05:05,777::lvm::290::Storage.Misc.excCmd::(cmd) /usr/bin/taskset
--cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/lvm pvs --config ' devices {
preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1
write_cache_state=0 disable_after_error_count=3 filter = [
'\''r|.*|'\'' ] }
[ovirt-users] engine gives JBAS015011 java.lang.NullPointerException
Hi, One of our older ovirt engine 3.5.5 (CentOS 7) machines had a raid controller faillure and no recent ovirt-engine backup. So I recovered the files + database and reinstalled with ovirt-engine-3.5.6.2-1. But I can't get it to work. engine-setup runs fine but in the server.log I keep getting these and the webinterface is blank. 2016-05-05 17:24:43,431 INFO [org.apache.coyote.ajp.AjpProtocol] (MSC service thread 1-4) Starting Coyote AJP/1.3 on ajp--127.0.0.1-8702 2016-05-05 17:24:43,493 INFO [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on /127.0.0.1:8703 2016-05-05 17:24:43,704 INFO [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on /127.0.0.1:8706 2016-05-05 17:24:43,720 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) JBAS015012: Started FileSystemDeploymentService for directory /var/lib/ovirt-engine/jboss_runtime/deployments 2016-05-05 17:24:43,893 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010400: Bound data source [java:/ENGINEDataSource] 2016-05-05 17:24:43,899 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015011: Scan of /var/lib/ovirt-engine/jboss_runtime/deployments threw Exception: java.lang.NullPointerException at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.getDeploymentTimestamp(FileSystemDeploymentService.java:682) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.getDeploymentTimestamp(FileSystemDeploymentService.java:683) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.getDeploymentTimestamp(FileSystemDeploymentService.java:683) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.getDeploymentTimestamp(FileSystemDeploymentService.java:683) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.scanDirectory(FileSystemDeploymentService.java:525) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.scan(FileSystemDeploymentService.java:336) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentScanRunnable.run(FileSystemDeploymentService.java:149) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_101] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [rt.jar:1.7.0_101] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [rt.jar:1.7.0_101] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_101] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_101] at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA.jar:2.0.0.GA] 2016-05-05 17:24:44,141 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.1.Final "Brontes" started in 3718ms - Started 136 of 160 services (23 services are passive or on-demand) 2016-05-05 17:24:48,918 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015011: Scan of /var/lib/ovirt-engine/jboss_runtime/deployments threw Exception: java.lang.NullPointerException at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.getDeploymentTimestamp(FileSystemDeploymentService.java:682) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] at org.jboss.as.server.deployment.scanner.FileSystemDeploymentService.getDeploymentTimestamp(FileSystemDeploymentService.java:683) [jboss-as-deployment-scanner-7.1.1.Final.jar:7.1.1.Final] Anyone seen these errors before? I have all the files/directories of the old drive but all the permissions are gone :-( Met vriendelijke groet, With kind regards, Jorick Astrego Netbulae Virtualization Experts Tel: 053 20 30 270 i...@netbulae.euStaalsteden 4-3A KvK 0
Re: [ovirt-users] vdsmd and libvirtd services failed to start
> On 05 May 2016, at 16:18, Bhaskarakiran wrote: > > Hi, > > I have a HCI setup running on 3 nodes and created 6 VM's. Was running IO > (like dd and linux untar) on those VM's overnight. Next day i saw that for 2 > of the nodes vdsmd and libvirtd services failed and if manually started, they > don't come up. All the VM's state has changed to 'unknown' and failed to > migrate. Can someone help looking at the logs and figure out the RCA. Let me > know what all logs are needed, i can post the same. Hi, I saw your other bugs you reported already. In all cases it doesn’t seem like ovirt’s fault (except the buggy vdsm recovery flow), the underlying reason is that something got broken in either libvirt or qemu. For that you better enable libvirt debug logging as the default level is not so useful. You can find more details about logging at http://www.ovirt.org/develop/developer-guide/vdsm/log-files/ Once you have that please share/send vdsm.log (it’s rotated often, so check the times to cover the time from VM creation all the way to failure), libvirt.log with debug info and VM’s qemu log from /var/log/libvirt/qemu/.log Thanks, michal > > Thanks, > Bhaskarakiran. > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
