Re: [ovirt-users] Issue while importing the existing storage domain

2016-04-21 Thread Nir Soffer 
On Thu, Apr 21, 2016 at 8:44 AM, Idan Shaby  wrote:
> Hi Satheesaran,
>
> Please file a BZ and attach all the relevant logs so we can investigate it
> properly.

In particular we would like to see sanlock.log (/var/log/sanlock.log)
and glusterfs logs
(/var/log/glusterfs/server:path*.log

Nir

>
>
> Regards,
> Idan
>
> On Wed, Apr 20, 2016 at 12:42 PM, SATHEESARAN  wrote:
>>
>> Hi All,
>>
>> I was testing the gluster geo-replication on RHEV storage domain backed by
>> gluster volume.
>> In this case, storage domain ( data domain ) was created with gluster
>> replica 3 volume.
>>
>> The VMs additional disks are carved out from this storage domain.
>>
>> Now I have geo-replicated[1] the gluster volume to the remote volume.
>> When I try importing this storage domain in another RHEVM instance, it
>> fails with error "internal engine error"
>>  I see the following error in engine.log
>>
>> 
>> 2016-04-20 05:13:47,685 ERROR
>> [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
>> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Failed in 'DetachStorageDomainVDS' method
>> 2016-04-20 05:13:47,708 ERROR
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Correlation ID: null, Call Stack: null,
>> Custom Event ID: -1, Message: VDSM command failed: Cannot acquire host id:
>> (u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262, 'Sanlock
>> lockspace add failure', 'Sanlock exception'))
>> 2016-04-20 05:13:47,708 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
>> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Command
>> 'org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand'
>> return value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=661,
>> message=Cannot acquire host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e',
>> SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock
>> exception'))]]'
>> 
>>
>> The complete logs are available in the fpaste[2]
>> Attaching the part of vdsm log to this mail
>>
>> [1] - geo-replication is the feature in glusterfs where the contents of
>> volume are asynchronously replicated in remote volume.
>> This is used for disaster-recovery workflow
>>
>> [2] - https://paste.fedoraproject.org/357701/11448771/
>>
>> Thanks,
>> Satheesaran S
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Issue while importing the existing storage domain

2016-04-21 Thread Amit Aviram 
Adding users list, dropped from the reply.

On Wed, Apr 20, 2016 at 1:49 PM, Amit Aviram  wrote:

> Hi sasundar.
> IIUC, The storage that is being imported is a backup that was generated by
> other process than VDSM?
> If that is the case, the files inside might have other user as an owner,
> which might cause permission problems to VDSM while trying to write to it.
>
> Can you try changing the owner back to VDSM and try importing again?
>
> Please let us know if that did the trick.
>
> Thanks, Amit.
>
> On Wed, Apr 20, 2016 at 12:42 PM, SATHEESARAN  wrote:
>
>> Hi All,
>>
>> I was testing the gluster geo-replication on RHEV storage domain backed
>> by gluster volume.
>> In this case, storage domain ( data domain ) was created with gluster
>> replica 3 volume.
>>
>> The VMs additional disks are carved out from this storage domain.
>>
>> Now I have geo-replicated[1] the gluster volume to the remote volume.
>> When I try importing this storage domain in another RHEVM instance, it
>> fails with error "internal engine error"
>>  I see the following error in engine.log
>>
>> 
>> 2016-04-20 05:13:47,685 ERROR
>> [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
>> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Failed in 'DetachStorageDomainVDS'
>> method
>> 2016-04-20 05:13:47,708 ERROR
>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Correlation ID: null, Call Stack:
>> null, Custom Event ID: -1, Message: VDSM command failed: Cannot acquire
>> host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262,
>> 'Sanlock lockspace add failure', 'Sanlock exception'))
>> 2016-04-20 05:13:47,708 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
>> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Command
>> 'org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand'
>> return value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=661,
>> message=Cannot acquire host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e',
>> SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock
>> exception'))]]'
>> 
>>
>> The complete logs are available in the fpaste[2]
>> Attaching the part of vdsm log to this mail
>>
>> [1] - geo-replication is the feature in glusterfs where the contents of
>> volume are asynchronously replicated in remote volume.
>> This is used for disaster-recovery workflow
>>
>> [2] - https://paste.fedoraproject.org/357701/11448771/
>>
>> Thanks,
>> Satheesaran S
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] Issue while importing the existing storage domain

2016-04-20 Thread Idan Shaby 
Hi Satheesaran,

Please file a BZ and attach all the relevant logs so we can investigate it
properly.


Regards,
Idan

On Wed, Apr 20, 2016 at 12:42 PM, SATHEESARAN  wrote:

> Hi All,
>
> I was testing the gluster geo-replication on RHEV storage domain backed by
> gluster volume.
> In this case, storage domain ( data domain ) was created with gluster
> replica 3 volume.
>
> The VMs additional disks are carved out from this storage domain.
>
> Now I have geo-replicated[1] the gluster volume to the remote volume.
> When I try importing this storage domain in another RHEVM instance, it
> fails with error "internal engine error"
>  I see the following error in engine.log
>
> 
> 2016-04-20 05:13:47,685 ERROR
> [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Failed in 'DetachStorageDomainVDS'
> method
> 2016-04-20 05:13:47,708 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Correlation ID: null, Call Stack:
> null, Custom Event ID: -1, Message: VDSM command failed: Cannot acquire
> host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262,
> 'Sanlock lockspace add failure', 'Sanlock exception'))
> 2016-04-20 05:13:47,708 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
> (ajp-/127.0.0.1:8702-3) [20f6ea4c] Command
> 'org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand'
> return value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=661,
> message=Cannot acquire host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e',
> SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock
> exception'))]]'
> 
>
> The complete logs are available in the fpaste[2]
> Attaching the part of vdsm log to this mail
>
> [1] - geo-replication is the feature in glusterfs where the contents of
> volume are asynchronously replicated in remote volume.
> This is used for disaster-recovery workflow
>
> [2] - https://paste.fedoraproject.org/357701/11448771/
>
> Thanks,
> Satheesaran S
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] Issue while importing the existing storage domain

2016-04-20 Thread SATHEESARAN 

Hi All,

I was testing the gluster geo-replication on RHEV storage domain backed 
by gluster volume.
In this case, storage domain ( data domain ) was created with gluster 
replica 3 volume.


The VMs additional disks are carved out from this storage domain.

Now I have geo-replicated[1] the gluster volume to the remote volume.
When I try importing this storage domain in another RHEVM instance, it 
fails with error "internal engine error"

 I see the following error in engine.log


2016-04-20 05:13:47,685 ERROR 
[org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-3) 
[20f6ea4c] Failed in 'DetachStorageDomainVDS' method
2016-04-20 05:13:47,708 ERROR 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(ajp-/127.0.0.1:8702-3) [20f6ea4c] Correlation ID: null, Call Stack: 
null, Custom Event ID: -1, Message: VDSM command failed: Cannot acquire 
host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e', 
SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock 
exception'))
2016-04-20 05:13:47,708 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-3) 
[20f6ea4c] Command 
'org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand' return 
value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=661, 
message=Cannot acquire host id: 
(u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262, 
'Sanlock lockspace add failure', 'Sanlock exception'))]]'



The complete logs are available in the fpaste[2]
Attaching the part of vdsm log to this mail

[1] - geo-replication is the feature in glusterfs where the contents of 
volume are asynchronously replicated in remote volume.

This is used for disaster-recovery workflow

[2] - https://paste.fedoraproject.org/357701/11448771/

Thanks,
Satheesaran S
BindingXMLRPC::INFO::2016-04-20 
10:42:10,604::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request 
handler for 127.0.0.1:37704
Thread-4816::INFO::2016-04-20 
10:42:10,605::xmlrpc::84::vds.XMLRPCServer::(_process_requests) Request handler 
for 127.0.0.1:37704 started
Thread-4816::INFO::2016-04-20 
10:42:10,611::xmlrpc::92::vds.XMLRPCServer::(_process_requests) Request handler 
for 127.0.0.1:37704 stopped
jsonrpc.Executor/0::ERROR::2016-04-20 
10:42:11,407::task::866::Storage.TaskManager.Task::(_setError) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 49, in wrapper
res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 805, in forcedDetachStorageDomain
self._deatchStorageDomainFromOldPools(sdUUID)
  File "/usr/share/vdsm/storage/hsm.py", line 781, in 
_deatchStorageDomainFromOldPools
dom.acquireHostId(pool.id)
  File "/usr/share/vdsm/storage/sd.py", line 533, in acquireHostId
self._clusterLock.acquireHostId(hostId, async)
  File "/usr/share/vdsm/storage/clusterlock.py", line 234, in acquireHostId
raise se.AcquireHostIdFailure(self._sdUUID, e)
AcquireHostIdFailure: Cannot acquire host id: 
(u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262, 'Sanlock 
lockspace add failure', 'Sanlock exception'))
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::885::Storage.TaskManager.Task::(_run) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Task._run: 
14b4ecd2-41f1-4cf3-bb21-8ba5e433f
1c7 (u'89061d19-fb76-47c9-a4aa-22b0062b769e', 
u'----') {} failed - stopping task
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::1246::Storage.TaskManager.Task::(stop) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::stopping in state preparing (force 
False)
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::993::Storage.TaskManager.Task::(_decref) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::ref 1 aborting True
jsonrpc.Executor/0::INFO::2016-04-20 
10:42:11,408::task::1171::Storage.TaskManager.Task::(prepare) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::aborting: Task is aborted: 'Cannot 
acquir
e host id' - code 661
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::1176::Storage.TaskManager.Task::(prepare) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Prepare: aborted: Cannot acquire 
host id
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::993::Storage.TaskManager.Task::(_decref) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::ref 0 aborting True
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::928::Storage.TaskManager.Task::(_doAbort) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Task._doAbort: force False
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,409::resourceManager::980::Storage.ResourceManager.Owner::(cancelAll) 
Owner.cancelAll requests {}
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,409::task::595::Storage.TaskManager.Task::(_updateState)

[ovirt-users] Issue while importing the existing storage domain

2016-04-20 Thread SATHEESARAN 

Hi All,

I was testing the gluster geo-replication on RHEV storage domain backed 
by gluster volume.
In this case, storage domain ( data domain ) was created with gluster 
replica 3 volume.


The VMs additional disks are carved out from this storage domain.

Now I have geo-replicated[1] the gluster volume to the remote volume.
When I try importing this storage domain in another RHEVM instance, it 
fails with error "internal engine error"

 I see the following error in engine.log


2016-04-20 05:13:47,685 ERROR 
[org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-3) 
[20f6ea4c] Failed in 'DetachStorageDomainVDS' method
2016-04-20 05:13:47,708 ERROR 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(ajp-/127.0.0.1:8702-3) [20f6ea4c] Correlation ID: null, Call Stack: 
null, Custom Event ID: -1, Message: VDSM command failed: Cannot acquire 
host id: (u'89061d19-fb76-47c9-a4aa-22b0062b769e', 
SanlockException(-262, 'Sanlock lockspace add failure', 'Sanlock 
exception'))
2016-04-20 05:13:47,708 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-3) 
[20f6ea4c] Command 
'org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand' return 
value 'StatusOnlyReturnForXmlRpc [status=StatusForXmlRpc [code=661, 
message=Cannot acquire host id: 
(u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262, 
'Sanlock lockspace add failure', 'Sanlock exception'))]]'



The complete logs are available in the fpaste[2]
Attaching the part of vdsm log to this mail

[1] - geo-replication is the feature in glusterfs where the contents of 
volume are asynchronously replicated in remote volume.

This is used for disaster-recovery workflow

[2] - https://paste.fedoraproject.org/357701/11448771/

Thanks,
Satheesaran S
BindingXMLRPC::INFO::2016-04-20 
10:42:10,604::xmlrpc::73::vds.XMLRPCServer::(handle_request) Starting request 
handler for 127.0.0.1:37704
Thread-4816::INFO::2016-04-20 
10:42:10,605::xmlrpc::84::vds.XMLRPCServer::(_process_requests) Request handler 
for 127.0.0.1:37704 started
Thread-4816::INFO::2016-04-20 
10:42:10,611::xmlrpc::92::vds.XMLRPCServer::(_process_requests) Request handler 
for 127.0.0.1:37704 stopped
jsonrpc.Executor/0::ERROR::2016-04-20 
10:42:11,407::task::866::Storage.TaskManager.Task::(_setError) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 49, in wrapper
res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 805, in forcedDetachStorageDomain
self._deatchStorageDomainFromOldPools(sdUUID)
  File "/usr/share/vdsm/storage/hsm.py", line 781, in 
_deatchStorageDomainFromOldPools
dom.acquireHostId(pool.id)
  File "/usr/share/vdsm/storage/sd.py", line 533, in acquireHostId
self._clusterLock.acquireHostId(hostId, async)
  File "/usr/share/vdsm/storage/clusterlock.py", line 234, in acquireHostId
raise se.AcquireHostIdFailure(self._sdUUID, e)
AcquireHostIdFailure: Cannot acquire host id: 
(u'89061d19-fb76-47c9-a4aa-22b0062b769e', SanlockException(-262, 'Sanlock 
lockspace add failure', 'Sanlock exception'))
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::885::Storage.TaskManager.Task::(_run) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Task._run: 
14b4ecd2-41f1-4cf3-bb21-8ba5e433f
1c7 (u'89061d19-fb76-47c9-a4aa-22b0062b769e', 
u'----') {} failed - stopping task
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::1246::Storage.TaskManager.Task::(stop) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::stopping in state preparing (force 
False)
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::993::Storage.TaskManager.Task::(_decref) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::ref 1 aborting True
jsonrpc.Executor/0::INFO::2016-04-20 
10:42:11,408::task::1171::Storage.TaskManager.Task::(prepare) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::aborting: Task is aborted: 'Cannot 
acquir
e host id' - code 661
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::1176::Storage.TaskManager.Task::(prepare) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Prepare: aborted: Cannot acquire 
host id
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::993::Storage.TaskManager.Task::(_decref) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::ref 0 aborting True
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,408::task::928::Storage.TaskManager.Task::(_doAbort) 
Task=`14b4ecd2-41f1-4cf3-bb21-8ba5e433f1c7`::Task._doAbort: force False
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,409::resourceManager::980::Storage.ResourceManager.Owner::(cancelAll) 
Owner.cancelAll requests {}
jsonrpc.Executor/0::DEBUG::2016-04-20 
10:42:11,409::task::595::Storage.TaskManager.Task::(_updateState)