Hi all,
I'm writing to ask about the following problem (in a HC HE oVirt 3.6.7
GlusterFS 3.7.17 3-hosts-replica-with-arbiter sharded-volumes setup all on
CentOS 7.2):
>From /var/log/messages:
Dec 9 15:27:46 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
Dec 9 15:27:47 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:Extracting Engine VM OVF
from the OVF_STORE
Dec 9 15:27:47 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore:OVF_STORE volume path:
/rhev/data-center/mnt/glusterSD/shockley.gluster.private:_enginedomain/1d60fd45-507d-4a78-8294-d642b3178ea3/images/22a172de-698e-4cc5-bff0-082882fb3347/8738287c-8a25-4a2a-a53a-65c366a972a1
Dec 9 15:27:47 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Found an
OVF for HE VM, trying to convert
Dec 9 15:27:47 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config:Got vm.conf
from OVF_STORE
Dec 9 15:27:47 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Current state
EngineUp (score: 3400)
Dec 9 15:27:47 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Best remote host
read.mgmt.private (id: 2, score: 3400)
Dec 9 15:27:48 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection
established
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection
established
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection
established
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection
established
Dec 9 15:27:48 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection closed
Dec 9 15:27:48 shockley ovirt-ha-broker: INFO:mem_free.MemFree:memFree: 7392
Dec 9 15:27:50 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
Dec 9 15:27:52 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
Dec 9 15:27:54 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
Dec 9 15:27:55 shockley ovirt-ha-broker:
INFO:cpu_load_no_engine.EngineHealth:System load total=0.1234, engine=0.0364,
non-engine=0.0869
Dec 9 15:27:57 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Initializing VDSM
Dec 9 15:27:57 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Connecting the
storage
Dec 9 15:27:58 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
Dec 9 15:27:58 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting storage
server
Dec 9 15:27:58 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting storage
server
Dec 9 15:27:59 shockley ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Refreshing the
storage domain
Dec 9 15:27:59 shockley ovirt-ha-broker:
INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection
established
Dec 9 15:27:59 shockley journal: vdsm jsonrpc.JsonRpcServer ERROR Internal
server error#012Traceback (most recent call last):#012 File
"/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest#012 res = method(**params)#012 File
"/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod#012 result =
fn(*methodArgs)#012 File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in
status#012 return self._gluster.volumeStatus(volumeName, brick,
statusOption)#012 File "/usr/share/vdsm/gluster/api.py", line 86, in
wrapper#012 rv = func(*args, **kwargs)#012 File
"/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus#012
statusOption)#012 File "/usr/share/vdsm/supervdsm.py", line 50, in
__call__#012 return callMethod()#012 File "/usr/share/vdsm/supervdsm.py",
line 48, in <lambda>#012 **kwargs)#012 File "<string>", line 2, in
glusterVolumeStatus#012 File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _ca
llmethod#012 raise convert_to_error(kind, result)#012KeyError: 'device'
>From /var/log/vdsm/vdsm.log:
jsonrpc.Executor/1::ERROR::2016-12-09
15:27:46,870::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
jsonrpc.Executor/5::ERROR::2016-12-09
15:27:48,627::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
jsonrpc.Executor/7::ERROR::2016-12-09
15:27:50,164::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
jsonrpc.Executor/0::ERROR::2016-12-09
15:27:52,804::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
jsonrpc.Executor/5::ERROR::2016-12-09
15:27:54,679::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
jsonrpc.Executor/2::ERROR::2016-12-09
15:27:58,349::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
jsonrpc.Executor/4::ERROR::2016-12-09
15:27:59,169::__init__::538::jsonrpc.JsonRpcServer::(_serveRequest) Internal
server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 533, in
_serveRequest
res = method(**params)
File "/usr/share/vdsm/rpc/Bridge.py", line 275, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/gluster/apiwrapper.py", line 117, in status
return self._gluster.volumeStatus(volumeName, brick, statusOption)
File "/usr/share/vdsm/gluster/api.py", line 86, in wrapper
rv = func(*args, **kwargs)
File "/usr/share/vdsm/gluster/api.py", line 407, in volumeStatus
statusOption)
File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
**kwargs)
File "<string>", line 2, in glusterVolumeStatus
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in
_callmethod
raise convert_to_error(kind, result)
KeyError: 'device'
>From /var/log/vdsm/supervdsm.log:
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
MainProcess|jsonrpc.Executor/5::ERROR::2016-12-09
15:27:48,625::supervdsmServer::120::SuperVdsm.ServerCallback::(wrapper) Error
in wrapper
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
MainProcess|jsonrpc.Executor/7::ERROR::2016-12-09
15:27:50,163::supervdsmServer::120::SuperVdsm.ServerCallback::(wrapper) Error
in wrapper
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
MainProcess|jsonrpc.Executor/0::ERROR::2016-12-09
15:27:52,803::supervdsmServer::120::SuperVdsm.ServerCallback::(wrapper) Error
in wrapper
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
MainProcess|jsonrpc.Executor/5::ERROR::2016-12-09
15:27:54,677::supervdsmServer::120::SuperVdsm.ServerCallback::(wrapper) Error
in wrapper
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
MainProcess|jsonrpc.Executor/2::ERROR::2016-12-09
15:27:58,348::supervdsmServer::120::SuperVdsm.ServerCallback::(wrapper) Error
in wrapper
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
MainProcess|jsonrpc.Executor/4::ERROR::2016-12-09
15:27:59,168::supervdsmServer::120::SuperVdsm.ServerCallback::(wrapper) Error
in wrapper
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer", line 118, in wrapper
res = func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer", line 534, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/gluster/cli.py", line 352, in volumeStatus
return _parseVolumeStatusDetail(xmltree)
File "/usr/share/vdsm/gluster/cli.py", line 216, in _parseVolumeStatusDetail
'device': value['device'],
KeyError: 'device'
Please note that the whole oVirt cluster is working (apparently) as it should,
but due to a known limitation with split-GlusterFS-network setup
(http://lists.ovirt.org/pipermail/users/2016-August/042119.html solved in
https://gerrit.ovirt.org/#/c/60083/ but maybe not backported to 3.6.x or
present only in nightly later than 3.6.7, right?) GlusterFS volumes are being
managed from the hosts commandline only, while the oVirt Engine webui is used
only to monitor them.
The GlusterFS part is currently experiencing some recurring NFS crashes (using
internal GlusterFS NFS support, not NFS-Ganesha) as reported in Gluster users
mailing list and in Bugzilla
(http://www.gluster.org/pipermail/gluster-users/2016-December/029357.html and
https://bugzilla.redhat.com/show_bug.cgi?id=1381970 without any feedback
insofar...) but only on not-oVirt-related volumes.
Finally, I can confirm that checking all oVirt-related and not-oVirt-related
GlusterFS volumes from the hosts commandline with:
vdsClient -s localhost glusterVolumeStatus volumeName=nomevolume
always succeeds without errors.
Many thanks in advance for any advice (please note that I'm planning to upgrade
from 3.6.7 to latest nightly 3.6.10.x as soon as the corresponding RHEV gets
announced, then later on all the way up to 4.1.0 as soon as it stabilizes; on
GlusterFS-side I'd like to upgrade asap to 3.8.x but I cannot find any hint on
oVirt 3.6.x compatibility...).
Best regards,
Giuseppe
PS: please keep my address in to/copy since I still have problems receiving
oVirt mailing list messages on Hotmail.
_______________________________________________
Users mailing list
[email protected]
http://lists.phx.ovirt.org/mailman/listinfo/users
