Hi Martin,
Thanks for responding, and sorry for the long delay in getting back
to you. Where would I find the libvirt log? I checked on the host and
I have individual log files in /var/log/libvirt/qemu/<vmName>.log, but
nothing for the guest I'm trying to migrate.
I did some more digging and found these errors, which seem to point
to some sort of issue communicating with the JsonRpcServer module. I've
put the full vdsm.log output from the migration attempt up at
http://krissy.islandadmin.ca/public/vdsm.txt but the interesting parts
seems to be:
jsonrpc.Executor/3::DEBUG::2017-02-24
01:01:36,415::__init__::530::jsonrpc.JsonRpcServer::(_handle_request)
Calling 'Host.getAllVmIoTunePolicies' in bridge with {}
jsonrpc.Executor/3::ERROR::2017-02-24
01:01:36,436::__init__::550::jsonrpc.JsonRpcServer::(_handle_request)
Internal server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line
545, in _handle_request
res = method(**params)
File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 195,
in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/API.py", line 1358, in getAllVmIoTunePolicies
io_tune_policies_dict = self._cif.getAllVmIoTunePolicies()
File "/usr/share/vdsm/clientIF.py", line 447, in getAllVmIoTunePolicies
vm_io_tune_policies[v.id] = {'policy': v.getIoTunePolicy(),
File "/usr/share/vdsm/virt/vm.py", line 2772, in getIoTunePolicy
qos = self._getVmPolicy()
File "/usr/share/vdsm/virt/vm.py", line 2746, in _getVmPolicy
metadata_xml = self._dom.metadata(
File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line
47, in __getattr__
% self.vmid)
NotConnectedError: VM u'3390a529-e0c3-4f1f-9c8f-7595b2b3f38f' was not
started yet or was shut down
jsonrpc.Executor/3::INFO::2017-02-24
01:01:36,437::__init__::513::jsonrpc.JsonRpcServer::(_serveRequest) RPC
call Host.getAllVmIoTunePolicies failed (error -32603) in 0.03 seconds
Thread-351015::DEBUG::2017-02-24
01:01:36,471::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = '';
<rc> = 0
Thread-351015::DEBUG::2017-02-24
01:01:36,471::fileUtils::209::Storage.fileUtils::(createdir) Creating
directory:
/var/run/vdsm/storage/b22b3666-f8d6-4beb-ac19-f2a6c8f49271/a8da74c5-24d8-46c1-9a70-92e3045a3c18
Thread-351015::DEBUG::2017-02-24
01:01:36,472::blockSD::1186::Storage.StorageDomain::(createImageLinks)
Creating symlink from
/dev/b22b3666-f8d6-4beb-ac19-f2a6c8f49271/2e00f0ba-5d93-4c3b-af37-b53dcc68a43f
to /v
Thread-351015::DEBUG::2017-02-24
01:01:36,472::blockSD::1150::Storage.StorageDomain::(linkBCImage)
Creating symlink from
/var/run/vdsm/storage/b22b3666-f8d6-4beb-ac19-f2a6c8f49271/a8da74c5-24d8-46c1-9a70-92e304
...
jsonrpc.Executor/6::DEBUG::2017-02-24
01:01:37,645::resourceManager::241::Storage.ResourceManager.Request::(grant)
ResName=`Storage.b22b3666-f8d6-4beb-ac19-f2a6c8f49271`ReqID=`108242ef-f04e-4b9b-8a08-8f9ed1fae9
jsonrpc.Executor/6::DEBUG::2017-02-24
01:01:37,645::task::829::Storage.TaskManager.Task::(resourceAcquired)
Task=`a9e9fcd9-65fe-4652-96d1-8afc2cf47269`::_resourcesAcquired:
Storage.b22b3666-f8d6-4beb-ac19-f2a6c
jsonrpc.Executor/6::DEBUG::2017-02-24
01:01:37,645::task::995::Storage.TaskManager.Task::(_decref)
Task=`a9e9fcd9-65fe-4652-96d1-8afc2cf47269`::ref 1 aborting False
jsonrpc.Executor/6::DEBUG::2017-02-24
01:01:37,645::fileUtils::190::Storage.fileUtils::(cleanupdir) Removing
directory:
/var/run/vdsm/storage/b22b3666-f8d6-4beb-ac19-f2a6c8f49271/9cdfe4ab-c23e-42cd-8f45-2ca573a
jsonrpc.Executor/6::DEBUG::2017-02-24
01:01:37,646::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/taskset
--cpu-list 0-15 /usr/bin/sudo -n /usr/sbin/lvm lvs --config ' devices {
preferred_names = ["^/dev/mapper
Thread-351015::ERROR::2017-02-24
01:01:37,656::vm::767::virt.vm::(_startUnderlyingVm)
vmId=`3390a529-e0c3-4f1f-9c8f-7595b2b3f38f`::Failed to start a migration
destination vm
Traceback (most recent call last):
File "/usr/share/vdsm/virt/vm.py", line 733, in _startUnderlyingVm
self._completeIncomingMigration()
File "/usr/share/vdsm/virt/vm.py", line 3113, in
_completeIncomingMigration
self._incomingMigrationFinished.isSet(), usedTimeout)
File "/usr/share/vdsm/virt/vm.py", line 3196, in
_attachLibvirtDomainAfterMigration
raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid
'3390a529-e0c3-4f1f-9c8f-7595b2b3f38f'
Thread-351015::INFO::2017-02-24
01:01:37,658::vm::1330::virt.vm::(setDownStatus)
vmId=`3390a529-e0c3-4f1f-9c8f-7595b2b3f38f`::Changed state to Down: VM
failed to migrate (code=8)
Thread-351015::INFO::2017-02-24
01:01:37,658::guestagent::430::virt.vm::(stop)
vmId=`3390a529-e0c3-4f1f-9c8f-7595b2b3f38f`::Stopping connection
Thread-351015::DEBUG::2017-02-24
01:01:37,660::__init__::209::jsonrpc.Notification::(emit) Sending event
{"params": {"notify_time": 5250206560,
"3390a529-e0c3-4f1f-9c8f-7595b2b3f38f": {"status": "Down", "timeOf
Thread-351015::DEBUG::2017-02-24
01:01:37,660::vm::777::virt.vm::(_startUnderlyingVm)
vmId=`3390a529-e0c3-4f1f-9c8f-7595b2b3f38f`::Releasing incoming
migration semaphore
On 2017-02-15 05:16 AM, Martin Sivak wrote:
Hi Charles,
I am not sure about the migration issue, we need more logs [1] to be
able to tell, but the SSL issue is probably just annoying and not a
functionality bug (see
https://bugzilla.redhat.com/show_bug.cgi?id=1412583).
Best regards
[1] libvirt log for the domain, the VM process might be crashing for some reason
--
Martin Sivak
SLA / oVirt
On Wed, Feb 15, 2017 at 3:45 AM, Charles Tassell <[email protected]> wrote:
Hi Everyone,
I recently installed oVirt 4.0 on two servers with an iSCSI backstore, and
I'm having two issues I'm having trouble tracking down. First, the initial
datastore that the hosted engine is running on won't connect. It just says
"Unattached" in the GUI, and I can't find an error message anywhere. Not a
huge deal, it just means I can't see the hosted engine itself in the GUI.
More importantly: I can't migrate VMs. They fail with errors like this in
the vdsm.log of the destination host:
Thread-60381::ERROR::2017-02-14
21:55:47,921::vm::767::virt.vm::(_startUnderlyingVm)
vmId=`59f235cc-3ab2-4dae-b460-62be850e3d5d`::Failed to start a migration
destination vm
Traceback (most recent call last):
File "/usr/share/vdsm/virt/vm.py", line 733, in _startUnderlyingVm
self._completeIncomingMigration()
File "/usr/share/vdsm/virt/vm.py", line 3113, in
_completeIncomingMigration
self._incomingMigrationFinished.isSet(), usedTimeout)
File "/usr/share/vdsm/virt/vm.py", line 3196, in
_attachLibvirtDomainAfterMigration
raise MigrationError(e.get_error_message())
MigrationError: Domain not found: no domain with matching uuid
'59f235cc-3ab2-4dae-b460-62be850e3d5d'
If I shut the VM down and start it on the host it works fine, it just
won't migrate. I notice I'm also seeing a lot (like every 7 seconds) of
this error:
JsonRpc (StompReactor)::ERROR::2017-02-14
21:57:33,984::betterAsyncore::113::vds.dispatcher::(recv) SSL error during
reading data: unexpected eof
Which I can't seem to diagnose. Any ideas?
_______________________________________________
Users mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
[email protected]
http://lists.ovirt.org/mailman/listinfo/users
