[ovirt-users] Re: cannot migrate hosted engine to sandybridge hypervisor

[Douglas Duckworth]

Thanks!

No big deal as it's now working.

On Fri, Aug 24, 2018, 9:04 AM Simone Tiraboschi  wrote:

>
>
> On Fri, Aug 24, 2018 at 3:06 AM Douglas Duckworth 
> wrote:
>
>> Rebooted the hosted engine VM
>>
>> Migration now works
>>
>> Guess that's needed after changing cluster CPU type
>>
>
> Yes, exactly: we already have an open bug about that:
> https://bugzilla.redhat.com/show_bug.cgi?id=1585986
> 
>
>
>>
>> Thanks,
>>
>> Douglas Duckworth, MSc, LFCS
>> HPC System Administrator
>> Scientific Computing Unit
>> Weill Cornell Medicine
>> 1300 York - LC-502
>> E: d...@med.cornell.edu
>> O: 212-746-6305
>> F: 212-746-8690
>>
>>
>> On Thu, Aug 23, 2018 at 8:46 PM, Douglas Duckworth <
>> dod2...@med.cornell.edu> wrote:
>>
>>> Hi
>>>
>>> I am unable to migrate to hypervisor that's sandy bridge CPU.
>>>
>>> The cluster supports CPUs that old.
>>>
>>> This is the message on host when migration fails:
>>>
>>> Aug 23 20:40:14 lyon vdsm[9648]: WARN Worker blocked: >> name=periodic/1 running >> action= at
>>> 0x7f2f8054b350> timeout=15, duration=15 at 0x7f2f8048d290> task#=79 at
>>> 0x7f2f80540490>, traceback:#012File: "/usr/lib64/python2.7/threading.py",
>>> line 785, in __bootstrap#012  self.__bootstrap_inner()#012File:
>>> "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012
>>> self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in
>>> run#012  self.__target(*self.__args, **self.__kwargs)#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in
>>> run#012  ret = func(*args, **kwargs)#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012
>>> self._execute_task()#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in
>>> _execute_task#012  task()#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in
>>> __call__#012  self._callable()#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", line 220, in
>>> __call__#012  self._func()#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", line 580, in
>>> __call__#012  stats = hostapi.get_stats(self._cif,
>>> self._samples.stats())#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 77, in
>>> get_stats#012  ret['haStats'] = _getHaInfo()#012File:
>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 182, in
>>> _getHaInfo#012  stats = instance.get_all_stats()#012File:
>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>>> line 94, in get_all_stats#012  stats =
>>> broker.get_stats_from_storage()#012File:
>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>> line 135, in get_stats_from_storage#012  result =
>>> self._proxy.get_stats()#012File: "/usr/lib64/python2.7/xmlrpclib.py", line
>>> 1233, in __call__#012  return self.__send(self.__name, args)#012File:
>>> "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request#012
>>> verbose=self.__verbose#012File: "/usr/lib64/python2.7/xmlrpclib.py", line
>>> 1273, in request#012  return self.single_request(host, handler,
>>> request_body, verbose)#012File: "/usr/lib64/python2.7/xmlrpclib.py", line
>>> 1303, in single_request#012  response =
>>> h.getresponse(buffering=True)#012File: "/usr/lib64/python2.7/httplib.py",
>>> line 1113, in getresponse#012  response.begin()#012File:
>>> "/usr/lib64/python2.7/httplib.py", line 444, in begin#012  version, status,
>>> reason = self._read_status()#012File: "/usr/lib64/python2.7/httplib.py",
>>> line 400, in _read_status#012  line = self.fp.readline(_MAXLINE +
>>> 1)#012File: "/usr/lib64/python2.7/socket.py", line 476, in readline#012
>>> data = self._sock.recv(self._rbufsize)
>>> Aug 23 20:43:45 lyon vdsm[9648]: WARN ping was deprecated in favor of
>>> ping2 and confirmConnectivity
>>> Aug 23 20:43:45 lyon vdsm[9648]: WARN Attempting to add an existing net
>>> user: ovirtmgmt/adf14389-1563-4b1a-9af6-4b40370a825b
>>> Aug 23 20:43:46 lyon lldpad: recvfrom(Event interface): No buffer space
>>> available
>>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered blocking
>>> state
>>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered disabled
>>> state
>>> Aug 23 20:43:46 lyon kernel: device vnet0 entered promiscuous mode
>>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered blocking
>>> state
>>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered forwarding
>>> state
>>> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4326]
>>> manager: (vnet0): new Tun device
>>> (/org/freedesktop/NetworkManager/Devices/57)
>>> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4336]
>>> device 

[ovirt-users] Re: cannot migrate hosted engine to sandybridge hypervisor

[Simone Tiraboschi]

On Fri, Aug 24, 2018 at 3:06 AM Douglas Duckworth 
wrote:

> Rebooted the hosted engine VM
>
> Migration now works
>
> Guess that's needed after changing cluster CPU type
>

Yes, exactly: we already have an open bug about that:
https://bugzilla.redhat.com/show_bug.cgi?id=1585986


>
> Thanks,
>
> Douglas Duckworth, MSc, LFCS
> HPC System Administrator
> Scientific Computing Unit
> Weill Cornell Medicine
> 1300 York - LC-502
> E: d...@med.cornell.edu
> O: 212-746-6305
> F: 212-746-8690
>
>
> On Thu, Aug 23, 2018 at 8:46 PM, Douglas Duckworth <
> dod2...@med.cornell.edu> wrote:
>
>> Hi
>>
>> I am unable to migrate to hypervisor that's sandy bridge CPU.
>>
>> The cluster supports CPUs that old.
>>
>> This is the message on host when migration fails:
>>
>> Aug 23 20:40:14 lyon vdsm[9648]: WARN Worker blocked: > name=periodic/1 running > action= at
>> 0x7f2f8054b350> timeout=15, duration=15 at 0x7f2f8048d290> task#=79 at
>> 0x7f2f80540490>, traceback:#012File: "/usr/lib64/python2.7/threading.py",
>> line 785, in __bootstrap#012  self.__bootstrap_inner()#012File:
>> "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner#012
>> self.run()#012File: "/usr/lib64/python2.7/threading.py", line 765, in
>> run#012  self.__target(*self.__args, **self.__kwargs)#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in
>> run#012  ret = func(*args, **kwargs)#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run#012
>> self._execute_task()#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in
>> _execute_task#012  task()#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in
>> __call__#012  self._callable()#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", line 220, in
>> __call__#012  self._func()#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/virt/sampling.py", line 580, in
>> __call__#012  stats = hostapi.get_stats(self._cif,
>> self._samples.stats())#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 77, in
>> get_stats#012  ret['haStats'] = _getHaInfo()#012File:
>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 182, in
>> _getHaInfo#012  stats = instance.get_all_stats()#012File:
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>> line 94, in get_all_stats#012  stats =
>> broker.get_stats_from_storage()#012File:
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 135, in get_stats_from_storage#012  result =
>> self._proxy.get_stats()#012File: "/usr/lib64/python2.7/xmlrpclib.py", line
>> 1233, in __call__#012  return self.__send(self.__name, args)#012File:
>> "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request#012
>> verbose=self.__verbose#012File: "/usr/lib64/python2.7/xmlrpclib.py", line
>> 1273, in request#012  return self.single_request(host, handler,
>> request_body, verbose)#012File: "/usr/lib64/python2.7/xmlrpclib.py", line
>> 1303, in single_request#012  response =
>> h.getresponse(buffering=True)#012File: "/usr/lib64/python2.7/httplib.py",
>> line 1113, in getresponse#012  response.begin()#012File:
>> "/usr/lib64/python2.7/httplib.py", line 444, in begin#012  version, status,
>> reason = self._read_status()#012File: "/usr/lib64/python2.7/httplib.py",
>> line 400, in _read_status#012  line = self.fp.readline(_MAXLINE +
>> 1)#012File: "/usr/lib64/python2.7/socket.py", line 476, in readline#012
>> data = self._sock.recv(self._rbufsize)
>> Aug 23 20:43:45 lyon vdsm[9648]: WARN ping was deprecated in favor of
>> ping2 and confirmConnectivity
>> Aug 23 20:43:45 lyon vdsm[9648]: WARN Attempting to add an existing net
>> user: ovirtmgmt/adf14389-1563-4b1a-9af6-4b40370a825b
>> Aug 23 20:43:46 lyon lldpad: recvfrom(Event interface): No buffer space
>> available
>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered blocking
>> state
>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered disabled
>> state
>> Aug 23 20:43:46 lyon kernel: device vnet0 entered promiscuous mode
>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered blocking
>> state
>> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered forwarding
>> state
>> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4326]
>> manager: (vnet0): new Tun device
>> (/org/freedesktop/NetworkManager/Devices/57)
>> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4336]
>> device (vnet0): state change: unmanaged -> unavailable (reason
>> 'connection-assumed', sys-iface-state: 'external')
>> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4341]
>> device (vnet0): state change: unavailable -> disconnected (reason 'none',
>> sys-iface-state: 'external')
>> Aug 23 20:43:46 lyon dbus[1396]: [system] Activating via systemd: service
>> name='org.freedesktop.machine1' unit='dbus-org.freedesktop.machine1.service'
>> Aug 23 20:43:46 lyon systemd: Cannot add dependency job for 

[ovirt-users] Re: cannot migrate hosted engine to sandybridge hypervisor

[Douglas Duckworth]

Rebooted the hosted engine VM

Migration now works

Guess that's needed after changing cluster CPU type

Thanks,

Douglas Duckworth, MSc, LFCS
HPC System Administrator
Scientific Computing Unit
Weill Cornell Medicine
1300 York - LC-502
E: d...@med.cornell.edu
O: 212-746-6305
F: 212-746-8690


On Thu, Aug 23, 2018 at 8:46 PM, Douglas Duckworth 
wrote:

> Hi
>
> I am unable to migrate to hypervisor that's sandy bridge CPU.
>
> The cluster supports CPUs that old.
>
> This is the message on host when migration fails:
>
> Aug 23 20:40:14 lyon vdsm[9648]: WARN Worker blocked:  name=periodic/1 running  action= object at 0x7f2f8054b310> at 0x7f2f8054b350> timeout=15, duration=15 at
> 0x7f2f8048d290> task#=79 at 0x7f2f80540490>, traceback:#012File:
> "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap#012
> self.__bootstrap_inner()#012File: "/usr/lib64/python2.7/threading.py",
> line 812, in __bootstrap_inner#012  self.run()#012File:
> "/usr/lib64/python2.7/threading.py", line 765, in run#012
> self.__target(*self.__args, **self.__kwargs)#012File:
> "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194,
> in run#012  ret = func(*args, **kwargs)#012File: 
> "/usr/lib/python2.7/site-packages/vdsm/executor.py",
> line 301, in _run#012  self._execute_task()#012File:
> "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in
> _execute_task#012  task()#012File: 
> "/usr/lib/python2.7/site-packages/vdsm/executor.py",
> line 391, in __call__#012  self._callable()#012File:
> "/usr/lib/python2.7/site-packages/vdsm/virt/periodic.py", line 220, in
> __call__#012  self._func()#012File: "/usr/lib/python2.7/site-
> packages/vdsm/virt/sampling.py", line 580, in __call__#012  stats =
> hostapi.get_stats(self._cif, self._samples.stats())#012File:
> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 77, in
> get_stats#012  ret['haStats'] = _getHaInfo()#012File:
> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 182, in
> _getHaInfo#012  stats = instance.get_all_stats()#012File:
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
> line 94, in get_all_stats#012  stats = 
> broker.get_stats_from_storage()#012File:
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
> line 135, in get_stats_from_storage#012  result = 
> self._proxy.get_stats()#012File:
> "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__#012  return
> self.__send(self.__name, args)#012File: "/usr/lib64/python2.7/xmlrpclib.py",
> line 1591, in __request#012  verbose=self.__verbose#012File:
> "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request#012  return
> self.single_request(host, handler, request_body, verbose)#012File:
> "/usr/lib64/python2.7/xmlrpclib.py", line 1303, in single_request#012
> response = h.getresponse(buffering=True)#012File:
> "/usr/lib64/python2.7/httplib.py", line 1113, in getresponse#012
> response.begin()#012File: "/usr/lib64/python2.7/httplib.py", line 444, in
> begin#012  version, status, reason = self._read_status()#012File:
> "/usr/lib64/python2.7/httplib.py", line 400, in _read_status#012  line =
> self.fp.readline(_MAXLINE + 1)#012File: "/usr/lib64/python2.7/socket.py",
> line 476, in readline#012  data = self._sock.recv(self._rbufsize)
> Aug 23 20:43:45 lyon vdsm[9648]: WARN ping was deprecated in favor of
> ping2 and confirmConnectivity
> Aug 23 20:43:45 lyon vdsm[9648]: WARN Attempting to add an existing net
> user: ovirtmgmt/adf14389-1563-4b1a-9af6-4b40370a825b
> Aug 23 20:43:46 lyon lldpad: recvfrom(Event interface): No buffer space
> available
> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered blocking
> state
> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered disabled
> state
> Aug 23 20:43:46 lyon kernel: device vnet0 entered promiscuous mode
> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered blocking
> state
> Aug 23 20:43:46 lyon kernel: ovirtmgmt: port 2(vnet0) entered forwarding
> state
> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4326]
> manager: (vnet0): new Tun device (/org/freedesktop/
> NetworkManager/Devices/57)
> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4336]
> device (vnet0): state change: unmanaged -> unavailable (reason
> 'connection-assumed', sys-iface-state: 'external')
> Aug 23 20:43:46 lyon NetworkManager[1658]:   [1535071426.4341]
> device (vnet0): state change: unavailable -> disconnected (reason 'none',
> sys-iface-state: 'external')
> Aug 23 20:43:46 lyon dbus[1396]: [system] Activating via systemd: service
> name='org.freedesktop.machine1' unit='dbus-org.freedesktop.
> machine1.service'
> Aug 23 20:43:46 lyon systemd: Cannot add dependency job for unit
> lvm2-lvmetad.socket, ignoring: Unit is masked.
> Aug 23 20:43:46 lyon systemd: Starting Virtual Machine and Container
> Registration Service...
> Aug 23 20:43:46 lyon dbus[1396]: [system] Successfully activated service
> 'org.freedesktop.machine1'
> Aug 23 20:43:46 lyon