[ovirt-users] Re: Unable to start VMs after upgrade from 4.4.9 to 4.4.10
Hi Ales,
thank you so much for the feedback and for the possible workaround.
For the moment I have tweaked the bridge to perform the port mirror.
Best regards
Christoph
Am 28.03.22 um 07:08 schrieb Ales Musil:
Hi,
this is already fixed in oVirt 4.5 [0]. Possible workaround might be
to downgrade the iproute-tc package below version 5.15.0-3.
Regards,
Ales
[0]
https://github.com/oVirt/vdsm/commit/503a98139d65daed856d97a8c6d7f875bcbe42df
On Fri, Mar 25, 2022 at 1:15 PM Christoph Timm wrote:
Hello again,
I'm quite sure that this is a bug
Using up to date CentOS Stream 8 with vdsm-4.40.100.2-1.el8 and
oVirt 4.4.10.
The VM is getting into unresponsive state during the power down if
port mirror is activated in the vNIC profile of the VM.
The supervdsm.log on the host is showing the following error
during power off.
This was working in < 4.4.10. So I would say that this is a bug.
Where should I report this?
MainProcess|libvirt/events::DEBUG::2022-03-25
08:44:39,670::supervdsm_server::95::SuperVdsm.ServerCallback::(wrapper)
call unsetPortMirroring with ('probe_traffic', 'vnet2') {}
MainProcess|libvirt/events::DEBUG::2022-03-25
08:44:39,670::cmdutils::130::root::(exec_cmd) /sbin/tc filter show
dev probe_traffic parent : (cwd None)
MainProcess|libvirt/events::DEBUG::2022-03-25
08:44:39,678::cmdutils::138::root::(exec_cmd) SUCCESS: =
b''; = 0
MainProcess|libvirt/events::ERROR::2022-03-25
08:44:39,678::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
Error in unsetPortMirroring
Traceback (most recent call last):
File
"/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line
97, in wrapper
res = func(*args, **kwargs)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 104, in unsetPortMirroring
acts = _delTarget(network, QDISC_INGRESS, target)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 50, in _delTarget
fs = list(filters(network, parent))
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 174, in filters
for filt in _filters(dev, parent=parent, out=out):
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 199, in _iterate
yield module.parse(tokens)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py", line
99, in parse
data[data['kind']] = _filter_cls_parser(tokens)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py", line
119, in _parse_u32
_parser.consume(tokens, '???')
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py",
line 36, in consume
raise TCParseError('Found %s, expected %s' % (found, expected))
vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected
('???',)
Best regards
Christoph
Am 25.03.22 um 07:42 schrieb Christoph Timm:
Good morning,
please note that I got a bit further with my investigation.
It might be a bug in the VDSM as I see the following while
powering down the problematic VM:
MainProcess|libvirt/events::ERROR::2022-03-24
20:43:48,740::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
Error in unsetPortMirroring
Traceback (most recent call last):
File
"/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line
97, in wrapper
res = func(*args, **kwargs)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 104, in unsetPortMirroring
acts = _delTarget(network, QDISC_INGRESS, target)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 50, in _delTarget
fs = list(filters(network, parent))
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 174, in filters
for filt in _filters(dev, parent=parent, out=out):
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 199, in _iterate
yield module.parse(tokens)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 99, in parse
data[data['kind']] = _filter_cls_parser(tokens)
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 119, in _parse_u32
_parser.consume(tokens, '???')
File
"/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py",
line 36, in consume
raise TCParseError('Found %s, expected %s' % (found, expected))
vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected
('???',)
I already manage to find out that this is related to the specific
vNIC configuration which is using the port mirror flag.
My plan is remove and add the port mirror from the vNIC as the
first step. After that I will recreate
[ovirt-users] Re: Unable to start VMs after upgrade from 4.4.9 to 4.4.10
Hi,
this is already fixed in oVirt 4.5 [0]. Possible workaround might be to
downgrade the iproute-tc package below version 5.15.0-3.
Regards,
Ales
[0]
https://github.com/oVirt/vdsm/commit/503a98139d65daed856d97a8c6d7f875bcbe42df
On Fri, Mar 25, 2022 at 1:15 PM Christoph Timm wrote:
> Hello again,
>
> I'm quite sure that this is a bug
>
> Using up to date CentOS Stream 8 with vdsm-4.40.100.2-1.el8 and oVirt
> 4.4.10.
> The VM is getting into unresponsive state during the power down if port
> mirror is activated in the vNIC profile of the VM.
> The supervdsm.log on the host is showing the following error during power
> off.
>
> This was working in < 4.4.10. So I would say that this is a bug. Where
> should I report this?
>
> MainProcess|libvirt/events::DEBUG::2022-03-25
> 08:44:39,670::supervdsm_server::95::SuperVdsm.ServerCallback::(wrapper)
> call unsetPortMirroring with ('probe_traffic', 'vnet2') {}
> MainProcess|libvirt/events::DEBUG::2022-03-25
> 08:44:39,670::cmdutils::130::root::(exec_cmd) /sbin/tc filter show dev
> probe_traffic parent : (cwd None)
> MainProcess|libvirt/events::DEBUG::2022-03-25
> 08:44:39,678::cmdutils::138::root::(exec_cmd) SUCCESS: = b''; = 0
> MainProcess|libvirt/events::ERROR::2022-03-25
> 08:44:39,678::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
> Error in unsetPortMirroring
> Traceback (most recent call last):
> File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line
> 97, in wrapper
> res = func(*args, **kwargs)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 104, in unsetPortMirroring
> acts = _delTarget(network, QDISC_INGRESS, target)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 50, in _delTarget
> fs = list(filters(network, parent))
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 174, in filters
> for filt in _filters(dev, parent=parent, out=out):
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 199, in _iterate
> yield module.parse(tokens)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py", line
> 99, in parse
> data[data['kind']] = _filter_cls_parser(tokens)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py", line
> 119, in _parse_u32
> _parser.consume(tokens, '???')
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py", line
> 36, in consume
> raise TCParseError('Found %s, expected %s' % (found, expected))
> vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected ('???',)
>
>
> Best regards
> Christoph
>
>
> Am 25.03.22 um 07:42 schrieb Christoph Timm:
>
> Good morning,
>
> please note that I got a bit further with my investigation.
>
> It might be a bug in the VDSM as I see the following while powering down
> the problematic VM:
>
> MainProcess|libvirt/events::ERROR::2022-03-24
> 20:43:48,740::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
> Error in unsetPortMirroring
> Traceback (most recent call last):
> File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py", line
> 97, in wrapper
> res = func(*args, **kwargs)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 104, in unsetPortMirroring
> acts = _delTarget(network, QDISC_INGRESS, target)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 50, in _delTarget
> fs = list(filters(network, parent))
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 174, in filters
> for filt in _filters(dev, parent=parent, out=out):
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
> line 199, in _iterate
> yield module.parse(tokens)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py", line
> 99, in parse
> data[data['kind']] = _filter_cls_parser(tokens)
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py", line
> 119, in _parse_u32
> _parser.consume(tokens, '???')
> File "/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py", line
> 36, in consume
> raise TCParseError('Found %s, expected %s' % (found, expected))
> vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected ('???',)
>
> I already manage to find out that this is related to the specific vNIC
> configuration which is using the port mirror flag.
>
> My plan is remove and add the port mirror from the vNIC as the first step.
> After that I will recreate the vNIC.
>
> Any recommendations?
>
> Best regards
> Christoph
>
> Am 24.03.22 um 12:31 schrieb Christoph Timm:
>
> Hi List,
>
> I receive the following error while starting some of our VMs after the
> upgrade to 4.4.10.
>
> VM v4-probe is down with error. Exit message: internal error: process
> exited while connecting to monitor: 2022-03-24T11:27:23.098838Z qemu-kvm:
> -blockdev
>
[ovirt-users] Re: Unable to start VMs after upgrade from 4.4.9 to 4.4.10
Hello again,
I'm quite sure that this is a bug
Using up to date CentOS Stream 8 with vdsm-4.40.100.2-1.el8 and oVirt
4.4.10.
The VM is getting into unresponsive state during the power down if port
mirror is activated in the vNIC profile of the VM.
The supervdsm.log on the host is showing the following error during
power off.
This was working in < 4.4.10. So I would say that this is a bug. Where
should I report this?
MainProcess|libvirt/events::DEBUG::2022-03-25
08:44:39,670::supervdsm_server::95::SuperVdsm.ServerCallback::(wrapper)
call unsetPortMirroring with ('probe_traffic', 'vnet2') {}
MainProcess|libvirt/events::DEBUG::2022-03-25
08:44:39,670::cmdutils::130::root::(exec_cmd) /sbin/tc filter show dev
probe_traffic parent : (cwd None)
MainProcess|libvirt/events::DEBUG::2022-03-25
08:44:39,678::cmdutils::138::root::(exec_cmd) SUCCESS: = b''; = 0
MainProcess|libvirt/events::ERROR::2022-03-25
08:44:39,678::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
Error in unsetPortMirroring
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py",
line 97, in wrapper
res = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 104, in unsetPortMirroring
acts = _delTarget(network, QDISC_INGRESS, target)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 50, in _delTarget
fs = list(filters(network, parent))
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 174, in filters
for filt in _filters(dev, parent=parent, out=out):
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 199, in _iterate
yield module.parse(tokens)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 99, in parse
data[data['kind']] = _filter_cls_parser(tokens)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 119, in _parse_u32
_parser.consume(tokens, '???')
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py",
line 36, in consume
raise TCParseError('Found %s, expected %s' % (found, expected))
vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected ('???',)
Best regards
Christoph
Am 25.03.22 um 07:42 schrieb Christoph Timm:
Good morning,
please note that I got a bit further with my investigation.
It might be a bug in the VDSM as I see the following while powering
down the problematic VM:
MainProcess|libvirt/events::ERROR::2022-03-24
20:43:48,740::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
Error in unsetPortMirroring
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py",
line 97, in wrapper
res = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 104, in unsetPortMirroring
acts = _delTarget(network, QDISC_INGRESS, target)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 50, in _delTarget
fs = list(filters(network, parent))
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 174, in filters
for filt in _filters(dev, parent=parent, out=out):
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 199, in _iterate
yield module.parse(tokens)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 99, in parse
data[data['kind']] = _filter_cls_parser(tokens)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 119, in _parse_u32
_parser.consume(tokens, '???')
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py",
line 36, in consume
raise TCParseError('Found %s, expected %s' % (found, expected))
vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected ('???',)
I already manage to find out that this is related to the specific vNIC
configuration which is using the port mirror flag.
My plan is remove and add the port mirror from the vNIC as the first
step. After that I will recreate the vNIC.
Any recommendations?
Best regards
Christoph
Am 24.03.22 um 12:31 schrieb Christoph Timm:
Hi List,
I receive the following error while starting some of our VMs after
the upgrade to 4.4.10.
VM v4-probe is down with error. Exit message: internal error: process
exited while connecting to monitor: 2022-03-24T11:27:23.098838Z
qemu-kvm: -blockdev
{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-3-format"}:
Failed to get "write" lock
Is another process using the image
[/rhev/data-center/mnt/lxskinner:_exports_skinner__2tb__1/28da0c79-b6f1-4740-bd24-e7bafcb62c75/images/90b28e7e-98a3-4f80-a136-c5a52c4a3e05/c1450651-b1e5-47fd-906b-8ebd62ace8a4]?.
The example disk is located on a NFS share.
Any idea how to tell oVirt that there is no other process
[ovirt-users] Re: Unable to start VMs after upgrade from 4.4.9 to 4.4.10
Good morning,
please note that I got a bit further with my investigation.
It might be a bug in the VDSM as I see the following while powering down
the problematic VM:
MainProcess|libvirt/events::ERROR::2022-03-24
20:43:48,740::supervdsm_server::99::SuperVdsm.ServerCallback::(wrapper)
Error in unsetPortMirroring
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py",
line 97, in wrapper
res = func(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 104, in unsetPortMirroring
acts = _delTarget(network, QDISC_INGRESS, target)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 50, in _delTarget
fs = list(filters(network, parent))
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 174, in filters
for filt in _filters(dev, parent=parent, out=out):
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/__init__.py",
line 199, in _iterate
yield module.parse(tokens)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 99, in parse
data[data['kind']] = _filter_cls_parser(tokens)
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/filter.py",
line 119, in _parse_u32
_parser.consume(tokens, '???')
File "/usr/lib/python3.6/site-packages/vdsm/network/tc/_parser.py",
line 36, in consume
raise TCParseError('Found %s, expected %s' % (found, expected))
vdsm.network.tc._parser.TCParseError: Found not_in_hw, expected ('???',)
I already manage to find out that this is related to the specific vNIC
configuration which is using the port mirror flag.
My plan is remove and add the port mirror from the vNIC as the first
step. After that I will recreate the vNIC.
Any recommendations?
Best regards
Christoph
Am 24.03.22 um 12:31 schrieb Christoph Timm:
Hi List,
I receive the following error while starting some of our VMs after the
upgrade to 4.4.10.
VM v4-probe is down with error. Exit message: internal error: process
exited while connecting to monitor: 2022-03-24T11:27:23.098838Z
qemu-kvm: -blockdev
{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-3-format"}:
Failed to get "write" lock
Is another process using the image
[/rhev/data-center/mnt/lxskinner:_exports_skinner__2tb__1/28da0c79-b6f1-4740-bd24-e7bafcb62c75/images/90b28e7e-98a3-4f80-a136-c5a52c4a3e05/c1450651-b1e5-47fd-906b-8ebd62ace8a4]?.
The example disk is located on a NFS share.
Any idea how to tell oVirt that there is no other process using this
disk?
Best regards and thx
Christoph
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/SIDIS42UA5DUTS3GF5W7XBQFCNVMN3YG/
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/3DD4UJSQXJENYYYP762Q5Z3TTHE7GKBH/
