[ovirt-users] Re: Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

Replying to my own post here, I've verified that it's currently not possible to 
do a greenfield deploy of oVirt 4.4.0 w/ hosted engine on EPYC CPUs due to the 
system setting a requirement of 'virt-ssbd' for the final HE definition after 
the move to shared storage. The local HE runs perfectly through the setup 
process because it correctly uses 'amd-ssbd' but unfortunately that doesn't 
stick after the disks are moved.

You can work around it via 'virsh -r dumpxml HostedEngine > /tmp/he.xml', then 
editing that file to simply change 'virt-ssbd' to 'amd-ssbd'. Start it via 
'virsh create /tmp/he.xml' and now it runs fine. You can get into the admin 
interface and if you want to run something hacky could at this point change the 
cluster CPU type from 'Secure AMD EPYC' to just 'AMD EPYC', take everything 
down and bring it back up cleanly... hosted engine will now work because the 
requirement for virt-ssbd (not available on EPYC when using CentOS 8.1 or 
presumably RHEL 8, amd-ssbd is needed) is gone.
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/VFZT6QCQLEDHCRL75LL4TJWOLV7F43GG/

[ovirt-users] Re: Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

Oof, I butchered his last name, but the quote I found was this:

"Details: Tom Lendacky from AMD writes[4] -- "The idea behind
'virt-ssbd' was to provide an architectural method for a guest to do
SSBD when 'amd-ssbd' isn't present.  The 'amd-ssbd' feature will use
SPEC_CTRL which is intended to not be intercepted and will be fast.
The use of 'virt-ssbd' will always be intercepted and therefore will
not be as fast.  So a guest should be presented with 'amd-ssbd', if
available, in preference to 'virt-ssbd'.

It seems for these EPYC CPUs with qemu on 8.1, virt-ssbd isn't an option 
anymore and it has to be amd-ssbd for the VM to work?
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/I6ZZ53I6HC7HIULXO4XKMLSJH4RYFH2S/

[ovirt-users] Re: Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

I should have also mentioned that I found an existing report for the issue I'm 
having on RedHat's Bugzilla:  
https://bugzilla.redhat.com/show_bug.cgi?id=1783180

Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IDIH7L3QLX6TPER6IY2KTUSMWGVIGP6W/

[ovirt-users] Re: Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

Hi Lucia,

>>> the cluster is set to use the
secure variant of the CPU type but your host does not support all the
necessary flags.

For my deployments the system is auto-detecting and using the 'Secure AMD EPYC' 
CPU type.  Note that the HostedEngineLocal VM does run with 'amd-ssbd' and 
works fine (amd-ssbd is supposed to be faster/preferred over virt-ssbd 
according to another thread I found with a comment from Tom Landsky at AMD). So 
the CPUs can definitely run in a a secure configuration, they just need the 
'amd-ssbd' flag instead of 'virt-ssbd'.  I don't know why the local HE VM runs 
with the correct flag, but then the final one does not... If I can figure that 
out I'll be much further along towards a resolution.

Thanks!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KYQLKVPQINZ4JRFE3JGJTGGQV5GTPTVF/

[ovirt-users] Re: Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

Thank you Lucia, that's good news to hear. Is this something that might be 
fixed up in the 4.4.1 release whenever it comes along, and/or something simple 
to patch to get a clean deploy using 4.4.0? I have the servers and time to 
experiment with for now. If I can somehow edit that 
'4:Secure AMD 
EPYC:svm,nx,ibpb,ssbd,model_EPYC:EPYC,+ibpb,+virt-ssbd:x86_64; '

line to instead use amd-ssbd before it's added to the vdc_options table, and 
thus used for the HostedEngine config, I think it'd work. I could be wildly 
wrong, I'm basing that guess on the fact that all I have to do is swap 
'virt-ssbd' with 'amd-ssbd' in the xml and can start the failed-to-deploy 
HostedEngine VM right up.

Thank you!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/IRFGL6Y7G2YOQKW5VDIOGUDJ7BCEC2OS/

[ovirt-users] Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

Hello all,

I have some EPYC servers that are not yet in production, so I wanted to go 
ahead and move them off of 4.3 (which was working) to 4.4. I flattened and 
reinstalled the hosts with CentOS 8.1 Minimal and installed all updates. Some 
very simple networking, just a bond and two iSCSI interfaces. After adding the 
oVirt 4.4 repo and installing the requirements, I run 'hosted-engine --deploy' 
and proceed through the setup. Everything looks as though it is going nicely 
and the local HE starts and runs perfectly. After copying the HE disks out to 
storage, the system tries to start it there but is using a different CPU 
definition and it's impossible to start it. At this point I'm stuck but hoping 
someone knows the fix, because this is as vanilla a deployment as I could 
attempt and it appears EPYC CPUs are a no-go right now with 4.4.

When the HostedEngineLocal VM is running, the CPU definition is:
  
EPYC-IBPB
AMD















  

Once the HostedEngine VM is defined and trying to start, the CPU definition is 
simply:

  
EPYC




  

  

On attempts to start it, the host is logging this error:  "CPU is incompatible 
with host CPU: Host CPU does not provide required features: virt-ssbd".

So, the HostedEngineLocal VM works because it has a requirement set for  
'amd-ssbd' instead of 'virt-ssbd', and a VM requiring 'virt-ssbd' can't run on 
EPYC CPUs with CentOS 8.1.  As mentioned, the HostedEngine ran fine on oVirt 
4.3 with CentOS 7.8, and on 4.3 the cpu definition also required 'virt-ssbd', 
so I can only imagine that perhaps this is due to the more recent 4.x kernel 
that I now need HE to require 'amd-ssbd' instead?

Any clues to help with this? I can completely wipe/reconfigure the hosts as 
needed so I'm willing to try whatever so that I can move forward with a 4.4 
deployment.

Thanks!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KZHDCDE6JYADDMFSZD6AXYBP6SPV4TGA/

[ovirt-users] Re: Issues deploying 4.4 with HE on new EPYC hosts

[Mark R]

Thank you for the suggestion, Ryan it looks like AVIC is disabled by 
default though (/sys/module/kvm_amd/parameters/avic is 0).
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FUFXVPUT4AY4BB3M5E3B7N5E5OHBRZRL/

[ovirt-users] Re: New oVirt Install - Host Engine Deployment Fails

[Mark R]

Just to chime in here, trying a fresh hosted engine deploy today on AMD EPYC 
servers, the exact same error kills the deploy.  So it's not you or your 
CPUs/servers, there's a bug in the Ansible routine that was introduced with 
4.4.2 that makes hosted engine deployment impossible.

I was able to deploy hosted engine on these servers using 4.4.1 without issue 
but hit a different (non-oVirt, rather Linux kernel) issue. As I was given a 
test kernel to work around that, I wanted to try deployment again so I paved 
the server, completely fresh install of CentOS 8.2 w/ all updates then added 
the kernel that fixes the bonding/bridge issue. The hosted engine deploy will 
reliably fail every time on that step with that error, this is a regression 
with 4.4.2.

Regards,
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JWGMJ6VTRRNAFUCU4PK2RXUW3WQOUYM6/

[ovirt-users] iSCSI multipath with separate subnets... still not possible in 4.4.x?

[Mark R]

I'm looking through quite a few bug reports and mailing list threads, but want 
to make sure I'm not missing some recent development.  It appears that doing 
iSCSI with two separate, non-routed subnets is still not possible with 4.4.x. I 
have the dead-standard iSCSI setup with two separate switches, separate 
interfaces on hosts and storage, and separate subnets that have no gateway and 
are completely unreachable except from directly attached interfaces.

The hosted-engine comes up with multiple paths and everything is perfect, but 
that's because Ansible/hosted-engine deploy script have configured things 
correctly.  Once you need to import or add new storage domains, it's not 
possible to do so in a way that gets both paths connected *and* persists across 
host reboots. Following the docs to create 'iSCSI Multipath" bonds (plural, you 
can _not_ create a single bond that includes both interfaces and hope things 
route correctly... oVirt will try to connect from the interface for storage 
network A to the target on storage network B, which can't happen since they are 
not routed (and should not be). So, there's nothing in the docs about how you 
can accomplish multipathing, but there are a few mailing list messages that say 
"just create two separate "iSCSI Multipath" bonds in the datacenter, one for 
each of your two interfaces. You can do this, and you'll get hopeful that 
things might work now. You can do discovery and it succeeds, because no
  more trying to connect to unreachable targets. However, and big caveat, 
there's no way to tell this new/imported domain, "Oh, use this other interface 
as well, so you have redundant paths". Once the domain is attached and 
activated, you have a single path. You can then manage the domain, do a 
discovery, see a path that isn't connected yet, and log into it as well. Now 
you have two paths, is everything right with the world?!?  Nope, it's 
impossible to persist that connection, it will be gone on next reboot and 
you'll always have to manually visit each host, do discovery, and login. 
Nothing in the UI allows you to "Save" that second connection in a way that it 
will be used again. Clicking "OK" does not, and going back to the "iSCSI 
Multipath" area of the Data Center you can't edit each of the bonds and make 
sure each logical network has every possible target checked, because these 
targets you've manually logged into are never listed in that area of the UI.

So I really, really hope I'm wrong because I'd like to move past this snag and 
onto the next one (which is that bond interfaces in 4.4.x will not allow you to 
attach additional networks... works great in 4.3, appears broken in 4.4.x). 
But, no sense chasing that yet if iSCSI multipath isn't possible, which is 
looking likely.

Has anyone had success, running iSCSI in by far the most common setup out 
there, but also in a way oVirt really doesn't want to let you? This is driving 
me nuts, I've paved and rebuilt these hosts dozens of times now, trying 
different methods in the hopes of getting multipath that persists.
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/QUIEWQLKB3JHQTLSWMWEFTKDR36AFOO7/

[ovirt-users] Re: status of oVirt 4.4.x and CentOS 8.2

[Mark R]

Hello Ales,

The nmstate and NetworkManager versions are as follows, installed from repos 
that are added while installing oVirt/HE:

Name : nmstate
Version  : 0.2.10
Release  : 1.el8
Architecture : noarch
Size : 32 k
Repository   : @System
From repo: ovirt-4.4-copr:copr.fedorainfracloud.org:nmstate:nmstate-0.2


Name : NetworkManager
Version  : 1.22.14
Release  : 1.el8
Architecture : x86_64
Repository   : @System
From repo: 
ovirt-4.4-copr:copr.fedorainfracloud.org:networkmanager:NetworkManager-1.22

The NM debug output is 2K lines (I only included logs from the point I clicked 
"OK" to apply the new network until it failed). I pastebin'ed it at 
https://controlc.com/f3a08a1b , use pass ovirt44 to view it. I didn't want to 
throw that much content into a forum post and can't add an attachment when 
using the list's web interface to post.

Regards,
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/PGHZOEQOZOCLG3TFEUQSIDXFFRILWPIG/

[ovirt-users] Re: status of oVirt 4.4.x and CentOS 8.2

[Mark R]

The error with a bit more info from the events page, after adding network to an 
interface fails:

VDSM rack4slot11.domain.com command HostSetupNetworksVDS failed: Internal 
JSON-RPC error: {'reason': 'Unexpected failure of libnm when running the 
mainloop: run execution'}

Sorry, should have included that in the other message.

Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/3BHB7KKSSTHCN2AN5ZAIY5MZVFI7IG36/

[ovirt-users] Re: status of oVirt 4.4.x and CentOS 8.2

[Mark R]

Hi Michal,

Might the openvswitch issues you mention be tied to the issue I'm having 
standing up a new 4.4 installation on 8.2, namely that you can create a network 
in the UI, but when you go to apply/attach it to a host (using the 
drag-and-drop wizard to try to add a new network onto the same bond ovirtmgmt 
uses), it fails?  So I can now install a new hosted-engine setup on 8.2 with 
EPYC CPUs, definitely forward progress, but I can't configure any networks 
beyond ovirtmgmt. Dragging a new network onto the bond and clicking "OK" gets 
"Error while executing action HostSetupNetworks: Unexpected exception".

Thanks,
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/XU2TX3OXQ3TVQNAGBOONX73CJMRBGGNY/

[ovirt-users] Re: status of oVirt 4.4.x and CentOS 8.2

[Mark R]

Hi Dominik,

My apologies for the delay, have been pushed to other projects this week.  I 
just verified that our CentOS 8.2 and oVirt 4.4.0 installs are fully up to 
date, and tried to apply a network configuration adding a tagged VLAN to the 
same bridge as ovirtmgmt.  Here are the logs:

```
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,832::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call 
setupNetworks with ({'DMZ0': {'vlan': '22', 'bonding': 'bond0', 'ipv6autoconf': 
False, 'bridged': 'true', 'dhcpv6': False, 'STP': 'no', 'mtu': 1500, 'switch': 
'legacy'}}, {}, {'connectivityTimeout': 120, 'commitOnSuccess': True, 
'connectivityCheck': 'true'}) {}
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:53:59,833::api::220::root::(setupNetworks) Setting up network according to 
configuration: networks:{'DMZ0': {'vlan': '22', 'bonding': 'bond0', 
'ipv6autoconf': False, 'bridged': 'true', 'dhcpv6': False, 'STP': 'no', 'mtu': 
1500, 'switch': 'legacy'}}, bondings:{}, options:{'connectivityTimeout': 120, 
'commitOnSuccess': True, 'connectivityCheck': 'true'}
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,844::cmdutils::130::root::(exec_cmd) /sbin/tc qdisc show (cwd None)
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,852::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,899::vsctl::74::root::(commit) Executing commands: /usr/bin/ovs-vsctl 
--timeout=5 --oneline --format=json -- list Bridge -- list Port -- list 
Interface
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,899::cmdutils::130::root::(exec_cmd) /usr/bin/ovs-vsctl --timeout=5 
--oneline --format=json -- list Bridge -- list Port -- list Interface (cwd None)
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,908::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:53:59,913::netconfpersistence::58::root::(setNetwork) Adding network 
ovirtmgmt({'bridged': True, 'stp': False, 'mtu': 1500, 'bonding': 'bond0', 
'defaultRoute': True, 'bootproto': 'none', 'dhcpv6': True, 'ipv6autoconf': 
True, 'ipaddr': '172.17.96.51', 'netmask': '255.255.255.0', 'gateway': 
'172.17.96.1', 'switch': 'legacy', 'nameservers': ['x.x.x.x ', 
'x.x.x.x ']})
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:53:59,914::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': 
['eno33np0', 'ens2f0np0'], 'options': 'mode=4', 'switch': 'legacy', 'hwaddr': 
'bc:97:e1:24:c5:40'})
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:53:59,914::netconfpersistence::58::root::(setNetwork) Adding network 
DMZ0({'vlan': 22, 'bonding': 'bond0', 'ipv6autoconf': False, 'bridged': True, 
'dhcpv6': False, 'mtu': 1500, 'switch': 'legacy', 'defaultRoute': False, 'stp': 
False, 'bootproto': 'none', 'nameservers': []})
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:53:59,914::netconfpersistence::69::root::(setBonding) Adding bond0({'nics': 
['eno33np0', 'ens2f0np0'], 'options': 'mode=4', 'switch': 'legacy', 'hwaddr': 
'bc:97:e1:24:c5:40'})
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:53:59,916::commands::153::common.commands::(start) /usr/bin/taskset 
--cpu-list 0-63 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe (cwd None)
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:54:00,271::hooks::122::root::(_runHooksDir) 
/usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err=b''
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:54:00,271::configurator::190::root::(_setup_nmstate) Processing setup 
through nmstate
MainProcess|jsonrpc/3::INFO::2020-06-23 
21:54:00,297::configurator::192::root::(_setup_nmstate) Desired state: 
{'interfaces': [{'name': 'DMZ0', 'type': 'linux-bridge', 'state': 'up', 'mtu': 
1500, 'bridge': {'port': [{'name': 'bond0.22'}], 'options': {'stp': {'enabled': 
False}}}, 'ipv4': {'enabled': False}, 'ipv6': {'enabled': False}}, {'name': 
'bond0', 'type': 'bond', 'state': 'up', 'mac-address': 'bc:97:e1:24:c5:40', 
'link-aggregation': {'slaves': ['eno33np0', 'ens2f0np0'], 'options': {}, 
'mode': '802.3ad'}, 'mtu': 1500}, {'vlan': {'id': 22, 'base-iface': 'bond0'}, 
'name': 'bond0.22', 'type': 'vlan', 'state': 'up', 'mtu': 1500, 'ipv4': 
{'enabled': False}, 'ipv6': {'enabled': False}}, {'name': 'ovirtmgmt'}]}
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:54:00,351::checkpoint::121::root::(create) Checkpoint 
/org/freedesktop/NetworkManager/Checkpoint/1 created for all devices: 60
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:54:00,351::netapplier::239::root::(_add_interfaces) Adding new interfaces: 
['DMZ0', 'bond0.22']
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:54:00,354::netapplier::251::root::(_edit_interfaces) Editing interfaces: 
['eno33np0', 'ovirtmgmt', 'bond0', 'ens2f0np0']
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:54:00,357::nmclient::136::root::(execute_next_action) Executing NM action: 
func=add_connection_async
MainProcess|jsonrpc/3::DEBUG::2020-06-23 
21:54:00,364::connection::329::root::(_add_connection_callback) Connection 
adding succeeded: dev=DMZ0

[ovirt-users] Re: status of oVirt 4.4.x and CentOS 8.2

[Mark R]

Follow up, as well as the logged error from supervdsm.log, I see this in dmesg 
so it appears that it very briefly does create DMZ0 and the bond0.22 interface, 
but rolls everything back.

```
[ 2205.551920] IPv6: ADDRCONF(NETDEV_UP): DMZ0: link is not ready
[ 2205.558366] IPv6: ADDRCONF(NETDEV_UP): bond0.22: link is not ready
[ 2205.655177] DMZ0: port 1(bond0.22) entered blocking state
[ 2205.655179] DMZ0: port 1(bond0.22) entered disabled state
```
Both ports for the bond are up and active (of couse, as this is currently the 
first host of the HE deployment so I couldn't manage it if they weren't).

```
# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: bc:97:e1:24:c5:40
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 2
Actor Key: 21
Partner Key: 3
Partner Mac Address: 0a:33:5e:69:1f:1e

Slave Interface: eno33np0
MII Status: up
Speed: 25000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: bc:97:e1:24:c5:40
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: bc:97:e1:24:c5:40
port key: 21
port priority: 255
port number: 1
port state: 61
details partner lacp pdu:
system priority: 4096
system mac address: 0a:33:5e:69:1f:1e
oper key: 3
port priority: 8192
port number: 65
port state: 61

Slave Interface: ens2f0np0
MII Status: up
Speed: 25000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: b0:26:28:cd:ec:d0
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: bc:97:e1:24:c5:40
port key: 21
port priority: 255
port number: 2
port state: 61
details partner lacp pdu:
system priority: 4096
system mac address: 0a:33:5e:69:1f:1e
oper key: 3
port priority: 32768
port number: 33
port state: 61
```

Thanks,
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/FY5227PI4PUFPOMTCE3UY3R7J6NHRQH3/

[ovirt-users] Re: 4.4.x - VLAN-based logical network can't attach to bonds?

[Mark R]

> Can you please create a bug
>  for oVirt with
> reproduction steps?
> 

Created https://bugzilla.redhat.com/show_bug.cgi?id=1860479 
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KHM3KU6LG6MYVSLBH76FSBJ67ITUWMQP/

[ovirt-users] Re: 4.4.x - VLAN-based logical network can't attach to bonds?

[Mark R]

Hi Dominik,

> Can you please create a bug
>  for oVirt with
> reproduction steps?

Yes, I'll do so. Rebuilding the host fresh now so I have a clean slate as I go 
through the steps for the report.  I did try NM 1.26 from the COPR repo you 
linked, but the issue persisted.

> .. if the new NetworkManager 1.26 from ...
> would fix the problem already.

I installed 1.26 and tried to add the network but it failed.

Thanks!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/AICQLMYKETZ63BUTTI25E53B76ZM32T3/

[ovirt-users] Re: 4.4.x - VLAN-based logical network can't attach to bonds?

[Mark R]

Hello Ales,

> can you please share supervdsm.log from the relevant host?

Absolutely! Here is the log from the point of opening the "Setup Host Networks" 
UI, as I just drag the "LegacyDMZ" network over onto the same bond that's 
running ovirtmgmt and click "OK".

#-8<--8<-8<

MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,627::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper) call 
get_lldp_info with ({'devices': []},) {}
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,631::routes::115::root::(get_gateway) The gateway 172.17.96.1 is 
duplicated for the device ovirtmgmt
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,634::routes::115::root::(get_gateway) The gateway 172.17.96.1 is 
duplicated for the device ovirtmgmt
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,635::cmdutils::130::root::(exec_cmd) /sbin/tc qdisc show (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,643::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,644::cmdutils::130::root::(exec_cmd) /sbin/tc class show dev eno34np1 
classid 0:1388 (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,649::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,649::cmdutils::130::root::(exec_cmd) /sbin/tc class show dev ens2f1np1 
classid 0:1388 (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,655::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,699::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-lldp -i 
eno33np0 adminStatus (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,706::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,706::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-tlv -n -i 
eno33np0 (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,712::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,713::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-lldp -i 
eno34np1 adminStatus (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,718::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,719::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-tlv -n -i 
eno34np1 (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,724::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,725::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-lldp -i 
ens2f0np0 adminStatus (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,729::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,730::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-tlv -n -i 
ens2f0np0 (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,735::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,736::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-lldp -i 
ens2f1np1 adminStatus (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,740::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,741::cmdutils::130::root::(exec_cmd) /usr/sbin/lldptool get-tlv -n -i 
ens2f1np1 (cwd None)
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,745::cmdutils::138::root::(exec_cmd) SUCCESS:  = b'';  = 0
MainProcess|jsonrpc/4::DEBUG::2020-07-23 
13:37:35,746::supervdsm_server::100::SuperVdsm.ServerCallback::(wrapper) return 
get_lldp_info with {'eno33np0': {'enabled': True, 'tlvs': [{'type': 1, 'name': 
'Chassis ID', 'properties': {'chassis ID subtype': 'MAC', 'chassis ID': 
'8c:04:ba:e9:21:40'}}, {'type': 2, 'name': 'Port ID', 'properties': {'port ID 
subtype': 'Ifname', 'port ID': 'ethernet1/1/16'}}, {'type': 3, 'name': 'Time to 
Live', 'properties': {'time to live': '120'}}, {'type': 4, 'name': 'Port 
Description', 'properties': {'port description': 'Part of VLT LACP trunk to 
rack4slot15 - em3'}}, {'type': 5, 'name': 'System Name', 'properties': {'system 
name': 'TOR-4-A'}}, {'type': 6, 'name': 'System Description', 'properties': 
{'system description': 'Dell EMC Networking OS10 Enterprise.'}}, {'type': 7, 
'name': 'System Capabilities', 'properties': {'system capabilities': 'Repeater, 
Bridge, Router', 'enabled capabilities': 'Repeater, Bridge, Router'}}, {'type': 
8, 'name': 'Management Add
 ress', 'properties': {'management address subtype': 'IPv4', 'management 
address': '192.168.3.1', 'interface numbering subtype': 'Ifindex', 'interface 
numbering': '9'}}, {'type': 8, 'name': 'Management Address', 'properties': 
{'management address subtype': 'IPv6', 'management address': 
'fe80::8e04:baff:fee9:2140', 'interface numbering 

[ovirt-users] 4.4.x - VLAN-based logical network can't attach to bonds?

[Mark R]

Hello all,

New 4.4.1 hosted-engine install. Pre-deploy, the host already had bond0 and its 
iSCSI interfaces configured. The deploy correctly sets up the ovirtmgmt 
interface using the bond, and when everything finishes up it's working as 
expected.  However, I then need to create a VLAN-based network and attach to 
the same bond0 interface. Editing the host networks and dragging the VLAN 102 
interface to the bond alongside ovirtmgmt, then clicking "OK" results in a 
failure every time. The error returned is:

VDSM ovirt5.domain.com command HostSetupNetworksVDS failed: Internal 
JSON-RPC error: {'reason': 'Unexpected failure of libnm when running the 
mainloop: run execution'}

I can break the bond and apply this an any other VLAN-based network at will, 
but then it's not possible to add the interface I removed to create the bond 
again. That may be by design and it's not supposed to be possible to attach an 
interface and create a bond once you've already assigned logical networks to 
it. The error there is "Error while executing action HostSetupNetworks: 
Interface already in use".

I'm just putting out feelers to see if this is a known issue that other people 
are hitting, or are other folks with hosted-engine 4.4.x deploys readily 
creating that initial bond0 interface and assigning any VLAN-based logical 
networks they want w/o issue?  These same hosts (they still aren't in prod so I 
rebuild them at will) run 4.3 with no issues and setting up the exact same 
network configurations works flawlessly, it just becomes an issue on 4.4.1. 
This host was freshly installed today and has all updates.

Thanks,
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/XF7PANDZBAH5J7FSJFUEUAOL6T7XBRHL/

[ovirt-users] Re: iSCSI multipath with separate subnets... still not possible in 4.4.x?

[Mark R]

> I don't use them as multipathing generally works without OVirt bonds in 
> my setup, I configured multipathd directly to use round robin e.g.

Thanks, Uwe. Am I understanding correctly that you're just letting your nodes 
attach to the iSCSI storage on their own by leaving "node.startup = automatic" 
in /etc/iscsi/iscsid.conf so the hosts attach to all known targets as they 
boot, long before oVirt services ever attempt to connect them? I've considered 
flipping that to automatic, it's currently on "manual" as the idea was "let's 
let oVirt connect and manage the storage it wants".

As another poster below mentioned, going the route of two separate iSCSI bonds 
in the "iSCSI Multipath" section does work when you're adding new storage 
domains. The aspect he talks about, where you connect both paths and save it, 
isn't possible if you import an existing storage domain. When importing, the UI 
won't expose the "Add" button that's available when creating a new domain, so 
you can't add redundant paths. You can import the storage, then edit it and 
discover/login to the other path, but that does _not_ save to the database and 
will not persist across reboots or connect on other hosts you add to the 
cluster (have to login manually on each). You can't edit your iSCSI bonds and 
check the box for these manually logged in targets either, they'll never 
populate in that part of the UI so can't be selected. I think it's just a UI 
issue because some very easy fidling in the database makes it work exactly as 
you'd expect (and as it does for domains you newly add instead of importing
 ).

Sorry, rambling, but I am curious about your "node.startup" setting in 
iscid.conf.  If left at 'automatic' (the default), are your hosts attaching all 
the disks as they boot and oVirt doesn't mind that? It could be the path I'll 
take as honestly I'd much prefer configuring the storage connections directly.

Thanks!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4ES7TX44S7FAODE6RTF34MKH7YUANVH5/

[ovirt-users] Re: 4.4.x - VLAN-based logical network can't attach to bonds?

[Mark R]

Following up, as noted in the bug report ( 
https://bugzilla.redhat.com/show_bug.cgi?id=1860479 ) this doesn't appear to be 
oVirt / VDSM having issues. You can replicate the same inability to attach a 
bond to a bridge by simply booting the 8.2.2004 installation media and trying a 
few manual 'ip' commands. With older 8.1 or any 7.x install media, there's no 
problem, but starting with 8.2, you can't attach a bond to a bridge anymore.

It looks like there may be some similar reports on bugzilla recently. I just 
wanted to note here on the list that this isn't an oVirt problem, looks to be 
an issue with the OS (kernel?).

Thanks again!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KUUZJTAF5CWWIUFZPTZMKZFY5QIXUNKI/

[ovirt-users] Re: Failed to execute stage 'Closing up': Command '/usr/share/ovirt-engine-keycloak/bin/kk_cli.sh' failed to execute

[Mark R]

Hi Martin,

Not OP, but I did just deploy a brand new oVirt 4.5.4 HE setup. As you 
mentioned, there's an issue during deployment with the latest 
java-11-openjdk-headless. I had to do the deploy with "hosted-engine --deploy 
--ansible-extra-vars=he_pause_before_engine_setup=true". Once the deployment 
paused, I was able to hop into the local VM and "dnf downgrade -y 
java-11-openjdk-headless-11.0.15.0.10-3.el8 && dnf versionlock add 
java-11-openjdk-headless", after which I could remove the lockfile from /tmp on 
the deploy host and everything went great.

That said, I now wonder if I need to leave java-11-openjdk-headless locked to 
that version on the HE VM, or should I remove the versionlock and allow it to 
update?  Do you know if keycloak in general is having issues with the latest 
jdk, or is it just a deploy-time problem?

Thanks!
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WTA56ZGG5V4HCCNIWZUC2A54KV3PNCCO/

[ovirt-users] Re: Latest 4.5.5 centos node install username password not working.

[Mark R]

If you're signing into Keycloak at "https:///ovirt-engine/" and 
clicking "Administration Portal", then the user is indeed admin@ovirt.
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KWMC363YZAJJIXKVH43GONZKEXYZGP4A/

[ovirt-users] Re: Latest 4.5.5 centos node install username password not working.

[Mark R]

On the installation screen, did you remember to click the "Root Password" item, 
set a password, and if using the el9 iso, remove the checkbox on "Lock root 
account"?
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/3YYVCLOEYW7C5VOUHIX3VF6CD6QDMBNZ/

[ovirt-users] Re: How restore nodes ovirt UP from NonResponsive and VMs executing

[Mark R]

Hello Carlos,

Can I ask what kind of shared storage you have for these nodes?  If iSCSI, are 
your targets all configured to allow multi-initiator access? If you're on iSCSI 
and don't allow multi-initiator, then only one node at a time can be connected.

Regards,
Mark
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/5USRBV6CPR45OOB5RXW4I3DBKWWMYEKU/

[ovirt-users] Re: Cannot update oVirt Node

[Mark R]

To add a "me too" this same problem bit us when updating our 4.5 cluster today. 
It will surely hit anyone once their hosted engine updates ansible to 2.16.2, 
which breaks the host upgrades until you patch the specific file mentioned by 
Marco.

In our case, after we'd upgraded the hosted-engine VM, we went through and 
upgraded all of the nodes one-by-one including reboots, and then realized after 
we thought we were done that none of the packages on the nodes get upgraded at 
all. From the UI side of things, you'd think it had worked (hosts goes 
maintenance, says updating but quickly goes to rebooting, comes back up and 
activates showing no updates needed until the next time you issue "check for 
upgrades" against it).
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/WW3QEMZDVYBQHVVSFCOQ7NV7VXKPNBIW/

[ovirt-users] Re: Ovirt 4.5 HA over NFS fails when a single host goes down

[Mark R]

You must not have an even number of HE-capable nodes. What you're running into 
is a classic split-brain scenario, with only two nodes allowed to run the HE, 
and one of the nodes down, the surviving host does not have quorum so does not 
know it can safely power off the other machine (because obviously this 
surviving node, from its viewpoint, may have somehow become isolated from the 
network while the other host is happily alive and running the engine and 
controlling everything).

In clustering, you _never_ want two, or four, or six of something. One, three, 
five, etc... because it must be impossible to have a "tie" situation when the 
decisions are being made on which hosts are needing to be fenced.
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/UEDSNIXJZXS5NSO7JARJKP56PT3YK3UB/

[ovirt-users] Re: Ovirt 4.5 HA over NFS fails when a single host goes down

[Mark R]

That is wild and definitely not in line with the behavior we saw when a 
motherboard failed in one of our small 3-node oVirt 4.5 clusters. The bad host 
hard-froze, shortly after that one of the surviving nodes fenced it via IPMI, 
and all the VMs that had been running on it were started on surviving hosts.

It sounds like your experience is that a single node fails (even one with no 
VMs on it at the time), and the surviving hosts shut down (or do they kill) the 
VMs that they have running. Is that accurate?
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/YKZUXIEAF5YUD3D5MTCAD7EUA25GB2QQ/