Sverker, is this bug blocking you, or can you work around it?

On Thu, Sep 3, 2020 at 8:52 PM Dominik Holler <dhol...@redhat.com> wrote:

> Sverker, thanks!
>
> On Thu, Sep 3, 2020 at 6:50 PM Sverker Abrahamsson <
> sver...@abrahamsson.com> wrote:
>
>> Hi Dominik,
>> bug filed at https://bugzilla.redhat.com/show_bug.cgi?id=1875520. I'm
>> doing a new install to get fresh vdsm and supervdsm logs which will be
>> attached as soon as they've failed.
>> /Sverker
>> Den 2020-09-03 kl. 18:03, skrev Dominik Holler:
>>
>>
>>
>> On Thu, Sep 3, 2020 at 12:42 PM Sverker Abrahamsson <
>> sver...@abrahamsson.com> wrote:
>>
>>> Hi Ales,
>>> this is a CentOS 8 so my impression was that you always have
>>> NetworkManager then? At least my attempt to remove it failed miserably.
>>>
>>
>> Yes, on CentOS 8 hosts oVirt requires the interfaces managed by
>> NetworkManager.
>>
>>
>>> The enp4s0 config was created by the install, so it should be controlled
>>> by NetworkManager.
>>>
>>
>> This should work. Can you please report a bug on vdsm [1]?
>> Would be helpful if the vdsm.log and supervdsm.log would be attached to
>> this bug.
>>
>> [1]
>>   https://bugzilla.redhat.com/enter_bug.cgi?product=vdsm
>>
>>
>>
>>> /Sverker
>>> Den 2020-09-03 kl. 12:29, skrev Ales Musil:
>>>
>>>
>>>
>>> On Thu, Sep 3, 2020 at 12:21 PM Sverker Abrahamsson <
>>> sver...@abrahamsson.com> wrote:
>>>
>>>> Hi Ales,
>>>> right now I have a manually created ovirtmgmt bridge (virbr0 and vnet0
>>>> seems to be created during the failed attempt to deploy hosted engine):
>>>>
>>>> [root@h1-mgmt ~]# nmcli con show
>>>> NAME              UUID                                  TYPE      DEVICE
>>>> enp4s0            af7ccb53-011b-4c36-998a-1878b4ae7100  ethernet  enp4s0
>>>> Bridge ovirtmgmt  9a0b07c0-2983-fe97-ec7f-ad2b51c3a3f0  bridge
>>>> ovirtmgmt
>>>> virbr0            aa593151-2c12-4cf7-985b-f105b3575d09  bridge    virbr0
>>>> enp4s0.4000       ecc8064d-18c1-99b7-3fe4-9c5a593ece6f  vlan
>>>> enp4s0.4000
>>>> vnet0             a6db45bd-93c8-4c37-85fc-0c58ba3e9d00  tun       vnet0
>>>> [root@h1-mgmt ~]# nmstatectl show
>>>> ---
>>>> dns-resolver:
>>>>   config:
>>>>     search: []
>>>>     server:
>>>>     - 213.133.98.98
>>>>   running:
>>>>     search: []
>>>>     server:
>>>>     - 213.133.98.98
>>>> route-rules:
>>>>   config: []
>>>> routes:
>>>>   config:
>>>>   - destination: 0.0.0.0/0
>>>>     metric: -1
>>>>     next-hop-address: 144.76.84.65
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 0
>>>>   - destination: ::/0
>>>>     metric: -1
>>>>     next-hop-address: fe80::1
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 0
>>>>   running:
>>>>   - destination: 0.0.0.0/0
>>>>     metric: 100
>>>>     next-hop-address: 144.76.84.65
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: 144.76.84.65/32
>>>>     metric: 100
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: 172.27.1.0/24
>>>>     metric: 425
>>>>     next-hop-address: ''
>>>>     next-hop-interface: ovirtmgmt
>>>>     table-id: 254
>>>>   - destination: 192.168.1.0/24
>>>>     metric: 0
>>>>     next-hop-address: ''
>>>>     next-hop-interface: virbr0
>>>>     table-id: 254
>>>>   - destination: 2a01:4f8:192:1148::/64
>>>>     metric: 100
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: ::/0
>>>>     metric: 100
>>>>     next-hop-address: fe80::1
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: fe80::/64
>>>>     metric: 100
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: ff00::/8
>>>>     metric: 256
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 255
>>>> interfaces:
>>>> - name: ;vdsmdummy;
>>>>   type: linux-bridge
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: DE:D3:A8:24:27:F6
>>>>   mtu: 1500
>>>> - name: br-int
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: 6E:37:94:63:E0:4B
>>>>   mtu: 1500
>>>> - name: enp4s0
>>>>   type: ethernet
>>>>   state: up
>>>>   ethernet:
>>>>     auto-negotiation: true
>>>>     duplex: full
>>>>     speed: 1000
>>>>   ipv4:
>>>>     address:
>>>>     - ip: 144.76.84.73
>>>>       prefix-length: 32
>>>>     dhcp: false
>>>>     enabled: true
>>>>   ipv6:
>>>>     address:
>>>>     - ip: 2a01:4f8:192:1148::2
>>>>       prefix-length: 64
>>>>     - ip: fe80::62a4:4cff:fee9:4ac
>>>>       prefix-length: 64
>>>>     auto-dns: true
>>>>     auto-gateway: true
>>>>     auto-routes: true
>>>>     autoconf: true
>>>>     dhcp: true
>>>>     enabled: true
>>>>   mac-address: 60:A4:4C:E9:04:AC
>>>>   mtu: 1500
>>>> - name: enp4s0.4000
>>>>   type: vlan
>>>>   state: up
>>>>   ipv4:
>>>>     dhcp: false
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: 60:A4:4C:E9:04:AC
>>>>   mtu: 1500
>>>>   vlan:
>>>>     base-iface: enp4s0
>>>>     id: 4000
>>>> - name: lo
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mtu: 65536
>>>> - name: ovirtmgmt
>>>>   type: linux-bridge
>>>>   state: up
>>>>   bridge:
>>>>     options:
>>>>       group-forward-mask: 0
>>>>       mac-ageing-time: 300
>>>>       multicast-snooping: true
>>>>       stp:
>>>>         enabled: false
>>>>         forward-delay: 15
>>>>         hello-time: 2
>>>>         max-age: 20
>>>>         priority: 32768
>>>>     port:
>>>>     - name: enp4s0.4000
>>>>       stp-hairpin-mode: false
>>>>       stp-path-cost: 100
>>>>       stp-priority: 32
>>>>   ipv4:
>>>>     address:
>>>>     - ip: 172.27.1.1
>>>>       prefix-length: 24
>>>>     dhcp: false
>>>>     enabled: true
>>>>   ipv6:
>>>>     autoconf: false
>>>>     dhcp: false
>>>>     enabled: false
>>>>   mac-address: 60:A4:4C:E9:04:AC
>>>>   mtu: 1500
>>>> - name: ovs-system
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: D2:D3:5B:BA:58:38
>>>>   mtu: 1500
>>>> - name: virbr0
>>>>   type: linux-bridge
>>>>   state: up
>>>>   bridge:
>>>>     options:
>>>>       group-forward-mask: 0
>>>>       mac-ageing-time: 300
>>>>       multicast-snooping: true
>>>>       stp:
>>>>         enabled: true
>>>>         forward-delay: 2
>>>>         hello-time: 2
>>>>         max-age: 20
>>>>         priority: 32768
>>>>     port:
>>>>     - name: virbr0-nic
>>>>       stp-hairpin-mode: false
>>>>       stp-path-cost: 100
>>>>       stp-priority: 32
>>>>     - name: vnet0
>>>>       stp-hairpin-mode: false
>>>>       stp-path-cost: 100
>>>>       stp-priority: 32
>>>>   ipv4:
>>>>     address:
>>>>     - ip: 192.168.1.1
>>>>       prefix-length: 24
>>>>     dhcp: false
>>>>     enabled: true
>>>>   ipv6:
>>>>     autoconf: false
>>>>     dhcp: false
>>>>     enabled: false
>>>>   mac-address: 52:54:00:C0:A5:20
>>>>   mtu: 1500
>>>> - name: virbr0-nic
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: 52:54:00:C0:A5:20
>>>>   mtu: 1500
>>>> - name: vnet0
>>>>   type: unknown
>>>>   state: up
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: FE:16:3E:1B:21:0B
>>>>   mtu: 1500
>>>>
>>>> Reverting the configuration back to not have the static bridge and a
>>>> reboot I get this:
>>>>
>>>> [root@h1-mgmt ~]# nmcli con show
>>>> NAME         UUID                                  TYPE      DEVICE
>>>> enp4s0       af7ccb53-011b-4c36-998a-1878b4ae7100  ethernet  enp4s0
>>>> enp4s0.4000  ecc8064d-18c1-99b7-3fe4-9c5a593ece6f  vlan      enp4s0.4000
>>>> [root@h1-mgmt ~]# nmstatectl show
>>>> ---
>>>> dns-resolver:
>>>>   config:
>>>>     search: []
>>>>     server:
>>>>     - 213.133.98.98
>>>>   running:
>>>>     search: []
>>>>     server:
>>>>     - 213.133.98.98
>>>> route-rules:
>>>>   config: []
>>>> routes:
>>>>   config:
>>>>   - destination: 0.0.0.0/0
>>>>     metric: -1
>>>>     next-hop-address: 144.76.84.65
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 0
>>>>   - destination: ::/0
>>>>     metric: -1
>>>>     next-hop-address: fe80::1
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 0
>>>>   running:
>>>>   - destination: 0.0.0.0/0
>>>>     metric: 100
>>>>     next-hop-address: 144.76.84.65
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: 144.76.84.65/32
>>>>     metric: 100
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: 172.27.1.0/24
>>>>     metric: 400
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0.4000
>>>>     table-id: 254
>>>>   - destination: 2a01:4f8:192:1148::/64
>>>>     metric: 100
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: ::/0
>>>>     metric: 100
>>>>     next-hop-address: fe80::1
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: fe80::/64
>>>>     metric: 100
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 254
>>>>   - destination: ff00::/8
>>>>     metric: 256
>>>>     next-hop-address: ''
>>>>     next-hop-interface: enp4s0
>>>>     table-id: 255
>>>> interfaces:
>>>> - name: ;vdsmdummy;
>>>>   type: linux-bridge
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: B2:9E:E0:61:71:88
>>>>   mtu: 1500
>>>> - name: br-int
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: 6E:37:94:63:E0:4B
>>>>   mtu: 1500
>>>> - name: enp4s0
>>>>   type: ethernet
>>>>   state: up
>>>>   ethernet:
>>>>     auto-negotiation: true
>>>>     duplex: full
>>>>     speed: 1000
>>>>   ipv4:
>>>>     address:
>>>>     - ip: 144.76.84.73
>>>>       prefix-length: 32
>>>>     dhcp: false
>>>>     enabled: true
>>>>   ipv6:
>>>>     address:
>>>>     - ip: 2a01:4f8:192:1148::2
>>>>       prefix-length: 64
>>>>     - ip: fe80::62a4:4cff:fee9:4ac
>>>>       prefix-length: 64
>>>>     auto-dns: true
>>>>     auto-gateway: true
>>>>     auto-routes: true
>>>>     autoconf: true
>>>>     dhcp: true
>>>>     enabled: true
>>>>   mac-address: 60:A4:4C:E9:04:AC
>>>>   mtu: 1500
>>>> - name: enp4s0.4000
>>>>   type: vlan
>>>>   state: up
>>>>   ipv4:
>>>>     address:
>>>>     - ip: 172.27.1.1
>>>>       prefix-length: 24
>>>>     dhcp: false
>>>>     enabled: true
>>>>   ipv6:
>>>>     autoconf: false
>>>>     dhcp: false
>>>>     enabled: false
>>>>   mac-address: 60:A4:4C:E9:04:AC
>>>>   mtu: 1500
>>>>   vlan:
>>>>     base-iface: enp4s0
>>>>     id: 4000
>>>> - name: lo
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mtu: 65536
>>>> - name: ovs-system
>>>>   type: unknown
>>>>   state: down
>>>>   ipv4:
>>>>     enabled: false
>>>>   ipv6:
>>>>     enabled: false
>>>>   mac-address: A2:35:7A:6C:B7:EF
>>>>   mtu: 1500
>>>>
>>>> /Sverker
>>>>
>>>
>>> Was the enp4s0 always managed by NetworkManager or only after the
>>> attempt to make the ovirtmgmt? If not that would explain the failure.
>>> Also the workaround would be to configure the interface via
>>> NetworkManager and then run the host deploy again.
>>>
>>> Thanks,
>>> Ales
>>>
>>>
>>>
>>>> Den 2020-09-03 kl. 11:54, skrev Ales Musil:
>>>>
>>>>
>>>>
>>>> On Thu, Sep 3, 2020 at 11:51 AM Sverker Abrahamsson via Users <
>>>> users@ovirt.org> wrote:
>>>>
>>>>> Hi Dominik
>>>>> That is my issue, I don't get to where I can get the ovirtmgmt bridge
>>>>> established because vdsm insists on creating it. It used to be possible to
>>>>> create that bridge statically and vdsm would just skip it but seems to be
>>>>> broken now.
>>>>>
>>>>> If it would be possible to use OVN for the management network that
>>>>> would solve my issue and would be the preferable solution, but as you 
>>>>> write
>>>>> that isn't possible which was what I suspected.
>>>>>
>>>>> Do you have any other suggestion on how to solve this issue? That I
>>>>> get the external interface untagged and the internal network tagged is not
>>>>> possible to change.
>>>>>
>>>>> /Sverker
>>>>>
>>>>
>>>> Hello Sverker,
>>>>
>>>> can you please share output from "nmcli con show" and "nmstatectl show"?
>>>>
>>>> Thank you.
>>>> Regards,
>>>> Ales
>>>>
>>>>> Den 2020-09-03 kl. 10:52, skrev Dominik Holler:
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Sep 2, 2020 at 10:38 PM Sverker Abrahamsson via Users <
>>>>> users@ovirt.org> wrote:
>>>>>
>>>>>> Well, unforturnatly I don't have a choise since it is out of my
>>>>>> control.
>>>>>> I only have one physical network port where the external traffic is
>>>>>> untagged and the internal vlan is tagged. If I could run with OVN
>>>>>>
>>>>>
>>>>> OVN is for VM traffic only, not usable for the management network.
>>>>>
>>>>>
>>>>>> instead I wouldn't need that tagged vlan, but I haven't been able to
>>>>>> get
>>>>>> that to work neither.
>>>>>>
>>>>>>
>>>>> Please let us know if OVN does not work for VM traffic for you.
>>>>>
>>>>>
>>>>>> It's perfectly possible to have both tagged and untagged traffic on
>>>>>> the
>>>>>> same switch port, issue is that vdsm tries to take control over the
>>>>>> network without being able to be flexible enough.. I'm attempting now
>>>>>> to
>>>>>> have ovirtmgmt bridge created before, that used to be possible but
>>>>>> according to previous mails on the list it went broken somewhere at
>>>>>> 4.x.
>>>>>>
>>>>>> /Sverker
>>>>>>
>>>>>> Den 2020-09-02 kl. 21:39, skrev Strahil Nikolov:
>>>>>> > Switchports can either be tagged or untagged.
>>>>>> > I'm not sure that your setup is supported at all.
>>>>>> >
>>>>>> > Best Regards,
>>>>>> > Strahil Nikolov
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > В сряда, 2 септември 2020 г., 20:41:57 Гринуич+3, Sverker
>>>>>> Abrahamsson via Users <users@ovirt.org> написа:
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > Pretty formatting the "desired state" it seems that vdsm tries to
>>>>>> remove
>>>>>> > the ip of my underlying interface, that is enp4s0:
>>>>>> >
>>>>>
>>>>> > {
>>>>>> >      'interfaces': [{
>>>>>> >              'name': 'enp4s0',
>>>>>> >              'state': 'up',
>>>>>> >              'mtu': 1500
>>>>>> >          }, {
>>>>>> >              'vlan': {
>>>>>> >                  'id': 4000,
>>>>>> >                  'base-iface': 'enp4s0'
>>>>>> >              },
>>>>>> >              'name': 'enp4s0.4000',
>>>>>> >              'type': 'vlan',
>>>>>> >              'state': 'up',
>>>>>> >              'mtu': 1500,
>>>>>> >              'ipv4': {
>>>>>> >                  'enabled': False
>>>>>> >              },
>>>>>> >              'ipv6': {
>>>>>> >                  'enabled': False
>>>>>> >              }
>>>>>> >          }, {
>>>>>> >              'name': 'ovirtmgmt',
>>>>>> >              'type': 'linux-bridge',
>>>>>> >              'state': 'up',
>>>>>> >              'mtu': 1500,
>>>>>> >              'bridge': {
>>>>>> >                  'port': [{
>>>>>> >                          'name': 'enp4s0.4000'
>>>>>> >                      }
>>>>>> >                  ],
>>>>>> >                  'options': {
>>>>>> >                      'stp': {
>>>>>> >                          'enabled': False
>>>>>> >                      }
>>>>>> >                  }
>>>>>> >              },
>>>>>> >              'ipv4': {
>>>>>> >                  'enabled': True,
>>>>>> >                  'address': [{
>>>>>> >                          'ip': '172.27.1.1',
>>>>>> >                          'prefix-length': 24
>>>>>> >                      }
>>>>>> >                  ],
>>>>>> >                  'dhcp': False
>>>>>> >              },
>>>>>> >              'ipv6': {
>>>>>> >                  'enabled': False
>>>>>> >              }
>>>>>> >          }
>>>>>> >      ],
>>>>>> >      'dns-resolver': {
>>>>>> >          'config'
>>>>>> >          : {
>>>>>> >              'server': ['213.133.98.98']
>>>>>> >          }
>>>>>> >      }
>>>>>> > }
>>>>>> >
>>>>>>
>>>>>
>>>>> Thanks, this is helpful information.
>>>>> Can you please share the getCapabilities result sent from vdsm to
>>>>> Engine directly before the setupNetworks request,
>>>>> and the parameters of the setupNetworks request from Engine to vdsm?
>>>>> Both are in the vdsm.log during adding the host.
>>>>>
>>>>>
>>>>>> >
>>>>>> > This is my interfaces before vdsm attemtpts to change the config:
>>>>>> >
>>>>>> > enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>>>>>> >          inet 144.76.84.73  netmask 255.255.255.255  broadcast
>>>>>> 0.0.0.0
>>>>>> >          inet6 fe80::62a4:4cff:fee9:4ac  prefixlen 64  scopeid
>>>>>> 0x20<link>
>>>>>> >          inet6 2a01:4f8:192:1148::2  prefixlen 64  scopeid
>>>>>> 0x0<global>
>>>>>> >          ether 60:a4:4c:e9:04:ac  txqueuelen 1000  (Ethernet)
>>>>>> >          RX packets 293442  bytes 385541799 (367.6 MiB)
>>>>>> >          RX errors 0  dropped 0  overruns 0  frame 0
>>>>>> >          TX packets 91095  bytes 31160348 (29.7 MiB)
>>>>>> >          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>>>>>> >          device interrupt 17  memory 0xf7d00000-f7d20000
>>>>>> >
>>>>>> > enp4s0.4000: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>>>>>> >          inet 172.27.1.1  netmask 255.255.255.0  broadcast
>>>>>> 172.27.1.255
>>>>>> >          inet6 fe80::62a4:4cff:fee9:4ac  prefixlen 64  scopeid
>>>>>> 0x20<link>
>>>>>> >          ether 60:a4:4c:e9:04:ac  txqueuelen 1000  (Ethernet)
>>>>>> >          RX packets 0  bytes 0 (0.0 B)
>>>>>> >          RX errors 0  dropped 0  overruns 0  frame 0
>>>>>> >          TX packets 13  bytes 938 (938.0 B)
>>>>>> >          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>>>>>> >
>>>>>> > I.e. enp4s0 is the external interface that must not be changed,
>>>>>> bridge
>>>>>> > must be created on the vlan interface. I would prefer to create the
>>>>>> > bridge manually and not through vdsm if that is possible.
>>>>>> >
>>>>>> > /Sverker
>>>>>> >
>>>>>> > Den 2020-09-02 kl. 19:14, skrev Sverker Abrahamsson via Users:
>>>>>> >> Hi,
>>>>>> >> I'm attempting to install hosted engine but getting this failure:
>>>>>> >>
>>>>>> >> [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg":
>>>>>> >> "The host has been set in non_operational status, deployment
>>>>>> errors:
>>>>>> >> code 505: Host h1-mgmt.limetransit.com installation failed.
>>>>>> Failed to
>>>>>> >> configure management network on the host.,    code 1120: Failed to
>>>>>> >> configure management network on host h1-mgmt.limetransit.com due
>>>>>> to
>>>>>> >> setup networks failure., code 9000: Failed to verify Power
>>>>>> Management
>>>>>> >> configuration for Host h1-mgmt.limetransit.com.,    code 10802:
>>>>>> VDSM
>>>>>> >> h1-mgmt.limetransit.com command HostSetupNetworksVDS failed:
>>>>>> Internal
>>>>>> >> JSON-RPC error: {'reason': 'Unexpected failure of libnm when
>>>>>> running
>>>>>> >> the mainloop: run execution'},   fix accordingly and re-deploy."}
>>>>>> >>
>>>>>> >> Looking in vdsm.log I find this which I believe is the root cause:
>>>>>> >>
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >>
>>>>>> 16:38:25,897::supervdsm_server::93::SuperVdsm.ServerCallback::(wrapper)
>>>>>> >> call setupNetworks with ({'ovirtmgmt': {'vlan': '4000', 'netmask':
>>>>>> >> '255.255.255.0', 'ipv6autoconf': False,
>>>>>> >> 'nic': 'enp4s0', 'bridged': 'true', 'ipaddr': '172.27.1.1',
>>>>>> >> 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'mtu': 1500,
>>>>>> >> 'switch': 'legacy'}}, {}, {'connectivityTimeout': 120,
>>>>>> >> 'commitOnSuccess': True, 'connectivityCh
>>>>>> >> eck': 'true'}) {}
>>>>>> >> MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >> 16:38:25,897::api::220::root::(setupNetworks) Setting up network
>>>>>> >> according to configuration: networks:{'ovirtmgmt': {'vlan': '4000',
>>>>>> >> 'netmask': '255.255.255.0', 'ipv6autoconf': Fal
>>>>>> >> se, 'nic': 'enp4s0', 'bridged': 'true', 'ipaddr': '172.27.1.1',
>>>>>> >> 'defaultRoute': True, 'dhcpv6': False, 'STP': 'no', 'mtu': 1500,
>>>>>> >> 'switch': 'legacy'}}, bondings:{}, options:{'connectivityTimeout':
>>>>>> >> 120, 'commitOnSuccess':
>>>>>> >> True, 'connectivityCheck': 'true'}
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:25,902::cmdutils::130::root::(exec_cmd) /sbin/tc qdisc show
>>>>>> (cwd
>>>>>> >> None)
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:25,906::cmdutils::138::root::(exec_cmd) SUCCESS: <err> = b'';
>>>>>> >> <rc> = 0
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:25,945::vsctl::74::root::(commit) Executing commands:
>>>>>> >> /usr/bin/ovs-vsctl --timeout=5 --oneline --format=json -- list
>>>>>> Bridge
>>>>>> >> -- list Port -- list Interface
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:25,945::cmdutils::130::root::(exec_cmd) /usr/bin/ovs-vsctl
>>>>>> >> --timeout=5 --oneline --format=json -- list Bridge -- list Port --
>>>>>> >> list Interface (cwd None)
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:25,952::cmdutils::138::root::(exec_cmd) SUCCESS: <err> = b'';
>>>>>> >> <rc> = 0
>>>>>> >> MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >> 16:38:25,957::netconfpersistence::58::root::(setNetwork) Adding
>>>>>> >> network ovirtmgmt({'vlan': 4000, 'netmask': '255.255.255.0',
>>>>>> >> 'ipv6autoconf': False, 'nic': 'enp4s0', 'bridged': True
>>>>>> >> , 'ipaddr': '172.27.1.1', 'defaultRoute': True, 'dhcpv6': False,
>>>>>> >> 'mtu': 1500, 'switch': 'legacy', 'stp': False, 'bootproto': 'none',
>>>>>> >> 'nameservers': ['213.133.98.98']})
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:25,958::commands::153::common.commands::(start)
>>>>>> /usr/bin/taskset
>>>>>> >> --cpu-list 0-7 /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe
>>>>>> >> (cwd None)
>>>>>> >> MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >> 16:38:26,154::hooks::122::root::(_runHooksDir)
>>>>>> >> /usr/libexec/vdsm/hooks/before_network_setup/50_fcoe: rc=0 err=b''
>>>>>> >> MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >> 16:38:26,155::configurator::195::root::(_setup_nmstate) Processing
>>>>>> >> setup through nmstate
>>>>>> >> MainProcess|jsonrpc/0::INFO::2020-09-02
>>>>>> >> 16:38:26,175::configurator::197::root::(_setup_nmstate) Desired
>>>>>> state:
>>>>>> >> {'interfaces': [{'name': 'enp4s0', 'state': 'up', 'mtu': 1500},
>>>>>> >> {'vlan': {'id': 4000, 'base-iface': 'enp4s0'}
>>>>>> >> , 'name': 'enp4s0.4000', 'type': 'vlan', 'state': 'up', 'mtu':
>>>>>> 1500,
>>>>>> >> 'ipv4': {'enabled': False}, 'ipv6': {'enabled': False}}, {'name':
>>>>>> >> 'ovirtmgmt', 'type': 'linux-bridge', 'state': 'up', 'mtu': 1500,
>>>>>> >> 'bridge': {'port': [
>>>>>> >> {'name': 'enp4s0.4000'}], 'options': {'stp': {'enabled': False}}},
>>>>>> >> 'ipv4': {'enabled': True, 'address': [{'ip': '172.27.1.1',
>>>>>> >> 'prefix-length': 24}], 'dhcp': False}, 'ipv6': {'enabled':
>>>>>> False}}],
>>>>>> >> 'dns-resolver': {'config'
>>>>>> >> : {'server': ['213.133.98.98']}}}
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,217::checkpoint::121::root::(create) Checkpoint
>>>>>> >> /org/freedesktop/NetworkManager/Checkpoint/1 created for all
>>>>>> devices: 60
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,218::netapplier::239::root::(_add_interfaces) Adding new
>>>>>> >> interfaces: ['ovirtmgmt']
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,221::netapplier::251::root::(_edit_interfaces) Editing
>>>>>> >> interfaces: ['enp4s0.4000', 'enp4s0']
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,224::nmclient::136::root::(execute_next_action) Executing
>>>>>> NM
>>>>>> >> action: func=add_connection_async
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,231::connection::329::root::(_add_connection_callback)
>>>>>> >> Connection adding succeeded: dev=ovirtmgmt
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,232::nmclient::136::root::(execute_next_action) Executing
>>>>>> NM
>>>>>> >> action: func=commit_changes_async
>>>>>> >> MainProcess|jsonrpc/0::ERROR::2020-09-02
>>>>>> >> 16:38:26,233::nmclient::200::root::(quit) NM main-loop aborted:
>>>>>> >> Connection update failed: error=nm-connection-error-quark:
>>>>>> >> ipv6.gateway: gateway cannot be set if there are no addresses
>>>>>> >> configured (7), dev=enp4s0/<enum NM_DEVICE_STATE_ACTIVATED of type
>>>>>> >> NM.DeviceState>
>>>>>> >> MainProcess|jsonrpc/0::DEBUG::2020-09-02
>>>>>> >> 16:38:26,246::checkpoint::164::root::(rollback) Checkpoint
>>>>>> >> /org/freedesktop/NetworkManager/Checkpoint/1 rollback executed:
>>>>>> >>
>>>>>> dbus.Dictionary({dbus.String('/org/freedesktop/NetworkManager/Devices/6'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/7'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/8'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/9'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/3'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/2'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/4'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/1'):
>>>>>> >> dbus.UInt32(0),
>>>>>> >> dbus.String('/org/freedesktop/NetworkManager/Devices/5'):
>>>>>> >> dbus.UInt32(0)}, signature=dbus.Signature('su'))
>>>>>> >> MainProcess|jsonrpc/0::ERROR::2020-09-02
>>>>>> >>
>>>>>> 16:38:31,251::supervdsm_server::97::SuperVdsm.ServerCallback::(wrapper)
>>>>>> >> Error in setupNetworks
>>>>>> >> Traceback (most recent call last):
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/vdsm/supervdsm_server.py",
>>>>>> >> line 95, in wrapper
>>>>>> >>      res = func(*args, **kwargs)
>>>>>> >>    File "/usr/lib/python3.6/site-packages/vdsm/network/api.py",
>>>>>> line
>>>>>> >> 241, in setupNetworks
>>>>>> >>      _setup_networks(networks, bondings, options, net_info)
>>>>>> >>    File "/usr/lib/python3.6/site-packages/vdsm/network/api.py",
>>>>>> line
>>>>>> >> 266, in _setup_networks
>>>>>> >>      networks, bondings, options, net_info, in_rollback
>>>>>> >>    File
>>>>>> >>
>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py",
>>>>>> >> line 154, in setup
>>>>>> >>      _setup_nmstate(networks, bondings, options, in_rollback)
>>>>>> >>    File
>>>>>> >>
>>>>>> "/usr/lib/python3.6/site-packages/vdsm/network/netswitch/configurator.py",
>>>>>> >> line 199, in _setup_nmstate
>>>>>> >>      nmstate.setup(desired_state, verify_change=not in_rollback)
>>>>>> >>    File "/usr/lib/python3.6/site-packages/vdsm/network/nmstate.py",
>>>>>> >> line 63, in setup
>>>>>> >>      state_apply(desired_state, verify_change=verify_change)
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/libnmstate/deprecation.py",
>>>>>> >> line 40, in wrapper
>>>>>> >>      return func(*args, **kwargs)
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py",
>>>>>> >> line 96, in wrapped
>>>>>> >>      ret = func(*args, **kwargs)
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py",
>>>>>> >> line 73, in apply
>>>>>> >>      state.State(desired_state), verify_change, commit,
>>>>>> rollback_timeout
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py",
>>>>>> >> line 163, in _apply_ifaces_state
>>>>>> >>      con_profiles=ifaces_add_configs + ifaces_edit_configs,
>>>>>> >>    File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
>>>>>> >>      next(self.gen)
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py",
>>>>>> >> line 232, in _setup_providers
>>>>>> >>      mainloop.run(timeout=MAINLOOP_TIMEOUT)
>>>>>> >>    File
>>>>>> "/usr/lib/python3.6/site-packages/libnmstate/nm/nmclient.py",
>>>>>> >> line 177, in run
>>>>>> >>      f"Unexpected failure of libnm when running the mainloop:
>>>>>> {err}"
>>>>>> >> libnmstate.error.NmstateLibnmError: Unexpected failure of libnm
>>>>>> when
>>>>>> >> running the mainloop: run execution
>>>>>> >>
>>>>>> >> My host have a main network interface and a vlan interface on the
>>>>>> same
>>>>>> >> physical interface which is an internal management network which
>>>>>> >> should be used as management network. The vlan interface doesn't
>>>>>> have
>>>>>> >> any ipv6 interface, but nmclient fails on setting ipv6 gateway.
>>>>>> This
>>>>>> >> seems like a bug?
>>>>>> >>
>>>>>> >> /Sverker
>>>>>> >>
>>>>>> >> _______________________________________________
>>>>>> >> discuss mailing list
>>>>>> >> disc...@openvswitch.org
>>>>>> >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>>>>> >> _______________________________________________
>>>>>> >> Users mailing list -- users@ovirt.org
>>>>>> >> To unsubscribe send an email to users-le...@ovirt.org
>>>>>> >> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>> >> oVirt Code of Conduct:
>>>>>> >> https://www.ovirt.org/community/about/community-guidelines/
>>>>>> >> List Archives:
>>>>>> >>
>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/R4YSTOQM7I4XLCWQJOJH5DKZFX56IDAZ/
>>>>>> > _______________________________________________
>>>>>> > Users mailing list -- users@ovirt.org
>>>>>> > To unsubscribe send an email to users-le...@ovirt.org
>>>>>> > Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>> > oVirt Code of Conduct:
>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>> > List Archives:
>>>>>> >
>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/UMXFPVLJHYOO3UR5PI24EPIXC3KATDPQ/
>>>>>> >
>>>>>> _______________________________________________
>>>>>> Users mailing list -- users@ovirt.org
>>>>>> To unsubscribe send an email to users-le...@ovirt.org
>>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>>> oVirt Code of Conduct:
>>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>>> List Archives:
>>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/RQIPLZGVOHXU5D2VPKSVPX7NEPYKGTRW/
>>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list -- users@ovirt.org
>>>>> To unsubscribe send an email to users-le...@ovirt.org
>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html
>>>>> oVirt Code of Conduct:
>>>>> https://www.ovirt.org/community/about/community-guidelines/
>>>>> List Archives:
>>>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/ED2YKQAKVG66JOMX3JCNXPMWK6OX73ET/
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Ales Musil
>>>>
>>>> Software Engineer - RHV Network
>>>>
>>>> Red Hat EMEA <https://www.redhat.com>
>>>>
>>>> amu...@redhat.com    IM: amusil
>>>> <https://red.ht/sig>
>>>>
>>>>
>>>
>>> --
>>>
>>> Ales Musil
>>>
>>> Software Engineer - RHV Network
>>>
>>> Red Hat EMEA <https://www.redhat.com>
>>>
>>> amu...@redhat.com    IM: amusil
>>> <https://red.ht/sig>
>>>
>>>
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/A3DOZNSL5Y6RVVWBD7AIS7AUJTDIKN42/

Reply via email to