Re: [ovirt-users] Network config

Tue, 27 Jan 2015 10:17:35 -0800 


On 27.1.2015 19:10, Ondřej Svoboda wrote:
From the first log (journal): VDSM managed to start, only it is being run too early (it should be waiting patiently for its supervdsm component to start responding). 

Are there no "RTNETLINK answers: File exists" errors anymore?

Second log: This is normal operation.


So the host stopped responding? It should have come up, assuming that 
it obtained the same address from DHCP (which it should, but please 
check).



Could you try using static network configuration first? And when it 
succeeds you can try DHCP again.


===


About the GUI: Is your host's status now "Not Operational"? In that 
case oVirt cannot tell anything about the status of the host's 
interfaces, it may only reflect the status before communication was lost.



Please file a bug if this is really the case I described :-) It would 
then be a matter of discussion if oVirt should display a yellow 
"unknown" status of the interfaces, or keep displaying the last known 
one. (The first solution would make it obvious that we really lost the 
host and go looking for the host's status to discover it is "Not 
Operational". The other is informative as well, just in a different way.)



I should add that the "yellow, unknown" state is just a suggestion, it 
need not exist yet.



On 27.1.2015 16:31, Jorick Astrego wrote:


Done, maybe this helps:

    Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
    unified_network_persistence_upgrade
    Jan 27 16:20:26 test2 vdsmd_init_common.sh: vdsm: Running
    restore_nets
    Jan 27 16:21:47 test2 systemd-logind: New session 5 of user root.
    Jan 27 16:21:47 test2 systemd: Starting Session 5 of user root.
    Jan 27 16:21:47 test2 systemd: Started Session 5 of user root.
    Jan 27 16:21:55 test2 systemd: vdsmd.service operation timed out.
    Terminating.
    Jan 27 16:21:55 test2 journal: End of file while reading data:
    Input/output error
    Jan 27 16:21:55 test2 systemd: Failed to start Virtual Desktop
    Server Manager.
    Jan 27 16:21:55 test2 systemd: Unit vdsmd.service entered failed
    state.
    Jan 27 16:21:55 test2 systemd: vdsmd.service holdoff time over,
    scheduling restart.
    Jan 27 16:21:55 test2 systemd: Stopping Virtual Desktop Server
    Manager...
    Jan 27 16:21:55 test2 systemd: Starting Virtual Desktop Server
    Manager...
    Jan 27 16:21:55 test2 vdsmd_init_common.sh: vdsm: Running mkdirs

Still:

    MainThread::INFO::2015-01-27
    16:21:56,964::netconfpersistence::158::root::(_clearDisk)
    Clearing /var/run/vdsm/netconf/nets/ and /var/run/vdsm/netconf/bonds/
    MainThread::DEBUG::2015-01-27
    16:21:56,964::netconfpersistence::166::root::(_clearDisk) No
    existent config to clear.
    MainThread::INFO::2015-01-27
    16:21:56,964::netconfpersistence::182::root::(save) Saved new
    config RunningConfig({}, {}) to /var/run/vdsm/netconf/nets/ and
    /var/run/vdsm/netconf/bonds/
    MainThread::DEBUG::2015-01-27
    16:21:56,965::netconfpersistence::134::root::(_getConfigs)
    Non-existing config set.

And when I want to reapply setup networks in ovirt I get:


    Error while executing action Commit Network changes: Network
    error during communication with the Host.


Every interface and bond is still green and visible in the webgui by 
the way. That is also strange, every interface of the host is bonded 
and up while in reality the host has lost it's network config and 
doesn't respond anymore.


Shouldn't they all be red on communication failure to the host?

Cheers,

Jorick

On 01/27/2015 04:01 PM, Ondřej Svoboda wrote:

This is weird, like there were multiple dhclients on the bond. Can 
you stop VDSM (systemctl stop vdsmd supervdsmd) and maybe `killall 
-SIGKILL dhclient` and restart VDSM afterwards?


On 27.1.2015 14:43, Jorick Astrego wrote:

Ah yes, I forgot to disable it as I switched to foreman provisioning.


Maybe it can be auto detected and disabled in the "install host" 
scripts when ovirt installs vdsm and sets up the networking. For 
people who forget to disable it.....


But now I get this:

    sourceRoute::DEBUG::2015-01-27
    14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
    0.0.0.0/0 via ***.***.***.*** dev bond0 table 174351414 (cwd None)
    sourceRoute::DEBUG::2015-01-27
    14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
    'RTNETLINK answers: File exists\n'; <rc> = 2
    sourceRoute::ERROR::2015-01-27
    14:33:57,420::sourceroute::88::root::(configure) ip binary
    failed during source route configuration: ['RTNETLINK answers:
    File exists']
    sourceRoute::WARNING::2015-01-27
    14:33:57,420::utils::129::root::(rmFile) File:
    /var/run/vdsm/trackedInterfaces/bond0 already removed

    Jan 27 14:38:19 test3.netbulae.test network[3042]: RTNETLINK
    answers: File exists
    Jan 27 14:38:19 test3.netbulae.test systemd[1]:
    network.service: control process exited, code=exited status=1
    Jan 27 14:38:19 test3.netbulae.test systemd[1]: Failed to start
    LSB: Bring up/down networking.
    -- Subject: Unit network.service has failed
    -- Defined-By: systemd
    -- Support:
    http://lists.freedesktop.org/mailman/listinfo/systemd-devel

    -- 
    -- Unit network.service has failed.
    -- 
    -- The result is failed.

    Jan 27 14:38:19 test3.netbulae.test systemd[1]: Unit
    network.service entered failed state.



Bond0 is still there though

    sourceRoute::DEBUG::2015-01-27
    14:33:57,410::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
    Responding to DHCP response in
    /var/run/vdsm/sourceRoutes/1422365637
    sourceRoute::INFO::2015-01-27
    14:33:57,416::sourceroute::78::root::(configure) Configuring
    gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
    255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
    bond0
    sourceRoute::DEBUG::2015-01-27
    14:33:57,416::utils::739::root::(execCmd) /sbin/ip -4 route add
    0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
    sourceRoute::DEBUG::2015-01-27
    14:33:57,420::utils::759::root::(execCmd) FAILED: <err> =
    'RTNETLINK answers: File exists\n'; <rc> = 2
    sourceRoute::ERROR::2015-01-27
    14:33:57,420::sourceroute::88::root::(configure) ip binary
    failed during source route configuration: ['RTNETLINK answers:
    File exists']
    sourceRoute::WARNING::2015-01-27
    14:33:57,420::utils::129::root::(rmFile) File:
    /var/run/vdsm/trackedInterfaces/bond0 already removed
    sourceRoute::DEBUG::2015-01-27
    14:37:52,085::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
    Responding to DHCP response in
    /var/run/vdsm/sourceRoutes/1422365872
    sourceRoute::INFO::2015-01-27
    14:37:52,090::sourceroute::78::root::(configure) Configuring
    gateway - ip: **.***.**.**, network: 10.100.100.0/24, subnet:
    255.255.255.0, gateway: **.***.**.**, table: 174351414, device:
    bond0
    sourceRoute::DEBUG::2015-01-27
    14:37:52,090::utils::739::root::(execCmd) /sbin/ip -4 route add
    0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd None)
    sourceRoute::DEBUG::2015-01-27
    14:37:52,094::utils::759::root::(execCmd) FAILED: <err> =
    'RTNETLINK answers: File exists\n'; <rc> = 2
    sourceRoute::ERROR::2015-01-27
    14:37:52,094::sourceroute::88::root::(configure) ip binary
    failed during source route configuration: ['RTNETLINK answers:
    File exists']
    sourceRoute::WARNING::2015-01-27
    14:37:52,094::utils::129::root::(rmFile) File:
    /var/run/vdsm/trackedInterfaces/bond0 already removed


If you need more info, let me know.


On 01/27/2015 02:20 PM, Ondřej Svoboda wrote:

Jorick,


just what comes to my mind: do you have NetworkManager running, by 
any chance? VDSM and NetworkManager don't mix well currently so 
please try


# systemctl stop NetworkManager
# systemctl disable NetworkManager

Regards,
Ondra Svoboda

On 27.1.2015 14:10, Jorick Astrego wrote:


On 01/27/2015 01:56 PM, Koen Vanoppen wrote:

Hello everybody,


Just wanted to know if anybody else has the problem, that when 
you restart your network configuration of a hypervisor, all the 
settings concerning the bonding are gone...

Do I file a bug report for this or is this already a known issue?

Kind regards,

Koen


Hmm,

I cannot restart the network service from the host for some reason:

    Jan 27 14:00:27 test2 systemd: network.service: control
    process exited, code=exited status=1
    Jan 27 14:00:27 test2 systemd: Failed to start LSB: Bring
    up/down networking.


    systemctl status network.service
    network.service - LSB: Bring up/down networking
       Loaded: loaded (/etc/rc.d/init.d/network)
       Active: failed (Result: exit-code) since Tue 2015-01-27
    14:00:27 CET; 1min 56s ago
      Process: 4113 ExecStart=/etc/rc.d/init.d/network start
    (code=exited, status=1/FAILURE)

    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test network[4113]: RTNETLINK
    answers: File exists
    Jan 27 14:00:27 test2.netbulae.test systemd[1]:
    network.service: control process exited, code=exited status=1
    Jan 27 14:00:27 test2.netbulae.test systemd[1]: Failed to
    start LSB: Bring up/down networking.
    Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
    network.service entered failed state.

    -- Subject: Unit network.service has failed
    -- Defined-By: systemd
    -- Support:
    http://lists.freedesktop.org/mailman/listinfo/systemd-devel

    -- 
    -- Unit network.service has failed.
    -- 
    -- The result is failed.

    Jan 27 14:00:27 test2.netbulae.test systemd[1]: Unit
    network.service entered failed state.


    In supervdsm I find lots and lots of these:


    sourceRoute::DEBUG::2015-01-27
    14:02:40,802::utils::739::root::(execCmd) /sbin/ip -4 route
    add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
    None)
    sourceRoute::DEBUG::2015-01-27
    14:02:40,806::utils::759::root::(execCmd) FAILED: <err> =
    'RTNETLINK answers: File exists\n'; <rc> = 2
    sourceRoute::ERROR::2015-01-27
    14:02:40,806::sourceroute::88::root::(configure) ip binary
    failed during source route configuration: ['RTNETLINK
    answers: File exists']
    sourceRoute::WARNING::2015-01-27
    14:02:40,807::utils::129::root::(rmFile) File:
    /var/run/vdsm/trackedInterfaces/bond0 already removed
    sourceRoute::DEBUG::2015-01-27
    14:07:09,047::sourceroutethread::39::root::(process_IN_CLOSE_WRITE_filePath)
    Responding to DHCP response in
    /var/run/vdsm/sourceRoutes/1422364029
    sourceRoute::INFO::2015-01-27
    14:07:09,052::sourceroute::78::root::(configure) Configuring
    gateway - ip: **.***.**.***, network: **.***.**.**0/24,
    subnet: 255.255.255.0, gateway: **.***.**.**, table:
    174351414, device: bond0
    sourceRoute::DEBUG::2015-01-27
    14:07:09,052::utils::739::root::(execCmd) /sbin/ip -4 route
    add 0.0.0.0/0 via **.***.**.** dev bond0 table 174351414 (cwd
    None)
    sourceRoute::DEBUG::2015-01-27
    14:07:09,057::utils::759::root::(execCmd) FAILED: <err> =
    'RTNETLINK answers: File exists\n'; <rc> = 2
    sourceRoute::ERROR::2015-01-27
    14:07:09,057::sourceroute::88::root::(configure) ip binary
    failed during source route configuration: ['RTNETLINK
    answers: File exists']
    sourceRoute::WARNING::2015-01-27
    14:07:09,057::utils::129::root::(rmFile) File:
    /var/run/vdsm/trackedInterfaces/bond0 already removed



The hosts are all Centos 7 freshly provisioned. With these vdsm 
packages:


    vdsm-python-zombiereaper-4.16.10-8.gitc937927.el7.noarch
    vdsm-python-4.16.10-8.gitc937927.el7.noarch
    vdsm-yajsonrpc-4.16.10-8.gitc937927.el7.noarch
    vdsm-4.16.10-8.gitc937927.el7.x86_64
    vdsm-xmlrpc-4.16.10-8.gitc937927.el7.noarch
    vdsm-jsonrpc-4.16.10-8.gitc937927.el7.noarch
    vdsm-cli-4.16.10-8.gitc937927.el7.noarch





Met vriendelijke groet, With kind regards,

Jorick Astrego*

Netbulae Virtualization Experts *
------------------------------------------------------------------------
Tel: 053 20 30 270      i...@netbulae.eu        Staalsteden 4-3A        KvK 
08198180

Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW 
NL821234584B01



------------------------------------------------------------------------



_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users




_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users





Met vriendelijke groet, With kind regards,

Jorick Astrego*

Netbulae Virtualization Experts *
------------------------------------------------------------------------
Tel: 053 20 30 270      i...@netbulae.eu        Staalsteden 4-3A        KvK 
08198180

Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW 
NL821234584B01



------------------------------------------------------------------------



_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users




_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users





Met vriendelijke groet, With kind regards,

Jorick Astrego*

Netbulae Virtualization Experts *
------------------------------------------------------------------------
Tel: 053 20 30 270      i...@netbulae.eu        Staalsteden 4-3A        KvK 
08198180

Fax: 053 20 30 271 	www.netbulae.eu 	7547 TA Enschede 	BTW 
NL821234584B01



------------------------------------------------------------------------



_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users




_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users



_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users






















					Reply via email to