Re: [ovirt-users] multiple ip routing table issue

2017-11-22 Thread Edward Clay 
On Wed, 2017-11-22 at 10:46 +0200, Edward Haas wrote:
> On Wed, Nov 22, 2017 at 1:26 AM, Edward Clay  m> wrote:
> > On Tue, 2017-11-21 at 16:01 -0700, Edward Clay wrote:
> > > On Wed, 2017-11-22 at 00:17 +0200, Edward Haas wrote:
> > > > On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay  > > > oup.com> wrote:
> > > > > On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
> > > > > > On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay  > > > > > k2group.com> wrote:
> > > > > > > Hello,
> > > > > > > 
> > > > > > > We have an issue where hosts are configured with the
> > > > > > > public facing nework interface as the ovirtmgmt network
> > > > > > > and it's default route is added to a ovirt created table
> > > > > > > but not to the main routing table.  From my searching
> > > > > > > I've found this snippet from https://www.ovirt.org/develo
> > > > > > > p/release-management/features/network/multiple-gateways/
> > > > > > > which seems to explain why I can't ping anything or communicate 
> > > > > > > with any other system needing a default route.
> > > > > > 
> > > > > > By default, the default route is set on the ovirtmgmt
> > > > > > network (the default one, defined on the interface/ip which
> > > > > > you added the host to Engine).
> > > > > > Do you have a different network set up which you will like
> > > > > > to set the default route on?
> > > > > > 
> > > > > >  
> > > > > > > "And finally, here's the host's main routing table. Any
> > > > > > > traffic coming in to the host will use the ip rules and
> > > > > > > an interface's routing table. The main routing table is
> > > > > > > only used for traffic originating from the host."
> > > > > > > 
> > > > > > > I'm seeing the following main and custom ovirt created
> > > > > > > tables.
> > > > > > > 
> > > > > > > main:
> > > > > > > # ip route show table main
> > > > > > > 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106 
> > > > > > > 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src
> > > > > > > 10.4.16.15 
> > > > > > > 1.1.1.0/24 dev PUBLICB proto kernel scope link src
> > > > > > > 1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002 
> > > > > > > 169.254.0.0/16 dev enp3s0 scope link metric 1003 
> > > > > > > 169.254.0.0/16 dev enp7s0 scope link metric 1004 
> > > > > > > 169.254.0.0/16 dev enp3s0.106 scope link metric 1020 
> > > > > > > 169.254.0.0/16 dev PRIVATE scope link metric 1022 
> > > > > > > 169.254.0.0/16 dev PUBLIC scope link metric 1024 
> > > > > > > 
> > > > > > > table 1138027711
> > > > > > > # ip route show table 1138027711
> > > > > > > default via 1.1.1.1 dev PUBLIC
> > > > > > > 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
> > > > > > > 
> > > > > > > If I manually execute the following command to add the
> > > > > > > default route as well to the main table I can ping ouside
> > > > > > > of the local network.
> > > > > > > 
> > > > > > > ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
> > > > > > > 
> > > > > > > If I attempt to modify the /etc/sysconfig/network-
> > > > > > > scripts/route-PUBLIC ad reboot the server ad one would
> > > > > > > think this file is recreated by vdsm on boot.
> > > > > > > 
> > > > > > > What I'm looking for is the correct way to setup a
> > > > > > > default gateway for the main routing table so the hosts
> > > > > > > can get OS updates and communicate with the outside
> > > > > > > world.
> > > > > > 
> > > > > > Providing the output from "ip addr" may help clear up some
> > > > > > things.
> > > > > > It looks like you have on the host the default route set as
> > > > > > 10.4.16.1 (on enp3s0.106), could you elaborate what this
> > > > > > interface is?
> > > > > 
> > > > > We have setup vlan taging to utilize the 2 internetal network
> > > > > interfaces (originally enp6s0 and enp7s0) to be configured
> > > > > with mulitiple networks each.  We eventually added 10Gb nics
> > > > > to all servers to improve san glusterfs performance which is
> > > > > enp3s0 which replaced enp6s0 in our setup.
> > > > > 
> > > > > enp3s0.106 = ovirtmgmt network access to private internal
> > > > > networks only
> > > > > enp3s0.206 = private network bridge PRIVATE used for private
> > > > > internal network access for VMs
> > > > > enp7s0.606 = is used for public access for both VMs (bridge)
> > > > > and each host/cp/san in our ovirt setup named PUBLIC
> > > > > 
> > > > > # ip addr show
> > > > > 1: lo:  mtu 65536 qdisc noqueue state
> > > > > UNKNOWN qlen 1
> > > > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > > > inet 127.0.0.1/8 scope host lo
> > > > >valid_lft forever preferred_lft forever
> > > > > inet6 ::1/128 scope host 
> > > > >valid_lft forever preferred_lft forever
> > > > > 2: enp6s0:  mtu 1500 qdisc
> > > > > pfifo_fast state UP qlen 1000
> > > > > link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> > > > > inet6

Re: [ovirt-users] multiple ip routing table issue

2017-11-22 Thread Edward Haas 
On Wed, Nov 22, 2017 at 1:26 AM, Edward Clay 
wrote:

> On Tue, 2017-11-21 at 16:01 -0700, Edward Clay wrote:
>
> On Wed, 2017-11-22 at 00:17 +0200, Edward Haas wrote:
>
>
>
> On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay 
> wrote:
>
>
> On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
>
>
>
> On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay 
> wrote:
>
> Hello,
>
> We have an issue where hosts are configured with the public facing nework
> interface as the ovirtmgmt network and it's default route is added to a
> ovirt created table but not to the main routing table. From my searching
> I've found this snippet from https://www.ovirt.org/develop/
> release-management/features/network/multiple-gateways/ which seems to
> explain why I can't ping anything or communicate with any other system
> needing a default route.
>
>
> By default, the default route is set on the ovirtmgmt network (the default
> one, defined on the interface/ip which you added the host to Engine).
> Do you have a different network set up which you will like to set the
> default route on?
>
>
>
> "And finally, here's the host's main routing table. Any traffic coming in
> to the host will use the ip rules and an interface's routing table. The
> main routing table is only used for traffic originating from the host."
>
> I'm seeing the following main and custom ovirt created tables.
>
> main:
> # ip route show table main
> 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106
> 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src 10.4.16.15
> 1.1.1.0/24 dev PUBLICB proto kernel scope link src 1.1.1.1 169.254.0.0/16
> dev enp6s0 scope link metric 1002
> 169.254.0.0/16 dev enp3s0 scope link metric 1003
> 169.254.0.0/16 dev enp7s0 scope link metric 1004
> 169.254.0.0/16 dev enp3s0.106 scope link metric 1020
> 169.254.0.0/16 dev PRIVATE scope link metric 1022
> 169.254.0.0/16 dev PUBLIC scope link metric 1024
>
> table 1138027711
> # ip route show table 1138027711
> default via 1.1.1.1 dev PUBLIC
> 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
>
> If I manually execute the following command to add the default route as
> well to the main table I can ping ouside of the local network.
>
> ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
>
> If I attempt to modify the /etc/sysconfig/network-scripts/route-PUBLIC ad
> reboot the server ad one would think this file is recreated by vdsm on boot.
>
> What I'm looking for is the correct way to setup a default gateway for the
> main routing table so the hosts can get OS updates and communicate with the
> outside world.
>
>
> Providing the output from "ip addr" may help clear up some things.
> It looks like you have on the host the default route set as 10.4.16.1 (on
> enp3s0.106), could you elaborate what this interface is?
>
>
> We have setup vlan taging to utilize the 2 internetal network interfaces
> (originally enp6s0 and enp7s0) to be configured with mulitiple networks
> each. We eventually added 10Gb nics to all servers to improve san glusterfs
> performance which is enp3s0 which replaced enp6s0 in our setup.
>
> enp3s0.106 = ovirtmgmt network access to private internal networks only
> enp3s0.206 = private network bridge PRIVATE used for private internal
> network access for VMs
> enp7s0.606 = is used for public access for both VMs (bridge) and each
> host/cp/san in our ovirt setup named PUBLIC
>
> # ip addr show
> 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN qlen 1
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
>valid_lft forever preferred_lft forever
> inet6 ::1/128 scope host
>valid_lft forever preferred_lft forever
> 2: enp6s0:  mtu 1500 qdisc pfifo_fast
> state UP qlen 1000
> link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> inet6 fe80::225:90ff:fe38:d62c/64 scope link
>valid_lft forever preferred_lft forever
> 3: enp3s0:  mtu 1500 qdisc mq state UP
> qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> inet6 fe80::92e2:baff:fe1d:a400/64 scope link
>valid_lft forever preferred_lft forever
> 4: enp7s0:  mtu 1500 qdisc pfifo_fast
> state UP qlen 1000
> link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> 20: enp3s0.106@enp3s0:  mtu 1500 qdisc
> noqueue state UP qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> inet 10.4.16.15/24 brd 10.4.16.255 scope global enp3s0.106
>valid_lft forever preferred_lft forever
> 21: enp3s0.206@enp3s0:  mtu 1500 qdisc
> noqueue master PRIVATEB state UP qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> 22: PRIVATE:  mtu 1500 qdisc noqueue
> state UP qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> 23:

Re: [ovirt-users] multiple ip routing table issue

2017-11-21 Thread Edward Clay 
On Tue, 2017-11-21 at 16:01 -0700, Edward Clay wrote:
> On Wed, 2017-11-22 at 00:17 +0200, Edward Haas wrote:
> > On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay  > com> wrote:
> > > On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
> > > > On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay  > > > oup.com> wrote:
> > > > > Hello,
> > > > > 
> > > > > We have an issue where hosts are configured with the public
> > > > > facing nework interface as the ovirtmgmt network and it's
> > > > > default route is added to a ovirt created table but not to
> > > > > the main routing table.  From my searching I've found this
> > > > > snippet from https://www.ovirt.org/develop/release-management
> > > > > /features/network/multiple-gateways/ which seems to explain
> > > > > why I can't ping anything or communicate with any other
> > > > > system needing a default route.
> > > > 
> > > > By default, the default route is set on the ovirtmgmt network
> > > > (the default one, defined on the interface/ip which you added
> > > > the host to Engine).
> > > > Do you have a different network set up which you will like to
> > > > set the default route on?
> > > > 
> > > >  
> > > > > "And finally, here's the host's main routing table. Any
> > > > > traffic coming in to the host will use the ip rules and an
> > > > > interface's routing table. The main routing table is only
> > > > > used for traffic originating from the host."
> > > > > 
> > > > > I'm seeing the following main and custom ovirt created
> > > > > tables.
> > > > > 
> > > > > main:
> > > > > # ip route show table main
> > > > > 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106 
> > > > > 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src
> > > > > 10.4.16.15 
> > > > > 1.1.1.0/24 dev PUBLICB proto kernel scope link src
> > > > > 1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002 
> > > > > 169.254.0.0/16 dev enp3s0 scope link metric 1003 
> > > > > 169.254.0.0/16 dev enp7s0 scope link metric 1004 
> > > > > 169.254.0.0/16 dev enp3s0.106 scope link metric 1020 
> > > > > 169.254.0.0/16 dev PRIVATE scope link metric 1022 
> > > > > 169.254.0.0/16 dev PUBLIC scope link metric 1024 
> > > > > 
> > > > > table 1138027711
> > > > > # ip route show table 1138027711
> > > > > default via 1.1.1.1 dev PUBLIC
> > > > > 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
> > > > > 
> > > > > If I manually execute the following command to add the
> > > > > default route as well to the main table I can ping ouside of
> > > > > the local network.
> > > > > 
> > > > > ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
> > > > > 
> > > > > If I attempt to modify the /etc/sysconfig/network-
> > > > > scripts/route-PUBLIC ad reboot the server ad one would think
> > > > > this file is recreated by vdsm on boot.
> > > > > 
> > > > > What I'm looking for is the correct way to setup a default
> > > > > gateway for the main routing table so the hosts can get OS
> > > > > updates and communicate with the outside world.
> > > > 
> > > > Providing the output from "ip addr" may help clear up some
> > > > things.
> > > > It looks like you have on the host the default route set as
> > > > 10.4.16.1 (on enp3s0.106), could you elaborate what this
> > > > interface is?
> > > 
> > > We have setup vlan taging to utilize the 2 internetal network
> > > interfaces (originally enp6s0 and enp7s0) to be configured with
> > > mulitiple networks each.  We eventually added 10Gb nics to all
> > > servers to improve san glusterfs performance which is enp3s0
> > > which replaced enp6s0 in our setup.
> > > 
> > > enp3s0.106 = ovirtmgmt network access to private internal
> > > networks only
> > > enp3s0.206 = private network bridge PRIVATE used for private
> > > internal network access for VMs
> > > enp7s0.606 = is used for public access for both VMs (bridge) and
> > > each host/cp/san in our ovirt setup named PUBLIC
> > > 
> > > # ip addr show
> > > 1: lo:  mtu 65536 qdisc noqueue state
> > > UNKNOWN qlen 1
> > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > > inet 127.0.0.1/8 scope host lo
> > >valid_lft forever preferred_lft forever
> > > inet6 ::1/128 scope host 
> > >valid_lft forever preferred_lft forever
> > > 2: enp6s0:  mtu 1500 qdisc
> > > pfifo_fast state UP qlen 1000
> > > link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> > > inet6 fe80::225:90ff:fe38:d62c/64 scope link 
> > >valid_lft forever preferred_lft forever
> > > 3: enp3s0:  mtu 1500 qdisc mq
> > > state UP qlen 1000
> > > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > > inet6 fe80::92e2:baff:fe1d:a400/64 scope link 
> > >valid_lft forever preferred_lft forever
> > > 4: enp7s0:  mtu 1500 qdisc
> > > pfifo_fast state UP qlen 1000
> > > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > > 20: enp3s0.106@enp3s0:

Re: [ovirt-users] multiple ip routing table issue

2017-11-21 Thread Edward Clay 
On Wed, 2017-11-22 at 00:17 +0200, Edward Haas wrote:
> On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay  m> wrote:
> > On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
> > > On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay  > > p.com> wrote:
> > > > Hello,
> > > > 
> > > > We have an issue where hosts are configured with the public
> > > > facing nework interface as the ovirtmgmt network and it's
> > > > default route is added to a ovirt created table but not to the
> > > > main routing table.  From my searching I've found this snippet
> > > > from https://www.ovirt.org/develop/release-management/features/
> > > > network/multiple-gateways/ which seems to explain why I can't
> > > > ping anything or communicate with any other system needing a
> > > > default route.
> > > 
> > > By default, the default route is set on the ovirtmgmt network
> > > (the default one, defined on the interface/ip which you added the
> > > host to Engine).
> > > Do you have a different network set up which you will like to set
> > > the default route on?
> > > 
> > >  
> > > > "And finally, here's the host's main routing table. Any traffic
> > > > coming in to the host will use the ip rules and an interface's
> > > > routing table. The main routing table is only used for traffic
> > > > originating from the host."
> > > > 
> > > > I'm seeing the following main and custom ovirt created tables.
> > > > 
> > > > main:
> > > > # ip route show table main
> > > > 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106 
> > > > 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src
> > > > 10.4.16.15 
> > > > 1.1.1.0/24 dev PUBLICB proto kernel scope link src
> > > > 1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002 
> > > > 169.254.0.0/16 dev enp3s0 scope link metric 1003 
> > > > 169.254.0.0/16 dev enp7s0 scope link metric 1004 
> > > > 169.254.0.0/16 dev enp3s0.106 scope link metric 1020 
> > > > 169.254.0.0/16 dev PRIVATE scope link metric 1022 
> > > > 169.254.0.0/16 dev PUBLIC scope link metric 1024 
> > > > 
> > > > table 1138027711
> > > > # ip route show table 1138027711
> > > > default via 1.1.1.1 dev PUBLIC
> > > > 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
> > > > 
> > > > If I manually execute the following command to add the default
> > > > route as well to the main table I can ping ouside of the local
> > > > network.
> > > > 
> > > > ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
> > > > 
> > > > If I attempt to modify the /etc/sysconfig/network-
> > > > scripts/route-PUBLIC ad reboot the server ad one would think
> > > > this file is recreated by vdsm on boot.
> > > > 
> > > > What I'm looking for is the correct way to setup a default
> > > > gateway for the main routing table so the hosts can get OS
> > > > updates and communicate with the outside world.
> > > 
> > > Providing the output from "ip addr" may help clear up some
> > > things.
> > > It looks like you have on the host the default route set as
> > > 10.4.16.1 (on enp3s0.106), could you elaborate what this
> > > interface is?
> > 
> > We have setup vlan taging to utilize the 2 internetal network
> > interfaces (originally enp6s0 and enp7s0) to be configured with
> > mulitiple networks each.  We eventually added 10Gb nics to all
> > servers to improve san glusterfs performance which is enp3s0 which
> > replaced enp6s0 in our setup.
> > 
> > enp3s0.106 = ovirtmgmt network access to private internal networks
> > only
> > enp3s0.206 = private network bridge PRIVATE used for private
> > internal network access for VMs
> > enp7s0.606 = is used for public access for both VMs (bridge) and
> > each host/cp/san in our ovirt setup named PUBLIC
> > 
> > # ip addr show
> > 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
> > qlen 1
> > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > inet 127.0.0.1/8 scope host lo
> >valid_lft forever preferred_lft forever
> > inet6 ::1/128 scope host 
> >valid_lft forever preferred_lft forever
> > 2: enp6s0:  mtu 1500 qdisc
> > pfifo_fast state UP qlen 1000
> > link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> > inet6 fe80::225:90ff:fe38:d62c/64 scope link 
> >valid_lft forever preferred_lft forever
> > 3: enp3s0:  mtu 1500 qdisc mq
> > state UP qlen 1000
> > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > inet6 fe80::92e2:baff:fe1d:a400/64 scope link 
> >valid_lft forever preferred_lft forever
> > 4: enp7s0:  mtu 1500 qdisc
> > pfifo_fast state UP qlen 1000
> > link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> > 20: enp3s0.106@enp3s0:  mtu 1500
> > qdisc noqueue state UP qlen 1000
> > link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> > inet 10.4.16.15/24 brd 10.4.16.255 scope global enp3s0.106
> >valid_lft forever preferred_lft forever
> > 21:

Re: [ovirt-users] multiple ip routing table issue

2017-11-21 Thread Edward Haas 
On Tue, Nov 21, 2017 at 6:16 PM, Edward Clay 
wrote:

>
> On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
>
>
>
> On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay 
> wrote:
>
> Hello,
>
> We have an issue where hosts are configured with the public facing nework
> interface as the ovirtmgmt network and it's default route is added to a
> ovirt created table but not to the main routing table. From my searching
> I've found this snippet from https://www.ovirt.org/develop/
> release-management/features/network/multiple-gateways/ which seems to
> explain why I can't ping anything or communicate with any other system
> needing a default route.
>
>
> By default, the default route is set on the ovirtmgmt network (the default
> one, defined on the interface/ip which you added the host to Engine).
> Do you have a different network set up which you will like to set the
> default route on?
>
>
>
> "And finally, here's the host's main routing table. Any traffic coming in
> to the host will use the ip rules and an interface's routing table. The
> main routing table is only used for traffic originating from the host."
>
> I'm seeing the following main and custom ovirt created tables.
>
> main:
> # ip route show table main
> 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106
> 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src 10.4.16.15
> 1.1.1.0/24 dev PUBLICB proto kernel scope link src 1.1.1.1 169.254.0.0/16
> dev enp6s0 scope link metric 1002
> 169.254.0.0/16 dev enp3s0 scope link metric 1003
> 169.254.0.0/16 dev enp7s0 scope link metric 1004
> 169.254.0.0/16 dev enp3s0.106 scope link metric 1020
> 169.254.0.0/16 dev PRIVATE scope link metric 1022
> 169.254.0.0/16 dev PUBLIC scope link metric 1024
>
> table 1138027711
> # ip route show table 1138027711
> default via 1.1.1.1 dev PUBLIC
> 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
>
> If I manually execute the following command to add the default route as
> well to the main table I can ping ouside of the local network.
>
> ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
>
> If I attempt to modify the /etc/sysconfig/network-scripts/route-PUBLIC ad
> reboot the server ad one would think this file is recreated by vdsm on boot.
>
> What I'm looking for is the correct way to setup a default gateway for the
> main routing table so the hosts can get OS updates and communicate with the
> outside world.
>
>
> Providing the output from "ip addr" may help clear up some things.
> It looks like you have on the host the default route set as 10.4.16.1 (on
> enp3s0.106), could you elaborate what this interface is?
>
>
> We have setup vlan taging to utilize the 2 internetal network interfaces
> (originally enp6s0 and enp7s0) to be configured with mulitiple networks
> each. We eventually added 10Gb nics to all servers to improve san glusterfs
> performance which is enp3s0 which replaced enp6s0 in our setup.
>
> enp3s0.106 = ovirtmgmt network access to private internal networks only
> enp3s0.206 = private network bridge PRIVATE used for private internal
> network access for VMs
> enp7s0.606 = is used for public access for both VMs (bridge) and each
> host/cp/san in our ovirt setup named PUBLIC
>
> # ip addr show
> 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN qlen 1
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
>valid_lft forever preferred_lft forever
> inet6 ::1/128 scope host
>valid_lft forever preferred_lft forever
> 2: enp6s0:  mtu 1500 qdisc pfifo_fast
> state UP qlen 1000
> link/ether 00:25:90:38:d6:2c brd ff:ff:ff:ff:ff:ff
> inet6 fe80::225:90ff:fe38:d62c/64 scope link
>valid_lft forever preferred_lft forever
> 3: enp3s0:  mtu 1500 qdisc mq state UP
> qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> inet6 fe80::92e2:baff:fe1d:a400/64 scope link
>valid_lft forever preferred_lft forever
> 4: enp7s0:  mtu 1500 qdisc pfifo_fast
> state UP qlen 1000
> link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> 20: enp3s0.106@enp3s0:  mtu 1500 qdisc
> noqueue state UP qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> inet 10.4.16.15/24 brd 10.4.16.255 scope global enp3s0.106
>valid_lft forever preferred_lft forever
> 21: enp3s0.206@enp3s0:  mtu 1500 qdisc
> noqueue master PRIVATEB state UP qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> 22: PRIVATE:  mtu 1500 qdisc noqueue
> state UP qlen 1000
> link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ff
> 23: enp7s0.606@enp7s0:  mtu 1500 qdisc
> noqueue master PUBLICB state UP qlen 1000
> link/ether 00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff
> 24: PUBLIC:

Re: [ovirt-users] multiple ip routing table issue

2017-11-21 Thread Edward Clay 

On Tue, 2017-11-21 at 09:00 +0200, Edward Haas wrote:
> On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay  m> wrote:
> > Hello,
> > 
> > We have an issue where hosts are configured with the public facing
> > nework interface as the ovirtmgmt network and it's default route is
> > added to a ovirt created table but not to the main routing
> > table.  From my searching I've found this snippet from https://www.
> > ovirt.org/develop/release-management/features/network/multiple-
> > gateways/ which seems to explain why I can't ping anything or
> > communicate with any other system needing a default route.
> 
> By default, the default route is set on the ovirtmgmt network (the
> default one, defined on the interface/ip which you added the host to
> Engine).
> Do you have a different network set up which you will like to set the
> default route on?
> 
>  
> > "And finally, here's the host's main routing table. Any traffic
> > coming in to the host will use the ip rules and an interface's
> > routing table. The main routing table is only used for traffic
> > originating from the host."
> > 
> > I'm seeing the following main and custom ovirt created tables.
> > 
> > main:
> > # ip route show table main
> > 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106 
> > 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src 10.4.16.15 
> > 1.1.1.0/24 dev PUBLICB proto kernel scope link src
> > 1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002 
> > 169.254.0.0/16 dev enp3s0 scope link metric 1003 
> > 169.254.0.0/16 dev enp7s0 scope link metric 1004 
> > 169.254.0.0/16 dev enp3s0.106 scope link metric 1020 
> > 169.254.0.0/16 dev PRIVATE scope link metric 1022 
> > 169.254.0.0/16 dev PUBLIC scope link metric 1024 
> > 
> > table 1138027711
> > # ip route show table 1138027711
> > default via 1.1.1.1 dev PUBLIC
> > 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
> > 
> > If I manually execute the following command to add the default
> > route as well to the main table I can ping ouside of the local
> > network.
> > 
> > ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
> > 
> > If I attempt to modify the /etc/sysconfig/network-scripts/route-
> > PUBLIC ad reboot the server ad one would think this file is
> > recreated by vdsm on boot.
> > 
> > What I'm looking for is the correct way to setup a default gateway
> > for the main routing table so the hosts can get OS updates and
> > communicate with the outside world.
> 
> Providing the output from "ip addr" may help clear up some things.
> It looks like you have on the host the default route set as 10.4.16.1
> (on enp3s0.106), could you elaborate what this interface is?

We have setup vlan taging to utilize the 2 internetal network
interfaces (originally enp6s0 and enp7s0) to be configured with
mulitiple networks each.  We eventually added 10Gb nics to all servers
to improve san glusterfs performance which is enp3s0 which replaced
enp6s0 in our setup.
enp3s0.106 = ovirtmgmt network access to private internal networks
onlyenp3s0.206 = private network bridge PRIVATE used for private
internal network access for VMsenp7s0.606 = is used for public access
for both VMs (bridge) and each host/cp/san in our ovirt setup named
PUBLIC
# ip addr show1: lo:  mtu 65536 qdisc noqueue
state UNKNOWN qlen 1link/loopback 00:00:00:00:00:00 brd
00:00:00:00:00:00inet 127.0.0.1/8 scope host lo   valid_lft
forever preferred_lft foreverinet6 ::1/128 scope
hostvalid_lft forever preferred_lft forever2: enp6s0:
 mtu 1500 qdisc pfifo_fast state UP
qlen 1000link/ether 00:25:90:38:d6:2c brd
ff:ff:ff:ff:ff:ffinet6 fe80::225:90ff:fe38:d62c/64 scope
linkvalid_lft forever preferred_lft forever3: enp3s0:
 mtu 1500 qdisc mq state UP qlen
1000link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ffinet6
fe80::92e2:baff:fe1d:a400/64 scope linkvalid_lft forever
preferred_lft forever4: enp7s0:  mtu
1500 qdisc pfifo_fast state UP qlen 1000link/ether
00:25:90:38:d6:2d brd ff:ff:ff:ff:ff:ff20: enp3s0.106@enp3s0:
 mtu 1500 qdisc noqueue state UP qlen
1000link/ether 90:e2:ba:1d:a4:00 brd ff:ff:ff:ff:ff:ffinet
10.4.16.15/24 brd 10.4.16.255 scope global enp3s0.106   valid_lft
forever preferred_lft forever21: enp3s0.206@enp3s0:
 mtu 1500 qdisc noqueue master
PRIVATEB state UP qlen 1000link/ether 90:e2:ba:1d:a4:00 brd
ff:ff:ff:ff:ff:ff22: PRIVATE:  mtu
1500 qdisc noqueue state UP qlen 1000link/ether 90:e2:ba:1d:a4:00
brd ff:ff:ff:ff:ff:ff23: enp7s0.606@enp7s0:
 mtu 1500 qdisc noqueue master PUBLICB
state UP qlen 1000link/ether 00:25:90:38:d6:2d brd
ff:ff:ff:ff:ff:ff24: PUBLIC:  mtu 1500
qdisc noqueue state UP qlen 1000link/ether 00:25:90:38:d6:2d brd

Re: [ovirt-users] multiple ip routing table issue

2017-11-20 Thread Edward Haas 
On Tue, Nov 21, 2017 at 1:24 AM, Edward Clay 
wrote:

> Hello,
>
> We have an issue where hosts are configured with the public facing nework
> interface as the ovirtmgmt network and it's default route is added to a
> ovirt created table but not to the main routing table. From my searching
> I've found this snippet from https://www.ovirt.org/develop/
> release-management/features/network/multiple-gateways/ which seems to
> explain why I can't ping anything or communicate with any other system
> needing a default route.
>

By default, the default route is set on the ovirtmgmt network (the default
one, defined on the interface/ip which you added the host to Engine).
Do you have a different network set up which you will like to set the
default route on?


>
> "And finally, here's the host's main routing table. Any traffic coming in
> to the host will use the ip rules and an interface's routing table. The
> main routing table is only used for traffic originating from the host."
>
> I'm seeing the following main and custom ovirt created tables.
>
> main:
> # ip route show table main
> 10.0.0.0/8 via 10.4.16.1 dev enp3s0.106
> 10.4.16.0/24 dev enp3s0.106 proto kernel scope link src 10.4.16.15
> 1.1.1.0/24 dev PUBLICB proto kernel scope link src 1.1.1.1 169.254.0.0/16
> dev enp6s0 scope link metric 1002
> 169.254.0.0/16 dev enp3s0 scope link metric 1003
> 169.254.0.0/16 dev enp7s0 scope link metric 1004
> 169.254.0.0/16 dev enp3s0.106 scope link metric 1020
> 169.254.0.0/16 dev PRIVATE scope link metric 1022
> 169.254.0.0/16 dev PUBLIC scope link metric 1024
>
> table 1138027711
> # ip route show table 1138027711
> default via 1.1.1.1 dev PUBLIC
> 1.1.1.0/24 via 1.1.1.1 dev PUBLIC
>
> If I manually execute the following command to add the default route as
> well to the main table I can ping ouside of the local network.
>
> ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC
>
> If I attempt to modify the /etc/sysconfig/network-scripts/route-PUBLIC ad
> reboot the server ad one would think this file is recreated by vdsm on boot.
>
> What I'm looking for is the correct way to setup a default gateway for the
> main routing table so the hosts can get OS updates and communicate with the
> outside world.
>

Providing the output from "ip addr" may help clear up some things.
It looks like you have on the host the default route set as 10.4.16.1 (on
enp3s0.106), could you elaborate what this interface is?

Thanks,
Edy.


> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] multiple ip routing table issue

2017-11-20 Thread Edward Clay 
Hello,

We have an issue where hosts are configured with the public facing
nework interface as the ovirtmgmt network and it's default route is
added to a ovirt created table but not to the main routing table.  From
my searching I've found this snippet from https://www.ovirt.org/develop
/release-management/features/network/multiple-gateways/ which seems to
explain why I can't ping anything or communicate with any other system
needing a default route.

"And finally, here's the host's main routing table. Any traffic coming
in to the host will use the ip rules and an interface's routing table.
The main routing table is only used for traffic originating from the
host."

I'm seeing the following main and custom ovirt created tables.

main:
# ip route show table main
10.0.0.0/8 via 10.4.16.1 dev enp3s0.106 
10.4.16.0/24 dev enp3s0.106 proto kernel scope link src 10.4.16.15 
1.1.1.0/24 dev PUBLICB proto kernel scope link src
1.1.1.1 169.254.0.0/16 dev enp6s0 scope link metric 1002 
169.254.0.0/16 dev enp3s0 scope link metric 1003 
169.254.0.0/16 dev enp7s0 scope link metric 1004 
169.254.0.0/16 dev enp3s0.106 scope link metric 1020 
169.254.0.0/16 dev PRIVATE scope link metric 1022 
169.254.0.0/16 dev PUBLIC scope link metric 1024 

table 1138027711
# ip route show table 1138027711
default via 1.1.1.1 dev PUBLIC
1.1.1.0/24 via 1.1.1.1 dev PUBLIC

If I manually execute the following command to add the default route as
well to the main table I can ping ouside of the local network.

ip route add 0.0.0.0/0 via 1.1.1.1 dev PUBLIC

If I attempt to modify the /etc/sysconfig/network-scripts/route-PUBLIC
ad reboot the server ad one would think this file is recreated by vdsm
on boot.

What I'm looking for is the correct way to setup a default gateway for
the main routing table so the hosts can get OS updates and communicate
with the outside world.___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users