Hi Doug, Just a small note - the commands in the rc.local should be placed *before* the very last line (the touch ... command).
Thanks in advance, On Tue, Feb 12, 2019 at 10:56 PM Lev Veyde <[email protected]> wrote: > Hi Doug, > > Thanks, you're welcome! > > It's generally possible to add persistent rules by either using the legacy > /etc/rc.d/rc.local boot script (chmod +x /etc/rc.d/rc.local and put the > relevant commands at the end of the file, that way you can actually run any > commands) or by using the /etc/sysconfig/network-scripts/rule-<interface > name> e.g. rule-ovirtmgmt or rule-eno3 (just put in the file "to > 172.22.20.31 priority 32764" - without quotes), however of course this > needs to be tested to verify that it works. > > I forwarded the issue to the network team and hopefully they will update > soon. > > Thanks in advance, > > On Tue, Feb 12, 2019 at 10:17 PM Doug Maxfield <[email protected]> > wrote: > >> Lev, >> >> You have no idea how helpful you have been!!!!!! I’ve been having this >> issue for over 2 months and starting to cause major problems. >> >> >> >> Please let me know if you find a better solution. I’m going to wait to >> hear back from you before implementing. >> >> >> >> If we go with this workaround, is there a way to add the ip rule so it’s >> persistent? I know how to make the ip route add persistent. >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 2:14 PM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> Yes, it looks like it did. >> >> I'll try to catch somebody from my local oVirt/RHV networking team to >> take a look at this issue, maybe there is a better solution for that issue >> than my workaround. >> >> >> >> But meanwhile you can use this workaround (ip route add + ip rule add), >> just please be aware that these should be run after each host reboot, since >> these commands are not persistent. >> >> >> >> Thanks in advance, >> >> >> >> On Tue, Feb 12, 2019 at 10:07 PM Doug Maxfield < >> [email protected]> wrote: >> >> Lev, >> >> Here’s the results. It appears to fix the issue!! >> >> >> >> From non-working system: >> >> >> >> -bash-4.2# ip rule add to 172.22.20.31 priority 32764 >> >> >> >> -bash-4.2# traceroute 172.22.20.31 >> >> traceroute to 172.22.20.31 (172.22.20.31), 30 hops max, 60 byte packets >> >> 1 CenteraAmesAN1.emcins.com (172.21.12.61) 0.487 ms 1.082 ms 1.313 ms >> >> 2 192.168.90.9 (192.168.90.9) 1.507 ms 2.090 ms 2.294 ms >> >> 3 pdputopcomm01.emcins.com (172.22.20.31) 1.403 ms 1.401 ms 1.348 ms >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 2:01 PM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> OK, I think that I have a guess of what may have happened here... >> >> Can you please run the following command on the (non-working) host: >> >> ip rule add to 172.22.20.31 priority 32764 >> >> >> >> This needs to be run in addition to the ip route add ... command. >> >> >> >> And then please run the traceroute command and let me know if that fixed >> the issue. >> >> >> >> Thanks in advance, >> >> >> >> On Tue, Feb 12, 2019 at 9:30 PM Doug Maxfield <[email protected]> >> wrote: >> >> Lev, >> >> Here you go. >> >> >> >> From non-working system >> >> >> >> -bash-4.2# ip rule >> >> 0: from all lookup local >> >> 32764: from all to 172.21.0.0/16 iif ovirtmgmt lookup 2887058719 >> >> 32765: from 172.21.0.0/16 lookup 2887058719 >> >> 32766: from all lookup main >> >> 32767: from all lookup default >> >> >> >> >> >> From working system >> >> >> >> [root@paputopcomm04 ~]# ip rule >> >> 0: from all lookup local >> >> 32766: from all lookup main >> >> 32767: from all lookup default >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 12:35 PM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> That shouldn't really matter, since ovirtmgmt is just a bridged >> interface, and unless I'm missing something it shouldn't really affect >> routing. >> >> Can you please also provide the output of "ip rule" from both working and >> non-working hosts ? >> >> >> >> Thanks in advance, >> >> >> >> On Tue, Feb 12, 2019 at 8:01 PM Doug Maxfield <[email protected]> >> wrote: >> >> Lev, >> >> Here’s the ip route from the server that worked and didn’t work >> >> >> >> IP Route from server that worked >> >> >> >> [root@paputopcomm04 ~]# ip route >> >> default via 172.21.0.250 dev eno3 >> >> 169.254.0.0/16 dev eno3 scope link metric 1002 >> >> 169.254.0.0/16 dev eno4 scope link metric 1005 >> >> 172.17.0.0/16 dev eno4 proto kernel scope link src 172.17.53.18 >> >> 172.21.0.0/16 dev eno3 proto kernel scope link src 172.21.5.34 >> >> *172.22.20.31 via 172.21.12.61 dev eno3* >> >> 172.26.0.0/16 dev eno1 proto kernel scope link src 172.26.5.34 >> >> >> >> >> >> >> >> IP Route from server that didn't work >> >> >> >> -bash-4.2# ip route >> >> default via 172.21.0.250 dev ovirtmgmt >> >> 169.254.0.0/16 dev eno4 scope link metric 1004 >> >> 169.254.0.0/16 dev ovirtmgmt scope link metric 1021 >> >> 172.17.0.0/16 dev eno4 proto kernel scope link src 172.17.53.15 >> >> 172.21.0.0/16 dev ovirtmgmt proto kernel scope link src 172.21.5.31 >> >> *172.22.20.31 via 172.21.12.61 dev ovirtmgmt* >> >> 172.26.0.0/16 dev eno1 proto kernel scope link src 172.26.5.31 >> >> >> >> We have had our network team look and they don’t know. As I said, the >> was setup by Commvault. It’s almost like the ovirtmgmt interface will not >> allow a route add due to the way it’s configured? They, Commvault, are >> trying different things within their software to get this working, but it’s >> causing us more problems. I figured I would take a shot and see if >> someone, who works with the oVirt software, would have any ideas. >> >> >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 11:51 AM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> This is indeed quite weird... >> >> It looks like the OS for some reason disregards the static route that you >> have added. >> >> BTW, have you verified that it's indeed was added by running "ip route" >> following the "ip route add ..." command ? >> >> Maybe somebody from the network team has an idea of what may be the cause >> of this issue... >> >> >> >> Thanks in advance, >> >> >> >> On Tue, Feb 12, 2019 at 7:36 PM Doug Maxfield <[email protected]> >> wrote: >> >> Lev, >> >> I’m showing you the same outputs (ip a, ip route, traceroute, ip route >> add, and traceroute) from another server in this group that doesn’t use the >> ovirtmgmt for it’s default interface. >> >> >> >> [root@paputopcomm04 ~]# ip a >> >> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 >> >> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 >> >> inet 127.0.0.1/8 scope host lo >> >> valid_lft forever preferred_lft forever >> >> inet6 ::1/128 scope host >> >> valid_lft forever preferred_lft forever >> >> 2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP >> qlen 1000 >> >> link/ether 90:1b:0e:db:bc:c0 brd ff:ff:ff:ff:ff:ff >> >> inet 172.21.5.34/16 brd 172.21.255.255 scope global eno3 >> >> valid_lft forever preferred_lft forever >> >> inet6 fe80::921b:eff:fedb:bcc0/64 scope link >> >> valid_lft forever preferred_lft forever >> >> 3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP >> qlen 1000 >> >> link/ether 90:1b:0e:e8:da:03 brd ff:ff:ff:ff:ff:ff >> >> inet 172.26.5.34/16 brd 172.26.255.255 scope global eno1 >> >> valid_lft forever preferred_lft forever >> >> inet6 fe80::921b:eff:fee8:da03/64 scope link >> >> valid_lft forever preferred_lft forever >> >> 4: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN >> qlen 1000 >> >> link/ether 90:1b:0e:e8:da:04 brd ff:ff:ff:ff:ff:ff >> >> 5: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP >> qlen 1000 >> >> link/ether 90:1b:0e:db:bc:c1 brd ff:ff:ff:ff:ff:ff >> >> inet 172.17.53.18/16 brd 172.17.255.255 scope global eno4 >> >> valid_lft forever preferred_lft forever >> >> inet6 fe80::921b:eff:fedb:bcc1/64 scope link >> >> valid_lft forever preferred_lft forever >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# ip route >> >> default via 172.21.0.250 dev eno3 >> >> 169.254.0.0/16 dev eno3 scope link metric 1002 >> >> 169.254.0.0/16 dev eno4 scope link metric 1005 >> >> 172.17.0.0/16 dev eno4 proto kernel scope link src 172.17.53.18 >> >> 172.21.0.0/16 dev eno3 proto kernel scope link src 172.21.5.34 >> >> 172.26.0.0/16 dev eno1 proto kernel scope link src 172.26.5.34 >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# traceroute 172.22.20.31 >> >> traceroute to 172.22.20.31 (172.22.20.31), 30 hops max, 60 byte packets >> >> 1 ames-acc1-agg-1.emcins.com (172.21.0.246) 1.081 ms 1.816 ms 1.832 >> ms >> >> 2 192.168.200.30 (192.168.200.30) 0.639 ms 192.168.200.14 >> (192.168.200.14) 1.063 ms 1.577 ms >> >> 3 192.168.200.101 (192.168.200.101) 3.110 ms ames-sw-wanrtr.emcins.com >> (192.168.200.97) 3.191 ms 3.277 ms >> >> 4 192.168.100.26 (192.168.100.26) 4.146 ms 4.273 ms 4.394 ms >> >> 5 192.168.100.98 (192.168.100.98) 1.893 ms 2.364 ms 2.397 ms >> >> 6 192.168.100.50 (192.168.100.50) 5.707 ms 192.168.100.33 >> (192.168.100.33) 5.134 ms 192.168.100.50 (192.168.100.50) 4.105 ms >> >> 7 pdputopcomm01.emcins.com (172.22.20.31) 1.359 ms 1.374 ms 1.351 ms >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# ip route add 172.22.20.31 via 172.21.12.61 >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# >> >> [root@paputopcomm04 ~]# traceroute 172.22.20.31 >> >> traceroute to 172.22.20.31 (172.22.20.31), 30 hops max, 60 byte packets >> >> 1 CenteraAmesAN1.emcins.com (172.21.12.61) 6.184 ms 6.475 ms 6.686 ms >> >> 2 192.168.90.9 (192.168.90.9) 1.770 ms 1.979 ms 2.189 ms >> >> 3 pdputopcomm01.emcins.com (172.22.20.31) 1.405 ms * 1.354 ms >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 11:07 AM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> In most cases it should not be required, especially as my guess (again it >> would be easier if you could post the output of the "ip a" and "ip route") >> is that the eno1 is part of the ovirtmgmt bridge. >> >> What is the output of the "traceroute 172.22.20.31" that you get on the >> host after you have added the static route? >> >> Thanks in advance, >> >> >> >> >> >> On Tue, Feb 12, 2019 at 6:58 PM Doug Maxfield <[email protected]> >> wrote: >> >> Lev, >> >> One other question, since there are multiple enabled interfaces >> (ovirtmgmt and eno1), do I need to specify an interface for the route add. >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 10:50 AM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> What do you mean by "static route is refused"? >> >> What is the error message(s) you see? >> >> >> >> Can you please try to add the route with the following command: >> >> ip route add 172.22.20.31 via 172.21.12.61 >> >> >> >> If that doesn't work, then please provide more details, i.e. what is the >> exact error message(s) you see, the version of your OS and oVirt, output of >> "ip a" and "ip route" on your host(s), so that it will be easier to help >> you. >> >> >> >> Thanks in advance, >> >> >> >> On Tue, Feb 12, 2019 at 6:43 PM Doug Maxfield <[email protected]> >> wrote: >> >> Lev, >> >> Thanks for your response. >> >> >> >> If I attempt to manually add the route using route add: >> >> >> >> route add -net 172.22.20.31 netmask 255.255.255.255 gw 172.21.12.61 dev >> ovirtmgmt >> >> >> >> This static route is refused and all traffic is routed over the default >> gateway. >> >> >> >> But if I use a server that doesn’t use the ovirtmgmt interface and set >> the same static route, the required data is routed correctly over that >> static route. >> >> >> >> [image: Count On EMC] <http://www.emcins.com/> >> >> *Doug Maxfield *| Senior Operating Systems Analyst >> >> *EMC Insurance Companies *717 Mulberry St | Des Moines, IA 50265 >> Tel: 515.345.4507 | Fax: 866.331.1522 >> [email protected] | www.emcins.com >> >> >> >> >> >> *From:* Lev Veyde <[email protected]> >> *Sent:* Tuesday, February 12, 2019 10:34 AM >> *To:* Doug Maxfield <[email protected]> >> *Cc:* users <[email protected]> >> *Subject:* Re: [ovirt-users] Creating a static route >> >> >> >> Hi Doug, >> >> What is the exact problem you are having while attempting to add a static >> route on the hosts? >> >> >> >> Thanks in advance, >> >> >> >> On Tue, Feb 12, 2019 at 5:23 PM <[email protected]> wrote: >> >> Good Morning, >> New to oVirt. We are using this with a backup solution from Commvault. >> The issue that we are having is that we need to setup a static route for >> specific data between 2 remote sites. The ovirtmgmt is configured with the >> correct IP and gateway for the server. We need to route data over a >> different gateway so that we don't "max out" the default network connection >> between the 2 sites. Example below >> >> Ovirtmgmt IP - 172.21.5.31 >> Gateway IP - 172.21.0.250 >> >> We need any traffic with a destination of 172.22.20.31(Remote site) to >> route over this gateway, 172.21.12.61 >> >> There are multiple servers in this configuration. Servers that are not >> using the ovirtmgmt interface for their default, we are able to route the >> data. The only problem is with attempting to setup a different static >> route on the ovirtmgmt interfaces. >> >> Thanks in advance for any help!!! >> >> Doug >> _______________________________________________ >> Users mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/[email protected]/message/WGXSLHAP3X5OT7DOXX3PMRLIMZBKTHR7/ >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> >> >> NOTICE: This message (including any attachments) is intended for a >> specific individual and may contain information that is either confidential >> or legally protected. If you believe that it has been sent to you in >> error, please reply to the sender that you have received the message in >> error, then delete it. If you are not the intended recipient, you are >> hereby notified that any retention, dissemination, distribution, or copying >> of this communication is strictly prohibited. Thank you. EMC071856 >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> >> >> >> -- >> >> >> >> *Lev Veyde* >> >> Software Engineer, RHCE | RHCVA | MCITP >> >> Red Hat Israel >> >> <https://www.redhat.com> >> >> [email protected] | [email protected] >> >> <https://red.ht/sig> >> >> *TRIED. TESTED. TRUSTED.* <https://redhat.com/trusted> >> > > > -- > > Lev Veyde > > Software Engineer, RHCE | RHCVA | MCITP > > Red Hat Israel > > <https://www.redhat.com> > > [email protected] | [email protected] > <https://red.ht/sig> > TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> > -- Lev Veyde Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> [email protected] | [email protected] <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/JDK5LQRNBOQLAV7UJA37BKPOKFSY22GI/
