I just got some time to look at this again. Here are some entries from my systems, though first let me give an overview of the setup in case it helps:
Nodes: orldc-prod-netlb01 orldc-prod-netlb02 They have 2 interfaces each (these are VM's running RHEV/oVirt, btw) On orldc-prod-netlb01: ----- root@orldc-prod-netlb01:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:1a:4a:16:01:af inet addr:10.16.250.84 Bcast:10.16.250.255 Mask:255.255.255.0 inet6 addr: fe80::21a:4aff:fe16:1af/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:823698 errors:0 dropped:0 overruns:0 frame:0 TX packets:34753 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:233120483 (222.3 MiB) TX bytes:4205889 (4.0 MiB) eth1 Link encap:Ethernet HWaddr 00:1a:4a:16:01:b0 inet addr:10.16.251.60 Bcast:10.16.251.255 Mask:255.255.255.0 inet6 addr: fe80::21a:4aff:fe16:1b0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2579616 errors:0 dropped:0 overruns:0 frame:0 TX packets:1289074 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:180811247 (172.4 MiB) TX bytes:90488489 (86.2 MiB) eth1:1 Link encap:Ethernet HWaddr 00:1a:4a:16:01:b0 inet addr:10.16.251.59 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ----- On orldc-prod-netlb02: ----- root@orldc-prod-netlb02:/usr/local/zenloadbalancer/logs# ifconfig eth0 Link encap:Ethernet HWaddr 00:1a:4a:16:01:b1 inet addr:10.16.250.85 Bcast:10.16.250.255 Mask:255.255.255.0 inet6 addr: fe80::21a:4aff:fe16:1b1/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:825157 errors:0 dropped:4 overruns:0 frame:0 TX packets:32922 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:233049562 (222.2 MiB) TX bytes:3004798 (2.8 MiB) eth1 Link encap:Ethernet HWaddr 00:1a:4a:16:01:b2 inet addr:10.16.251.61 Bcast:10.16.251.255 Mask:255.255.255.0 inet6 addr: fe80::21a:4aff:fe16:1b2/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1290289 errors:0 dropped:0 overruns:0 frame:0 TX packets:1289030 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:90571721 (86.3 MiB) TX bytes:90460829 (86.2 MiB) eth1:1 Link encap:Ethernet HWaddr 00:1a:4a:16:01:b2 inet addr:10.16.251.59 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 ----- =================================== Older ucarp.log entries on orldc-prod-netlb01: ----- root@orldc-prod-netlb01:/usr/local/zenloadbalancer/logs# cat ucarp.log.1 Sep 6 12:09:43 orldc-prod-netlb02 ucarp[31586]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b0] Sep 6 12:09:43 orldc-prod-netlb02 ucarp[31586]: [WARNING] Switching to state: BACKUP Sep 6 12:09:43 orldc-prod-netlb02 ucarp[31586]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 12:09:51 orldc-prod-netlb02 ucarp[31586]: [WARNING] Switching to state: MASTER Sep 6 12:09:51 orldc-prod-netlb02 ucarp[31586]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] Sep 6 10:07:46 orldc-prod-netlb02 ucarp[427]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b0] Sep 6 10:07:46 orldc-prod-netlb02 ucarp[427]: [WARNING] Switching to state: BACKUP Sep 6 10:07:46 orldc-prod-netlb02 ucarp[427]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 10:07:53 orldc-prod-netlb02 ucarp[427]: [WARNING] Switching to state: MASTER Sep 6 10:07:53 orldc-prod-netlb02 ucarp[427]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] Sep 6 14:30:33 orldc-prod-netlb02 ucarp[1165]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b0] Sep 6 14:30:33 orldc-prod-netlb02 ucarp[1165]: [WARNING] Switching to state: BACKUP Sep 6 14:30:33 orldc-prod-netlb02 ucarp[1165]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 14:30:40 orldc-prod-netlb02 ucarp[1165]: [WARNING] Switching to state: MASTER Sep 6 14:30:40 orldc-prod-netlb02 ucarp[1165]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] Sep 6 14:34:10 orldc-prod-netlb02 ucarp[1802]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b0] Sep 6 14:34:10 orldc-prod-netlb02 ucarp[1802]: [WARNING] Switching to state: BACKUP Sep 6 14:34:10 orldc-prod-netlb02 ucarp[1802]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 14:34:17 orldc-prod-netlb02 ucarp[1802]: [WARNING] Switching to state: MASTER Sep 6 14:34:17 orldc-prod-netlb02 ucarp[1802]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] ----- Older ucarp.log entries on orldc-prod-netlb02: ----- root@orldc-prod-netlb02:/usr/local/zenloadbalancer/logs# cat ucarp.log.1 Sep 6 12:09:48 orldc-prod-netlb02 ucarp[31158]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b2] Sep 6 12:09:48 orldc-prod-netlb02 ucarp[31158]: [WARNING] Switching to state: BACKUP Sep 6 12:09:48 orldc-prod-netlb02 ucarp[31158]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 12:09:56 orldc-prod-netlb02 ucarp[31158]: [WARNING] Switching to state: MASTER Sep 6 12:09:56 orldc-prod-netlb02 ucarp[31158]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] Sep 6 10:13:38 orldc-prod-netlb02 ucarp[426]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b2] Sep 6 10:13:38 orldc-prod-netlb02 ucarp[426]: [WARNING] Switching to state: BACKUP Sep 6 10:13:38 orldc-prod-netlb02 ucarp[426]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 10:13:46 orldc-prod-netlb02 ucarp[426]: [WARNING] Switching to state: MASTER Sep 6 10:13:46 orldc-prod-netlb02 ucarp[426]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] Sep 6 14:30:43 orldc-prod-netlb02 ucarp[594]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b2] Sep 6 14:30:43 orldc-prod-netlb02 ucarp[594]: [WARNING] Switching to state: BACKUP Sep 6 14:30:43 orldc-prod-netlb02 ucarp[594]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 14:30:50 orldc-prod-netlb02 ucarp[594]: [WARNING] Switching to state: MASTER Sep 6 14:30:50 orldc-prod-netlb02 ucarp[594]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] Sep 6 14:34:20 orldc-prod-netlb02 ucarp[1142]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b2] Sep 6 14:34:20 orldc-prod-netlb02 ucarp[1142]: [WARNING] Switching to state: BACKUP Sep 6 14:34:20 orldc-prod-netlb02 ucarp[1142]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 6 14:34:27 orldc-prod-netlb02 ucarp[1142]: [WARNING] Switching to state: MASTER Sep 6 14:34:27 orldc-prod-netlb02 ucarp[1142]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] ----- ==================================== I decided to try to remove the virtual interface and go through the process all over again with logs. So, here's the logs from the most recently attempt: ----- root@orldc-prod-netlb01:/usr/local/zenloadbalancer/logs# tail -f ucarp.log zenlatency.log zenloadbalancer.log ==> ucarp.log <== Sep 21 12:24:02 orldc-prod-netlb02 ucarp[22673]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b0] Sep 21 12:24:02 orldc-prod-netlb02 ucarp[22673]: [WARNING] Switching to state: BACKUP Sep 21 12:24:02 orldc-prod-netlb02 ucarp[22673]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 21 12:24:09 orldc-prod-netlb02 ucarp[22673]: [WARNING] Switching to state: MASTER Sep 21 12:24:09 orldc-prod-netlb02 ucarp[22673]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] ***** ^^^ ****** ^^^ NOTICE that is says 'orldc-prod-netlb02' here which is the OTHER node. Is that normal? ***** ***** ==> zenlatency.log <== 16/09/21 12-24-02 Running start commands: Running: /sbin/ip addr del 10.16.251.59/24 dev eth1 label eth1:1 Stopping zeninotify with pid 1825 16/09/21 12-24-09 Running start commands: Running: /sbin/ip addr add 10.16.251.59/24 dev eth1 label eth1:1 Running Zen inotify syncronization service /usr/local/zenloadbalancer/app/zeninotify/zeninotify.pl & ==> zenloadbalancer.log <== Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - running '/sbin/ip link set eth1 up' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - sending '/usr/bin/arping -c 2 -A -I eth1 10.16.251.60' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - running '/sbin/ip link set eth1:1 up' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - running '/sbin/ip rule del from 10.16.251.59 table table_eth1' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - running '/sbin/ip rule add from 10.16.251.59 table table_eth1' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - sending '/usr/bin/arping -c 2 -A -I eth1 10.16.251.59' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - running '/sbin/ip route del default' Wed Sep 21 12:24:14 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - running '/sbin/ip route add default via 10.16.250.1 dev eth0 ' Wed Sep 21 12:24:18 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - Cluster configured on mode orldc-prod-netlb01 master and orldc-prod-netlb02 backup automatic failover Wed Sep 21 12:24:18 EDT 2016 - orldc-prod-netlb01.passur.local:444 - 192.168.202.171 - admin - Reload here <a href="index.cgi?id=3-3"><img src="img/icons/small/arrow_refresh.png"></a> to apply changes ----- ==> ucarp.log <== Sep 21 12:24:07 orldc-prod-netlb02 ucarp[21206]: [INFO] Local advertised ethernet address is [00:1a:4a:16:01:b2] Sep 21 12:24:07 orldc-prod-netlb02 ucarp[21206]: [WARNING] Switching to state: BACKUP Sep 21 12:24:07 orldc-prod-netlb02 ucarp[21206]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-stop.pl eth1 10.16.251.59] Sep 21 12:24:15 orldc-prod-netlb02 ucarp[21206]: [WARNING] Switching to state: MASTER Sep 21 12:24:15 orldc-prod-netlb02 ucarp[21206]: [WARNING] Spawning [/usr/local/zenloadbalancer/app/zenlatency/zenlatency-start.pl eth1 10.16.251.59] ==> zenlatency.log <== 16/09/21 12-24-07 Running start commands: RTNETLINK answers: Cannot assign requested address Running: /sbin/ip addr del 10.16.251.59/24 dev eth1 label eth1:1 Stopping zeninotify with pid 1173 16/09/21 12-24-15 Running start commands: Running: /sbin/ip addr add 10.16.251.59/24 dev eth1 label eth1:1 Running Zen inotify syncronization service /usr/local/zenloadbalancer/app/zeninotify/zeninotify.pl & ==> zenloadbalancer.log <== Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip route add 10.16.250.0/24 dev eth0 src 10.16.250.85 table table_eth0 ' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip rule add from 10.16.250.85 table table_eth0' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip link set eth0 up' Wed Sep 21 12:24:20 EDT 2016 - - - - sending '/usr/bin/arping -c 2 -A -I eth0 10.16.250.85' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip link set eth1:1 up' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip rule del from 10.16.251.59 table table_eth1' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip rule add from 10.16.251.59 table table_eth1' Wed Sep 21 12:24:20 EDT 2016 - - - - sending '/usr/bin/arping -c 2 -A -I eth1 10.16.251.59' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip route del default' Wed Sep 21 12:24:20 EDT 2016 - - - - running '/sbin/ip route add default via 10.16.250.1 dev eth0 ' ----- I hope this helps lead me down the right path. I'm going to start by migrating these to another hypervisor just to see if something may be related to that in some way. Thanks for the help! -- Chris On Wed, Sep 14, 2016 at 3:55 AM, Emilio Campos < emilio.campos.mar...@gmail.com> wrote: > Dear Christopher, if the issue is presented in your cluster at the moment > of the first configuration, then the problem must be related with VRRP > packets, if VRRP packets are not received then the cluster node can't take > the decession if the node is master or backup, so both cluster nodes will > be master. > > If the issue is present once the cluster is configured then you should be > experiencing some issues with NTP sync. > > Any case if you could send us ucarp log file in the directory > /usr/local/zenloadbalancer/logs/ and syslog in /var/log/syslog it could > help us to understand what you are experiencing. > > Thanks in advanced. > > 2016-09-12 19:34 GMT+02:00 Christopher Young <mexigaba...@gmail.com>: > >> I feel good about the switches since I have other ZenLB instances running >> fine. The appropriate MAC spoofing options are enabled for these VMs, so I >> believe that would be ok, but is there some method to verify that it is >> working (I'd like to have that as a check regardless)? >> >> I will get the logs from both systems and attach in another message, but >> so far I've tried to go through the standard process of creating the >> cluster several times with the same result (everything passes, but the >> cluster doesn't appear to work... I'll elaborate shortly). Thanks for the >> response. I'm hoping to solve this as soon as I can. >> >> On Wed, Sep 7, 2016 at 4:04 AM, Emilio Campos < >> emilio.campos.mar...@gmail.com> wrote: >> >>> Dear Christopher what kind of issue are you experiencing? pleaes could >>> you paste us some informatio? >>> >>> Related to the cluster configuration all the information is sent to log >>> /usr/local/zenloadbalancer/logs/zenloadbalancer.log and >>> /usr/local/zenloadbalancer/logs/ucarp.log >>> >>> Verify mac spoofing is enabled and VRRP protocol is not locked in >>> switches. >>> >>> Anyway if you paste us the error and logs we could give you more >>> information. >>> >>> Thanks! >>> >>> >>> 2016-09-06 20:51 GMT+02:00 Christopher Young <mexigaba...@gmail.com>: >>> >>>> I appear to have this issue and yet my content3-3.cgi seems to be >>>> correct. I have other clusters (both commercial EE and CE) working and >>>> macspoofing enabled on the systems (RHEV/oVirt), so this seems like an odd >>>> one for me: >>>> >>>> root@orldc-prod-netlb02:/usr/local/zenloadbalancer/www# sha1sum >>>> content3-3.cgi >>>> f859509c00db125fd35bf24fcc9d6a5ca8a90e68 content3-3.cgi >>>> root@orldc-prod-netlb02:/usr/local/zenloadbalancer/www# sha256sum >>>> content3-3.cgi >>>> 8c8c652637c12383b12357c1c67913619d9efb41b2572a47e07078f4861d80f0 >>>> content3-3.cgi >>>> >>>> I downloaded a fairly recent ISO, so it looks like the patched version >>>> was included. If there anything specific that I could look in the logs and >>>> config files that might help me troubleshoot the issue? >>>> >>>> Thanks, >>>> >>>> Chris >>>> >>>> On Tue, Apr 19, 2016 at 10:35 AM, Shawn Hawkins < >>>> shawk...@texasbankandtrust.com> wrote: >>>> >>>>> That did the trick. Thank you. >>>>> >>>>> *Shawn Hawkins | **Administrative Officer | * >>>>> *Network Administrator* >>>>> *Texas Bank and Trust | P.O. Box 3188 | Longview, TX 75606p. (903) >>>>> 237-5674 <%28903%29%20237-5674> | f. (903) 237-1875 <%28903%29%20237-1875> >>>>> | shawk...@texasbankandtrust.com <shawk...@texasbankandtrust.com>* >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> The above message is confidential and intended only for the use of the >>>>> person to whom it is addressed and may contain information that is >>>>> privileged and/or exempt from disclosure under applicable law. If the >>>>> reader of this message is not the intended recipient or the employee or >>>>> agent responsible for delivering the message to intended recipient, you >>>>> are >>>>> hereby notified that any dissemination, distribution, copying or other use >>>>> of the information contained herein is strictly prohibited. If you have >>>>> received this transmission in error, please notify the sender immediately >>>>> by telephone and destroy the material in its entirety, whether in >>>>> electronic or hard copy format. >>>>> >>>>> Thank You, >>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------ >>>>> Find and fix application performance issues faster with Applications >>>>> Manager >>>>> Applications Manager provides deep performance insights into multiple >>>>> tiers of >>>>> your business applications. It resolves application problems quickly >>>>> and >>>>> reduces your MTTR. Get your free trial! >>>>> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z >>>>> _______________________________________________ >>>>> Zenloadbalancer-support mailing list >>>>> Zenloadbalancer-support@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support >>>>> >>>>> >>>> >>>> ------------------------------------------------------------ >>>> ------------------ >>>> >>>> _______________________________________________ >>>> Zenloadbalancer-support mailing list >>>> Zenloadbalancer-support@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support >>>> >>>> >>> >>> >>> -- >>> Load balancer distribution - Open Source Project >>> http://www.zenloadbalancer.com >>> Distribution list (subscribe): zenloadbalancer-support@lists. >>> sourceforge.net >>> >>> ------------------------------------------------------------ >>> ------------------ >>> >>> _______________________________________________ >>> Zenloadbalancer-support mailing list >>> Zenloadbalancer-support@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support >>> >>> >> >> ------------------------------------------------------------ >> ------------------ >> What NetFlow Analyzer can do for you? Monitors network bandwidth and >> traffic >> patterns at an interface-level. Reveals which users, apps, and protocols >> are >> consuming the most bandwidth. Provides multi-vendor support for NetFlow, >> J-Flow, sFlow and other flows. Make informed decisions using capacity >> planning reports. http://sdm.link/zohodev2dev >> _______________________________________________ >> Zenloadbalancer-support mailing list >> Zenloadbalancer-support@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support >> >> > > > -- > Load balancer distribution - Open Source Project > http://www.zenloadbalancer.com > Distribution list (subscribe): zenloadbalancer-support@lists. > sourceforge.net > > ------------------------------------------------------------ > ------------------ > > _______________________________________________ > Zenloadbalancer-support mailing list > Zenloadbalancer-support@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support > >
------------------------------------------------------------------------------
_______________________________________________ Zenloadbalancer-support mailing list Zenloadbalancer-support@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support