The source ports appear to not be the issue. Not sure if this helps, but here
is a sanitized output of a tcpdump. Notice there are no packets going to any
of the farm servers. I noted that the source port was mostly 514, but could be
anything. Most firewalls send using 514 for the source port, some use random
high ports for the source port.
root@ZLB1<mailto:root@ZLB1>:~# tcpdump -len host (LogSource)
17:07:04.041125 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 204: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 162
17:07:04.041139 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 198: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 156
17:07:04.041151 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 197: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 155
17:07:04.041226 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 197: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 155
17:07:04.041245 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 197: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 155
17:07:04.043603 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 197: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 155
17:07:04.062748 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 188: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 146
17:07:04.062774 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 195: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 153
17:07:04.062782 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 196: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 154
17:07:04.081443 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 187: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 145
17:07:04.093583 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 160: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 118
17:07:04.093804 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 155: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 113
17:07:09.158890 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 197: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 155
17:07:09.158940 00:82:29:49:c4:0f > 00:dd:e3:ff:ff:20, ethertype IPv4 (0x0800),
length 225: (ZLB1) > (LogSource): ICMP (ZLB1) udp port 514 unreachable, length
191
17:07:09.198905 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 193: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 151
17:07:09.198942 00:82:29:49:c4:0f > 00:dd:e3:ff:ff:20, ethertype IPv4 (0x0800),
length 221: (ZLB1) > (LogSource): ICMP (ZLB1) udp port 514 unreachable, length
187
17:07:09.257895 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 160: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 118
17:07:19.598262 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 166: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 124
17:07:19.598294 00:82:29:49:c4:0f > 00:dd:e3:ff:ff:20, ethertype IPv4 (0x0800),
length 194: (ZLB1) > (LogSource): ICMP (ZLB1) udp port 514 unreachable, length
160
17:07:19.604182 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 197: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 155
17:07:19.604208 00:82:29:49:c4:0f > 00:dd:e3:ff:ff:20, ethertype IPv4 (0x0800),
length 225: (ZLB1) > (LogSource): ICMP (ZLB1) udp port 514 unreachable, length
191
17:07:19.624273 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 202: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 160
17:07:19.624307 00:82:29:49:c4:0f > 00:dd:e3:ff:ff:20, ethertype IPv4 (0x0800),
length 230: (ZLB1) > (LogSource): ICMP (ZLB1) udp port 514 unreachable, length
196
17:07:19.660342 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 196: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 154
17:07:19.689706 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 160: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 118
17:07:19.689773 00:dd:e3:ff:ff:20 > 00:82:29:49:c4:0f, ethertype IPv4 (0x0800),
length 155: (LogSource).514 > (ZLB1).514: SYSLOG local4.info, length: 113
RB
________________________________
From: Randy Baca [ra...@rbaca.com]
Sent: Tuesday, October 25, 2016 4:31 PM
To: zenloadbalancer-support@lists.sourceforge.net
Subject: Re: [Zenloadbalancer-support] UDP Port 514 Unreachable
* Syslog file shows no errors.
* iptables mangle and nat are as shown below.
* The unreachable response is not coming from any of the syslog servers in
the farm
* Health check is configured and working and showing "up" for all servers
according to farmguardian logs. I cannot find the word "down" in any of the
logs.
* No problems from the TCP farm, and there would not be because we have
very few TCP connections. Maybe none.
Could it be that the Zen is running out of DNAT ports? We are using DNAT in
order to keep the original source address in the syslog message. If it runs
out of ports it will need to time out before adding a new connection. Is this
configurable? Can I increase the DNAT port range or add another IP for DNAT?
RB
________________________________
From: Laura Garcia [nev...@gmail.com]
Sent: Tuesday, October 25, 2016 8:31 AM
To: zenloadbalancer-support@lists.sourceforge.net
Subject: Re: [Zenloadbalancer-support] UDP Port 514 Unreachable
Rules are the output lines of "iptables -L -t ...".
If there is any limitation issue in the system, it'll be shown in the syslog
file. But it doesn't seem to be the reason.
Are you sure that the unreachable response is from the Zen Load Balancer?
Do you have any health check configured?
Did you notice any problem from the L4 TCP farm?
Regards
Laura Garcia
Zen Load Balancer Team
www.zenloadbalancer.com<http://www.zenloadbalancer.com>
On Mon, Oct 24, 2016 at 8:17 PM, Randy Baca
<ra...@rbaca.com<mailto:ra...@rbaca.com>> wrote:
Not sure what you mean by rules. Where and how do I set rules for logging?
Here is the issue again:
• I have a lot of firewalls sending syslogs to the ZLB on udp port 514.
• The ZLB forwards (DNAT) the logs to a syslog server farm.
• Some of the firewalls are being rejected by the ZLB with a “udp port
514 unreachable” message.
Why would the ZLB reject an inbound syslog message from only some of the
firewalls? Everything else is working fine. There are no special
configurations for the sending firewalls. Everything is simply pointed at the
ZLB and it forwards everything just fine except for a few of the firewalls.
RB
From: Laura Garcia [mailto:nev...@gmail.com<mailto:nev...@gmail.com>]
Sent: Friday, October 21, 2016 3:24 PM
To:
zenloadbalancer-support@lists.sourceforge.net<mailto:zenloadbalancer-support@lists.sourceforge.net>
Subject: Re: [Zenloadbalancer-support] UDP Port 514 Unreachable
And which rules are from the firewall which is not being redirected?
Can you check the rules marks which the configured ones?
The reason for not being redirected is that the rules is not matching correctly.
You can incluye logging rules in order to check why the rules are notatching
for such firewall.
Regards
El 20 oct. 2016 8:20 p. m., "Randy Baca"
<ra...@rbaca.com<mailto:ra...@rbaca.com>> escribió:
Here is sanitized output. It all looks normal.
root@zenlb01<mailto:root@zenlb01>:~# iptables -L -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
MARK udp -- anywhere (ZLB VIP) statistic mode random
probability 1.00000000000 multiport dports syslog /* FARM_SyslogUDP_4_ */
MARK set 0x206
MARK udp -- anywhere (ZLB VIP) statistic mode random
probability 0.72727272706 multiport dports syslog /* FARM_SyslogUDP_3_ */
MARK set 0x205
MARK udp -- anywhere (ZLB VIP) statistic mode random
probability 0.45454545459 multiport dports syslog /* FARM_SyslogUDP_2_ */
MARK set 0x204
MARK udp -- anywhere (ZLB VIP) statistic mode random
probability 0.18181818165 multiport dports syslog /* FARM_SyslogUDP_1_ */
MARK set 0x203
MARK udp -- anywhere (ZLB VIP) statistic mode random
probability 0.03636363614<tel:0.03636363614> multiport dports syslog /*
FARM_SyslogUDP_0_ */ MARK set 0x200
MARK tcp -- anywhere (ZLB VIP) statistic mode random
probability 1.00000000000 multiport dports shell /* FARM_SyslogTCP_3_ */ MARK
set 0x208
MARK tcp -- anywhere (ZLB VIP) statistic mode random
probability 0.75000000000 multiport dports shell /* FARM_SyslogTCP_2_ */ MARK
set 0x207
MARK tcp -- anywhere (ZLB VIP) statistic mode random
probability 0.50000000000 multiport dports shell /* FARM_SyslogTCP_1_ */ MARK
set 0x202
MARK tcp -- anywhere (ZLB VIP) statistic mode random
probability 0.25000000000 multiport dports shell /* FARM_SyslogTCP_0_ */ MARK
set 0x201
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
root@zenlb01<mailto:root@zenlb01>:~#
root@zenlb01<mailto:root@zenlb01>:~#
root@zenlb01<mailto:root@zenlb01>:~# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp -- anywhere anywhere mark match 0x201
/* FARM_SyslogTCP_0_ */ to:(Server0):514
DNAT tcp -- anywhere anywhere mark match 0x202
/* FARM_SyslogTCP_1_ */ to:(Server1):514
DNAT tcp -- anywhere anywhere mark match 0x207
/* FARM_SyslogTCP_2_ */ to:(Server2):514
DNAT tcp -- anywhere anywhere mark match 0x208
/* FARM_SyslogTCP_3_ */ to:(Server3):514
DNAT udp -- anywhere anywhere mark match 0x200
/* FARM_SyslogUDP_0_ */ to:(Server0):514
DNAT udp -- anywhere anywhere mark match 0x203
/* FARM_SyslogUDP_1_ */ to:(Server1):514
DNAT udp -- anywhere anywhere mark match 0x204
/* FARM_SyslogUDP_2_ */ to:(Server2):514
DNAT udp -- anywhere anywhere mark match 0x205
/* FARM_SyslogUDP_3_ */ to:(Server3):514
DNAT udp -- anywhere anywhere mark match 0x206
/* FARM_SyslogUDP_4_ */ to:(Server4):514
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
RB
________________________________
From: Laura Garcia [nev...@gmail.com<mailto:nev...@gmail.com>]
Sent: Thursday, October 20, 2016 10:52 AM
To:
zenloadbalancer-support@lists.sourceforge.net<mailto:zenloadbalancer-support@lists.sourceforge.net>
Subject: Re: [Zenloadbalancer-support] UDP Port 514 Unreachable
Hi Randy, maybe the L4 rules are not generated properly for this certain
firewall client.
Could you check if the rules for this firewall has the same rules than the
others in the following commands?
iptables -L -t mangle
iptables -L -t nat
Regards.
Laura Garcia
Zen Load Balancer Team
www.zenloadbalancer.com<http://www.zenloadbalancer.com>
On Thu, Oct 20, 2016 at 7:29 PM, Randy Baca
<ra...@rbaca.com<mailto:ra...@rbaca.com>> wrote:
Hi,
I am running ZLB to load balance syslog messages coming from my firewalls to a
farm of log parsers. One firewall is sending syslogs but instead of
load-balancing the packets like the other firewalls ( all Cisco ASA) the ZLB
responds to the one firewall with this message:
10:21:25.683555 IP (firewall).514 > (ZLB VIP).514: SYSLOG
local4.info<http://local4.info>, length: 147
10:19:44.045419 IP (ZLB VIP) > (firewall): ICMP 10.251.253.50 udp port 514
unreachable, length 183
Does anyone know why this is happening? All the other firewalls are being
load-balanced properly.
RB
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Zenloadbalancer-support mailing list
Zenloadbalancer-support@lists.sourceforge.net<mailto:Zenloadbalancer-support@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Zenloadbalancer-support mailing list
Zenloadbalancer-support@lists.sourceforge.net<mailto:Zenloadbalancer-support@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Zenloadbalancer-support mailing list
Zenloadbalancer-support@lists.sourceforge.net<mailto:Zenloadbalancer-support@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support
------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Zenloadbalancer-support mailing list
Zenloadbalancer-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/zenloadbalancer-support
