OK, I got everything updated and we are up and running.
However, there appears to be an issue with disk queuing. When we fire up
rsyslog the remotebox is only receiving a short burst of events for a few
seconds and then about 1 or 2 events per second after that. After the few
seconds when the burst is over the disk queue on the local system then starts
creating spool files. They don't appear to be flushing. The local system
receives about 1k events per second. They are all making it to the local UDP
spoof but not the remotebox, even after playing around with the queue settings.
Here is the current conf:
module (load="imklog")
module (load="imuxsock")
module (load="imudp")
module (load="imtcp")
module (load="omudpspoof")
# Global Configuration
global (
parser.escapeControlCharactersCStyle="on"
workDirectory="/var/spool/rsyslog"
localHostname="hostname"
)
# In: 514/TCP (uses the same queue as UDP)
input(type="imtcp"
address="10.1.2.3"
port="514"
ruleset="ruleset_eth0_514")
# In: 514/UDP (uses the same queue as TCP)
input(type="imudp"
address="10.1.2.3"
port="514"
ruleset="ruleset_eth0_514")
# Out: UDP-spoof to the local and also forward to remote rsyslog
ruleset(name="ruleset_eth0_514"
queue.type="LinkedList") {
call action.local.udp515
call action.fwd.remotebox
stop
}
# Fwd to localhost:udp/515
ruleset(name="action.local.udp515") {
action(type="omudpspoof"
name="omudpspoof.udp515"
target="127.0.0.1"
port="515")
}
# Fwd to remotebox w/ compression and local disk queueing
ruleset(name="action.fwd.remotebox") {
action(type="omfwd"
name="omfwd.remotebox"
queue.spoolDirectory="/var/spool/rsyslog_tcp"
# queue.lowwatermark="2000"
# queue.highwatermark="1000000"
queue.workerthreads="2"
# queue.type="Disk"
queue.type="LinkedList"
# queue.filename="omfwd_remotebox"
# queue.size="100000"
queue.maxdiskspace="20G"
queue.saveonshutdown="on"
action.resumeretrycount="-1"
target="172.22.22.22"
port="514"
protocol="tcp"
compression.mode="stream:always")
}
________________________________________
From: [email protected] [[email protected]] on
behalf of Dave Caplinger [[email protected]]
Sent: Monday, October 05, 2015 5:40 AM
To: rsyslog-users
Subject: Re: [rsyslog] Complex forwarding and spoofing question
David's right, it's a separate package. Sorry about that. Here's a list of
all the 64-bit rsyslog packages (from 'yum search rsyslog') from a system I
have access to; please verify you see something similar from yours.
rsyslog-gnutls.x86_64 : TLS protocol support for rsyslog
rsyslog-gssapi.x86_64 : GSSAPI authentication and encryption support for rsyslog
rsyslog-mysql.x86_64 : MySQL support for rsyslog
rsyslog-pgsql.x86_64 : PostgresSQL support for rsyslog
rsyslog-relp.x86_64 : RELP protocol support for rsyslog
rsyslog-snmp.x86_64 : SNMP protocol support for rsyslog
rsyslog.x86_64 : Enhanced system logging and kernel message trapping daemons
rsyslog-elasticsearch.x86_64 : Provides the omelasticsearch module
rsyslog-mmjsonparse.x86_64 : mmjsonparse support
rsyslog-mmutf8fix.x86_64 : mmutf8fix support
rsyslog-udpspoof.x86_64 : Provides the omudpspoof module
Then, of course: 'yum install rsyslog-udpspoof'.
--
Dave Caplinger, Director, Technical Product Management | 402.361.3063 |
Solutionary — An NTT Group Security Company
> On Oct 3, 2015, at 1:02 AM, David Lang <[email protected]> wrote:
>
> It may be a separate package (I don't know how to query this through yum)
>
> in the meantime, try addding the following
>
> $template raw,"%rawmsg%"
>
> and then change the forwarding via localhost from omudpspoof to udp with the
> template raw.
>
> David Lang
>
> On Fri, 2 Oct 2015, Randy Baca wrote:
>
>> Date: Fri, 2 Oct 2015 23:46:27 +0000
>> From: Randy Baca <[email protected]>
>> Reply-To: rsyslog-users <[email protected]>
>> To: rsyslog-users <[email protected]>
>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>> Upgraded to 8.13. Looks like now the omudpspoof.so file is missing. Does
>> it not get installed automatically with the upgrade? I checked in
>> /lib64/rsyslog/ and it is not there. It also didn't like the number I had
>> set for the highwatermark so I removed it, and now this is the only error
>> left.
>>
>> [root@host etc]# /sbin/rsyslogd -N2
>> rsyslogd: version 8.13.0, config validation run (level 2), master config
>> /etc/rsyslog.conf
>> rsyslogd: could not load module '/lib64/rsyslog/omudpspoof.so', dlopen:
>> /lib64/rsyslog/omudpspoof.so: cannot open shared object file: No such file
>> or directory [v8.13.0 try http://www.rsyslog.com/e/2066 ]
>> rsyslogd: module name 'omudpspoof' is unknown [v8.13.0 try
>> http://www.rsyslog.com/e/2209 ]
>> rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 32:
>> errors occured in file '/etc/rsyslog.conf' around line 32 [v8.13.0 try
>> http://www.rsyslog.com/e/2207 ]
>>
>> module (load="imklog")
>> module (load="imuxsock")
>> module (load="imudp")
>> module (load="imtcp")
>> Module (load="omudpspoof")
>> # In: 514/TCP (uses the same queue as UDP)
>> input(type="imtcp"
>> address="10.1.2.3"
>> port="514"
>> ruleset="ruleset_eth0_514")
>> # In: 514/UDP (uses the same queue as TCP)
>> input(type="imudp"
>> address="10.1.2.3"
>> port="514"
>> ruleset="ruleset_eth0_514")
>> # Out: UDP-spoof to the local and also forward to remote rsyslog
>> ruleset(name="ruleset_eth0_514"
>> queue.type="LinkedList") {
>> call action.local.udp515
>> call action.fwd.remotebox
>> stop
>> }
>> # Fwd to localhost:udp/515
>> ruleset(name="action.local.udp515") {
>> action(type="omudpspoof"
>> name="omudpspoof.udp515"
>> target="127.0.0.1"
>> port="515")
>> }
>> # Fwd to remotebox w/ compression and local disk queueing
>> ruleset(name="action.fwd.remotebox") {
>> action(type="omfwd"
>> name="omfwd.remotebox"
>> queue.type="LinkedList"
>> queue.filename="omfwd_remotebox"
>> queue.size="10000"
>> queue.maxdiskspace="2G"
>> queue.saveonshutdown="on"
>> action.resumeretrycount="-1"
>> target="172.22.22.22"
>> port="514"
>> protocol="tcp"
>> compression.mode="stream:always")
>> }
>>
>>
>> ________________________________________
>> From: [email protected] [[email protected]]
>> on behalf of Dave Caplinger [[email protected]]
>> Sent: Friday, October 02, 2015 2:40 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>>
>> I think stream compression was added to rsyslog around version 7.2 or so.
>> If i recall correctly, CentOS 6 includes rsyslog v5.8, so you'll definitely
>> want to upgrade your rsyslog version.
>>
>> See http://www.rsyslog.com/rhelcentos-rpms/ for instructions on how to add
>> the v8-stable repo to your /etc/yum.repos.d/ directory.
>>
>> --
>> Dave Caplinger, Director, Technical Product Management | 402.361.3063 |
>> Solutionary — An NTT Group Security Company
>>
>>> On Oct 2, 2015, at 12:35 PM, Randy Baca <[email protected]> wrote:
>>>
>>> Thanks for the conf validation switch. Looks like we have a compatibility
>>> issue. Most of our existing syslog servers are running on CentOS 6. Will
>>> an upgrade of rsyslog resolve this, or do we need to convert the conf to
>>> legacy?
>>>
>>> module(load="imklog")
>>> module(load="imuxsock")
>>> module(load="imudp")
>>> module(load="imtcp")
>>> # In: 514/TCP (uses the same queue as UDP)
>>> input(type="imtcp"
>>> address="10.1.2.3"
>>> port="514"
>>> ruleset="ruleset_eth0_514")
>>> # In: 514/UDP (uses the same queue as TCP)
>>> input(type="imudp"
>>> address="10.1.2.3"
>>> port="514"
>>> ruleset="ruleset_eth0_514")
>>> # Out: UDP-spoof to the local and also forward to remote rsyslog
>>> ruleset(name="ruleset_eth0_514"
>>> queue.highwatermark 50000
>>> queue.fulldelaymark 20000
>>> queue.lowwatermark 2000
>>> queue.type="LinkedList") {
>>> call action.local.udp515
>>> call action.fwd.remotebox
>>> stop
>>> }
>>> # Fwd to localhost:udp/515
>>> ruleset(name="action.local.udp515") {
>>> action(type="omudpspoof"
>>> name="omudpspoof.local515"
>>> target="127.0.0.1"
>>> port="515")
>>> }
>>> # Fwd to remotebox w/ compression and local disk queueing
>>> ruleset(name="action.fwd.remotebox") {
>>> action(type="omfwd"
>>> name="omfwd.remotebox"
>>> queue.type="LinkedList"
>>> queue.filename="omfwd_remotebox"
>>> queue.size="10000"
>>> queue.maxdiskspace="2G"
>>> queue.saveonshutdown="on"
>>> action.resumeretrycount="-1"
>>> target="172.22.22.22"
>>> port="514"
>>> protocol="tcp"
>>> compression.mode="stream:always")
>>> }
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>> Randy Baca
>>>
>>> ________________________________________
>>> From: [email protected] [[email protected]]
>>> on behalf of David Lang [[email protected]]
>>> Sent: Thursday, October 01, 2015 7:53 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>>>
>>> I'm not seeing anything obviously wrong, but you left out part of the config
>>> (the module loading if nothing else)
>>>
>>> do
>>> rsyslogd -N2
>>> to have rsyslog do a syntax check of the config and see if it's happy.
>>>
>>> David Lang
>>>
>>> On Fri, 2 Oct 2015, Randy Baca wrote:
>>>
>>>> Date: Fri, 2 Oct 2015 00:26:11 +0000
>>>> From: Randy Baca <[email protected]>
>>>> Reply-To: rsyslog-users <[email protected]>
>>>> To: rsyslog-users <[email protected]>
>>>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>>>>
>>>> This is what I am using but nothing in a tcpdump going to local on port
>>>> 515 or remote on port 514.
>>>>
>>>> # In: 514/TCP (uses the same queue as UDP)
>>>> input(type="imtcp"
>>>> address="10.1.2.3"
>>>> port="514"
>>>> ruleset="ruleset_eth0_514")
>>>>
>>>> # In: 514/UDP (uses the same queue as TCP)
>>>> input(type="imudp"
>>>> address="10.1.2.3"
>>>> port="514"
>>>> ruleset="ruleset_eth0_514")
>>>>
>>>> # Queue: UDP-spoof to the local and also forward to remote rsyslog
>>>> ruleset(name="ruleset_eth0_514"
>>>> queue.highwatermark 50000
>>>> queue.fulldelaymark 20000
>>>> queue.lowwatermark 2000
>>>> queue.type="LinkedList") {
>>>> call action.local.udp515
>>>> call action.fwd.remotebox
>>>> stop
>>>> }
>>>>
>>>> # Fwd to localhost:udp/515
>>>> ruleset(name="action.local.udp515") {
>>>> action(type="omudpspoof"
>>>> name="omudpspoof.local515"
>>>> target="127.0.0.1"
>>>> port="515")
>>>> }
>>>>
>>>> # Fwd to remotebox w/ compression and local disk queueing
>>>> ruleset(name="action.fwd.remotebox") {
>>>> action(type="omfwd"
>>>> name="omfwd.remotebox"
>>>> queue.type="LinkedList"
>>>> queue.filename="omfwd_remotebox"
>>>> queue.size="10000"
>>>> queue.maxdiskspace="2G"
>>>> queue.saveonshutdown="on"
>>>> action.resumeretrycount="-1"
>>>> target="172.22.22.22"
>>>> port="514"
>>>> protocol="tcp"
>>>> compression.mode="stream:always")
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________________
>>>> From: [email protected]
>>>> [[email protected]] on behalf of Dave Caplinger
>>>> [[email protected]]
>>>> Sent: Thursday, October 01, 2015 2:09 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>>>>
>>>> Looks like it's "queue.fulldelaymark" (and I presume
>>>> "queue.lightdelaymark" may be related, but there's no description at
>>>> http://www.rsyslog.com/doc/master/rainerscript/queue_parameters.html so
>>>> I'm not sure yet what it does...)
>>>>
>>>> Thanks!
>>>>
>>>> --
>>>> Dave Caplinger, Director, Technical Product Management | 402.361.3063 |
>>>> Solutionary — An NTT Group Security Company
>>>>
>>>>> On Oct 1, 2015, at 4:03 PM, David Lang <[email protected]> wrote:
>>>>>
>>>>> There is a queue parameter that tells rsyslog that if the queue is larger
>>>>> than X, stop accepting inputs that can be delayed (like TCP) so that
>>>>> there is space left for a burst of traffic from inputs that can't be
>>>>> delayed (like UDP)
>>>>>
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>> DON'T LIKE THAT.
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>> DON'T LIKE THAT.
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>>> LIKE THAT.
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
