If you process inbound UDP and TCP syslog exactly the same way, you *could*
just add the TCP input and point it to the same initial ruleset, such as:
input(type="imtcp"
address="192.168.1.1"
port="514"
ruleset="ruleset_eth0_514")
(note I changed the name of that common ruleset from "ruleset_eth0_514_udp"; do
that also on the imudp input and the ruleset itself of course.)
The result would be two inputs feeding a common queue, which then calls two
output actions (with their own associated output queues).
We typically don't do this; we'd have a separate input queue (ruleset) for UDP
and TCP (and we use imptcp rather than imtcp). This helps us better isolate
the impact of traffic bursts of one input type over the other. For example, a
large, persistent volume of inbound TCP could potentially (eventually) overrun
the shared ruleset/queue and impact reception of UDP (which could in turn lead
to the kernel dropping logs it already successfully received). Particularly if
you are forwarding to a remote destination, your output queue is more likely to
drain more slowly than the input rate, especially if you ever enter disk-assist
buffering. (Consider disk-assist mode a "last resort" for short-term bursts or
intermittent connectivity problems only.)
--
Dave Caplinger, Director, Technical Product Management | 402.361.3063 |
Solutionary — An NTT Group Security Company
> On Oct 1, 2015, at 12:13 PM, Randy Baca <[email protected]> wrote:
>
> That looks similar to what we are doing, except we are using legacy commands.
>
> Also, we have two inputs, tcp and udp 514. When we create the new input for
> TCP, do we also have to create a new output for it, or will the outputs you
> note below cover both inputs?
>
> Regards,
>
>
>
> Randy Baca
>
> ________________________________________
> From: [email protected] [[email protected]]
> on behalf of Dave Caplinger [[email protected]]
> Sent: Thursday, October 01, 2015 5:36 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>
> No; you'll need to explicitly turn on compression and disk-assisted queueing.
>
> It's a little more complex this way, but if you're likely to do any kind of
> filtering or selective forwarding, I'd recommend a config that:
>
> 1) Defines your input (in this case UDP 514) and connects it to a named
> ruleset
> 2) That ruleset/queue then forwards to two destinations by calling additional
> rulesets
> 3) Define rulesets for each destination
> 4) Give everything descriptive names so you can implement impstats,
> troubleshoot later, etc.
>
> For example:
>
> # 514/UDP
> input(type="imudp"
> address="192.168.1.1"
> port="514"
> ruleset="ruleset_eth0_514_udp")
>
> # In: 514/UDP
> # Out: UDP-spoof to the other local syslog and also forward to remote rsyslog
> ruleset(name="ruleset_eth0_514_udp"
> queue.type="LinkedList") {
> call action.local.udp515
> call action.fwd.remotebox
> stop
> }
>
> # Fwd to localhost:udp/515
> ruleset(name="action.local.udp515") {
> action(type="omudpspoof"
> name="omudpspoof.local515"
> target="127.0.0.1"
> port="515"
> )
> }
>
> # Fwd to remote (10.11.12.13) via tcp/514 w/ compression and local disk
> queueing
> ruleset(name="action.fwd.remotebox") {
> action(type="omfwd"
> name="omfwd.remotebox"
> queue.type="LinkedList"
> queue.filename="omfwd_remotebox"
> queue.size="10000"
> queue.maxdiskspace="2G"
> queue.saveonshutdown="on"
> action.resumeretrycount="-1"
> target="10.11.12.13"
> port="514"
> protocol="tcp"
> compression.mode="stream:always"
> )
> }
>
>
> Note that the receiving remote rsyslog server will also need compression
> enabled, and you cannot receive both compressed and non-compressed log feeds
> on the same input. So if you need that, use multiple rulesets again: set up
> the compressed feed on one input/port/ruleset, and non-compressed on another.
>
> --
> Dave Caplinger, Director, Technical Product Management | 402.361.3063 |
> Solutionary — An NTT Group Security Company
>
>> On Sep 30, 2015, at 9:58 PM, Randy Baca <[email protected]> wrote:
>>
>> We need the spoofing because rsyslog adds a timestamp and an IP (127.0.0.1)
>> to the beginning of each message forwarded and it messes up the legacy
>> parsing.
>>
>> Does @@remotesystem perform compression and will it spool messages by
>> default in case the link goes down?
>>
>> RB
>>
>> -----Original Message-----
>> From: [email protected]
>> [mailto:[email protected]] On Behalf Of David Lang
>> Sent: Wednesday, September 30, 2015 6:14 PM
>> To: rsyslog-users <[email protected]>
>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>>
>> On Thu, 1 Oct 2015, Randy Baca wrote:
>>
>>> I have a complex issue to resolve and I would like to use rsyslog for it.
>>> Hopefully someone has done this.
>>>
>>>
>>>
>>> Currently we have a number of remote sites and each site has a single
>>> syslog server collecting events from multiple hosts. We need to keep the
>>> flow going to the legacy syslog servers and also forward all events to a
>>> central server at one of the sites. The legacy syslog servers need to stay
>>> running until several other projects get completed; maybe a year.
>>>
>>>
>>>
>>> This is what we want to do. We have pieces of it working but can't get
>>> everything at once.
>>>
>>> * Set up rsyslog on the same server that is running the legacy service.
>>> * Change the legacy service to run on udp/515.
>>> * Configure rsyslog to receive on existing ports udp/tcp 514. We are
>>> essentially injecting it between the sending hosts and the legacy service.
>>> * Configure rsyslog to forward and spoof (omudpspoof) to 127.0.0.1 on
>>> udp/515. We need the exact original message because of all the custom
>>> parsers written over the years.
>>> * Add another action to forward all messages to the central location.
>>> Some sites are not high bandwidth, so we will need compression, TCP for
>>> reliability, and spooling in case the link goes down or one of the
>>> endpoints reboots.
>>> * The server at the central location will itself then forward to multiple
>>> services locally.
>>>
>>> Is this do-able? Can someone get me started with an example config?
>>
>> it's doable, but you shouldn't need to do the spoofing. Spoofing is only
>> needed if the receiving system ignores the text of the log message and
>> instead uses the IP address in the packet.
>>
>> It's as simple as two outputs
>>
>> @localhost:515
>> @remotesystem
>>
>> or @@remotesystem if you want to use TCP forwarding
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This
>> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
>> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
