> On Oct 9, 2015, at 8:01 AM, Dave Caplinger <[email protected]> > wrote: > ... It's unlikely that the stats queue will get >
... congested. I'll second David's suggestion to capture the original log line in a variable and add separate metadata (including site ID) as early in the log stream as possible, for example by using JSON. Then at point 'D' immediately before the SEIM, use a template to only send the original logline along, and it's none the wiser. By this method you have not modified the original log, you've only enriched it with additional information. I understand that at this point you are bolting this new log transport into the middle of your existing processes and maintaining backwards compatibility is paramount; we faced a similar situation when we converted over a thousand remote sites from syslog-ng collectors to rsyslog ones. If you haven't already run into this (especially if the central organization does not control the configs of sending log sources), you'll discover that log sources are liars about time. They're frequently significantly out of sync with (or simply have no) authoritative time source. So you do have to preserve the logline the source sent, but you can't necessarily trust it's timestamp. Even if the timestamp is accurate, you'll have buffering delays and possible connectivity outages that can result in disk buffering and further delay. So logs could arrive at your SEIM minutes or hours later than the source generated them. So for what it's worth, I definitely support David's recommendation of adding metatdata (while preserving the original logline)! -- Dave Caplinger, Director, Technical Product Management Solutionary — An NTT Group Security Company _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
