Side-note: I agree with mostolog on the advantages of componentication for fault isolation. Just another user case...
Rainer Sent from phone, thus brief. Am 23.11.2016 14:47 schrieb "David Lang" <da...@lang.hm>: > On Wed, 23 Nov 2016, mosto...@gmail.com wrote: > > However, if you really want to go this way, one thing you can do is to >>> make use of the multicast mac feature in ethernet to distribute the same >>> logs to multiple systems/containers and have each container throw away all >>> logs except what it's configured to handle. >>> >>> This lets you add/remove log processing at any time and even have >>> multiple systems processing the same logs in different ways >>> >>> https://www.usenix.org/conference/lisa12/technical-sessions/ >>> presentation/lang_david >>> >> Network traffic x2 >> Actually, we are using a similar environment for other things, but I >> don't think that's the way to go. >> > > This doesn't need to double the network traffic in the way you are > thinking. The IP address that the senders deliver to is shared across all > your processing boxes. The switch replicates the traffic on it's backbone > and delivers it to each machine. > > with your current approach you do > > sender -> rsyslog -> redis -> logstash -> ES > > so there are 3-4 copies of the logs (depending on if sender and rsyslog > are the same box) > > if instead you did > > sender -> multicast mac to rsyslog -> ES > > there would only be two copies of the logs on the wire at any point > (although N copies total going into the rsyslog box, but that's only on the > interface to those boxes) > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
