On Jan 15, 2015, at 3:08 PM, Radu Gheorghe <[email protected]> wrote: > > On Thu, Jan 15, 2015 at 9:28 PM, David Lang <[email protected]> wrote: > >> I'm missing something here. If rsyslog has a queue for the destination, >> and the delivery to the destination is via TCP, how is a pull any better >> than a push? if the destination accepts data at a faster pace than it can >> really handle, why would the pull be any better? If the destination only >> accepts data at the rate it can handle, then the traffic will backup into >> the rsyslog queue > > ... > > Either way, I'm glad to see that there are other points in favor of having > a pull model as well (like firewalls). >
I'm also interested in what problems the pull model is going to solve... we do quite a bit of data collection via a pull model from systems that don't speak syslog directly by adding agents on the same system running Rsyslog (and then feed the data locally into Rsyslog). If the device you pull from is not Rsyslog, the method will vary a lot depending on what the remote device is. For example, many firewall and intrusion detection/prevention systems such as CheckPoint, Cisco/NetRanger, etc. have their own proprietary data collection protocols and mechanisms. (And you can probably imagine lots of other devices that don't speak syslog that still generate log-like data you might want to collect.) I guess it would be handy if Rsyslog could do this work, but it seems like it would add a *lot* of complexity for "pull modules" that are going to be even harder to keep up-to-date than the existing lineup of input- and output- modules. If the use case is strictly to have one Rsyslog instance pull from a remote Rsyslog instance in order to get around firewall outbound connectivity limitations (the remote can't connect to the receiver), that seems like a very specific low-ocurrance situation (but much easier to maintain). What situation would this be useful in? Remote cloud-hosted systems that you want to collect logs from inside your enterprise network but they can't connect in because of your corporate firewall policy? Or is the purpose to force buffer management and DAQ to happen at the remote side? (So you don't accept data at the puller only to have to drop it later when a downstream output queue or main queue fills up?) Somewhat tangental to this discussion but related to rsyslog wish-list items: If I understand correctly, if an Rsyslog queue is in DAQ mode sending to a output module (because the output is temporarily unavailable, or not emptying the output queue quickly enough), then the output will start getting messages out of order as the Rsyslog sends some current messages from the front of the queue as well as some from the on-disk back of the queue. I presume this is an optimization to help get the backlog delivered and try to get out of DAQ mode ASAP. It would be handy (for me at least) if we could optionally turn that off for an output queue in order to deliver the queued messages in-order even if there is an additional disk write penalty to pay (for longer). - Dave _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
