Vielen Dank, Rainer. I'm glad to hear -- es tüt mir gern :-) -- that there's a proper way to isolate the blocking network socket-opening calls in a worker thread. And when I track down whoever it was who wanted to add @@192.168.2.100 in the first place, I'm sure they'll be glad to hear it, too.
Rainer Gerhards wrote: > The logging behavior you observed, where messages were sometimes written > immediately and sometimes delayed, was a direct consequence of a > misconfigured remote forwarding rule in your rsyslog.conf: *.* @@ > 192.168.2.100. > Here's a breakdown of the core issue: > * Unreachable Destination: The @@192.168.2.100 directive attempts to > forward all log messages via TCP to 192.168.2.100. Since this IP address > was unreachable or not accepting connections, rsyslog encountered > connection failures. > * Synchronous Blocking: Without an explicitly configured action queue for > this specific remote destination, rsyslog's default behavior for that > action is synchronous. This means the main rsyslog process, responsible for > handling all log messages, would directly attempt to establish a TCP > connection to 192.168.2.100. Your observation of a 134-second timeout > indicates that during each connection attempt, the main rsyslog process was > blocked, preventing any other log messages from being processed and written > to local files. > * Buffering and Retry Cycle: While the connection attempt was ongoing, > incoming log messages would be buffered internally. Once the 134-second > timeout elapsed, the connection attempt failed. Rsyslog then entered a > retry cycle, waiting for 30 seconds before attempting another connection. > During this 30-second waiting period, the main process was unblocked, > allowing buffered messages to be written and new messages to be processed > normally, until the next blocking connection attempt. > * Action Queues as the Solution: The mechanism designed to prevent such > blocking is rsyslog's action queues. Instead of the main thread directly > handling potentially slow or blocking operations like remote forwarding, > action queues allow rsyslog to enqueue messages for a specific destination. > A dedicated worker thread then handles the actual delivery to that > destination asynchronously. > If you had configured an action queue for the 192.168.2.100 destination, > the main rsyslog process would have simply placed messages into this queue > (a very fast, non-blocking operation). The dedicated worker thread for that > queue would then attempt to send messages to 192.168.2.100. If the > destination was unreachable, only this worker thread would be blocked by > the connection timeout, while the main rsyslog process would continue to > process and write other log messages to local files without interruption. > Messages for the unreachable host would accumulate in the queue until the > connection was re-established or the queue limits were reached. > Commenting out the problematic line resolved your issue by eliminating the > blocking, synchronous attempts to connect to the unreachable remote host. > > > Rainer > > Steve Summit via rsyslog <[email protected]> schrieb am Fr., 30. > Mai 2025, 19:15: > > > I wrote: > > > Sometimes it writes messages to the relevant log files immediately; > > > sometimes it seems to treasure them up in a buffer somewhere and > > > write them some time later. > > > > I've discovered that the source of the problem was the line > > > > *.* @@192.168.2.100 > > > > which someone had added to my rsyslog.conf. > > > > But 192.168.2.100 was unreachable or not accepting connections. > > Evidently, the timeout for those connections amounts to 134 > > seconds, and during each connection attempt, all other logging > > stops. Evidently, rsyslogd waits 30 seconds (during which time > > logging is normal) before attempting the remote connection again. > > > > Commenting out the 92.168.2.100 line made my problem go away.
_______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
