Package: rsyslog
Version: 8.2504.0-1
Severity: important
Dear Maintainer,
I would like to report a regression affecting the omfwd module in the
rsyslog package shipped with Debian 13 (Trixie).
Environment
* Distribution: Debian GNU/Linux 13 (Trixie)
* Architecture: amd64
* Kernel: 6.12.94+deb13-amd64
* Package: rsyslog
* Version: 8.2504.0-1
What led up to the situation?
I configured rsyslog as a central log collector. The server receives syslog
messages through TCP and forwards them using the omfwd action to another
syslog server.
The problem appears when multiple clients are forwarding logs through the
same rsyslog instance.
What exactly did you do?
I configured rsyslog with an imtcp input and an omfwd TCP forwarding action.
Minimal configuration:
$DebugFile /var/log/remote/rsyslog.debug
$DebugLevel 2
module(load="imtcp")
input(type="imtcp" port="514" ruleset="test")
template(
name="DynFile"
type="string"
string="/var/log/remote/%HOSTNAME%.log"
)
ruleset(name="test") {
action(
type="omfile"
dynaFile="DynFile"
createDirs="on"
)
action(
type="omfwd"
target="172.31.0.6"
port="514"
protocol="tcp"
)
}
I then forwarded logs from multiple servers (approximately 10 clients).
What was the outcome?
The rsyslog debug output contains repeated messages indicating that the
number of active targets changed, even though the number of active targets
did not change.
Example:
omfwd: [wrkr 0] TCPSendInitTarget established connection to 172.31.0.6:514
omfwd: [wrkr 0] number of active targets changed from 0 to 1
omfwd: [wrkr 0] number of active targets changed from 1 to 1
omfwd: [wrkr 0] number of active targets changed from 1 to 1
.
The messages with:
number of active targets changed from 1 to 1
are incorrect because there was no actual change.
Rsyslog immediately starts rate-limiting its own internal messages:
rsyslogd[internal_messages]: XX messages lost due to rate-limiting (500
allowed within 5 seconds)
The forwarding function continues to work, but rsyslog produces a large
amount of unnecessary internal logging and can discard diagnostic messages
due to its own rate limiter.
What outcome did you expect instead?
The message:
number of active targets changed should only be generated when the active
target count actually changes.
For example:
number of active targets changed from 0 to 1
is expected after establishing a connection.
However:
number of active targets changed from 1 to 1 should not be generated.
Additional information
This appears to be an upstream regression in rsyslog.
The issue has already been fixed upstream by commit:
https://github.com/rsyslog/rsyslog/commit/d6d340aaad6ea414a2fd4cf07d3f98c5c9
334194
Commit message:
omfwd regression fix: avoid false active target change log message
The upstream commit explains:
Commit ffaf6dc added proper variable sync, but dropped the check if active
count had actually changed. As such, the output was always generated, which
could pollute the log heavily.
The fix restores the missing check so the message is only emitted when the
active target count actually changes.
Could this fix please be considered for inclusion in the Debian rsyslog
package, either by backporting commit
d6d340aaad6ea414a2fd4cf07d3f98c5c9334194 or by updating to a version
containing this fix (8.2506.0 or later)?
Thank you for your work maintaining rsyslog in Debian.
-- System Information:
Debian Release: 13.5
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500,
'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 6.12.94+deb13-amd64 (SMP w/8 CPU threads; PREEMPT)
-- no debconf information
