Hey Folks,
My advance apologies for the rather beginner questions and a wall of text. I'm
not accustomed to sharing code with enterprise projects that large scale
integrations. I'm also not a C-developer (or really a developer at all beyond
bash scripting). I'm also still testing this to make sure it'll integrate
correctly with my systems (and hopefully others). Once I'm done verifying this
works for both rsyslog (stability, performance, etc.) and my systems, I'll
submit this if it's of interest.
High Level Overview
We use the impstats module to monitor the performance of rsyslog in our SAM
system. One of the predominant issues we have on two of our servers is our
pipelines are periodically updated to incorporate new pipelines. This is
notable for applications such as Kafka (our use case), where each topic we
connect to has it's own configuration (SAS Key, topic, broker, etc.) The long
standing issue between rsyslog and our SAM system is rsyslog's implementation
of the JSON format. Pre-processed, the implementation of rsyslog's JSON isn't
technically incorrect, but it lacks compatibility with dynamic discovery in our
SAM system. This is because rsyslog outputs a new line for each statistic,
rather than concat'ing the statistics into a single JSON object.
The way I explained this to GPT-5 was this:
Mon Sep 15 10:32:45 2025: action-0-builtin:omfile: origin=core.action
processed=4444 failed=0 suspended=0 suspended.duration=0 resumed=0
[...]
Mon Sep 15 10:32:45 2025: action-3-builtin:omfile: origin=core.action
processed=1111 failed=0 suspended=0 suspended.duration=0 resumed=0
should be turned into (these logs aren't identical):
{"timedate": "Mon Sep 15 10:52:54 2025", "stats": [{ "name":
"action-0-builtin:omfile", "origin": "core.action", "processed": 2641,
"failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 },{ "name":
"imtcp(514)", "origin": "imtcp", "submitted": 0 },{ "name": "w1\/imtcp",
"origin": "imtcp", "runs": 0, "read": 0, "emptyread": 0, "starvation_protect":
0, "accept": 0 }]}
The difference lies in the stats: [] JSON array. This can be iterated through
in programs like Zabbix to dynamically discover (create) or delete items
automatically, rather than having to manually create new items each time we
update the Kafka pipelines to incorporate new environments.
To do this, we took the original impstats.c plugin code and added a new format
type called "Prometheus". (Prometheus is a common monitoring application used
in conjunction with something like Grafana, for those who don't use it.) This
keeps the legacy formats intact (legacy, cee, json, json-es, etc.) allowing
backwards compatibility with current installations, while enabling the new
format for individuals seeking this functionality. The new impstats.c file
modifies implements some new functions such as generatePrometheusStats()
Again, I still have some testing to do, code to remove (some debug print
statements), etc. But the modified impstats.c file can be seen here
(https://github.com/MRedbourne-BPSI/rsyslog/blob/main/plugins/impstats/impstats.c)
Note: I know rsyslog is AI-friendly. The prometheus section was generated in
part by GPT-5, with a lot of prompting and describing issues that were arising
(specifically: UAF bugs with elastic strings.) It's been the better part of a
decade since I've done any C-development, and nothing with mutexs, threads, or
anything remotely complex.
Which brings me to my final questions:
1.
Assuming this is stable and performant, what is the procedure to submit this to
rsyslog? I'm assuming I just fork the repository and create a pull request for
impstats.c?
2.
(Again, assume this is stable/performant): What is the likelihood that this
type of contribution is accepted? It's not a particularly novel approach, and I
haven't seen much discussion around this in the community.
3.
Does rsyslog have a testing pipeline (unit files, etc.) that might help me test
functionality? I compiled this in the available stable tarball (8.2508.0) via
Rocky EL9.
4.
Would this be better contributed as an individual plugin (eg:
impstats-prometheus) or as a drop-in replacement for the current impstats
module?
________________________________________
This e-mail communication (including any or all attachments) is intended only
for the use of the person or entity to which it is addressed and may contain
confidential and/or privileged material. If you are not the intended recipient
of this e-mail, any use, review, retransmission, distribution, dissemination,
copying, printing, or other use of, or taking of any action in reliance upon
this e-mail, is strictly prohibited. If you have received this e-mail in error,
please contact the sender and delete the original and any copy of this e-mail
and any printout thereof, immediately. If you have any questions or concerns,
please contact our Customer Service Desk at 1-877-274-2349. Your co-operation
is appreciated.
Le présent courriel (y compris toute pièce jointe) s'adresse uniquement à son
destinataire, qu'il soit une personne ou un organisme, et pourrait comporter
des renseignements privilégiés ou confidentiels. Si vous n'êtes pas le
destinataire du courriel, il est interdit d'utiliser, de revoir, de
retransmettre, de distribuer, de disséminer, de copier ou d'imprimer ce
courriel, d'agir en vous y fiant ou de vous en servir de toute autre façon. Si
vous avez reçu le présent courriel par erreur, prière de communiquer avec
l'expéditeur et d'éliminer l'original du courriel, ainsi que toute copie
électronique ou imprimée de celui-ci, immédiatement. Si vous avez des questions
ou des préoccupations, veuillez contacter notre centre de service à la
clientèle au 1-877-274-2349. Nous sommes reconnaissants de votre collaboration.
________________________________________
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
