Hey Rainer & David, Rainer: (I did see your last message with the message that just came in a few minutes ago) Thank you for the insights Rainer. On the note of documentation, the only "complaint" (it's not really a complaint though) that I really had was the load of googling necessary to find the various devel packages I needed on a RHEL9/Rocky9 build from source. That was a bit aggravating, but I wasn't making life easy on me either. The rest of it was relatively simple. I'll grab a copy of my bash_history and see if we can start by at least documenting high level packages contributors need for RHEL-based builds. (I've seen in the past that you also have docker images... I've never used docker and I'm loathed to learn it.)
The only other consideration I have for rsyslog is implementing the statistics counter for imkafka in the same method as omkafka implements it. (We use both modules. omkafka on rsyslog-dedicated servers, imkafka for syslog processors going upstream to a SIEM.) David: We're using log.file as the output destination primarily. At least, that's where our monitoring programs capture from. As far as I can tell from doLogToFile(), this isn't bound by any $MaxMessageSize as it doesn't follow the normal rsyslog queue, processing rules, etc. It's just I/O from RAM to disk. If we were bound to log.syslog="on", that may be a concern. Which, is probably is a concern for "future-proofing" reasons. The only thoughts I have around handling log.syslog message truncation would be to either: 1/ Set a warning in Docs to increase $MaxMessageSize to some higher value (Eg: 8k to say 32k+) 2/ Make format="json-array" (or whatever it ends up being named) mutually exclusive with log.syslog="on" This is what it looks like in monitoring (currently): ZBX Monitor - Imgur<https://imgur.com/a/zbx-monitor-5Gx4vJJ>. The monitor just periodically queries the specified log file for new entries. ________________________________ From: David Lang <[email protected]> Sent: Monday, September 15, 2025 9:57 PM To: Redbourne,Michael <[email protected]> Cc: David Lang via rsyslog <[email protected]>; David Lang <[email protected]> Subject: Re: [rsyslog] Code Contributions - New Format for impstats.c CAUTION: The Sender is located Outside The Organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Redbourne,Michael wrote: > Renaming it would probably be a good idea for something more fitting... > likely json-array or something. Prometheus is a bit specific. I'll see what > Rainer has to say; see if he has any input on what I'm doing versus what he's > currently working on. I named it Prometheus because that's what our > monitoring systems generally use, and they're all just complex JSON arrays. question, how large will this message be? how quickly do you run into maxmessagesize and how do you handle that? > Thank you for the reminder to update the docs. I'll do that now soi it > describes the added functionality and how to use it. note that the docs have been going through a major series of changes in the last month or so, so if you modify the doc page from the tarball, you may have to redo it once you see the current state in git (Rainer has been splitting pages up, I'm not sure if he's dont impstats yet)) David Lang > > > ________________________________ > From: rsyslog <[email protected]> on behalf of David Lang via > rsyslog <[email protected]> > Sent: Monday, September 15, 2025 12:27 PM > To: David Lang via rsyslog <[email protected]> > Cc: David Lang <[email protected]> > Subject: Re: [rsyslog] Code Contributions - New Format for impstats.c > > CAUTION: The Sender is located Outside The Organization. Do not click links > or open attachments unless you recognize the sender and know the content is > safe. > > > a couple more things > > 1. make sure you update the documentation along with your feature > > 2. work from the git repo rather than a stable release snapshot. > > David Lang > > On Sun, 14 Sep 2025, David Lang via rsyslog wrote: > >> Date: Sun, 14 Sep 2025 19:23:57 -0700 (PDT) >> From: David Lang via rsyslog <[email protected]> >> To: "Redbourne,Michael via rsyslog" <[email protected]> >> Cc: David Lang <[email protected]> >> Subject: Re: [rsyslog] Code Contributions - New Format for impstats.c >> >> Redbourne,Michael wrote: >> >>> 1. Assuming this is stable and performant, what is the procedure to submit >>> this to rsyslog? I'm assuming I just fork the repository and create a pull >>> request for impstats.c? >> >> Yes. fork and PR >> >>> 2. (Again, assume this is stable/performant): What is the likelihood that >>> this type of contribution is accepted? It's not a particularly novel >>> approach, and I haven't seen much discussion around this in the community. >> >> I know that Rainer has been doing some work related to Prometheus, so I would >> suggest that you check what's happening there. There may be some overlap, or >> you may need to name things differently (or it may be just fine, Rainer would >> need to comment) >> >> Rsyslog doesn't handle processing data in arrays well, which may be why this >> hasn't been in place before. >> >>> 3. Does rsyslog have a testing pipeline (unit files, etc.) that might help >>> me test functionality? I compiled this in the available stable tarball >>> (8.2508.0) via Rocky EL9. >> >> Yes, when you submit the PR, it will trigger tests. The tests are in the >> codebase, so you should be able to run them yourself >> >>> 4. Would this be better contributed as an individual plugin (eg: >>> impstats-prometheus) or as a drop-in replacement for the current impstats >>> module? >> >> This is probably best done as a modification of the existing impstats module, >> you don't want to duplicate everything (and the problem would then be >> maintaining it over time) >> >> Just make sure to maintain backwards compatibility. Rsyslog goes to a lot of >> trouble to make sure that people can upgrade without knowing about new >> features and modules and not have their system break. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841383666%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=xscaGVFzg%2FZtpUcP%2FzRtTaOKB6Zu0InTJJ%2BwZ1NdlTU%3D&reserved=0<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841402701%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=EK7FLlAGb%2FI5FVpkBkDhrQzFuEMQOrjSKC4qOCSLx20%3D&reserved=0><https://lists.adiscon.net/mailman/listinfo/rsyslog> >> https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841416465%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=VZzUDCl7fQWu%2FMPBDB27T7gRwdkb9%2BVPSzk6ABM6XmA%3D&reserved=0<https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841429904%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=54tmiNogO7ugGzKRlUmwjsoUfBJGtk%2FH7oF%2FNwRdbQc%3D&reserved=0><http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow >> https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841448666%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Lbjc53TBaozS7NRTqqln6DbBnJLL8qTbjQTGc2CnOzk%3D&reserved=0<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841466852%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=aoDghQtAW6m9fk0tVdo0zUq%2BTUC8%2Fxjzs4eTg6e%2FAmc%3D&reserved=0><https://twitter.com/rgerhards> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841481134%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=8kXmtvI9RWA63gjHAHdjbEJ0z%2F6FWhVKwk9RIiANkGI%3D&reserved=0<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841494863%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=hl9lGyXnJyRuKKAJsq8xSE6CCge9L3Ar6OzPBTPDpEw%3D&reserved=0><https://lists.adiscon.net/mailman/listinfo/rsyslog> > https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841508509%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=OnIQdzEqnw0OYO6vzsFvtHVK6%2FZcqq6ccAlxlgs%2FiKU%3D&reserved=0<https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.rsyslog.com%2Fprofessional-services%2F&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841522968%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=HXITIpLkNwQ2tNtuIUyXFj31rVqzCDIjweDX3fVrCIE%3D&reserved=0><http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow > https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841536413%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=y0XG5z4FOf7qFTYi0JqvCcHzd53j1NGpvVR%2BTI5GdFA%3D&reserved=0<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Frgerhards&data=05%7C02%7Cmichael.redbourne%40bulletproofsi.com%7Cf9ada1d7c9164e33d3a108ddf44f1b47%7C9a63d13853ea411bbe8458b7e2570747%7C1%7C0%7C638935342841549902%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=j1N0pvMHpPtzhTFDx38OR2lsgzuKdMADz6Sicf9uyGU%3D&reserved=0><https://twitter.com/rgerhards> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. > ________________________________________ > This e-mail communication (including any or all attachments) is intended only > for the use of the person or entity to which it is addressed and may contain > confidential and/or privileged material. If you are not the intended > recipient of this e-mail, any use, review, retransmission, distribution, > dissemination, copying, printing, or other use of, or taking of any action in > reliance upon this e-mail, is strictly prohibited. If you have received this > e-mail in error, please contact the sender and delete the original and any > copy of this e-mail and any printout thereof, immediately. If you have any > questions or concerns, please contact our Customer Service Desk at > 1-877-274-2349. Your co-operation is appreciated. > > Le pr?sent courriel (y compris toute pi?ce jointe) s'adresse uniquement ? son > destinataire, qu'il soit une personne ou un organisme, et pourrait comporter > des renseignements privil?gi?s ou confidentiels. Si vous n'?tes pas le > destinataire du courriel, il est interdit d'utiliser, de revoir, de > retransmettre, de distribuer, de diss?miner, de copier ou d'imprimer ce > courriel, d'agir en vous y fiant ou de vous en servir de toute autre fa?on. > Si vous avez re?u le pr?sent courriel par erreur, pri?re de communiquer avec > l'exp?diteur et d'?liminer l'original du courriel, ainsi que toute copie > ?lectronique ou imprim?e de celui-ci, imm?diatement. Si vous avez des > questions ou des pr?occupations, veuillez contacter notre centre de service ? > la client?le au 1-877-274-2349. Nous sommes reconnaissants de votre > collaboration. > ________________________________________ > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
