Hi Michael, only a partial part 2 messages. I have tried to teach the rsyslog dev assistant a bit helpfulness for your case, but I am not sure if it went for better or worse ;-)
In case you would like to have try, it is here: https://www.rsyslog.com/tool_rsyslog-dev-assistant Unfortunately, I now need to look at other tasks scheduled for today. I hope to be back with better doc and AI soon. In the meantime just keep asking if s/t is unclear. Thx, Rainer El lun, 15 sept 2025 a las 9:07, Rainer Gerhards (<[email protected]>) escribió: > > Hi Michael, and David, > > @david: thanks for your great and timely responses, as always. I > really appreciate them, even though I haven't been very vocal about it > since the pandemics... > > @michael: thanks for your questions and welcome. We highly appreciate > your interest. > > David already pointed you in the right direction. I will now answer in > two steps: > > 1. immediately for the most pressing things > 2. a bit later after I added some developer docs > > #2 is because I think you asked questions that probably many potential > contributors have, so I would like to have the answer better > accessible. Also make sure they are sufficiently well. I would > appreciate feedback on all material and answers. Note that I am so > deeply involved for many years with the project that I may overlook > the obvious. Just tell me than! > > Immediate needs: > > Prometheus: there has already work done already, and native prometheus > format is supported by impstats. imhttp now also provides an endpoint > for publishing these values. A sidecar container w/ external process > is also coming up for native cloud integration. > > See this PR: https://github.com/rsyslog/rsyslog/pull/5983 > > Note that we do NOT use JSON arrays - I simple was not aware this also > works with Prometheus. > --- > > Workflow: > As David said: fork repo and keep it in sync. This is best practice. > Right now the development is rapid and the base you work on may > otherwise become outdated. > > Create Pull requests, this will also automatically trigger AI review > (usually very useful) and ~40 CI test runs executing each one > executing 1000+ tests. Good at catching regression, sometime there are > flakes (test the fail albeit there is not real issue). I'll usually > baby-sit such flakes. With PR, no harm can be done, I or co-maintainer > will always rdo manual review before final merge (so the > responsibility ultimately is mine if something goes wrong). > --- > > Likelihood of acceptance: > We are highly interested in contributions of all types, be it small or > large, code, doc or otherwise. We are also interested to learn use > cases and recently in missing feature for cloud-native. Good Feature > requests can in a number of cases be addressed by AI workflow for > rapid implementation (does not work for large features but > increasingly good for small and smallest). > > Often small things have big effect, so do not be shy. > --- > > Seprate Plugin: definitely not, you took the right approach. Beside > prometheus, I think your change makes a lot of sense. Jsut be sure to > make it configurable. We insist on backwards compatibility. > --- > > Note: besides the general "rsyslog assitant" AI advisor at > https://rsyslog.ai I also work on a developer-focussed AI, > experimentally available at > > https://chatgpt.com/g/g-68ab335c662c8191853b11fe69877f45-ryslog-dev-assistant?model=gpt-5 > > But be warned: it is in its infancy, the quality is not yet very good. > > As it looks, I dug a bit deeper than I originally intended, so the > part 2 is probably no longer that much ;-) > > Any questions pls let us know. > > I would strongly suggest to open either a PR or Feature issue on > github - this platform is better suited to to development work,e.g. > because of interlinking capabilities. > > Once again: many thanks for your interest, and this for sure will > become an important addition! > > Rainer > > > > > El lun, 15 sept 2025 a las 3:58, Redbourne,Michael via rsyslog > (<[email protected]>) escribió: > > > > Hey Folks, > > > > My advance apologies for the rather beginner questions and a wall of text. > > I'm not accustomed to sharing code with enterprise projects that large > > scale integrations. I'm also not a C-developer (or really a developer at > > all beyond bash scripting). I'm also still testing this to make sure it'll > > integrate correctly with my systems (and hopefully others). Once I'm done > > verifying this works for both rsyslog (stability, performance, etc.) and my > > systems, I'll submit this if it's of interest. > > > > High Level Overview > > We use the impstats module to monitor the performance of rsyslog in our SAM > > system. One of the predominant issues we have on two of our servers is our > > pipelines are periodically updated to incorporate new pipelines. This is > > notable for applications such as Kafka (our use case), where each topic we > > connect to has it's own configuration (SAS Key, topic, broker, etc.) The > > long standing issue between rsyslog and our SAM system is rsyslog's > > implementation of the JSON format. Pre-processed, the implementation of > > rsyslog's JSON isn't technically incorrect, but it lacks compatibility with > > dynamic discovery in our SAM system. This is because rsyslog outputs a new > > line for each statistic, rather than concat'ing the statistics into a > > single JSON object. > > > > The way I explained this to GPT-5 was this: > > > > Mon Sep 15 10:32:45 2025: action-0-builtin:omfile: origin=core.action > > processed=4444 failed=0 suspended=0 suspended.duration=0 resumed=0 > > [...] > > Mon Sep 15 10:32:45 2025: action-3-builtin:omfile: origin=core.action > > processed=1111 failed=0 suspended=0 suspended.duration=0 resumed=0 > > > > should be turned into (these logs aren't identical): > > > > {"timedate": "Mon Sep 15 10:52:54 2025", "stats": [{ "name": > > "action-0-builtin:omfile", "origin": "core.action", "processed": 2641, > > "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 },{ > > "name": "imtcp(514)", "origin": "imtcp", "submitted": 0 },{ "name": > > "w1\/imtcp", "origin": "imtcp", "runs": 0, "read": 0, "emptyread": 0, > > "starvation_protect": 0, "accept": 0 }]} > > > > The difference lies in the stats: [] JSON array. This can be iterated > > through in programs like Zabbix to dynamically discover (create) or delete > > items automatically, rather than having to manually create new items each > > time we update the Kafka pipelines to incorporate new environments. > > > > To do this, we took the original impstats.c plugin code and added a new > > format type called "Prometheus". (Prometheus is a common monitoring > > application used in conjunction with something like Grafana, for those who > > don't use it.) This keeps the legacy formats intact (legacy, cee, json, > > json-es, etc.) allowing backwards compatibility with current installations, > > while enabling the new format for individuals seeking this functionality. > > The new impstats.c file modifies implements some new functions such as > > generatePrometheusStats() > > > > Again, I still have some testing to do, code to remove (some debug print > > statements), etc. But the modified impstats.c file can be seen here > > (https://github.com/MRedbourne-BPSI/rsyslog/blob/main/plugins/impstats/impstats.c) > > > > Note: I know rsyslog is AI-friendly. The prometheus section was generated > > in part by GPT-5, with a lot of prompting and describing issues that were > > arising (specifically: UAF bugs with elastic strings.) It's been the better > > part of a decade since I've done any C-development, and nothing with > > mutexs, threads, or anything remotely complex. > > > > Which brings me to my final questions: > > > > 1. > > Assuming this is stable and performant, what is the procedure to submit > > this to rsyslog? I'm assuming I just fork the repository and create a pull > > request for impstats.c? > > 2. > > (Again, assume this is stable/performant): What is the likelihood that this > > type of contribution is accepted? It's not a particularly novel approach, > > and I haven't seen much discussion around this in the community. > > 3. > > Does rsyslog have a testing pipeline (unit files, etc.) that might help me > > test functionality? I compiled this in the available stable tarball > > (8.2508.0) via Rocky EL9. > > 4. > > Would this be better contributed as an individual plugin (eg: > > impstats-prometheus) or as a drop-in replacement for the current impstats > > module? > > > > ________________________________________ > > This e-mail communication (including any or all attachments) is intended > > only for the use of the person or entity to which it is addressed and may > > contain confidential and/or privileged material. If you are not the > > intended recipient of this e-mail, any use, review, retransmission, > > distribution, dissemination, copying, printing, or other use of, or taking > > of any action in reliance upon this e-mail, is strictly prohibited. If you > > have received this e-mail in error, please contact the sender and delete > > the original and any copy of this e-mail and any printout thereof, > > immediately. If you have any questions or concerns, please contact our > > Customer Service Desk at 1-877-274-2349. Your co-operation is appreciated. > > > > Le présent courriel (y compris toute pièce jointe) s'adresse uniquement à > > son destinataire, qu'il soit une personne ou un organisme, et pourrait > > comporter des renseignements privilégiés ou confidentiels. Si vous n'êtes > > pas le destinataire du courriel, il est interdit d'utiliser, de revoir, de > > retransmettre, de distribuer, de disséminer, de copier ou d'imprimer ce > > courriel, d'agir en vous y fiant ou de vous en servir de toute autre façon. > > Si vous avez reçu le présent courriel par erreur, prière de communiquer > > avec l'expéditeur et d'éliminer l'original du courriel, ainsi que toute > > copie électronique ou imprimée de celui-ci, immédiatement. Si vous avez des > > questions ou des préoccupations, veuillez contacter notre centre de service > > à la clientèle au 1-877-274-2349. Nous sommes reconnaissants de votre > > collaboration. > > ________________________________________ > > _______________________________________________ > > rsyslog mailing list > > https://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
