Hey Folks,
We've begun using the kafka modules from rsyslog (imkafka and omkafka) to help
distribute some event loads. I noticed that my writer ("RSYSLOG01" +
"RSYSLOG02") will fully track the pstats available for omkafka (and
imtcp/imudp). However, when I attempt to get statistics on the final processor
(imkafka), those statistics are missing, despite loading the impstats module at
the beginning of rsyslog. To be clear, the file (/var/log/impstats, written in
file mode 0644) is created, and I can get stats from every other module used
(omfwd, imjournal, etc), it's just specifically imkafka that doesn't provide
stats.
raw.githubusercontent.com/rsyslog/rsyslog/master/plugins/omkafka/omkafka.c<https://raw.githubusercontent.com/rsyslog/rsyslog/master/plugins/omkafka/omkafka.c>
raw.githubusercontent.com/rsyslog/rsyslog/master/plugins/imkafka/imkafka.c<https://raw.githubusercontent.com/rsyslog/rsyslog/master/plugins/imkafka/imkafka.c>
I believe lines 36-38, 49, 62, 64-82 in omkafka would be crucial to implement
equivalent - or roughly equivalent - stats in imkafka. This is in addition to
instantiating the statsobj, calls to STATSCOUNTER_INC(*), etc. However, it's
been many years since I've looked at C code... or mutexs. I'm not the right
person for this. In short, the imkafka module doesn't appear to import any of
the necessary stats headers or variables necessary that omkafka does bring in.
Is it possible to open an improvement request to generate the statistics for
imkafka in the same manner that's currently done for omkafka?
In addition, the use of Kafka necessitates the use of port 9092 and/or 9093 (we
use SSL, so 9093). Can a *supplementary & optional* dnf-compatible module be
written as an installable feature for those of us who use SE Linux? See below
for the SE Linux context re-writes I used on my local servers. I say
supplementary (eg: optional) module because others will use RHEL and they may
have custom configs that we don't want to overwrite by forcing a dependency for
example. The SE Linux configuration to remap 9092/9093 (which exist as
"tcontext=system_u:object_r:unreserved_port_t:s0") to syslogd_port_t is merely
(shell commands - tested on RHEL 9):
semanage port -a -t syslogd_port_t -p tcp 9092
semanage port -a -t syslogd_port_t -p tcp 9093
Without the context re-write and assuming SE Linux is in enforce/target,
rsyslog will fail on { name_connect } to a target kafka broker. Not everyone
knows how SE Linux works or how to specifically whitelist the options. If that
can't be done, would it instead be something that could be included in the
rsyslog Kafka docs as a foot note?
Cheers,
Mike Redbourne
________________________________________
This e-mail communication (including any or all attachments) is intended only
for the use of the person or entity to which it is addressed and may contain
confidential and/or privileged material. If you are not the intended recipient
of this e-mail, any use, review, retransmission, distribution, dissemination,
copying, printing, or other use of, or taking of any action in reliance upon
this e-mail, is strictly prohibited. If you have received this e-mail in error,
please contact the sender and delete the original and any copy of this e-mail
and any printout thereof, immediately. If you have any questions or concerns,
please contact our Customer Service Desk at 1-877-274-2349. Your co-operation
is appreciated.
Le pr?sent courriel (y compris toute pi?ce jointe) s'adresse uniquement ? son
destinataire, qu'il soit une personne ou un organisme, et pourrait comporter
des renseignements privil?gi?s ou confidentiels. Si vous n'?tes pas le
destinataire du courriel, il est interdit d'utiliser, de revoir, de
retransmettre, de distribuer, de diss?miner, de copier ou d'imprimer ce
courriel, d'agir en vous y fiant ou de vous en servir de toute autre fa?on. Si
vous avez re?u le pr?sent courriel par erreur, pri?re de communiquer avec
l'exp?diteur et d'?liminer l'original du courriel, ainsi que toute copie
?lectronique ou imprim?e de celui-ci, imm?diatement. Si vous avez des questions
ou des pr?occupations, veuillez contacter notre centre de service ? la
client?le au 1-877-274-2349. Nous sommes reconnaissants de votre collaboration.
________________________________________
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
