Quite interested on this thread
http://blog.gerhards.net/2016/11/would-creating-simple-linux-log-file.html
IMHO: the "issue" we're having with rsyslog is understanding the code
rather than performance. Why you'll invest time doing a "faster reader"
if you already have a "fast-enough processor"? Has anyone complained
about speed? What would be the benefit from such development? Will it
increase reading speed by 2x? 4x? 10x?
I suggest you taking this approach:
* Read file line by line (consider also it handles multiline)
* Measure speed (lines/sec, MB/sec, chars/sec...
* Read the same file with imfile with similar reading configuration
* Measure speed (lines/sec, MB/sec, chars/sec...
* Compare both times and think what you can do to make it faster :P
I have not found a good tool yet (I've written or seen written a
couple over the years)
What are the issues with imfile? I don't know when rsyslog development
started, but perhaps time for refactoring has come.
I suspect that a stripped down compile of rsyslog (no input modules
other than imfile, especially no imjournal, etc) would end up being
competitive to just about any special-purpose program.
I don't know if I understood properly: is there such a thing as
--disable-imjournal? will this make rsyslog faster?
Is there any place where I can see "everything than can be disabled in
order to make rsyslog even faster"?
IMHO, The biggest problem with using rsyslog to do this is the same
problem we have with using rsyslog to create /dev/log in containers,
the fact that the config is fixed at startup time.
That's somehow why I was asking about imhiredis.
I don't know yet how rsyslog works internally, but I'll love being able
to reload/add/remove configuration/pipelines during execution.
Having a command socket that rsyslog listened to that would let you
add/remove inputs (files or unix sockets), but didn't allow you to
change anything else in the config would let you easily tell rsyslog
to start watching a new container or file as needed, and then stop
watching so that it doesn't prevent the container or directory from
going away when the app/container is removed.
It's all this because rsyslogs polls for new files/dir changes or
something like that?
TBH, I'll love having more expertise to better contribute.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
