I agree with many of the points about the mission statement. I would really like to see Rsyslog continue on the path of becoming one of the defacto unified data transport solutions. To me, a true unified data transport needs to not just be able to send the messages to a destination, but control how it arrives there as well. I need to dig into it more, but things like the log normalization functionality appears to play a strong role in that part as well.
I would note that I think the idea of trying to add the ability to use another language as part of the process to build modules or plugins to be a bad idea. I know this wasn't specifically what Radu was suggesting, but it has been suggested before. One of the fundamental strengths, in my opinion, of Rsyslog is the speed it is able to process the materials being passed to it. Like it or not, no other language is going to be able to parse at the levels that C can handle. That is just a simple reality. That being said, I do agree the documentation needs work. That was one of the primary reasons I agreed to help Rainer with the Rsyslog-doc project. My plan is that once the existing documentation is converted over to the new configuration I can work on updating, clarifying and adding to the documentation to help resolve this. It was why I had been asking people to email this distro with a [doc] or [docs] in the subject line detailing issues or concerns they have with the documentation. It will be extremely helpful in trying to work on improving the quality of it. Do you have a process that you had to fight through to get working? Write up instructions on how you did it and email it with the [docs] tag to the Distro. Found an error? Email it to the distro if that documentation set hasn't yet been completed in the Git repo. If it is done in the Git repo, feel free to submit a PR. -- James -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Rainer Gerhards Sent: Wednesday, January 22, 2014 4:26 AM To: rsyslog-users Subject: Re: [rsyslog] rsyslog mission statement - was: logo just for the records: I've read this posting, like it very much but need some time to digest before I can craft a useful response. Rainer On Wed, Jan 22, 2014 at 7:48 AM, Radu Gheorghe <[email protected]>wrote: > Hi, > > I think I need to make my thoughts clear regarding the flexibility > topic, because I saw the same position by Otis and I saw some > reactions to my earlier Email. > > Let's imagine logging tools are cars. Rsyslog seems to be one that's > resource-efficient (light) and has a good top speed (fast). Handling > (configuration) is not that easy and all-terrain abilities (adding new > modules) are also difficult. Those two things are the flexibility I > was talking about. > > My journal + mongodb example was intentionally chosen as something > that rsyslog *can* do, but it's not that easy. We can look at the causes: > documentation is far from where we want it, names like "imuxsock" or > "mmnormalize" are far from being intuitive. In car terms, this is what > I mean by "handling is difficult": it can take a turn, but requires > some skill. > > Writing a new module is possible, or course, but again: documentation > can be more detailed and writing this C code is nowhere near as easy > as, say, Ruby. In car terms, this is what I mean by "all terrain > ability": it can go on some gravel, but I wouldn't cross a river with it. > > Can we improve on flexibility? Of course we can and it's a good thing > to do. If a car can't take a turn without flipping over it's useless > and if it can only go on track only a handful of people will buy it. > But we can't take a Ferrari and make it a VW Touareg. We can improve > it in lots of ways, make it handle better (improve docs) or even add > 4-wheel drive (improve omprog, add more modules) but ultimately there > are some design decisions that limit how far we [want to] go in that > direction, especially when there are good SUVs out there on one hand > and people need what rsyslog offers on the other. > > Making rsyslog (more) flexible by helping users configure and extend > it easier is definitely something that we should do. But making it a > mission > (ie: a top priority) out of this? I think it's not credible, nor is it > realistic. I'd rather rsyslog as a tool that's light, fast and > reliable > (mission!) and that can also do a lot of stuff. > > Like a McLaren F1: can still smoke supercars after 25 years, with a > couple of buddies and their backpacks on-board. Its mission was to be > the ultimate driver's car, and not to be a usable/flexible > grand-tourer. Flexibility comes in as P2, and it's important so you can > actually use the thing. > > I hope I didn't bore you with my comparison, I tried my best to make > myself clear. On the upside, the designer guy I talked to gave me the first > logo. > I'll post it on a different thread. > > Best regards, > Radu > > > 2014/1/21 Rainer Gerhards <[email protected]> > > > Sorry folks, i had some very time critical things on my agenda... i > > overlooked a dependency ;) will rejoin this great discussion tomorrow. > > Just so that you know i am very interested. Actually, its kind of a > > reality check for me. Flexibility was always high on ny agenda, it > probably > > has slipped for performance without me noticing. I'd like to get > > most of both. Maybe with the upcoming non-c interface... > > > > Keep the thoughts flowing! > > > > Rainer > > > > Sent from phone, thus brief. > > Am 21.01.2014 19:35 schrieb "Dave Caplinger" < > > [email protected] > > >: > > > > > On Jan 20, 2014, at 4:24 PM, David Lang <[email protected]> wrote: > > > > > > > On Mon, 20 Jan 2014, Radu Gheorghe wrote: > > > > > > > >> I'm not saying rsyslog shouldn't do flexibility. I think it's > > > uberimportant > > > >> and it's worth investing in. I'm saying we should go with one > > > >> of the directions where rsyslog is pushed to that: > > > >> - is already partially accomplished (so it's credible) > > > >> - has the potential to go > > > >> - last but not least, where we want it to go :) > > > >> > > > >> I thought something that includes the words fast, light and > processing > > > will > > > >> do, given the history of rsyslog and where it seems to go now > > > >> with > v8. > > > > > > > > I see rsyslog as being the core of a logging system, it gathers > > > > logs > > from > > > > (almost) anything, and delivers them to (almost) anything. It > > > > can > > modify > > > and > > > > filter the messages along the way. > > > > > > This is similar to my own "customer testimonial." The three main > reasons > > I > > > switched to rsyslog are: > > > > > > 1) Much higher performance. > > > 2) It has DAQ, detailed pstats, TLS, RELP, and now log-signing > > > support > so > > > it's reliable in the sense that logs that get in are not going to > > > get > > lost > > > someplace mysteriously. (Even drops outside your control become > > > manageable/correctable.) > > > 3) Property replacement, JSON, and filtering, allow you to modify > > > and route logs as you like. > > > > > > Going with the R-theme (since 'R' initially meant Reliable) that > > > gives > > me: > > > > > > * Reliable > > > * Rapid > > > * Routing > > > > > > (Back to the logo: borrowing from another of Rainer's interests > > > that I happen to share, maybe R is for Rocket [with apologies to > > > Ray > Bradbury].) > > > > > > Contrast this with logstash, which is extremely flexible: it can > connect > > > just about any input to just about any output, like pipe/grep/awk/etc. > > for > > > log streams. It's a "log format translator", but not necessarily > > > a high-performance "log router". > > > > > > - Dave > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > > you DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
