On Wed, 22 Jan 2014, Boylan, James wrote:
I agree with many of the points about the mission statement. I would really
like to see Rsyslog continue on the path of becoming one of the defacto unified
data transport solutions. To me, a true unified data transport needs to not
just be able to send the messages to a destination, but control how it arrives
there as well. I need to dig into it more, but things like the log
normalization functionality appears to play a strong role in that part as well.
I would note that I think the idea of trying to add the ability to use another
language as part of the process to build modules or plugins to be a bad idea.
I know this wasn't specifically what Radu was suggesting, but it has been
suggested before. One of the fundamental strengths, in my opinion, of Rsyslog
is the speed it is able to process the materials being passed to it. Like it
or not, no other language is going to be able to parse at the levels that C
can handle. That is just a simple reality.
very true, the one possible exception that I see is the mm modules. Once rsyslog
gains the ability to use many threads for this, a less efficient language could
be used, with rsyslog starting multiple threads to handle the load (cpu
permitting)
That being said, I do agree the documentation needs work. That was one of the
primary reasons I agreed to help Rainer with the Rsyslog-doc project. My plan
is that once the existing documentation is converted over to the new
configuration I can work on updating, clarifying and adding to the
documentation to help resolve this. It was why I had been asking people to
email this distro with a [doc] or [docs] in the subject line detailing issues
or concerns they have with the documentation. It will be extremely helpful in
trying to work on improving the quality of it.
Yes, there is pretty good configuration documentation (although it's frequently
hard to find the right place in it), but very little developer documentation,
especially with the massive changes that have been happening.
David Lang
Do you have a process that you had to fight through to get working? Write up
instructions on how you did it and email it with the [docs] tag to the Distro.
Found an error? Email it to the distro if that documentation set hasn't yet
been completed in the Git repo. If it is done in the Git repo, feel free to
submit a PR.
-- James
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Rainer Gerhards
Sent: Wednesday, January 22, 2014 4:26 AM
To: rsyslog-users
Subject: Re: [rsyslog] rsyslog mission statement - was: logo
just for the records: I've read this posting, like it very much but need some
time to digest before I can craft a useful response.
Rainer
On Wed, Jan 22, 2014 at 7:48 AM, Radu Gheorghe <[email protected]>wrote:
Hi,
I think I need to make my thoughts clear regarding the flexibility
topic, because I saw the same position by Otis and I saw some
reactions to my earlier Email.
Let's imagine logging tools are cars. Rsyslog seems to be one that's
resource-efficient (light) and has a good top speed (fast). Handling
(configuration) is not that easy and all-terrain abilities (adding new
modules) are also difficult. Those two things are the flexibility I
was talking about.
My journal + mongodb example was intentionally chosen as something
that rsyslog *can* do, but it's not that easy. We can look at the causes:
documentation is far from where we want it, names like "imuxsock" or
"mmnormalize" are far from being intuitive. In car terms, this is what
I mean by "handling is difficult": it can take a turn, but requires
some skill.
Writing a new module is possible, or course, but again: documentation
can be more detailed and writing this C code is nowhere near as easy
as, say, Ruby. In car terms, this is what I mean by "all terrain
ability": it can go on some gravel, but I wouldn't cross a river with it.
Can we improve on flexibility? Of course we can and it's a good thing
to do. If a car can't take a turn without flipping over it's useless
and if it can only go on track only a handful of people will buy it.
But we can't take a Ferrari and make it a VW Touareg. We can improve
it in lots of ways, make it handle better (improve docs) or even add
4-wheel drive (improve omprog, add more modules) but ultimately there
are some design decisions that limit how far we [want to] go in that
direction, especially when there are good SUVs out there on one hand
and people need what rsyslog offers on the other.
Making rsyslog (more) flexible by helping users configure and extend
it easier is definitely something that we should do. But making it a
mission
(ie: a top priority) out of this? I think it's not credible, nor is it
realistic. I'd rather rsyslog as a tool that's light, fast and
reliable
(mission!) and that can also do a lot of stuff.
Like a McLaren F1: can still smoke supercars after 25 years, with a
couple of buddies and their backpacks on-board. Its mission was to be
the ultimate driver's car, and not to be a usable/flexible
grand-tourer. Flexibility comes in as P2, and it's important so you can
actually use the thing.
I hope I didn't bore you with my comparison, I tried my best to make
myself clear. On the upside, the designer guy I talked to gave me the first
logo.
I'll post it on a different thread.
Best regards,
Radu
2014/1/21 Rainer Gerhards <[email protected]>
Sorry folks, i had some very time critical things on my agenda... i
overlooked a dependency ;) will rejoin this great discussion tomorrow.
Just so that you know i am very interested. Actually, its kind of a
reality check for me. Flexibility was always high on ny agenda, it
probably
has slipped for performance without me noticing. I'd like to get
most of both. Maybe with the upcoming non-c interface...
Keep the thoughts flowing!
Rainer
Sent from phone, thus brief.
Am 21.01.2014 19:35 schrieb "Dave Caplinger" <
[email protected]
:
On Jan 20, 2014, at 4:24 PM, David Lang <[email protected]> wrote:
On Mon, 20 Jan 2014, Radu Gheorghe wrote:
I'm not saying rsyslog shouldn't do flexibility. I think it's
uberimportant
and it's worth investing in. I'm saying we should go with one
of the directions where rsyslog is pushed to that:
- is already partially accomplished (so it's credible)
- has the potential to go
- last but not least, where we want it to go :)
I thought something that includes the words fast, light and
processing
will
do, given the history of rsyslog and where it seems to go now
with
v8.
I see rsyslog as being the core of a logging system, it gathers
logs
from
(almost) anything, and delivers them to (almost) anything. It
can
modify
and
filter the messages along the way.
This is similar to my own "customer testimonial." The three main
reasons
I
switched to rsyslog are:
1) Much higher performance.
2) It has DAQ, detailed pstats, TLS, RELP, and now log-signing
support
so
it's reliable in the sense that logs that get in are not going to
get
lost
someplace mysteriously. (Even drops outside your control become
manageable/correctable.)
3) Property replacement, JSON, and filtering, allow you to modify
and route logs as you like.
Going with the R-theme (since 'R' initially meant Reliable) that
gives
me:
* Reliable
* Rapid
* Routing
(Back to the logo: borrowing from another of Rainer's interests
that I happen to share, maybe R is for Rocket [with apologies to
Ray
Bradbury].)
Contrast this with logstash, which is extremely flexible: it can
connect
just about any input to just about any output, like pipe/grep/awk/etc.
for
log streams. It's a "log format translator", but not necessarily
a high-performance "log router".
- Dave
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
