commenting on some things i have painful experience with :-) -----Original Message----- From: Xuri Nagarin <[email protected]> Reply-To: rsyslog-users <[email protected]> Date: Monday, February 3, 2014 at 2:07 PM To: rsyslog-users <[email protected]> Subject: Re: [rsyslog] name games...
>Chiming in late (and ranting) but I hear you when folks try to paint it was a good rant though... :-) >syslog as old and obsolete. It is part of a new trend (atleast in the >Silicon Valley here) where people are busy re-inventing the wheel when >there really is no need to. With Syslog, looks like most people don't >even understand it properly. I often hear issues like "Syslog has a >limitation of 1024 bytes so we switched to this new cool tool". >Obviously, these are the exact same people who probably never bother >reading a RFC in their lives. > >For the record and hopefully Google search picks this up, Syslog does >NOT have a 1024 byte limit. > >http://tools.ietf.org/html/rfc5424#page-9 >"Syslog message size limits are dictated by the syslog transport >mapping in use. There is no upper limit per se. Each transport mapping >defines the minimum maximum required message length support, and the >minimum maximum MUST be at least 480 octets in length." > >For the networking ignorant, even you used UDP, "The field size sets a >theoretical limit of 65,535 bytes (8 byte header +65,527 bytes of >data) for a UDP datagram. The practical limit for the data length >which is imposed by the underlying IPv4 protocol is 65,507 >bytes(65,535 - 8 byte UDP header - 20 byte IP header)." > >Find me a log transport mechanism that can use UDP/TCP/TLS - both >RSyslog and Syslog-NG implement them with nice features like disk >buffering. > >There are folks who complain that Syslog has no structure. NEWS - >Syslog is a transport mechanism, you can fit in structured messages, >unstructured messages or use it to transport pigeon feathers - it is >up to you. > >In general, logging from non-webapp doesn't seem to be cool anymore so >no one pays attention. I routinely see broken Syslog messages from >products by Oracle, Cisco, Juniper, Palo Alto firewall etc etc. Guess >what, if people cannot get them to stick to Syslog, what makes you >think they will produce logs that can be consistently consumed by >logstash or whatever new comes around the block? tell me about it. i've got just under a hundred lines of regexp/fixups for cisco logs alone...different timestamp formats, multi-line logs, etc. no two devices do it the same way because they are different acquisitions/dev teams. :-( there is a high level initiative to "fix" a lot of things like this as products converge, but it takes years to accomplish. however, as you point out this it not vendor specific -- and not going away in our lifetime. this is a big reason flexibility in the logging tool is essential, so the infra admin can fix upstream problems and keep users happy in an imperfect world. >There is a lot of new techies who don't have any experience with >Unix/Linux or existing products/tools to solve problems. So they write >a new tool (often with poor documentation or community support) and >other folks like them join in on the new "cool" stuff. A lot of these >tools are written in Java and performance simply sucks. No one notices >because either they don't have high volumes or they just throw a ton >of hardware at the problem - either case, bad engineering. Even with >well written code, code written in Java can be 100x slower than C. My >co-worker wrote a netflow-to-syslog collector/translator in Java. It >ate up 100% CPU on a massive 16-core server for X amount of traffic. >For the same amount of traffic, the tool re-written in C hums at >something like 2% CPU utilization. > >I have to run flume to ingest Syslog into Hadoop. Each instance is >memory and CPU hungry (not to mention brittle and buggy). i'm in a similar boat with logstash, effectively running a hub and spoke model with rsyslog receiving fixed data from load balanced logstash instances around the globe. i say load balanced because i really have to scale out logstash as it's a fat java app. things have gotten a lot better in the latest milestones, but it still likes to crash on a whim...unlike rsyslog which despite ingesting much more data is far faster, and the last time i restarted was a planned upgrade. >I'd say, let the dust settle down with all these new tools - no need >to rename rsyslog. Add new modules or improve existing ones to do >normalization and make it easier to add output modules so people can >write modules for their consumer layer (search, map-reduce, CEP etc >etc). Make RSyslog a platform for log apps and you will kick >everyone's butt. > >My 2 cents worth. huge +1 > > > >On Mon, Feb 3, 2014 at 12:51 AM, Rainer Gerhards ><[email protected]> wrote: >> OK folks, thanks again for the feedback. Looks like I need to stick with >> "rsyslog", even though I am quite unhappy with the capitalization. But >>as >> it looks changing it doesn't change anything either ;) I don't see that >> "Rsyslog" gets us more. If at all, "rsysLog" would, as the key word to >> break is "syslog" IMHO. But anyhow... ;) I still keep the changed >>tagline. >> >> Rainer >> >> >> On Fri, Jan 31, 2014 at 9:29 PM, David Lang <[email protected]> wrote: >> >>> I don't think that anyone is talking about changing the binaries, we >>>are >>> just talking about changing english language usage. >>> >>> I think that doing Rsyslog instead of rsyslog de-emphisises syslog >>>enough, >>> but as long as people don't get dogmatic about it, I'm not that worried >>> about things. >>> >>> David Lang >>> >>> >>> On Fri, 31 Jan 2014, robert s wrote: >>> >>> I agree with Radu, and David, I think capitalization are a bit much, >>>> specially with UNIX being case sensitive, its just looks odd. >>>> >>>> Robert >>>> >>>> >>>> On Fri, Jan 31, 2014 at 1:49 PM, Orangepeel Beef >>>> <[email protected]> wrote: >>>> >>>>> I'm a network engineer and we use rsyslog as our centralized syslog >>>>> server. we collect logs not only from systems but tons of networking >>>>> gear. after rsyslog gets it we send it into SEC for alerting, then >>>>> logstash for indexing. Anyone who says syslog is dead is definitely >>>>>not >>>>> a >>>>> networking person. good luck trying to get logstash or sysd on a >>>>>switch, >>>>> router, firewall, network appliance.... yada yada. >>>>> >>>>> also I'm curious, do you think the syslog-ng guys are worried about >>>>>this >>>>> as >>>>> well? >>>>> >>>>> >>>>> >>>>> On Jan 31, 2014 10:00 AM, "Radu Gheorghe" <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>>> 2014-01-31 Rainer Gerhards <[email protected]>: >>>>>> >>>>>> On Fri, Jan 31, 2014 at 5:48 PM, Radu Gheorghe < >>>>>>> [email protected] >>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>> >>>>>>> 2014-01-31 Rainer Gerhards <[email protected]>: >>>>>>>> >>>>>>>> OK, thanks everyone for the feedback. Much appreciated! >>>>>>>>> >>>>>>>>> So changing the name is the crazy (and bad) idea that I thought >>>>>>>>>it >>>>>>>>> >>>>>>>> is. >>>>> >>>>>> But >>>>>>>> >>>>>>>>> thankfully the thread brought up other pressing problems. >>>>>>>>> >>>>>>>>> On the name: obviously, it needs to stay the same. I have taken >>>>>>>>>the >>>>>>>>> liberty, though, to change the tagline and capitalization of >>>>>>>>> >>>>>>>> rsyslog. >>>>> >>>>>> Right >>>>>>>> >>>>>>>>> now, I write "RSysLog - a *r*ocket-fast *sys*tem for *log* >>>>>>>>> >>>>>>>> processing". >>>>> >>>>>> It >>>>>>>> >>>>>>>>> follows the good ole industry standard way of renaming things >>>>>>>>> >>>>>>>> without >>>>> >>>>>> changing the name ;) It's not final yet and if anyone has a good >>>>>>>>> >>>>>>>> argument >>>>>>> >>>>>>>> why it would be a really terrible idea, we can change this back. >>>>>>>>> >>>>>>>> Otherwise, >>>>>>>> >>>>>>>>> I'll roll this out over the next couple of weeks. >>>>>>>>> >>>>>>>>> >>>>>>>> I think rsyslog or Rsyslog is what people know and expect. >>>>>>>>RSysLog is >>>>>>>> trying to hard IMO. >>>>>>>> >>>>>>>> >>>>>>> comment appreciated, but if that's the only concern, I'd like to >>>>>>>go for >>>>>>> >>>>>> it. >>>>> >>>>>> Especially if it awakes people. I'd intentionally like to break the >>>>>>> >>>>>> "mental >>>>> >>>>>> syslog connection" and if SysLog helps in that sense, that's good. >>>>>>> >>>>>>> >>>>>>> I think it doesn't break anything, because very few people will >>>>>> understand >>>>>> what it means (because R, Sys and Log are all abbreviations). I >>>>>>would >>>>>> expect most people to be ignoring/get annoyed by the strange >>>>>>capitals. >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>myriad >>>>>> >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>you >>>>> DON'T LIKE THAT. >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>you >>>>> DON'T LIKE THAT. >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>if you DON'T LIKE THAT. >_______________________________________________ >rsyslog mailing list >http://lists.adiscon.net/mailman/listinfo/rsyslog >http://www.rsyslog.com/professional-services/ >What's up with rsyslog? Follow https://twitter.com/rgerhards >NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
