> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of [email protected] > Sent: Thursday, July 15, 2010 10:01 AM > To: rsyslog-users > Subject: Re: [rsyslog] NEW rsyslog.conf format > > On Thu, 15 Jul 2010, Rainer Gerhards wrote: > > > After writing this, it just hit me that we could stay within the > current > > config notation by introducing > > > > $begin <type> > > $end <type> > > I have no problem with this. I view is as symatically eqivalent to the > {}(), just more characters to type ;-)
Jupp, but the big difference is that the current parser can handle that. Also, if we really stick with the language, I personally think this is more consistent with the rest of the language (but that's again an issue of personal taste). > > $name <obj> > > I had even suggested NAME <name> block and USE <name> syntaxes Ack, didn't mention it explicitely, but definitely kept it on my mind ;) > > the only question is if we need to explicitly state the type or if it's > good enough to be able to just nest the scope. I think you can get away > with just nesting the scope, but I could be wrong and if so defining > the > type is a fairly cheap way of working around the issue. Again a taste issue first: I think it is less confusing for a human to know what exactly the block defines. But there is also a hard argument: if I know it is an action, I can disallow input or global statements inside this block (and vice versa)! So we do not only get the scoping, but also the capability to not permit irrelevant statements within the scope. The only thing I need to change is add a type to each config statement (in the statement table), but this is not much work (given the fact that I need to change some internals in any way to support the scoping). Rainer > > > The sample below could then be: > > > > ============================================================== > > $ModLoad ommysql # load the output driver (use ompgsql for > PostgreSQL) > > $ModLoad imudp # network reception > > $UDPServerRun 514 # start a udp server at port 514 > > $ModLoad imuxsock # local message reception > > > > $WorkDirectory /rsyslog/work # default location for work (spool) > files > > > > $Begin action > > $ActionQueueType LinkedList # use asynchronous processing > > $ActionQueueFileName dbq # set file name, also enables disk mode > > $ActionResumeRetryCount -1 # infinite retries on insert failure > > # for PostgreSQL replace :ommysql: by :ompgsql: below: > > *.* :ommysql:hostname,dbname,userid,password; > > $End action > > ============================================================== > > > > This would probably be possible to implement within the constraints > of the > > current config parser. We could also add a directive > > > > $StrictScoping on > > > > to instruct rsyslog to disallow and $Action<whatever> directives > outside of > > scoped blocks (this could be good to guard against accidential global > > directives). > > probably a good idea. > > > That still requires reworking of the internals, but would not need a > new > > (real) grammar, so it would be considerable less work. I still need > to see > > how it would work with more complex configs, but assuming we have > things like > > > > $Begin Ruleset > > $Begin Rule > > $Begin Input > > > > I think the same paradigm could probably be used for everything (and > that > > than would eliminate the need for a new grammar). > > sounds good. > > David Lang > > > Rainer > > > >> -----Original Message----- > >> From: [email protected] [mailto:rsyslog- > >> [email protected]] On Behalf Of Rainer Gerhards > >> Sent: Thursday, July 15, 2010 9:42 AM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] NEW rsyslog.conf format > >> > >> Hi David, > >> > >> thanks for the well-crafted mail. My concerns for this proposal were > >> (and > >> are) mainly based on the "under the cover" changes. Other than that, > I > >> think > >> you are right, except that it boils down to personal taste. > >> > >> Let me ignore the internal changes for now. > >> > >> Throwing in "{}" creates a very compact, ultra-brief config > language. > >> But it > >> would definitely work. I just have to admit it does not fit my > >> *personal* > >> taste because it carries a lot of implicit scoping as well (like > what > >> exactly > >> is meant to be scoped by the {} -- an action, or an input, or a > >> ruleset...). > >> But I think the semantics of this format are close to any other > config > >> format > >> that fits rsyslog. So I think it is probably worth giving it a try, > so > >> that > >> we make at least some progress. The only thing I really would like > to > >> change > >> is to use "()" instead of "{}" because I would like to reserve "{}" > for > >> future use, where it may fit the user's expectation better than > simple > >> parenthesis. But I guess that's not really a problem. > >> > >> One thing though, that is on my feature list is that I would like to > >> use a > >> more standard parser, that means one that can becreated with lex and > >> Bison. > >> While the hand-crafted parser is fine, it always is more work to > extend > >> and > >> enhance it. As the parser is no critical component, I'd prefer to > use > >> the > >> simpler approach. However, I need to check if I can find a suitable > >> grammar > >> for this proposal. This also works on the assumption. > >> > >> I'd also just concentrate on actions for now. So to double-check a > >> fairly > >> simple config in that format would look as follows: > >> > >> ============================================================== > >> $ModLoad ommysql # load the output driver (use ompgsql for > PostgreSQL) > >> $ModLoad imudp # network reception > >> $UDPServerRun 514 # start a udp server at port 514 > >> $ModLoad imuxsock # local message reception > >> > >> $WorkDirectory /rsyslog/work # default location for work (spool) > files > >> > >> ( > >> $ActionQueueType LinkedList # use asynchronous processing > >> $ActionQueueFileName dbq # set file name, also enables disk mode > >> $ActionResumeRetryCount -1 # infinite retries on insert failure > >> # for PostgreSQL replace :ommysql: by :ompgsql: below: > >> *.* :ommysql:hostname,dbname,userid,password; > >> ) > >> ============================================================== > >> > >> This is based on the second example in > >> http://www.rsyslog.com/doc-rsyslog_high_database_rate.html > >> > >> I will probably also begin to look at the internal changes. As it > >> looks, we > >> need to make them in any case. So it doesn't hurt to start with > them, > >> even > >> though there initially will be no external (user) visibility that > they > >> are > >> made. > >> > >> But at first, I'll start looking at how this may be processed by a > >> standard > >> flex/bison parser. From the work I already did, I know this is not > >> easy, but > >> could probably work. > >> > >> Feedback appreciated. > >> > >> Rainer > >> > >>> -----Original Message----- > >>> From: [email protected] [mailto:rsyslog- > >>> [email protected]] On Behalf Of [email protected] > >>> Sent: Tuesday, July 13, 2010 7:30 PM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] NEW rsyslog.conf format > >>> > >>> On Tue, 13 Jul 2010, Rainer Gerhards wrote: > >>> > >>>> To add some of the less obvious things: support for multiple > >>> listeners going > >>>> to different outputs, in an easy to use way. Support for > >> explicitely > >>>> specified concurrency (or serialization) for high-speed > >> environments, > >>> support > >>>> for different queues, and tying of different queues to different > >>> object > >>>> classes (inputs, message processors, actions). Flexibility to > >> support > >>>> configuration for even unexpected plugins that we may not even > know > >>> about > >>>> (because some third party writes them and never publishes them). > >>>> > >>>> But I begin to agree that we, the community, as a whole have very > >>> different > >>>> needs. I have gotten the impression that it is probably a good > idea > >>> to stop > >>>> the current effort and re-start it with a requirements definition. > >> I > >>> have > >>>> tried to digest the discussions on config format we had over the > >>> year, but > >>>> sometimes it looks like the only consensus we have is that the > >>> current format > >>>> is ugly and hard to use. The solutions are very wide-ranging. I > >> have > >>> even > >>>> briefly looked at syslog-ng, and seen a lot of the pain again that > >>> makes me > >>>> dislike that format (but I'll probably still have a closer look > and > >>> will try > >>>> to write up my detailed position why I do not find buying into > this > >>> format is > >>>> a good idea). > >>>> > >>>> All in all, I think the best way to re-start our thinking about > the > >>> config > >>>> format is to set up a web site where we gather user feedback on > >>>> > >>>> a) what problems they have with the config format > >>>> (not what they just dislike, but real problems, an example > >>>> From myself: it is nearly impossible to get the sequence right > >>>> To bind inputs to the right rulesets AND use ruleset inclusion) > >>>> b) which alternatives they see > >>>> > >>>> With all this being on a web site, it may be possible to craft a > >>> better > >>>> decision in 6 to 9 month, assuming we were able to gain sufficient > >>> feedback > >>>> during that time. > >>>> > >>>> An alternative would be to create a config parser interface, so > >> that > >>>> everybody could code up his favorite config language. However, > this > >>> seems to > >>>> be impractical, as many of the ideas floating around (Lua, syslog- > >> ng > >>> style) > >>>> require not just different config parsers, but a different rsyslog > >>> processing > >>>> core as well. Obviously a complete rewrite in the case of Lua, > less > >>> for > >>>> syslog-ng, but still considerate. For the syslog-ng style we would > >>> need to > >>>> create named filters, something that I really find questionable. > >>> Also, we > >>>> would need to add an interim layer between inputs and rulesets, > >>> something > >>>> that eats performance. In any case, this are not config-parser > only > >>> changes. > >>>> > >>>> Of course, I could just pick one of the alternatives. However, it > >>> doesn't > >>>> make sense to invest a couple of month to do the config system > >>> "right", if we > >>>> know that a lot of folks will still be unhappy after we have done > >>> this. > >>> > >>> one thing that's really good about the current rsyslog config is > that > >>> it > >>> makes doing simple things simple, especially for people used to > >> classic > >>> syslog. > >>> > >>> as you say, we need to say what the problems there are with the > >>> existing > >>> config format and look at how to solve those. We don't necessarily > >> need > >>> to > >>> change everything. > >>> > >>> weaknesses that I know of > >>> > >>> inability to easily/clearly define if-then-else > >>> > >>> inability to easily/clearly define/use rulesets > >>> > >>> inability to easily have multiple conditions that go to the same > >>> destination (this can be implemented via rulesets, see above) > >>> > >>> unclear scope rules for config options > >>> > >>> inability to easily/clearly define per-input rules/filters (this is > >>> part > >>> of the scope problem above) > >>> > >>> > >>> I don't think that this requires an entirely new config language. I > >>> think > >>> that almost everything can be solved with a couple simple changes > to > >>> the > >>> config language (although as we found when I proposed this on June > 25 > >>> there are more drastic changes under the covers to > >>> check/correct/document > >>> what gets changed when a config option is processed) > >>> > >>> > >>> Ignoring for the moment the problem of changing how the config > >> options > >>> are > >>> processed (and the fact that you would need to know what data > >>> structures > >>> are managed/created/modified) what does the following proposal > _not_ > >>> do? > >>> > >>> (pasted in from the mail on June 25 since that's now quite a ways > >> back > >>> in > >>> the archives ;-) > >>> > >>> > >>> > >>> I would propose the following (more or less in order of difficulty) > >>> > >>> introduce scoping > >>> > >>> whenever you see a "{" in the config file, save the current > config > >>> options to a stack. when you see a "}" pop the config options from > >> the > >>> stack (undoing any changes in between) introduce statement blocks > >>> change the parser so that wherever it currently allows one > action > >>> make > >>> it accept a {} block of actions (treat them as if they were > multiple > >>> actions with & as the selector) > >>> > >>> introduce named actions > >>> > >>> something like sub NAME <action> to define and use NAME to use > >>> > >>> introduce if-then-else > >>> > >>> internally you could convert it to > >>> > >>> ruleset { > >>> if condition { > >>> block > >>> discard} > >>> { block } } > >>> > >>> introduct the ability to implcitly define a ruleset > >>> > >>> if an action is a condition (i.e. nested configutations) then > >>> implicitly > >>> create a new ruleset to resolve the nesting. > >>> > >>> > >>> with these capabilities available I think that this will allow for > >>> straightforward config files representing very complex > configurations > >>> with > >>> very little change internally to rsyslog. > >>> > >>> I suspect that the result is going to have some bottlenecks in > >> complex > >>> configurations due to all the different rulesets and the passing of > >>> messages between them, but once the capability is in place in the > >>> config > >>> file the implementation under the covers could change later to be > >>> better > >>> optimized. > >>> > >>> as far as the rsyslog config being hard to understand, I think > there > >>> are > >>> two conditions here. > >>> > >>> 1. very simple configs where the fact that rsyslog can use a > >>> traditional > >>> syslog config file (with a header on it that seldom needs to > change) > >> is > >>> a > >>> huge advantage > >>> > >>> 2. complex configs where the inability to define scope and nest > >> things > >>> is > >>> a major problem and makes the resulting config hard to write. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
