2015-03-12 5:55 GMT+01:00 singh.janmejay <[email protected]>: > On Thu, Mar 12, 2015 at 9:19 AM, David Lang <[email protected]> wrote: > > On Thu, 12 Mar 2015, singh.janmejay wrote: > > > >> Tried re-ordering it? Put the one with /port first? > > > > > > no, lognorm rules are not supposed to be order dependent, so I didn't try > > that (especially after finding things failing to parse with rsyslog that > > worked manually) > > In case of input strings being matching-rule-wise disjoint, you are > right, order won't matter. But when they are not disjoint, order does > matter, because the first one to match the string wins. > > Consider this rulebase: > rule=:%ip:ipv4%%last:rest% > rule=:%ip:ipv4%%junk:char-sep:/%/%port:number% > > If you write it the way I have above, you'll end up matching first > rule for input 10.20.30.40/5 > > But if you write it this way: > rule=:%ip:ipv4%%junk:char-sep:/%/%port:number% > rule=:%ip:ipv4%%last:rest% > > You'll end up matching the first one. >
This shouldn't happen. The theory is: Let i be the current index to be looked at at the line. If for i a parser is selected, parsers shall be tried first (in theory, according to parser ordering, but I think this is not yet fully implemented). If a parser fits, processing is advanced to next tree node. If the node at i does not have a parser (or all parsers failed, I think [but not sure]), advance to next node basded on character match. The order of apperance of rules inside the rulebase should not affect this. If it does, it's either not yet implemented or a bug. this is also why I don't like the "rest" syntax -it always matches and thus terminates interpretation. > I know it appears order independent for your original rulebase, but > that is because fields are always tried first(in preference to > subtrees hanging off literals), and rest is a field, while '/' creates > a litteral-subtree. > > > > >> Yes, rest must get atleast one char to succeed. I'll create some new > >> tests without rest-capture (and see what fails). > > > > > > Ok, this can be worked around (but it's a bit ugly), any reason why rest > has > > to get at least one character? > > Yep, its annoying, it happens only for last token. > > The reason is, parsed-fragment length >= input-string is used as a > termination condition for ln_normalize recursion (see ln_normalizeRec) > and the last token identified when recursion terminates is not the > terminal-node, so its not considered a complete match(one that goes > till leaf of ptree). > > > > > David Lang > > > > > >> On Thu, Mar 12, 2015 at 1:09 AM, David Lang <[email protected]> wrote: > >>> > >>> I just upgraded to liblognorm 1.1.1 (unfortunantly I didn't get a > chance > >>> to > >>> compile it myself and test it earlier) > >>> > >>> I ran into two problems > >>> > >>> first, %last:rest% does not match if there is nothing left on the line > >>> > >>> i.e. a line that ends with an IP address will not match > >>> rule=:%ip:ipv4%%last:rest% > >>> > >>> secondly, liblognorm is selecting the rule that matches the least > amount > >>> of > >>> the message. > >>> > >>> so with these two rules > >>> > >>> rule=:%ip:ipv4%%last:rest% > >>> rule=:%ip:ipv4%/%port:number%%last:rest% > > I guess the hack I proposed above (using char-sep) can unblock you for > now, unless you hate its aesthetics too much :-). > > >>> > >>> 192.168.1.1/5 will get matched by the first rule, with '/5' in last, > even > >>> though the second rule would match it. If I remove the first rule, the > >>> second rule does match and the parse succeeds. > >>> > >>> David Lang > >>> > >>> > >>> On Fri, 6 Feb 2015, David Lang wrote: > >>> > >>>> While I'm working to build packages of this to test with, what happens > >>>> if > >>>> you descend into a ruleset like the following > >>>> > >>>> rule=:%ip:ipv4%%last:rest% > >>>> rule=:%ip:ipv4%/%port:number%%last:rest% > >>>> > >>>> will it work to find the match that has the least left in last? > >>>> > >>>> David Lang > >>>> > >>>> > >>>> On Fri, 6 Feb 2015, singh.janmejay wrote: > >>>> > >>>>> It's going to be in the coming release, just master build for now. > >>>>> > >>>>> -- > >>>>> Regards, > >>>>> Janmejay > >>>>> > >>>>> PS: Please blame the typos in this mail on my phone's uncivilized > soft > >>>>> keyboard sporting it's not-so-smart-assist technology. > >>>>> > >>>>> On Feb 6, 2015 6:37 AM, "David Lang" <[email protected]> wrote: > >>>>> > >>>>>> On Wed, 4 Feb 2015, singh.janmejay wrote: > >>>>>> > >>>>>> On Wed, Feb 4, 2015 at 6:22 PM, David Lang <[email protected]> wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Wed, 4 Feb 2015, singh.janmejay wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Wed, Feb 4, 2015 at 7:17 AM, David Lang <[email protected]> > wrote: > >>>>>>>> > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Field type 'descent' does this, but not exactly in the same > way. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> does it? I understood it to just be calling another ruleset on the > >>>>>>>> whole > >>>>>>>> line (doc problem again) > >>>>>>>> > >>>>>>>> > >>>>>>> It allows field to identify how remaining-text should be returned, > >>>>>>> which > >>>>>>> allows it to be parsed by remaining part of the rule which the > field > >>>>>>> belongs to. > >>>>>>> > >>>>>>> Here is a test which uses something similar to what you are trying > to > >>>>>>> do: > >>>>>>> https://github.com/rsyslog/liblognorm/blob/master/tests/ > >>>>>>> field_tokenized_recursive.sh#L41 > >>>>>>> > >>>>>>> (check 41 to EOF) > >>>>>>> > >>>>>> > >>>>>> This looks like it may do this, but it looks like it's not in the > >>>>>> release > >>>>>> yet. I'll have to compile from scratch. > >>>>>> > >>>>>> David Lang > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>>> myriad > >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >>>>>> DON'T LIKE THAT. > >>>>>> > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com/professional-services/ > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>> myriad > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >>>>> DON'T > >>>>> LIKE THAT. > >>>>> > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com/professional-services/ > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>>> DON'T > >>>> LIKE THAT. > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >>> of > >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T > >>> LIKE THAT. > >> > >> > >> > >> > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. > > > > -- > Regards, > Janmejay > http://codehunk.wordpress.com > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
