@Janmejay: I'll be honest - I <strike>don't</strike> didn't know if it'll handle escape sequences. I didn't test it earlier, just tested it now. Totally worked, woo! I did put in documentation - you can check the file. Actually included a little bugfix on existing documentation that kept it from compiling as well. I'm not against putting tests in at all, though I didn't because I didn't see any tests for the non-special functions, only regex and tokenization. I can throw them in. What I did do is test this against a couple thousand log lines that I actually needed to parse, just to make sure it worked.
On Mon Jan 26 2015 at 10:01:21 PM Kendall Green <[email protected]> wrote: > I like the 'or' option, precisely for doing type check condition when a > whole lot of fields exists in records. This is currently cumbersome and > quickly becomes a daunting mess of a Cartesian Product set rule base for > all the combination of fields that could have single values unquoted, or > possibly quoted. Not to mention how this use case caries over to other > scenarios where an or operation would be invaluable to type casting. > > %<tag>:<type>:or:<type>% > could be very useful, not just to solve the issue of which behavior should > be default, as it would be set by the syntax. > > For example, if type quoted-string is set first, then should check without > quotes up to space. > Wouldn't the default be for what the type is, so with quoted-string, then > it's quoted, unless an 'or' condition exists for an alternate expected data > type. > > With so very many fields in verbose messages, it is great to have a single > rule which would otherwise be an exponentially lengthy ruleset to > accommodate all the possible known type setting combinations. > %Description:quoted-string:or:word% > > An ''type:or:type" option could also be useful in other cases where > unpopulated fields exists with a default type value which doesn't match the > field when populated with specific typed value. > > %IP Address:ipv4:or:word% > The IP Address is provided, or a hyphen exists in the field when > unpopulated. In this scenario more specific literal matching would also be > nice option, which please correct me if literals already exists beyond > annotations. Having a char type match as char-sep somewhat resembles, where > field extraction only when the literal matches. The difference being that > the literal would be matched for field value not just up to that position. > To give a more strict rule: > %IP Address:ipv4:or:char:\x2d% > > Similarly, it would be good to have string type, like described for the > purposed char type above, but for capturing the string literal instead of > only the literal char. Rulebase could use string parse > enhancement with capture of literal string at specific field start > position within rulebase, since existing features could likely be used like > annotation fields. Additionally, please inform of any contributions for > the discussion regarding data type of fields to match string as a > string-to, as char-to / char-sep feature of char > separator on string, like the function, field($!path, string-or-char). So > please also elaborate on what has already been done for rulebase matching > string literals. Thanks! > > -Kendall > > > > On Mon, Jan 26, 2015 at 5:49 PM, David Lang <[email protected]> wrote: > > > I don't like the "or" option as I think it makes the rules harder to > read. > > unless you are doing this on a lot of fields in a line, just make a new > > line with the different type. > > > > We need feedback from others, but at the very least I think making this > an > > option to the standard quoted-string type would be better than a new type > > (the question is if this should be enabled by default or disabled by > > default) > > > > > > David Lang > > > > On Tue, 27 Jan 2015, Chris Schafer wrote: > > > > It comes back as a full fail. I thought about modifying that, but I > didn't > >> want to wreck anything currently in place. > >> A coworker of mine had a great idea for an "or" ability, going > >> %tag:or:quoted-string:word% where i attempts the first, and if that > fails, > >> goes to the second. However, that's not going to be easy, and I wanted > to > >> push this change before you guys got too many commits ahead. > >> > >> On Mon Jan 26 2015 at 4:43:02 PM David Lang <[email protected]> wrote: > >> > >> hmm, I'm wondering if we should do this for the normal quoted type? If > >>> you > >>> say > >>> quoted string and there isn't a quote does it just not match? > >>> > >>> David Lang > >>> > >>> On Tue, 27 Jan 2015, Chris Schafer wrote: > >>> > >>> This only handles " because that's what the current quoted string > does. > >>>> If it doesn't start with ", it implements the "word" functionality > >>>> > >>> (which I > >>> > >>>> shamelessly copied). The idea is to capture inputs where the source > >>>> > >>> system > >>> > >>>> only quotes it if it contains a space, but leaves it unquoted > otherwise. > >>>> Example: > >>>> No data = - > >>>> One Word = word > >>>> Two words+ = "Two Words" > >>>> > >>>> The function should handle all three. > >>>> Chris > >>>> > >>>> On Mon Jan 26 2015 at 4:36:25 PM David Lang <[email protected]> wrote: > >>>> > >>>> does this handle embedded quotes in the string? and do you handle > >>>>> > >>>> strings > >>> > >>>> starting with ' and " or just one of them? > >>>>> > >>>>> David Lang > >>>>> > >>>>> On Tue, 27 Jan 2015, Chris Schafer wrote: > >>>>> > >>>>> Date: Tue, 27 Jan 2015 00:30:54 +0000 > >>>>>> From: Chris Schafer <[email protected]> > >>>>>> Reply-To: rsyslog-users <[email protected]> > >>>>>> To: [email protected] > >>>>>> Subject: [rsyslog] New Pull request for liblognorm - additional > >>>>>> > >>>>> mmnormalize > >>>>> > >>>>>> functionality > >>>>>> > >>>>>> Just submitted the following pull request: > >>>>>> https://github.com/rsyslog/liblognorm/pull/20 > >>>>>> And I believe it could solve a lot of issues (at least, it solves a > >>>>>> lot > >>>>>> > >>>>> of > >>>>> > >>>>>> mine) surrounding mmnormalize parsing in rsyslog. I'm looking for > >>>>>> comments/issues/holy-crap-you-can't-code-what-are-you-doing, if you > >>>>>> > >>>>> guys > >>> > >>>> have any. This is my first time submitting a patch to a large project > >>>>>> > >>>>> (or > >>> > >>>> at least one where I didn't know the maintainer personally), so be > >>>>>> > >>>>> gentle > >>> > >>>> please :) > >>>>>> > >>>>>> Chris > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>>> > >>>>> myriad > >>> > >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>>>> DON'T LIKE THAT. > >>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com/professional-services/ > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>>>> myriad > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >>>>> DON'T LIKE THAT. > >>>>> > >>>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com/professional-services/ > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >>>> > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>> DON'T LIKE THAT. > >>> > >>>> > >>>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>> DON'T LIKE THAT. > >>> > >>> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >> DON'T LIKE THAT. > >> > >> _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
