Hrrm, thanks for verifiying. Very strange, then. I'm definitely not getting
any distribution of messages, but have verified that the serial (or SN) numbers
are incrementing.
Are you seeing the rest of the logic (writing to the different files) working,
perchance?
Thanks!
Robert
> Date: Thu, 24 Oct 2013 23:59:54 +0400
> From: [email protected]
> To: [email protected]
> Subject: Re: [rsyslog] Another approach to action load balancing
>
>
> Very strange, because it works for me with some 7.4 from git.
>
> set $!msg = "vpn_tunnel=\"N/A\" src_int=\"port6\" dst_int=\"port5\"
> app=\"N/A\" app_cat=\"N/A\" user=\"N/A\" group=\"N/A\" serial=1249572
> app-type=\"N/A\"";
> set $!msgid = re_extract($!msg, " serial=([0-9]*)", 0, 1, "0");
>
> I'm using this template to look at results:
>
> template(
> name="common1"
> type="string"
> string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg%
> _%$!%_\n"
> )
>
> And they are:
>
> _{ "msg": "vpn_tunnel=\"N\/A\" src_int=\"port6\" dst_int=\"port5\"
> app=\"N\/A\" app_cat=\"N\/A\" user=\"N\/A\" group=\"N\/A\"
> serial=1249572 app-type=\"N\/A\"", "msgid": "1249572" }_
>
>
> --
> Pavel Levshin
>
>
> 24.10.2013 23:44, Robert McIntyre:
> > Thanks, Andre! Downloaded and testing. Unfortunately, I'm just getting
> > almost nowhere with my regexes. :| I've pasted the appropriate part of the
> > config, and some sections of events for reference below
> >
> > Let me start with what *is* working. I can use re_match(), and it finds
> > the strings and routes appropriately (This is Option 1 in the config
> > below). But, for some reason, I can't get the re_extract() to pull a value
> > out (this is the commented out Option 2 in the config below). Based on
> > what I can see from debug logs, it never finds a match. I've tested the
> > regexs and strings with the rsyslog regex tester, and it looks like they
> > should be working.
> >
> > Any suggestions?
> >
> > # Option 1, this verifies that the regex's work, and do what is expected
> > if (re_match($msg,"serial=([0-9]*)"))
> > then set $!msgid = "0";
> > else if (re_match ($msg, "SN=([0-9]*)"))
> > then set $!msgid = "1";
> >
> > # Option 2, this should extract either the serial number, or SN from the
> > message, but doesn't
> > #set $!msgid = re_extract($msg, " serial=([0-9]*)", 0, 1, "0");
> > #if ($!msgid == "0")
> > # then set $!msgid = re_extract($msg, " SN=([0-9]*)", 0, 1, "0");
> >
> > if ($!msgid % 3 == '0')
> > then {
> > action(name="Act_File1"
> > type="omfile"
> > file="/syslogdata/testing/1.txt")
> > stop
> > }
> > if ($!msgid % 3 == '1')
> > then {
> > action(name="Act_File2"
> > type="omfile"
> > file="/syslogdata/testing/2.txt")
> > stop
> > }
> > if ($!msgid % 3 == '2')
> > then {
> > action(name="Act_File3"
> > type="omfile"
> > file="/syslogdata/testing/3.txt")
> > stop
> > }
> > action(name="Act_File4"
> > type="omfile"
> > file="/syslogdata/testing/4.txt")
> >
> > Extracts from logs:
> > vpn_tunnel="N/A" src_int="port6" dst_int="port5" SN=1718017351 app="N/A"
> > app_cat="N/A" user="N/A" group="N/A"
> >
> > vpn_tunnel="N/A" src_int="port6" dst_int="port5" app="N/A" app_cat="N/A"
> > user="N/A" group="N/A" serial=1249572 app-type="N/A"
> >
> > Thanks!
> > Robert
> >
> >> From: [email protected]
> >> Date: Thu, 24 Oct 2013 09:15:09 +0200
> >> To: [email protected]
> >> Subject: Re: [rsyslog] Another approach to action load balancing
> >>
> >> I am sorry for the missing rpm packages for 7.4.5. For some reason, my
> >> build
> >> environment failed to build packages for EHEL 6 x64.
> >> Packages have been created now and should be available in the repository.
> >>
> >> Best regards,
> >> Andre Lorbach
> >>
> >>> -----Original Message-----
> >>> From: [email protected] [mailto:rsyslog-
> >>> [email protected]] On Behalf Of Robert McIntyre
> >>> Sent: Thursday, October 24, 2013 12:43 AM
> >>> To: rsyslog-users
> >>> Subject: Re: [rsyslog] Another approach to action load balancing
> >>>
> >>> Make sense! Now just waiting for the RPMs get updated, and will give this
> >>> a
> >>> go!
> >>>
> >>> Thanks!
> >>> Robert
> >>> ________________________________
> >>> From: David Lang<mailto:[email protected]>
> >>> Sent: 10/23/2013 3:33 PM
> >>> To: rsyslog-users<mailto:[email protected]>
> >>> Subject: Re: [rsyslog] Another approach to action load balancing
> >>>
> >>> local variables only exist in the 7.5 branch.
> >>>
> >>> in 7.4 you only have $! variables.
> >>>
> >>> David Lang
> >>>
> >>> On Wed, 23 Oct 2013, Robert McIntyre wrote:
> >>>
> >>>> Date: Wed, 23 Oct 2013 11:48:19 -0700
> >>>> From: Robert McIntyre <[email protected]>
> >>>> Reply-To: rsyslog-users <[email protected]>
> >>>> To: rsyslog-users <[email protected]>
> >>>> Subject: Re: [rsyslog] Another approach to action load balancing
> >>>>
> >>>> So, I've had decent luck with Pavel's suggestion
> >>> (field($timegenerated,':',3), and it rotates around nicely based on the
> >>> second.
> >>>> I'm trying a slightly different approach, though, to try to get
> >>>> sub-second
> >>> rotation. My firewall logs have a log sequence number that I'd like to
> >>> use as
> >>> the input to my modulus, but I'm having trouble extracting it. Using the
> >>> rsyslog regex builder/tester, I came up with this regex:
> >>>> %msg:R,ERE,1,DFLT:SN=([0-9]*)--end%
> >>>>
> >>>> But, when I try to use it in my config it doesn't work. I've tried
> >>>> setting a
> >>> local variable:
> >>>> $.msgid = "%msg:R,ERE,1,DFLT:SN=([0-9]*)--end%"
> >>>> or
> >>>> # $.msgid = "msg:R,ERE,1,DFLT:SN=([0-9]*)--end"
> >>>> or
> >>>> set $.msgid = "msg:R,ERE,1,DFLT:SN=([0-9]*)--end"
> >>>> or
> >>>> set # $.msgid = "%msg:R,ERE,1,DFLT:SN=([0-9]*)--end%"
> >>>>
> >>>> But get config errors regardless.
> >>>>
> >>>> I've tried putting these variations directly in the if clause:
> >>>>
> >>>> if ("msg:R,ERE,1,DFLT:SN=([0-9]*)--end" % 3 == '0')
> >>>>
> >>>> (and all the iterations), but no luck with that.
> >>>>
> >>>> So, under 7.4.4, what is the recommended way to extract a string/number
> >>> from a message, and then use that extracted value in an expression?
> >>>> I've got some other lessons learned from this that I plan to write up
> >>>> for the
> >>> group, but want to get this final bit sorted first.
> >>>> Thanks!!!
> >>>> Robert
> >>>>
> >>>>
> >>>> Date: Wed, 23 Oct 2013 08:41:20 -0700
> >>>> From: [email protected]
> >>>> To: [email protected]
> >>>> Subject: Re: [rsyslog] Another approach to action load balancing
> >>>>
> >>>> there should be per the docs, but in practice there is not. At least
> >>>> not as it is accessed via the scripting variables. I think if you use
> >>>> it as a property in a template you get the higher precision.
> >>>>
> >>>> David Lang
> >>>>
> >>>> On Wed, 23 Oct 2013, Robert McIntyre wrote:
> >>>>
> >>>>> Thanks, Pavel! This works as expected. The docs say that
> >>>>> $timegenerated is "always in high resolution". Is that max
> >>>>> resolution seconds? I'm trying to figure out how to just see the
> >>>>> value of $timegenerated to see what format it is (I'm assuming
> >>>>> HH:MM:SS based on the field statement, but wonder if there's a .XX at
> >>> the end).
> >>>>> Thanks!!!
> >>>>> Robert
> >>>>>
> >>>>>
> >>>>>> Date: Wed, 23 Oct 2013 18:00:04 +0400
> >>>>>> From: [email protected]
> >>>>>> To: [email protected]
> >>>>>> Subject: Re: [rsyslog] Another approach to action load balancing
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Here is what you looked for:
> >>>>>>
> >>>>>> field($timegenerated,':',3);
> >>>>>>
> >>>>>> It is a number, so you can balance per second based on it. And it
> >>>>>> works with 7.4.4.
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Pavel Levshin
> >>>>>>
> >>>>>>
> >>>>>> 23.10.2013 17:12, Robert McIntyre:
> >>>>>>> Thanks, that's too bad. I was quite excited yesterday, thinking
> >>>>>>> about
> >>> the problem and reading the docs, but couldn't figure it out after
> >>> spending
> >>> some time with my test server.
> >>>>>>> I recall the other thread recently about the documentation, and how
> >>>>>>> to
> >>> make it clear what's applicable to which version, as well as what's
> >>> possible
> >>> across features (Rainer script crossed with property replacer in this
> >>> case).
> >>> This is an example of that issue.
> >>>>>>> I don't have much to contribute to this project other than
> >>>>>>> questions, the occasional answer for someone else, and thanks, so
> >>>>>>> I'll reiterate: thanks to everyone working on this project! :)
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>> Robert
> >>>>>>> ________________________________
> >>>>>>> From: Rainer Gerhards<mailto:[email protected]>
> >>>>>>> Sent: 10/23/2013 4:01 AM
> >>>>>>> To: rsyslog-users<mailto:[email protected]>
> >>>>>>> Subject: Re: [rsyslog] Another approach to action load balancing
> >>>>>>>
> >>>>>>> On Wed, Oct 23, 2013 at 12:41 PM, Pavel Levshin
> >>> <[email protected]>wrote:
> >>>>>>>> So, not all system properties are accessible from RainerScript, in
> >>>>>>>> 7.4.
> >>>>>>>> There is none having resolution of seconds. Here they are:
> >>>>>>>>
> >>>>>>>> $now (this is just a date, unfortunately) $year $month $day $hour
> >>>>>>>> $minute $myhostname
> >>>>>>>>
> >>>>>>>> And that's all. In 7.5, all is complicated right now.
> >>>>>>>>
> >>>>>>>> I am working on that ;) I could promise to add some $$nowseconds
> >>>>>>>> sysvar,
> >>>>>>> but looking at the current schedule I better do not do that...
> >>>>>>>
> >>>>>>> Rainer
> >>>>>>>
> >>>>>>>> --
> >>>>>>>> Pavel
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 23.10.2013 10:33, Pavel Levshin:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> It seemes that you are unable to access $uptime property (as
> >>>>>>>>> $$uptime, I suppose). The same is true for 7.4 and 7.5.5.
> >>>>>>>>>
> >>>>>>>>> It works for me, because there is a regression after latest fixes
> >>>>>>>>> for global variables. In HEAD, I can access $uptime (as $uptime),
> >>>>>>>>> but do not see any property without $ at start.
> >>>>>>>>>
> >>>>>>>>> As for more precise counter, it is timegenerated. But it is also
> >>>>>>>>> unusable because you cannot access subseconds from RainerScript.
> >>> AFAIK.
> >>>>>>>>>
> >>>>>>>> ______________________________**_________________
> >>>>>>>> rsyslog mailing list
> >>>>>>>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.a
> >>>>>>>> discon.net/mailman/listinfo/rsyslog>
> >>>>>>>> http://www.rsyslog.com/**professional-
> >>> services/<http://www.rsyslog
> >>>>>>>> .com/professional-services/> What's up with rsyslog? Follow
> >>>>>>>> https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing
> >>>>>>>> list, posts are ARCHIVED by a myriad of sites beyond our control.
> >>>>>>>> PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> >>>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> rsyslog mailing list
> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>>> http://www.rsyslog.com/professional-services/
> >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> >>>>>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >>>>>>> of
> >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >>> DON'T LIKE THAT.
> >>>>>>> _______________________________________________
> >>>>>>> rsyslog mailing list
> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>>> http://www.rsyslog.com/professional-services/
> >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> >>>>>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >>>>>>> of
> >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >>> DON'T LIKE THAT.
> >>>>>> _______________________________________________
> >>>>>> rsyslog mailing list
> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>> http://www.rsyslog.com/professional-services/
> >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> >>>>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >>> DON'T LIKE THAT.
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com/professional-services/
> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> >>>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> >>>>> sites
> >>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> >>> LIKE THAT.
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com/professional-services/
> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> >>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> >>>> sites
> >>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> >>> LIKE THAT.
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com/professional-services/
> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
> >>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> >>>> sites
> >>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> >>> LIKE THAT.
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >> DON'T LIKE THAT.
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> > LIKE THAT.
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
