On Sat, 23 May 2015, singh.janmejay wrote:
Actually for contains, I was thinking of function rather than keyword.
I was proposing we leave the already supported keyword alone, but
don't add anymore, and develop more things in the form of functions as
opposed to keywords, so that it doesn't complicate the language too
much (Im thinking from the pov of a new person trying to learn this,
we can make stdlib very large, no one really worries about it, but if
we have too many keywords, it's going to scare people away). I know
stdlib is a little bit of a stretch, but closest analogy that comes to
my mind right now.
So I meant 'contains' the function, not the keyword.
Or may be we can use has(array, elem) or include(array, elem) or
hasItem(array, elem) etc to avoid confusing people by having similarly
named keyword and function.
we shouldn't use the same string for functions and keywords.
The problem with functions is that the relationship of the various parameters is
not nearly as obvious
if you have lessthan(a,b), which needs to be less than the other in order for
it to be true? I know which way I would default to, but I've been bit by others
making other assumptions
for isin(a,b), is this asking if a contains b or if b contains a?
but if a[] contains b is unambiguous.
So if something can be expressed cleanly with keywords, they should be used
instead of functions.
having lots of functions to know about isn't any better than the same number of
keywords.
David Lang
Thoughts?
On Sat, May 23, 2015 at 12:04 AM, singh.janmejay
<[email protected]> wrote:
Ok, we'll have to get into a little bit of code here, I know no better
way to explain why it doesn't work for you, my apologies.
This is the form that the array that you built is loaded to:
https://github.com/rsyslog/rsyslog/blob/master/grammar/grammar.y#L215
We actually convert cnfarray to first element of that array for
expr-evaluation:
https://github.com/rsyslog/rsyslog/blob/master/grammar/rainerscript.c#L2314
Beyond this, all code assumes no cnfarray:
https://github.com/rsyslog/rsyslog/blob/master/runtime/msg.c#L4558
So cnfarray was actually meant for a different purpose, I haven't yet
dug into what that purpose is (@Rainer/David help?).
But its definitely not equal to json-array, which is why your loop doesn't work.
So in order to loop, you need a json array, which generally comes from
mmjsonparse or mmnormalize generated object-graph.
Take a look at the test in the PR I linked-to in this thread. That
actually loops over an array and decides if it should stop processing
a message depending on presence of a certain value in an array. That
should serve as a working example of how to use foreach, except you'll
probably have something useful in place of stop.
On Fri, May 22, 2015 at 11:12 PM, David Lang <[email protected]> wrote:
On Fri, 22 May 2015, David Boles (dboles) wrote:
Yes, it is a string. Taking your clarification that foreach operates on
arrays, I've tried the config below. When I push a log message from an
application I observe:
- The action with the msg1 template occurs
- The action with the msg2 template does not occur
- The action with the msg3 template occurs
What should be changed below to cause the loop to be traversed?
Thanks,
David Boles
--------
module(load="imuxsock") # local system logging support
module(load="imkmsg") # structured kernel logging support
module(load="mmjsonparse")
template(name="msg1" type="list") { constant(value="pre-foreach record\n")
}
template(name="msg2" type="list") { constant(value=" foreach record\n")
}
template(name="msg3" type="list") { constant(value="post-foreach
record\n") }
if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern') then {
action(type="mmjsonparse")
if $parsesuccess == "OK" then {
action(type="omfile" template="msg1"
file="/var/log/db_local_user_structured.log")
set $.myarray = [ "0", "1", "2", "3", "4", "5" ];
set $.index = 0;
there is no need for $.index = 0; it's not a counter, it just gets assigned
to each of the contents of myarray
foreach ($.index in $.myarray) do {
action(type="omfile" template="msg2"
file="/var/log/db_local_user_structured.log")
}
this action should happen 6 times as I understand it.
action(type="omfile" template="msg3"
file="/var/log/db_local_user_structured.log")
}
}
--------
having multiple action writing to the same file can result in some
interesting issues (at least conceptually, I'll let Rainer speak up if they
have been solved in practice)
what I would do is
$template custom,"%$.custom%"
ruleset(name="db_local_user_structured"){
action(type="omfile" template="custom"
file="/var/log/db_local_user_structured.log")
}
and then replace each of the actions in your example with
set $.custom = "message";
call custom
David Lang
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of singh.janmejay
Sent: Thursday, May 21, 2015 10:43 AM
To: rsyslog-users
Subject: Re: [rsyslog] Unable to use foreach
I think $!mse.element_indices is a string, right?
If its a string, you'll need to parse it to make it an array before you
can loop on it. Foreach works only with arrays. It can be array or anything
(string, object, numbers whatever), but it has to be an array.
You can use tokenized field-type to parse it and since you are already
using mmjsonparse it shouldn't be a problem.
On Thu, May 21, 2015 at 8:47 PM, David Boles (dboles) <[email protected]>
wrote:
Hi,
I am using rsyslog (v8.9) to process structured log data from umberlog
and Linux's printk_emit. In the log message is a field
"$!mse.element_indices" that can have values such as "0", "0 1", "0 1 2",
and so on. I would like to iterate over the delimited elements of that value
and had supposed that foreach would do something like that.
With the config below I comment/uncomment the foreach loop. When the
foreach loop is commented out rsyslog creates entries in both mongodb and
the file. When the foreach loop is uncommented, rsyslog produces nothing in
either destination.
Why does this use of foreach fail? What should I be doing to iterate?
Thanks,
David Boles
----------------------------------------------------------------------
--------
module(load="imuxsock")
module(load="imkmsg")
module(load="imtcp")
input(type="imtcp" port="10514")
module(load="mmjsonparse")
module(load="ommongodb")
kern.* /var/log/db_kernel.log
*.* /var/log/db_full.log
template(name="mse-structured-info" type="subtree" subtree="$!")
template(name="mse-all-info" type="list") {
property(name="jsonmesg" outname="msg") }
if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern') then
{
action(type="mmjsonparse")
if $parsesuccess == "OK" then {
set $!foo = $!mse.element_indices;
set $!amph = "toad";
# foreach ($.index in $!mse.element_indices) do {
# set $!amph = "turtle";
# }
action(type="ommongodb" server="somemachine.somewhere.org"
db="logs" collection="syslog"
template="mse-structured-info")
action(type="omfile" template="mse-all-info"
file="/var/log/db_local_user_structured.log")
}
}
$WorkDirectory /var/spool/rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL:
This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
