Cool, will add doc soon.
I see what you are saying about functions, but keywords make the
language's core harder to grasp, which affects language's learning
curve. Functions are very different, because every-time a user
encounters one, they know its just another procedure/function which
will do something or return something. To be able to use keywords
effectively, one needs to know very well where they fit in the
language's constructs. For functions, they already know that.
But I do see the point you are making about arg order.
How about this: In addition to current function-calling-style, we
support python or R like calling style too, so people can optionally
mention argument names?
Another option is to use object-oriented style:
$.array.includes("foo") etc. We can make this work similar to how
python or c++ do, first argument is assumed to variable(in our case
expression) the function is dispatched upon. It'll not solve all
problems, but it'll do a brilliant job of disambiguating 2-arg
functions, while keeping language core lean.
On Sat, May 23, 2015 at 1:06 AM, David Lang <[email protected]> wrote:
> On Sat, 23 May 2015, singh.janmejay wrote:
>
>> Oh, just recalled you mentioned documentation of array-subscripting.
>> Im unsure where to put it.
>>
>> I am not sure if I documented it at all at the time of implementing
>> it, because I just didn't know where to put it.
>>
>> Suggestions?
>
>
> I would put it in two places.
>
> 1. in the section that talks about variables
>
> 2. as an example in the section that talks about tests
>
> David Lang
>
>
>> On Sat, May 23, 2015 at 12:13 AM, singh.janmejay
>> <[email protected]> wrote:
>>>
>>> Actually for contains, I was thinking of function rather than keyword.
>>>
>>> I was proposing we leave the already supported keyword alone, but
>>> don't add anymore, and develop more things in the form of functions as
>>> opposed to keywords, so that it doesn't complicate the language too
>>> much (Im thinking from the pov of a new person trying to learn this,
>>> we can make stdlib very large, no one really worries about it, but if
>>> we have too many keywords, it's going to scare people away). I know
>>> stdlib is a little bit of a stretch, but closest analogy that comes to
>>> my mind right now.
>>>
>>> So I meant 'contains' the function, not the keyword.
>>>
>>> Or may be we can use has(array, elem) or include(array, elem) or
>>> hasItem(array, elem) etc to avoid confusing people by having similarly
>>> named keyword and function.
>>>
>>> Thoughts?
>>>
>>> On Sat, May 23, 2015 at 12:04 AM, singh.janmejay
>>> <[email protected]> wrote:
>>>>
>>>> Ok, we'll have to get into a little bit of code here, I know no better
>>>> way to explain why it doesn't work for you, my apologies.
>>>>
>>>> This is the form that the array that you built is loaded to:
>>>> https://github.com/rsyslog/rsyslog/blob/master/grammar/grammar.y#L215
>>>>
>>>> We actually convert cnfarray to first element of that array for
>>>> expr-evaluation:
>>>>
>>>> https://github.com/rsyslog/rsyslog/blob/master/grammar/rainerscript.c#L2314
>>>>
>>>> Beyond this, all code assumes no cnfarray:
>>>> https://github.com/rsyslog/rsyslog/blob/master/runtime/msg.c#L4558
>>>>
>>>> So cnfarray was actually meant for a different purpose, I haven't yet
>>>> dug into what that purpose is (@Rainer/David help?).
>>>>
>>>> But its definitely not equal to json-array, which is why your loop
>>>> doesn't work.
>>>>
>>>> So in order to loop, you need a json array, which generally comes from
>>>> mmjsonparse or mmnormalize generated object-graph.
>>>>
>>>> Take a look at the test in the PR I linked-to in this thread. That
>>>> actually loops over an array and decides if it should stop processing
>>>> a message depending on presence of a certain value in an array. That
>>>> should serve as a working example of how to use foreach, except you'll
>>>> probably have something useful in place of stop.
>>>>
>>>>
>>>> On Fri, May 22, 2015 at 11:12 PM, David Lang <[email protected]> wrote:
>>>>>
>>>>> On Fri, 22 May 2015, David Boles (dboles) wrote:
>>>>>
>>>>>> Yes, it is a string. Taking your clarification that foreach operates
>>>>>> on
>>>>>> arrays, I've tried the config below. When I push a log message from an
>>>>>> application I observe:
>>>>>>
>>>>>> - The action with the msg1 template occurs
>>>>>> - The action with the msg2 template does not occur
>>>>>> - The action with the msg3 template occurs
>>>>>>
>>>>>> What should be changed below to cause the loop to be traversed?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> David Boles
>>>>>>
>>>>>> --------
>>>>>> module(load="imuxsock") # local system logging support
>>>>>> module(load="imkmsg") # structured kernel logging support
>>>>>>
>>>>>> module(load="mmjsonparse")
>>>>>>
>>>>>> template(name="msg1" type="list") { constant(value="pre-foreach
>>>>>> record\n")
>>>>>> }
>>>>>> template(name="msg2" type="list") { constant(value=" foreach
>>>>>> record\n")
>>>>>> }
>>>>>> template(name="msg3" type="list") { constant(value="post-foreach
>>>>>> record\n") }
>>>>>>
>>>>>> if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern')
>>>>>> then {
>>>>>> action(type="mmjsonparse")
>>>>>> if $parsesuccess == "OK" then {
>>>>>>
>>>>>> action(type="omfile" template="msg1"
>>>>>> file="/var/log/db_local_user_structured.log")
>>>>>>
>>>>>> set $.myarray = [ "0", "1", "2", "3", "4", "5" ];
>>>>>> set $.index = 0;
>>>>>
>>>>>
>>>>>
>>>>> there is no need for $.index = 0; it's not a counter, it just gets
>>>>> assigned
>>>>> to each of the contents of myarray
>>>>>
>>>>>> foreach ($.index in $.myarray) do {
>>>>>> action(type="omfile" template="msg2"
>>>>>> file="/var/log/db_local_user_structured.log")
>>>>>> }
>>>>>
>>>>>
>>>>>
>>>>> this action should happen 6 times as I understand it.
>>>>>
>>>>>> action(type="omfile" template="msg3"
>>>>>> file="/var/log/db_local_user_structured.log")
>>>>>> }
>>>>>> }
>>>>>> --------
>>>>>
>>>>>
>>>>>
>>>>> having multiple action writing to the same file can result in some
>>>>> interesting issues (at least conceptually, I'll let Rainer speak up if
>>>>> they
>>>>> have been solved in practice)
>>>>>
>>>>> what I would do is
>>>>>
>>>>> $template custom,"%$.custom%"
>>>>>
>>>>> ruleset(name="db_local_user_structured"){
>>>>> action(type="omfile" template="custom"
>>>>> file="/var/log/db_local_user_structured.log")
>>>>> }
>>>>>
>>>>> and then replace each of the actions in your example with
>>>>>
>>>>> set $.custom = "message";
>>>>> call custom
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: [email protected]
>>>>>> [mailto:[email protected]] On Behalf Of singh.janmejay
>>>>>> Sent: Thursday, May 21, 2015 10:43 AM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] Unable to use foreach
>>>>>>
>>>>>> I think $!mse.element_indices is a string, right?
>>>>>>
>>>>>> If its a string, you'll need to parse it to make it an array before
>>>>>> you
>>>>>> can loop on it. Foreach works only with arrays. It can be array or
>>>>>> anything
>>>>>> (string, object, numbers whatever), but it has to be an array.
>>>>>>
>>>>>> You can use tokenized field-type to parse it and since you are already
>>>>>> using mmjsonparse it shouldn't be a problem.
>>>>>>
>>>>>> On Thu, May 21, 2015 at 8:47 PM, David Boles (dboles)
>>>>>> <[email protected]>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am using rsyslog (v8.9) to process structured log data from
>>>>>>> umberlog
>>>>>>> and Linux's printk_emit. In the log message is a field
>>>>>>> "$!mse.element_indices" that can have values such as "0", "0 1", "0 1
>>>>>>> 2",
>>>>>>> and so on. I would like to iterate over the delimited elements of
>>>>>>> that value
>>>>>>> and had supposed that foreach would do something like that.
>>>>>>>
>>>>>>> With the config below I comment/uncomment the foreach loop. When the
>>>>>>> foreach loop is commented out rsyslog creates entries in both mongodb
>>>>>>> and
>>>>>>> the file. When the foreach loop is uncommented, rsyslog produces
>>>>>>> nothing in
>>>>>>> either destination.
>>>>>>>
>>>>>>> Why does this use of foreach fail? What should I be doing to iterate?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> David Boles
>>>>>>>
>>>>>>>
>>>>>>> ----------------------------------------------------------------------
>>>>>>> --------
>>>>>>>
>>>>>>> module(load="imuxsock")
>>>>>>> module(load="imkmsg")
>>>>>>> module(load="imtcp")
>>>>>>> input(type="imtcp" port="10514")
>>>>>>> module(load="mmjsonparse")
>>>>>>> module(load="ommongodb")
>>>>>>>
>>>>>>> kern.* /var/log/db_kernel.log
>>>>>>> *.* /var/log/db_full.log
>>>>>>>
>>>>>>> template(name="mse-structured-info" type="subtree" subtree="$!")
>>>>>>>
>>>>>>> template(name="mse-all-info" type="list") {
>>>>>>> property(name="jsonmesg" outname="msg") }
>>>>>>>
>>>>>>> if ($fromhost-ip == '127.0.0.1' and $syslogfacility-text != 'kern')
>>>>>>> then
>>>>>>> {
>>>>>>> action(type="mmjsonparse")
>>>>>>> if $parsesuccess == "OK" then {
>>>>>>> set $!foo = $!mse.element_indices;
>>>>>>>
>>>>>>> set $!amph = "toad";
>>>>>>> # foreach ($.index in $!mse.element_indices) do {
>>>>>>> # set $!amph = "turtle";
>>>>>>> # }
>>>>>>> action(type="ommongodb" server="somemachine.somewhere.org"
>>>>>>> db="logs" collection="syslog"
>>>>>>> template="mse-structured-info")
>>>>>>>
>>>>>>> action(type="omfile" template="mse-all-info"
>>>>>>> file="/var/log/db_local_user_structured.log")
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> $WorkDirectory /var/spool/rsyslog
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com/professional-services/
>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
>>>>>>> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>>>>>> of
>>>>>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>>>>> DON'T
>>>>>>> LIKE THAT.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Janmejay
>>>>>> http://codehunk.wordpress.com
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com/professional-services/
>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
>>>>>> WELL:
>>>>>> This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
>>>>>> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>>>>>> LIKE
>>>>>> THAT.
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com/professional-services/
>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>>>>> myriad
>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>>>> DON'T
>>>>>> LIKE THAT.
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com/professional-services/
>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>>>> myriad of
>>>>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>>> DON'T
>>>>> LIKE THAT.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Janmejay
>>>> http://codehunk.wordpress.com
>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Janmejay
>>> http://codehunk.wordpress.com
>>
>>
>>
>>
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
--
Regards,
Janmejay
http://codehunk.wordpress.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
