Couple of points: $!all-json is a legacy property. It is better to access native json pathes.
Overall idea: to handle cases like this, the idea is that the data to be sent is put into differet subtress and a subtree template [1] is used for forwarding. IIRC, I did a blog post about this, but I have to admit I didn't find it on quick search :-(. As David said, this has not been discussed much so far. Out of my head, I don't even know if all pieces have the necessary plumbing (e.g. the parsers need to be able to store their parsing result into a specific subtree in order for the whole idea to make sense). If it's just a small thing that is missing, I may be able to add it before I depart for vacation. But please ask for this only if you intend to actually deploy and test it (non-stable versions are often not supported in enterprise envs), as I would like to save my limited pre-vacation time to things that really make a difference ;) Rainer [1] http://www.rsyslog.com/doc/rsyslog_conf_templates.html On Tue, Jul 16, 2013 at 8:07 AM, David Lang <[email protected]> wrote: > On Mon, 15 Jul 2013, Erik Steffl wrote: > > On 07/15/2013 09:37 PM, David Lang wrote: >> >>> On Mon, 15 Jul 2013, Erik Steffl wrote: >>> >>> Here's what I'm trying to do: >>>> >>>> receive cee message, put it into a json data structure and forward >>>> the whole thing to another server. >>>> >>>> input: <135> time host tag @cee{"orignal":"message"} >>>> >>>> output: <135> time host tag @cee{"cluster":"someName", "message": >>>> {"original":"message"}} >>>> >>>> So what I did was: >>>> >>>> set $!myClusterName = "someName"; >>>> >>>> then use these as part of the template: >>>> >>>> constant(value="\"cluster\":{"**) >>>> property("$!myClusterName") >>>> ... >>>> constant(value="\"message\":") >>>> property("$!all-json") >>>> >>>> and it almost works but $!myClusterName becomes part of the >>>> $!all-json. Looked at docs but it seems all variables that are set >>>> using set keyword become part of $!all-json. >>>> >>>> using property("msg") also sort of works but it includes @cee (which >>>> I don't think makes sense in that part of message, it's already all >>>> json). >>>> >>>> Any way to get $!all-json without the custom variables or "msg" >>>> property without @cee? Or should I be forwarding it in some entirely >>>> different way? >>>> >>> >>> what if you just do something like: >>> >>> set $!cluster = "somename"; >>> >>> won't this include the 'cluster:"somename"' string inside the all-json >>> string? >>> >> >> yes it will but I want exact opposite :) I am trying to not change the >> original message (it might have a field named "cluster" already) or it >> might not even be a cee json message etc. (I can figure out whether the >> message is json formatted or not but I have to set the variables before >> that since I need them no matter what the format it) >> >> trying to achieve something like (as described above but this might be >> more obvious), pseudo code follows: >> >> set $!message = $!all-json >> set $!cluster = "clusterName" >> >> after the two lines above the $!all-json has two fields - "message" >> where the value is original $!all-json and cluster, where the value is my >> variable (which of course does not work cause I (as far as I know) don't >> have a way to set the variable without changing $all-json. >> >> But only if the message is a cee message (otherwise it, of course does >> not make sense). If it's not cee message then I still need cluster name but >> $!all-json!message is just the original message string. >> >> I.e. I am not trying to add info to the parsed message (that works, >> discovered that during my failed attempts), I would like the intact parsed >> message to become part of outgoing message. >> >> Does that explain what I'm trying to do? Any ideas how to achieve that? >> > > Ok, that's a little simpler than what I was thinking you were doing. > > As far as I know, this is something that rsyslog does not currently handle > well, we need some way to blacklist items so they don't appear in all-json > without deleteing them entirely. There has occasionally be talk about > implementing more control along these lines, but there hasn't been enough > discussion to find good examples and people needing this control to discuss > possible syntax options with. > > > In the short term, I think the way to do what you are trying to do is to > output in two formats, depending on if the json parse was successful (there > is a variable set that says if the parse found json to parse), one where > you output all-json, the other where you format your own string, just > including the message body. > > I'm not understanding why you want the output to always be in json format, > but only include the variable if the input was json, I would have thought > you always wanted to have the variable there. > > If you are concerned about overwriting an existing variable, then what you > need to do is to reserve a subtree, and if you find that the subtree > exists, copy the contents of the subtree to subtree!old and then put your > variable in subtree!variable (taking advantage of the fact that json is a > tree structure, not just a name=variable flat structure. > > David Lang > > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
