On Tue, 16 Jul 2013, Rainer Gerhards wrote:
Couple of points:
$!all-json is a legacy property. It is better to access native json pathes.
Overall idea: to handle cases like this, the idea is that the data to be
sent is put into differet subtress and a subtree template [1] is used for
forwarding. IIRC, I did a blog post about this, but I have to admit I
didn't find it on quick search :-(.
the problem is similar to the issue I raised about trusted properties, how do
you know what you are dealing with.
It may be worth defining _rsyslog_local!* as a tree that is not included in
all-json so that people can define whatever they want (and I believe that 'real'
json trees cannot start with aa '_', so it wouldn't even be a namespace
conflict, assuming that rsyslog can tolerate it)
is the blog post you are thinking of
http://blog.gerhards.net/2012/03/json-and-rsyslog-templates.html
David Lang
As David said, this has not been discussed much so far. Out of my head, I
don't even know if all pieces have the necessary plumbing (e.g. the parsers
need to be able to store their parsing result into a specific subtree in
order for the whole idea to make sense). If it's just a small thing that is
missing, I may be able to add it before I depart for vacation. But please
ask for this only if you intend to actually deploy and test it (non-stable
versions are often not supported in enterprise envs), as I would like to
save my limited pre-vacation time to things that really make a difference ;)
Rainer
[1] http://www.rsyslog.com/doc/rsyslog_conf_templates.html
On Tue, Jul 16, 2013 at 8:07 AM, David Lang <[email protected]> wrote:
On Mon, 15 Jul 2013, Erik Steffl wrote:
On 07/15/2013 09:37 PM, David Lang wrote:
On Mon, 15 Jul 2013, Erik Steffl wrote:
Here's what I'm trying to do:
receive cee message, put it into a json data structure and forward
the whole thing to another server.
input: <135> time host tag @cee{"orignal":"message"}
output: <135> time host tag @cee{"cluster":"someName", "message":
{"original":"message"}}
So what I did was:
set $!myClusterName = "someName";
then use these as part of the template:
constant(value="\"cluster\":{"**)
property("$!myClusterName")
...
constant(value="\"message\":")
property("$!all-json")
and it almost works but $!myClusterName becomes part of the
$!all-json. Looked at docs but it seems all variables that are set
using set keyword become part of $!all-json.
using property("msg") also sort of works but it includes @cee (which
I don't think makes sense in that part of message, it's already all
json).
Any way to get $!all-json without the custom variables or "msg"
property without @cee? Or should I be forwarding it in some entirely
different way?
what if you just do something like:
set $!cluster = "somename";
won't this include the 'cluster:"somename"' string inside the all-json
string?
yes it will but I want exact opposite :) I am trying to not change the
original message (it might have a field named "cluster" already) or it
might not even be a cee json message etc. (I can figure out whether the
message is json formatted or not but I have to set the variables before
that since I need them no matter what the format it)
trying to achieve something like (as described above but this might be
more obvious), pseudo code follows:
set $!message = $!all-json
set $!cluster = "clusterName"
after the two lines above the $!all-json has two fields - "message"
where the value is original $!all-json and cluster, where the value is my
variable (which of course does not work cause I (as far as I know) don't
have a way to set the variable without changing $all-json.
But only if the message is a cee message (otherwise it, of course does
not make sense). If it's not cee message then I still need cluster name but
$!all-json!message is just the original message string.
I.e. I am not trying to add info to the parsed message (that works,
discovered that during my failed attempts), I would like the intact parsed
message to become part of outgoing message.
Does that explain what I'm trying to do? Any ideas how to achieve that?
Ok, that's a little simpler than what I was thinking you were doing.
As far as I know, this is something that rsyslog does not currently handle
well, we need some way to blacklist items so they don't appear in all-json
without deleteing them entirely. There has occasionally be talk about
implementing more control along these lines, but there hasn't been enough
discussion to find good examples and people needing this control to discuss
possible syntax options with.
In the short term, I think the way to do what you are trying to do is to
output in two formats, depending on if the json parse was successful (there
is a variable set that says if the parse found json to parse), one where
you output all-json, the other where you format your own string, just
including the message body.
I'm not understanding why you want the output to always be in json format,
but only include the variable if the input was json, I would have thought
you always wanted to have the variable there.
If you are concerned about overwriting an existing variable, then what you
need to do is to reserve a subtree, and if you find that the subtree
exists, copy the contents of the subtree to subtree!old and then put your
variable in subtree!variable (taking advantage of the fact that json is a
tree structure, not just a name=variable flat structure.
David Lang
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
