What about wrapDynamicObjects="on|off"? That is required regardless, right? (if we want to preserve backward compatibility). Unless we choose to change it anyway (which im fine with).
@Rainer: what do you think about foreach handling object? On Wed, Mar 23, 2016 at 1:01 AM, David Lang <[email protected]> wrote: > On Wed, 23 Mar 2016, singh.janmejay wrote: > >> Foreach can only work with arrays as of now. It can't work with >> objects (key-value pairs). So [{name: ... value: ...}, {..},...] is >> the only format that will work as of now. >> >> We can enhance foreach to work with objects. >> >> I can make a flag available at dyn-stats bucket level, which can >> control serialization format, but that would really be a hack. >> >> From single-responsibility pattern pov, impstats should be the only >> component that decides how to layout data for user to see. >> >> How about this: >> >> impstats(format="json" wrapDynamicObjects="on"...)? >> >> It defaults to off, which keeps backward compatibility. >> >> So what do you guys think about: >> - wrapDynamicObjects="on|off" >> - generating [{name: a, value: 10}, {...} ...] vs. {a: 10, ...} >> (foreach will handle the former out of the box, but later is concise, >> readable and light-weight in addition to being more json-y. >> - enhancing foreach to work with {a: 10, b: 20...} > > > If we can enhance foreach to work with the concise format, I would rather > wait for it instead of introducing the wrapping version. > > I'm thinking that foreach walks through arrays, rather than mixing concepts, > a foreachobject that gives us a name and contents for a {} list of objects > may be the right thing to do? > > foreach just returns a single object while foreachobject needs to return the > object and name. > > although, if we ever get the ability to address arrays directly, being able > to look at the array position would be the equivalent of the name. > > David Lang > > > >> On Wed, Mar 23, 2016 at 12:26 AM, David Lang <[email protected]> wrote: >>> >>> On Tue, 22 Mar 2016, singh.janmejay wrote: >>> >>>> How about this: we support a new flag in impstats which allows >>>> json-formatted stats-line to optionally use >>>> encapsulated/wrapped-layout? >>>> >>>> impstats(format="json" ...) generates >>>> >>>> >>>> {"name":"msg_per_host","origin":"dynstats.bucket","z-scribe1r-b":80,"SWEB10":67} >>>> >>>> however, >>>> >>>> impstats(format="json/w" ...) generates {"header": >>>> {"name":"msg_per_host","origin":"dynstats.bucket"}, "counters" : >>>> {"z-scribe1r-b":80,"SWEB10":67}} >>>> >>>> This is relevant, the serilization format we use right now mixes >>>> pre-defined keys with counter-names and it can affect regular static >>>> counters too. >>> >>> >>> >>> with the existing pstats output, there is no ability for user-defined >>> data >>> to become a tag name, so there is no potential for ambiguity. but with >>> dynastats, this is not a possibility, and the format we use should >>> prevent >>> problems. >>> >>> Also, just from a conceptual point of view, why should the bucket >>> contents >>> be at the same level as the bucket name? >>> >>> other than backwards compatibility, what advantage is there of the >>> current >>> version in JSON? the documentation uses the plain text equivalant, which >>> is >>> perfectly legitimate because there is an order to the line, and after you >>> get past the name and origin, everything else on the line is name-value >>> pairs of counters, again, no ambiguity. >>> >>> But with JSON, I don't believe that you can depend on tools maintaining >>> (or >>> even identifying) the order of the elements, and if you have multiple >>> elements with the same name, it's implementation dependent as to which >>> one >>> will be seen. >>> >>> So purely from a correctness and defensive programming point of view, I >>> think the current JSON serialization should be changed, with the old >>> format >>> no longer being an option. >>> >>> >>> >>> >>> As to the details of the new format >>> >>> What I'm wanting to do with the counters is something like >>> >>> if $!origin == "dynstats.bucket" then { >>> foreach $.tag $!counters { >>> /var/log/stats;format >>> } >>> } >>> >>> to output one line per counter. >>> >>> I'm very flexible in how to do this, but I would much rather be able to >>> do >>> this inside rsyslog than have to serialize things to an external script, >>> have it parse the json and process it. >>> >>> my initial thinking was just do >>> >>> counters: [ "z-scribe1r-b":80,"SWEB10":67 ] >>> >>> but as I'm typing this, I realize that doesn't work as I don't have a way >>> to >>> break $.tag down to reference the name and the value. >>> >>> I'd hate to have to do something like >>> >>> counters: [{"name":"z-scribe1r-b","value":80 >>> },{"name":"SWEB10","value":67}] >>> >>> this mirrors the misuse of XML that gives it such a horrible reputation. >>> But >>> unless we introduce some new function to rsyslog to break things down, I >>> don't see a better way. If we do need to do something like this, I sure >>> would not want to make it the default JSON, which would result in two >>> different formats. I hate the idea of starting to have different formats >>> because of subtypes of data (what is someone wants the cee version of >>> this >>> for example, you start to have orthoginal format decisions) >>> >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >> >> >> >> >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. -- Regards, Janmejay http://codehunk.wordpress.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
