On Fri, 2 Dec 2016, Bob Gregory wrote:

Evening all,

I've mostly finished my last personal project, so my thoughts are turning
to omriemann.

I'm trying to work out how we might configure the module. Riemann requires
that we send a protobuf encoded message containing a few pre-set fields,
plus whatever additional fields we feel like forwarding.

host: localhost
service: cpu-load-average/1m
state: ok
time: 1480661786
description: "everything is perfectly fine"
tags: ["laptop", "personal"]
metric: 0.58
ttl: 120
my-custom-field: 27

This makes it unusual for an rsyslog module: usually rsyslog is happy to
ship arbitrary strings to a destination and only cares about the _framing_
of your data: omelasticsearch, ommysql, omkafka, omrelp etc. all accept
some number of static parameters, plus a free-form template for the actual
message.

Omriemann, in order to be useful, will need to impose some structure on the
message itself.

How do people think we should configure the module so that people have
flexibility over the host, metric value, metric name, and tags on a
per-message basis?

use a parameter to pass the variable name to use for the field, and have a default if they aren't set.

Also, think hard about the need to set them on a per-message basis.

I guess the simplest thing that could possibly work is defining a simple
message format, eg. `host=foo; metric_f=0.6;
service=rsyslog.impstats/utime; timestamp=1480661786` that messages need to
conform to. We can then parse out the key/value pairs in the module and
encode them to protobuf.

no, that way lies madness (I did something very similar in the first iteration of omudpspoof, but in my defense that was before we had the action() cal)

Alternatively, we could set up the structure of the message in the config
itself, like this:

action(
  type="omriemann"
  host="$hostname"
  metric="$!metric.value"
  service="$!metric.name")

That seems more user-friendly, but rules out using custom fields. I guess
I'd have to create a new template per-field during module begin.

this is the right approach for the fixed fields. For defining custom fields, can you accept a JSON structure and do the right thing?

Given that the protobuf needs to be pre-defined and exist on both sides, how much flexibility do you really have?

On a related note, I think I remember seeing some discussion of conversion
functions recently. Some of the fields need to valid integers, floats, unix
timestamps etc. What's the best way of parsing those out?

you will be passed strings [1] and need to validate them (and figure out what to do if you are passed garbage)

[1] timestamps are a possible exception to this.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to