Hi Konstantinos, Thank you for the reply and for taking the time to look at the JSON parser code.
Ok. As you suggested, since I'm using custom scripts, changing their output to the Influx Line Protocol format is possible. And it seems easy too. Thank you for your time! Best, Damião On Wed, Aug 3, 2016 at 6:20 PM, Konstantinos Botsas <[email protected]> wrote: > Hi Damião, > > After looking at the code of Telegraf's JSON parser here: > > https://github.com/influxdata/telegraf/blob/master/plugins/parsers/json/parser.go#L20 > > if looks like the parser will attempt to unmarshall JSON input and return > a single metric. > > Performance and complexity concerns related to flattening JSON have been > the main reason that the JSON write path was deprecated in InfluxDB. > My understanding is that you are working on a custom script to output data > points, so I strongly encourage you to use Influx Line Protocol instead of > the JSON format. > > Thanks, > Kostas > > On Tue, Aug 2, 2016 at 8:26 PM, Damião Rodrigues <[email protected]> > wrote: > >> Hi Konstantinos, >> >> Thanks for the reply. Ok, that behavior fits what I've observed. Your >> first hunch was correct: it was a permission issue: telegraf was run by a >> user 'nobody', set by the s6-setuidgid program >> <http://skarnet.org/software/s6/s6-setuidgid.html>. >> >> By the way - actually bit unrelated :D - afterwards I've found another >> problem: if my script outputs multiple json lines, i.e. something like >> >> ``` >> {"key": "some_key_1", "value": 28.214, "unit": "ms"} >> {"key": "some_key_2", "value": 28.214, "unit": "ms"} >> ``` >> >> input.exec complains about an invalid format. This makes sense, since >> it's not actually correct json formatting. If I then try to output a more >> correct output, e.g.: >> >> ``` >> [{"key": "some_key_1", "value": 28.214, "unit": "ms"}, {"key": >> "some_key_2", "value": 28.214, "unit": "ms"}] >> ``` >> >> input.exec complains again with a de-marshaling error. What is the >> correct way to format multiple measurements out of a script monitored by >> input.exec? >> >> Thanks! >> >> Best regards, >> Damião >> >> On Tue, Aug 2, 2016 at 2:01 PM, Konstantinos Botsas <[email protected]> >> wrote: >> >>> Hi Damião, >>> >>> If your script "/home/myscript.py" does not complete before the timeout >>> that is configured in [inputs.exec] , an error "[exec: exit status 1 for >>> command X]" will be logged. >>> Additionally, the exec script should complete before the collection >>> interval is reached. The default interval is 10s and it is configurable >>> globally under the [agent] section as well as per-plugin. >>> >>> On Tue, Aug 2, 2016 at 2:51 PM, Damião Rodrigues <[email protected]> >>> wrote: >>> >>>> Hi kostas, >>>> >>>> Thanks for your reply. >>>> >>>> I have given the script full permissions (`chmod 777`) and the problem >>>> remains. The telegraf version I'm running is 'Telegraf - version 0.13.1'. >>>> >>>> The weirdest thing is I just tried the same with some dummy python >>>> script: >>>> >>>> ``` >>>> #!/usr/bin/env python >>>> import json >>>> from datetime import datetime >>>> dict_list = [] >>>> >>>> n = 66.666 >>>> s = datetime.now().second >>>> x = "boing" >>>> dict_list.append(dict(seconds=s,order=n,description=x)) >>>> >>>> print(json.dumps(dict_list[0])) >>>> ``` >>>> >>>> telegraf.conf: >>>> ``` >>>> [[inputs.exec]] >>>> command = "/home/dummy.py" >>>> data_format = "json" >>>> ##name_suffix = "_dummy" >>>> name_override = 'dummy' >>>> timeout = "10s" >>>> tag_keys = ['description'] >>>> ``` >>>> >>>> ... and it works perfectly! >>>> >>>> I've noticed that while the output of dummy.py is printed immediately >>>> to the stdout, the one from my_script takes a few seconds to show up. Could >>>> this be the problem? Doesn't the 'timeout' config handle this? >>>> >>>> >>>> On Tue, Aug 2, 2016 at 1:42 PM, <[email protected]> wrote: >>>> >>>>> On Tuesday, August 2, 2016 at 1:06:06 PM UTC+3, [email protected] >>>>> wrote: >>>>> > Hi all, >>>>> > >>>>> > I'm trying to capture the json output of a python script using the >>>>> Exec Input Plugin. For some reason, I'm getting the following error when >>>>> telegraf tries to execute the script and collect the metrics: >>>>> > >>>>> > ``` >>>>> > telegraf_1 | 2016-08-02T09:43:00.360767123Z 2016/08/02 09:43:00 >>>>> ERROR in input [exec]: exec: exit status 1 for command '/home/myscript.py' >>>>> > ``` >>>>> > >>>>> > The relevant entry of telegraf.conf file loos like the following: >>>>> > >>>>> > ``` >>>>> > [[inputs.exec]] >>>>> > command = "/home/myscript.py" >>>>> > data_format = "json" >>>>> > name_suffix = "_some_suffix" >>>>> > timeout = "10s" >>>>> > tag_keys = ["key", "unit"] >>>>> > ``` >>>>> > >>>>> > /home/myscript.py has execution permissions ('chmod +x') and outputs >>>>> the following: >>>>> > >>>>> > ``` >>>>> > {"key": "some_key", "value": 28.214, "unit": "ms"} >>>>> > >>>>> > ``` >>>>> > >>>>> > I've checked that the exit status of running `/home/myscript.py` is >>>>> 0. >>>>> > >>>>> > >>>>> > The weirdest part is that, if I run `$ telegraf -config >>>>> /etc/telegraf.conf` -test`, I get a successful output: >>>>> > >>>>> > ``` >>>>> > * Plugin: exec, Collection 1 >>>>> > > exec_some_suffix,host=13ceb7312093,key=some_key,unit=ms >>>>> value=19.539 1470132272694745614 >>>>> > ``` >>>>> > >>>>> > Do you have any idea of what could be happening? >>>>> >>>>> Hi, >>>>> >>>>> I have not been able to reproduce this behavior with the latest beta >>>>> of Telegraf. >>>>> >>>>> It is likely that when you execute telegraf by invoking the binary >>>>> from the command line (telegraf -config /etc/telegraf.conf) you are >>>>> running >>>>> it as your user who has privileges to execute the python script, while the >>>>> telegraf service runs as user "telegraf", which may not have sufficient >>>>> privileges to access /home/myscript.py >>>>> >>>>> >>>> >>> >>> >>> -- >>> Konstantinos Botsas >>> Support Engineer >>> InfluxData >>> >> >> > > > -- > Konstantinos Botsas > Support Engineer > InfluxData > -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CAF8xhAu4f2WERKmWimmpksm_ek_WY63JLGg_9Qf7%3Dn0h-6nuFw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
