[influxdb] [telegraf] exec input plugin not working with JSON

[a***@gmail.com]

Hi all,

I'm trying to capture the json output of a python script using the Exec Input Plugin. For some reason, I'm getting the following error when telegraf tries to execute the script and collect the metrics:

```
telegraf_1 | 2016-08-02T09:43:00.360767123Z 2016/08/02 09:43:00 ERROR in input [exec]: exec: exit status 1 for command '/home/myscript.py'
```

The relevant entry of telegraf.conf file loos like the following:

```
[[inputs.exec]]
command = "/home/myscript.py"
data_format = "json"
name_suffix = "_some_suffix"
timeout = "10s"
tag_keys = ["key", "unit"]
```

/home/myscript.py has execution permissions ('chmod +x') and outputs the following:

```
{"key": "some_key", "value": 28.214, "unit": "ms"}

```

I've checked that the exit status of running `/home/myscript.py` is 0.


The weirdest part is that, if I run `$ telegraf -config /etc/telegraf.conf` -test`, I get a successful output:

```
* Plugin: exec, Collection 1

exec_some_suffix,host=13ceb7312093,key=some_key,unit=ms value=19.539 1470132272694745614

```

Do you have any idea of what could be happening?

[Damião Rodrigues]

Hi kostas,

Thanks for your reply.

I have given the script full permissions (`chmod 777`) and the problem
remains. The telegraf version I'm running is 'Telegraf - version 0.13.1'.

The weirdest thing is I just tried the same with some dummy python script:

```
#!/usr/bin/env python
import json
from datetime import datetime
dict_list = []

n = 66.666
s = datetime.now().second
x = "boing"
dict_list.append(dict(seconds=s,order=n,description=x))

print(json.dumps(dict_list[0]))
```

telegraf.conf:
```
[[inputs.exec]]
command = "/home/dummy.py"
data_format = "json"
##name_suffix = "_dummy"
name_override = 'dummy'
timeout = "10s"
tag_keys = ['description']
```

... and it works perfectly!

I've noticed that while the output of dummy.py is printed immediately to
the stdout, the one from my_script takes a few seconds to show up. Could
this be the problem? Doesn't the 'timeout' config handle this?

Post by a***@gmail.com

Post by a***@gmail.com
Hi all,
I'm trying to capture the json output of a python script using the Exec

Input Plugin. For some reason, I'm getting the following error when

Post by a***@gmail.com
```
telegraf_1 | 2016-08-02T09:43:00.360767123Z 2016/08/02 09:43:00 ERROR

in input [exec]: exec: exit status 1 for command '/home/myscript.py'

Post by a***@gmail.com
```
```
[[inputs.exec]]
command = "/home/myscript.py"
data_format = "json"
name_suffix = "_some_suffix"
timeout = "10s"
tag_keys = ["key", "unit"]
```
/home/myscript.py has execution permissions ('chmod +x') and outputs the
```
{"key": "some_key", "value": 28.214, "unit": "ms"}
```
I've checked that the exit status of running `/home/myscript.py` is 0.
The weirdest part is that, if I run `$ telegraf -config
```
* Plugin: exec, Collection 1

exec_some_suffix,host=13ceb7312093,key=some_key,unit=ms value=19.539

1470132272694745614

Post by a***@gmail.com
```
Do you have any idea of what could be happening?

Hi,
I have not been able to reproduce this behavior with the latest beta of
Telegraf.
It is likely that when you execute telegraf by invoking the binary from
the command line (telegraf -config /etc/telegraf.conf) you are running it
as your user who has privileges to execute the python script, while the
telegraf service runs as user "telegraf", which may not have sufficient
privileges to access /home/myscript.py

[Damião Rodrigues]

Hi Konstantinos,

Thanks for the reply. Ok, that behavior fits what I've observed. Your first
hunch was correct: it was a permission issue: telegraf was run by a user
'nobody', set by the s6-setuidgid program
<http://skarnet.org/software/s6/s6-setuidgid.html>.

By the way - actually bit unrelated :D - afterwards I've found another
problem: if my script outputs multiple json lines, i.e. something like

```
{"key": "some_key_1", "value": 28.214, "unit": "ms"}
{"key": "some_key_2", "value": 28.214, "unit": "ms"}
```

input.exec complains about an invalid format. This makes sense, since it's
not actually correct json formatting. If I then try to output a more
correct output, e.g.:

```
[{"key": "some_key_1", "value": 28.214, "unit": "ms"}, {"key":
"some_key_2", "value": 28.214, "unit": "ms"}]
```

input.exec complains again with a de-marshaling error. What is the correct
way to format multiple measurements out of a script monitored by input.exec?

Thanks!

Best regards,
Damião

Hi Damião,
If your script "/home/myscript.py" does not complete before the timeout
that is configured in [inputs.exec] , an error "[exec: exit status 1 for
command X]" will be logged.
Additionally, the exec script should complete before the collection
interval is reached. The default interval is 10s and it is configurable
globally under the [agent] section as well as per-plugin.

Post by Damião Rodrigues
Hi kostas,
Thanks for your reply.
I have given the script full permissions (`chmod 777`) and the problem
remains. The telegraf version I'm running is 'Telegraf - version 0.13.1'.
```
#!/usr/bin/env python
import json
from datetime import datetime
dict_list = []
n = 66.666
s = datetime.now().second
x = "boing"
dict_list.append(dict(seconds=s,order=n,description=x))
print(json.dumps(dict_list[0]))
```
```
[[inputs.exec]]
command = "/home/dummy.py"
data_format = "json"
##name_suffix = "_dummy"
name_override = 'dummy'
timeout = "10s"
tag_keys = ['description']
```
... and it works perfectly!
I've noticed that while the output of dummy.py is printed immediately to
the stdout, the one from my_script takes a few seconds to show up. Could
this be the problem? Doesn't the 'timeout' config handle this?

Post by a***@gmail.com

Post by a***@gmail.com
Hi all,
I'm trying to capture the json output of a python script using the

Exec Input Plugin. For some reason, I'm getting the following error when

Post by a***@gmail.com
```
telegraf_1 | 2016-08-02T09:43:00.360767123Z 2016/08/02 09:43:00

ERROR in input [exec]: exec: exit status 1 for command '/home/myscript.py'

Post by a***@gmail.com
```
```
[[inputs.exec]]
command = "/home/myscript.py"
data_format = "json"
name_suffix = "_some_suffix"
timeout = "10s"
tag_keys = ["key", "unit"]
```
/home/myscript.py has execution permissions ('chmod +x') and outputs
```
{"key": "some_key", "value": 28.214, "unit": "ms"}
```
I've checked that the exit status of running `/home/myscript.py` is 0.
The weirdest part is that, if I run `$ telegraf -config
```
* Plugin: exec, Collection 1

exec_some_suffix,host=13ceb7312093,key=some_key,unit=ms value=19.539

1470132272694745614

Post by a***@gmail.com
```
Do you have any idea of what could be happening?

Hi,
I have not been able to reproduce this behavior with the latest beta of
Telegraf.
It is likely that when you execute telegraf by invoking the binary from
the command line (telegraf -config /etc/telegraf.conf) you are running it
as your user who has privileges to execute the python script, while the
telegraf service runs as user "telegraf", which may not have sufficient
privileges to access /home/myscript.py

--
Konstantinos Botsas
Support Engineer
InfluxData

[Damião Rodrigues]

Hi Konstantinos,

Thank you for the reply and for taking the time to look at the JSON parser
code.

Ok. As you suggested, since I'm using custom scripts, changing their output
to the Influx Line Protocol format is possible. And it seems easy too.

Thank you for your time!

Best,
Damião

Hi Damião,
https://github.com/influxdata/telegraf/blob/master/plugins/parsers/json/parser.go#L20
if looks like the parser will attempt to unmarshall JSON input and return
a single metric.
Performance and complexity concerns related to flattening JSON have been
the main reason that the JSON write path was deprecated in InfluxDB.
My understanding is that you are working on a custom script to output data
points, so I strongly encourage you to use Influx Line Protocol instead of
the JSON format.
Thanks,
Kostas

Post by Damião Rodrigues
Hi Konstantinos,
Thanks for the reply. Ok, that behavior fits what I've observed. Your
first hunch was correct: it was a permission issue: telegraf was run by a
user 'nobody', set by the s6-setuidgid program
<http://skarnet.org/software/s6/s6-setuidgid.html>.
By the way - actually bit unrelated :D - afterwards I've found another
problem: if my script outputs multiple json lines, i.e. something like
```
{"key": "some_key_1", "value": 28.214, "unit": "ms"}
{"key": "some_key_2", "value": 28.214, "unit": "ms"}
```
input.exec complains about an invalid format. This makes sense, since
it's not actually correct json formatting. If I then try to output a more
```
"some_key_2", "value": 28.214, "unit": "ms"}]
```
input.exec complains again with a de-marshaling error. What is the
correct way to format multiple measurements out of a script monitored by
input.exec?
Thanks!
Best regards,
Damião

Hi Damião,
If your script "/home/myscript.py" does not complete before the timeout
that is configured in [inputs.exec] , an error "[exec: exit status 1 for
command X]" will be logged.
Additionally, the exec script should complete before the collection
interval is reached. The default interval is 10s and it is configurable
globally under the [agent] section as well as per-plugin.

Post by Damião Rodrigues
Hi kostas,
Thanks for your reply.
I have given the script full permissions (`chmod 777`) and the problem
remains. The telegraf version I'm running is 'Telegraf - version 0.13.1'.
```
#!/usr/bin/env python
import json
from datetime import datetime
dict_list = []
n = 66.666
s = datetime.now().second
x = "boing"
dict_list.append(dict(seconds=s,order=n,description=x))
print(json.dumps(dict_list[0]))
```
```
[[inputs.exec]]
command = "/home/dummy.py"
data_format = "json"
##name_suffix = "_dummy"
name_override = 'dummy'
timeout = "10s"
tag_keys = ['description']
```
... and it works perfectly!
I've noticed that while the output of dummy.py is printed immediately
to the stdout, the one from my_script takes a few seconds to show up. Could
this be the problem? Doesn't the 'timeout' config handle this?

Post by a***@gmail.com

Post by a***@gmail.com
Hi all,
I'm trying to capture the json output of a python script using the

Exec Input Plugin. For some reason, I'm getting the following error when

Post by a***@gmail.com
```
telegraf_1 | 2016-08-02T09:43:00.360767123Z 2016/08/02 09:43:00

ERROR in input [exec]: exec: exit status 1 for command '/home/myscript.py'

Post by a***@gmail.com
```
```
[[inputs.exec]]
command = "/home/myscript.py"
data_format = "json"
name_suffix = "_some_suffix"
timeout = "10s"
tag_keys = ["key", "unit"]
```
/home/myscript.py has execution permissions ('chmod +x') and outputs
```
{"key": "some_key", "value": 28.214, "unit": "ms"}
```
I've checked that the exit status of running `/home/myscript.py` is

0.

Post by a***@gmail.com
The weirdest part is that, if I run `$ telegraf -config
```
* Plugin: exec, Collection 1

exec_some_suffix,host=13ceb7312093,key=some_key,unit=ms

value=19.539 1470132272694745614

Post by a***@gmail.com
```
Do you have any idea of what could be happening?

Hi,
I have not been able to reproduce this behavior with the latest beta
of Telegraf.
It is likely that when you execute telegraf by invoking the binary
from the command line (telegraf -config /etc/telegraf.conf) you are running
it as your user who has privileges to execute the python script, while the
telegraf service runs as user "telegraf", which may not have sufficient
privileges to access /home/myscript.py

--
Konstantinos Botsas
Support Engineer
InfluxData

--
Konstantinos Botsas
Support Engineer
InfluxData

[n***@gmail.com]

I have the same issue with multiline JSON.
I cant change to influxline as well since I need to convert my other scripts to write to influxline instead of json!!
Any suggestion to overcome this??