[prometheus-users] Re: Basic webhook for alertmanager/prometheus

[Danny de Waard]

I have a webhook service in place that works. And i can send variables to a 
script. But how do i get AlertManager (prometheus) to send something to 
that wqwebhook based on a rule.


Op vrijdag 27 maart 2020 08:32:06 UTC+1 schreef Brian Candler:
>
> The format of the webhook JSON body is fixed.  You can write a small proxy 
> which receives this JSON as input, and sends whatever you like to the 
> downstream server.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/44b621a8-26df-490a-9f18-d2a312ac70a0%40googlegroups.com.

[prometheus-users] Re: Basic webhook for alertmanager/prometheus

[Brian Candler]

To control which receivers get which alerts, you need to configure "routes 
".  You haven't 
shown that part of your configuration.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/555a2d45-f586-42e3-9784-327a84d6e666%40googlegroups.com.

[prometheus-users] Basic webhook for alertmanager/prometheus

[Danny de Waard]

As a complete noob i want to know how i can simply introdcue a webhook 
config for prometheus/alertmanager.
For certain alerts i want to send a webhook with 3 parameters to a server.
How do i implement this in prometheus/alertmanager.

I allready have some recievers in place

receivers:
  - name: 'default'
  - name: 'mail'
email_configs:
  - send_resolved: true
to: mailaddr...@domain.nl ##fill in your email
require_tls: false
  - name: 'all'
email_configs:
  - send_resolved: true
to: some.m...@domain.nl ##fill in your email
require_tls: false
  - send_resolved: true
to: "0123456...@sms.domain.nl" ##fill in your phonenumbers
require_tls: false
  - name: 'webhook'
webhook_configs:
  - url: 'http://127.0.0.1:9000'

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/6cfb7ba2-eaec-427d-9771-ed48f59a09b9%40googlegroups.com.

[prometheus-users] Re: Basic webhook for alertmanager/prometheus

[Brian Candler]

The format of the webhook JSON body is fixed.  You can write a small proxy 
which receives this JSON as input, and sends whatever you like to the 
downstream server.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/df04faef-f5e0-41a7-bd73-4d90cc54faa8%40googlegroups.com.

Re: [prometheus-users] Re: Data retention policy

[REMI DRUILHE]

Hi,

Thanks for the answer.

Just to add some context to what I want to do. We are trying to make our 
application (Prometheus being a part of it) compliant with GDPR. The 
default period for our data retention is 10 days but can be set by the user 
to 1h minimum. If the data are not deleted after 1 hour then, it is no 
longer compliant with what we declared to the national data protection 
authority.

>From your answer, I understand now that the parameter 
*storage.tsdb.retention.time* is not hard deadline for the data. Data might 
live after the retention time, at least 2 hours. So I launched a new test 
with storage.tsdb.retention.time equals to 5m. Then, I waited for 3 hours 
to see if the data were deleted but they are always in Prometheus. 

Is it possible to anticipate when the data will be deleted to update ours 
compliance with GDPR? Can we change the background schedule to have a 
better estimation of when the data will be deleted? Or should I use the 
delete method of Prometheus form an external component to be sure that the 
data are deleted on time?

Thanks,

Best regards.


Le jeudi 26 mars 2020 21:40:35 UTC+1, Christian Hoffmann a écrit :
>
> Hi, 
>
> not sure if it's this what you are hitting, but the following may help: 
>
> The retention time defines how long Prometheus is expected to keep 
> history. This does not imply that data older than this interval will 
> vanish immediately. 
> In other words: Once the interval has passed, Prometheus is free to 
> delete this data, but it isn't forced to. In practice, this will depend 
> on maintenance (compaction) running. 
>
> This document provides further details, especially: 
>
> https://prometheus.io/docs/prometheus/latest/storage/ 
> "Expired block cleanup happens on a background schedule. It may take up 
> to two hours to remove expired blocks. Expired blocks must be fully 
> expired before they are cleaned up." 
>
> Maybe this explains things for you? 
>
> Kind regards, 
> Christian 
>
> On 3/26/20 3:33 PM, REMI DRUILHE wrote: 
> > I did some others tests, on others operating systems (not sure it is 
> > relevant because of Docker but without any clues it is better than 
> > nothing). For anyone to test it, I also changed and reduced my 
> > docker-compose file to: 
> > 
> > | 
> > version:'3.4' 
> > | 
> > services: 
> >   prometheus: 
> > image: prom/prometheus:latest 
> > container_name: "prometheus" 
> > depends_on: 
> >   - cadvisor 
> > command: 
> >   - '--config.file=/etc/prometheus/prometheus.yml' 
> >   - '--storage.tsdb.retention.time=60s' 
> > volumes: 
> >   - /tmp/prometheus:/etc/prometheus 
> > ports: 
> >   - "9090:9090" 
> > network_mode: "host" 
> >   cadvisor: 
> > image: gcr.io/google-containers/cadvisor 
> > container_name: cadvisor 
> > volumes: 
> >   - /:/rootfs:ro 
> >   - /var/run:/var/run:ro 
> >   - /sys:/sys:ro 
> >   - /var/lib/docker/:/var/lib/docker:ro 
> >   - /dev/disk/:/dev/disk:ro 
> > ports: 
> >   - "8080:8080" 
> > network_mode: "host" 
> > 
> > And here is my prometheus.yml 
> > 
> > | 
> > scrape_configs: 
> > - job_name: cadvisor 
> >   scrape_interval: 5s 
> >   static_configs: 
> >   - targets: 
> > - localhost:8080 
> > | 
> > 
> > But, the result is still the same with the following curl request:  
> > 
> > | 
> > curl 
> > '
> http://172.21.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes=2020-03-26T00:01:00.000Z=2020-03-26T17:00:00.000Z=15s'
>  
>
> > | 
> > 
> > I can still get data that should have been deleted due to out of date of 
> > the timestamp and defined in the storage.tsdb.retention.time parameter. 
> > 
> > According to me, it is a bug. But before opening an issue on Prometehus, 
> > I prefer to ask for advices about it. 
> > 
> > Thanks, 
> > 
> > Best regards. 
> > 
> > Le mercredi 25 mars 2020 10:27:22 UTC+1, REMI DRUILHE a écrit : 
> > 
> > Anyone to help me on this subject? 
> > 
> > Thanks. 
> > 
> > Le mardi 24 mars 2020 13:57:05 UTC+1, REMI DRUILHE a écrit : 
> > 
> > Hello, 
> > 
> > I am using a Docker version of Prometheus (the latest one) in 
> > which I am setting the value for the retention time (option 
> > *storage.tsdb.retention.time*) at launch. I trying first with 
> > small values like *1m* and *60s* (for 1 minute) or *5m* (for 5 
> > minutes) but I was not able to see any deletion of the data 
> > after this period. Thus, I tested again this morning with an 
> > higher value: *2h* (for 2 hours). I have seen this information 
> > here 
> > <
> https://medium.com/@valyala/prometheus-storage-technical-terms-for-humans-4ab4de6c3d48>
>  
>
> > (in the retention section) that 2h is the minimum value for 
> > Prometheus, not sure if it is true or not. But even with this 
> > value, the data stored 4 hours ago can still be retrieved using 
> >  

Re: [prometheus-users] Re: Data retention policy

[Stuart Clark]

On 27/03/2020 13:55, REMI DRUILHE wrote:

Hi,

Thanks for the answer.

Just to add some context to what I want to do. We are trying to make 
our application (Prometheus being a part of it) compliant with GDPR. 
The default period for our data retention is 10 days but can be set by 
the user to 1h minimum. If the data are not deleted after 1 hour then, 
it is no longer compliant with what we declared to the national data 
protection authority.


From your answer, I understand now that the parameter 
*storage.tsdb.retention.time* is not hard deadline for the data. Data 
might live after the retention time, at least 2 hours. So I launched a 
new test with storage.tsdb.retention.time equals to 5m. Then, I waited 
for 3 hours to see if the data were deleted but they are always in 
Prometheus.


Is it possible to anticipate when the data will be deleted to update 
ours compliance with GDPR? Can we change the background schedule to 
have a better estimation of when the data will be deleted? Or should I 
use the delete method of Prometheus form an external component to be 
sure that the data are deleted on time?




What data are you expecting Prometheus to hold that would be subject to 
GDPR?


Prometheus stores monitoring metrics so wouldn't have anything related 
to personal data, so isn't in scope for GDPR...




Thanks,

Best regards.


Le jeudi 26 mars 2020 21:40:35 UTC+1, Christian Hoffmann a écrit :

Hi,

not sure if it's this what you are hitting, but the following may
help:

The retention time defines how long Prometheus is expected to keep
history. This does not imply that data older than this interval will
vanish immediately.
In other words: Once the interval has passed, Prometheus is free to
delete this data, but it isn't forced to. In practice, this will
depend
on maintenance (compaction) running.

This document provides further details, especially:

https://prometheus.io/docs/prometheus/latest/storage/

"Expired block cleanup happens on a background schedule. It may
take up
to two hours to remove expired blocks. Expired blocks must be fully
expired before they are cleaned up."

Maybe this explains things for you?

Kind regards,
Christian

On 3/26/20 3:33 PM, REMI DRUILHE wrote:
> I did some others tests, on others operating systems (not sure
it is
> relevant because of Docker but without any clues it is better than
> nothing). For anyone to test it, I also changed and reduced my
> docker-compose file to:
>
> |
> version:'3.4'
> |
> services:
>   prometheus:
>     image: prom/prometheus:latest
>     container_name: "prometheus"
>     depends_on:
>       - cadvisor
>     command:
>       - '--config.file=/etc/prometheus/prometheus.yml'
>       - '--storage.tsdb.retention.time=60s'
>     volumes:
>       - /tmp/prometheus:/etc/prometheus
>     ports:
>       - "9090:9090"
>     network_mode: "host"
>   cadvisor:
>     image: gcr.io/google-containers/cadvisor

>     container_name: cadvisor
>     volumes:
>       - /:/rootfs:ro
>       - /var/run:/var/run:ro
>       - /sys:/sys:ro
>       - /var/lib/docker/:/var/lib/docker:ro
>       - /dev/disk/:/dev/disk:ro
>     ports:
>       - "8080:8080"
>     network_mode: "host"
>
> And here is my prometheus.yml
>
> |
> scrape_configs:
> - job_name: cadvisor
>   scrape_interval: 5s
>   static_configs:
>   - targets:
>     - localhost:8080
> |
>
> But, the result is still the same with the following curl request:
>
> |
> curl
>

'http://172.21.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes=2020-03-26T00:01:00.000Z=2020-03-26T17:00:00.000Z=15s

'

> |
>
> I can still get data that should have been deleted due to out of
date of
> the timestamp and defined in the storage.tsdb.retention.time
parameter.
>
> According to me, it is a bug. But before opening an issue on
Prometehus,
> I prefer to ask for advices about it.
>
> Thanks,
>
> Best regards.
>
> Le mercredi 25 mars 2020 10:27:22 UTC+1, REMI DRUILHE a écrit :
>
>     Anyone to help me on this subject?
>
>     Thanks.
>
>     Le mardi 24 mars 2020 13:57:05 UTC+1, REMI DRUILHE a écrit :
>
>         Hello,
>
>         I am using a Docker version of Prometheus (the latest
one) in
>         which I am setting the value for the retention time (option
>         *storage.tsdb.retention.time*) at launch. I trying first
with
>         small values like *1m* and *60s* 

Re: [prometheus-users] Prometheus Usage

[Madhu Kuriti]

Ben, thanks for the prompt response, super helpful. I will add 
clarification to your question as soon as I have it.

On Friday, March 27, 2020 at 1:44:23 PM UTC-4, Ben Kochie wrote:
>
>
>
> On Fri, Mar 27, 2020, 18:35 Madhu Kuriti > 
> wrote:
>
>> Hi,
>> I have a few questions related to the practical use of this 
>>
>>- Process mapping, where are the shortcuts to adding new monitors?
>>
>> This was an ask from a colleague, reached back to him for clarification. 
> I think this may be in the context of configuring data sources via 
> alertmanager, but I am not sure.
> I'm not sure what you mean for this.
>
>
>>- How can we have multiple people working on this?
>>
>> I highly recommend a git workflow and Ansible.
>
>
>>- Any tips on how to best utilize Ansible?
>>
>>
> We have semi-official Ansible roles.
>
> https://github.com/cloudalchemy/ansible-prometheus
>
>>
>>- SNMP webhooks for Prometheus and alert manager?
>>
>> I found a few of these (SNMP exporter 
 as well as the SNMP webhook 
 for alerts) 

> I may have seen something like this as a alertmanager webhook receiver. 
> But I don't know where.
>
>>
>>- How can we forward the alerts?
>>
>> There are a bunch of alertmanager integration options. And webhooks for 
> making your own.
>
> thanks
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to promethe...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-users/29031f6a-96da-4963-8df7-ecf238a69e63%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/77426d23-62a6-45e6-999b-5c2a763edf48%40googlegroups.com.

Re: [prometheus-users] Prometheus - how to get systemd service level open/max fds limits?

[Christian Hoffmann]

Hi,

On 3/27/20 5:30 PM, rs vas wrote:
> Thanks Brian for the input! Yes, tried systemd collector but did not see
> the limits exposed through that collector.
Pretty sure the ncabatoff's process_exporter exposes such metrics:
https://github.com/ncabatoff/process-exporter/

However, there is no direct linking to systemd in this case, although
you can try to use data from both sources in the same PromQL query if
needed.

There also seem to be specialized systemd_exporters, although I don't
have any experience with them, e.g.:
https://github.com/povilasv/systemd_exporter

This one seems to expose systemd_process_open_fds (if enabled).


Kind regards,
Christian

> 
> For now quick way to do this is through text collector through Node
> exporter.
> 
> Please share if anyone has any ideas Thanks again!
> 
> On Fri, Mar 27, 2020 at 9:22 AM Brian Candler  > wrote:
> 
> https://github.com/prometheus/node_exporter#disabled-by-default
> 
> There is a systemd collector (disabled by default) which "Exposes
> service and system status from systemd" - but it doesn't appear to
> have metrics for the cgroup limits.
> 
> You can either extend it in go, or use textfile_collector as you said.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to prometheus-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/prometheus-users/c926dc71-e860-49ef-af30-909dad5be204%40googlegroups.com
> 
> .
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to prometheus-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/CAPs_Afgn4o7fxLN7UnekVSH14S%2Bx4DbeqHdB7yFO39RwPwODBA%40mail.gmail.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/d20aff8d-9d14-e5b0-c339-5cc33323c740%40hoffmann-christian.info.

[prometheus-users] Postman Exporter

[Sebastian Ebling]

Hi,

I created an exporter that runs Postman collections and returns metrics on 
those runs.

You can find the project here:
https://github.com/hudeldudel/postman-exporter

I would appreciate feedback.

The included example is very simple and could also have been done with the 
blackbox exporter. But if you know Postman a little better, you also know 
that a lot is possible with it that is difficult or impossible to achieve 
with the blackbox exporter.

I use it to monitor APIs that are secured with OAuth2 and mTLS.
In another case, a complete OpenID Connect login with code flow (including 
login to the IDP, cookie handling, CSRF token handling) is carried out and 
monitored.

regards

Sebastian

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/0a4ff8e8-e7a0-44e6-b357-d3288de33c81%40googlegroups.com.

[prometheus-users] Node Exporter & Systemd

[Jonathan Sloan]

Hello,q

I seem to be having some troubling getting NE to ignore a few devices 
'sr0/sda[1-3]. I've tried multiple things to try and get this work, exact 
for manually telling it to ignore sr0|sda1|sda2|sda3 etc. When using the 
below command.
 
node_exporter --web.listen-address=0.0.0.0:9100 --web.max-requests=16 --
collector.processes

It does not ignore sda[1-3] I know sr* is not part of the standard regex 
they use. But this standard regex doesn't appear to work with systemd 
because the \\d chars are removed from systemd.

/usr/sbin/node_exporter --web.listen-address=0.0.0.0:9100 --web.max-requests
=16 --collector.processes --collector.diskstats.ignored-devices=^(sr|ram|
loop|fd|(h|s|v|xv)d[a-z]|nvmed+nd+p)d+$ --collector.filesystem.ignored-mount
-points=^/(dev|proc|sys|var/lib/docker/.+)($|/) --collector.filesystem.
ignored-fs-types=^(rootfs|tmpfs|autofs|binfmt_misc|bpf|cgroup2?|configfs|
debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|nsfs|overlay|proc|procfs|
pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$


The issue appears to be around the char '\'. I believe systemd removes 
these chars and thus you need to encode them from my googling around with 
systemd-escape which I did shown below, but it still doesn't appear to 
work..

[root@linda058 20:16 compose_files]$ systemd-escape 
'^(sr|ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\\d+n\\d+p)\\d+$'
\x5e\x28sr\x7cram\x7cloop\x7cfd\x7c\x28h\x7cs\x7cv\x7cxv\x29d\x5ba\x2dz\x5d\x7cnvme\x5c\x5cd\x2bn\x5c\x5cd\x2bp\x29\x5c\x5cd\x2b\x24
[root@linda058 20:18 compose_files]$ systemd-escape -u 
'\x5e\x28sr\x7cram\x7cloop\x7cfd\x7c\x28h\x7cs\x7cv\x7cxv\x29d\x5ba\x2dz\x5d\x7cnvme\x5c\x5cd\x2bn\x5c\x5cd\x2bp\x29\x5c\x5cd\x2b\x24'
^(sr|ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\\d+n\\d+p)\\d+$


-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/9f85f96e-14c0-4a43-a622-449b4aee6c68%40googlegroups.com.

Re: [prometheus-users] Re: Data retention policy

[Ben Kochie]

Yes, this is the general guideline. User identifiable data should not be in
Prometheus. Mostly because it tends to be a cardinality problem.

On Fri, Mar 27, 2020, 14:58 Stuart Clark  wrote:

> On 27/03/2020 13:55, REMI DRUILHE wrote:
>
> Hi,
>
> Thanks for the answer.
>
> Just to add some context to what I want to do. We are trying to make our
> application (Prometheus being a part of it) compliant with GDPR. The
> default period for our data retention is 10 days but can be set by the user
> to 1h minimum. If the data are not deleted after 1 hour then, it is no
> longer compliant with what we declared to the national data protection
> authority.
>
> From your answer, I understand now that the parameter
> *storage.tsdb.retention.time* is not hard deadline for the data. Data
> might live after the retention time, at least 2 hours. So I launched a new
> test with storage.tsdb.retention.time equals to 5m. Then, I waited for 3
> hours to see if the data were deleted but they are always in Prometheus.
>
> Is it possible to anticipate when the data will be deleted to update ours
> compliance with GDPR? Can we change the background schedule to have a
> better estimation of when the data will be deleted? Or should I use the
> delete method of Prometheus form an external component to be sure that the
> data are deleted on time?
>
>
> What data are you expecting Prometheus to hold that would be subject to
> GDPR?
>
> Prometheus stores monitoring metrics so wouldn't have anything related to
> personal data, so isn't in scope for GDPR...
>
>
> Thanks,
>
> Best regards.
>
>
> Le jeudi 26 mars 2020 21:40:35 UTC+1, Christian Hoffmann a écrit :
>>
>> Hi,
>>
>> not sure if it's this what you are hitting, but the following may help:
>>
>> The retention time defines how long Prometheus is expected to keep
>> history. This does not imply that data older than this interval will
>> vanish immediately.
>> In other words: Once the interval has passed, Prometheus is free to
>> delete this data, but it isn't forced to. In practice, this will depend
>> on maintenance (compaction) running.
>>
>> This document provides further details, especially:
>>
>> https://prometheus.io/docs/prometheus/latest/storage/
>> "Expired block cleanup happens on a background schedule. It may take up
>> to two hours to remove expired blocks. Expired blocks must be fully
>> expired before they are cleaned up."
>>
>> Maybe this explains things for you?
>>
>> Kind regards,
>> Christian
>>
>> On 3/26/20 3:33 PM, REMI DRUILHE wrote:
>> > I did some others tests, on others operating systems (not sure it is
>> > relevant because of Docker but without any clues it is better than
>> > nothing). For anyone to test it, I also changed and reduced my
>> > docker-compose file to:
>> >
>> > |
>> > version:'3.4'
>> > |
>> > services:
>> >   prometheus:
>> > image: prom/prometheus:latest
>> > container_name: "prometheus"
>> > depends_on:
>> >   - cadvisor
>> > command:
>> >   - '--config.file=/etc/prometheus/prometheus.yml'
>> >   - '--storage.tsdb.retention.time=60s'
>> > volumes:
>> >   - /tmp/prometheus:/etc/prometheus
>> > ports:
>> >   - "9090:9090"
>> > network_mode: "host"
>> >   cadvisor:
>> > image: gcr.io/google-containers/cadvisor
>> > container_name: cadvisor
>> > volumes:
>> >   - /:/rootfs:ro
>> >   - /var/run:/var/run:ro
>> >   - /sys:/sys:ro
>> >   - /var/lib/docker/:/var/lib/docker:ro
>> >   - /dev/disk/:/dev/disk:ro
>> > ports:
>> >   - "8080:8080"
>> > network_mode: "host"
>> >
>> > And here is my prometheus.yml
>> >
>> > |
>> > scrape_configs:
>> > - job_name: cadvisor
>> >   scrape_interval: 5s
>> >   static_configs:
>> >   - targets:
>> > - localhost:8080
>> > |
>> >
>> > But, the result is still the same with the following curl request:
>> >
>> > |
>> > curl
>> > '
>> http://172.21.0.3:9090/api/v1/query_range?query=go_memstats_alloc_bytes=2020-03-26T00:01:00.000Z=2020-03-26T17:00:00.000Z=15s'
>>
>> > |
>> >
>> > I can still get data that should have been deleted due to out of date
>> of
>> > the timestamp and defined in the storage.tsdb.retention.time parameter.
>> >
>> > According to me, it is a bug. But before opening an issue on
>> Prometehus,
>> > I prefer to ask for advices about it.
>> >
>> > Thanks,
>> >
>> > Best regards.
>> >
>> > Le mercredi 25 mars 2020 10:27:22 UTC+1, REMI DRUILHE a écrit :
>> >
>> > Anyone to help me on this subject?
>> >
>> > Thanks.
>> >
>> > Le mardi 24 mars 2020 13:57:05 UTC+1, REMI DRUILHE a écrit :
>> >
>> > Hello,
>> >
>> > I am using a Docker version of Prometheus (the latest one) in
>> > which I am setting the value for the retention time (option
>> > *storage.tsdb.retention.time*) at launch. I trying first with
>> > small values like *1m* and *60s* (for 1 minute) or *5m* (for 5
>> > minutes) but I was not able to see any deletion of the data
>> > 

Re: [prometheus-users] Prometheus Usage

[Ben Kochie]

On Fri, Mar 27, 2020, 18:35 Madhu Kuriti  wrote:

> Hi,
> I have a few questions related to the practical use of this
>
>- Process mapping, where are the shortcuts to adding new monitors?
>
>
I'm not sure what you mean for this.


>- How can we have multiple people working on this?
>
> I highly recommend a git workflow and Ansible.


>- Any tips on how to best utilize Ansible?
>
>
We have semi-official Ansible roles.

https://github.com/cloudalchemy/ansible-prometheus

>
>- SNMP webhooks for Prometheus and alert manager?
>
> I may have seen something like this as a alertmanager webhook receiver.
But I don't know where.

>
>- How can we forward the alerts?
>
> There are a bunch of alertmanager integration options. And webhooks for
making your own.

thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/29031f6a-96da-4963-8df7-ecf238a69e63%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CABbyFmqKK6ju7NvS0MR6wyo5dVhdZFvPMRt0rTNvMpsz56NfFA%40mail.gmail.com.

Re: [prometheus-users] Prometheus - how to get systemd service level open/max fds limits?

[rs vas]

I was thinking one way could be use the textfile collector and expose those
metrics through node exporter? Not sure before going through that route if
there is anything Promethes exporters offer out-of-the-box?

On Thu, Mar 26, 2020 at 5:47 PM rs vas  wrote:

> Hello, I see node exporter exposed below metric for the file descriptors
> limits, but these are specific node exporter systemd service level limits.
> process_max_fds 1024
> process_open_fds 100
>
> I have another systemd proces running on the same aws ec2 instance, for
> example xyz-service, it has it's limits set. But how to expose these limits
> to Prometheus? Appreciate your input!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/779c4553-f711-4dbe-930c-ce7ff85b0ca6%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CAPs_AfiTMs-3FMnTizjXOegn%3DS4ZSBdM%3Dx7WpfJso%3DPFNom17A%40mail.gmail.com.

Re: [prometheus-users] Prometheus - how to get systemd service level open/max fds limits?

[Brian Candler]

https://github.com/prometheus/node_exporter#disabled-by-default

There is a systemd collector (disabled by default) which "Exposes service 
and system status from systemd" - but it doesn't appear to have metrics for 
the cgroup limits.

You can either extend it in go, or use textfile_collector as you said.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/c926dc71-e860-49ef-af30-909dad5be204%40googlegroups.com.

Re: [prometheus-users] Prometheus - how to get systemd service level open/max fds limits?

[rs vas]

Thanks Brian for the input! Yes, tried systemd collector but did not see
the limits exposed through that collector.

For now quick way to do this is through text collector through Node
exporter.

Please share if anyone has any ideas Thanks again!

On Fri, Mar 27, 2020 at 9:22 AM Brian Candler  wrote:

> https://github.com/prometheus/node_exporter#disabled-by-default
>
> There is a systemd collector (disabled by default) which "Exposes service
> and system status from systemd" - but it doesn't appear to have metrics for
> the cgroup limits.
>
> You can either extend it in go, or use textfile_collector as you said.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/c926dc71-e860-49ef-af30-909dad5be204%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/CAPs_Afgn4o7fxLN7UnekVSH14S%2Bx4DbeqHdB7yFO39RwPwODBA%40mail.gmail.com.