> As you said, someone else is hitting the /metrics endpoint, the health probes!
Is it worth asking for endpoints for healthy/ready so we don't trigger the exporter doing all the work and subsequently incur expenses just to check (from outside the container/instance) if the exporter is worth talking to? On Friday, 6 March 2020 03:59:56 UTC-5, Albert Serrallé Ríos wrote: > > Ok, found it... As you said, someone else is hitting the /metrics > endpoint, the health probes! > > (ノ_<。) > > https://github.com/prometheus/prometheus/issues/2831 > > Gonna use that feature now... > > Thanks for the help! > > On Friday, March 6, 2020 at 9:44:01 AM UTC+1, Brian Brazil wrote: >> >> On Fri, 6 Mar 2020 at 08:30, Albert Serrallé Ríos <[email protected]> >> wrote: >> >>> To make things easier, I've just removed everything except for one >>> metric: >>> >>> region: us-east-1metrics: >>> - aws_namespace: AWS/CloudFront >>> aws_metric_name: Requests >>> aws_statistics: [Sum] >>> aws_dimensions: [DistributionId, Region] >>> aws_dimensions_select: >>> Region: [Global] >>> >>> Then prometheus: >>> >>> global: >>> scrape_interval: 30m >>> scrape_timeout: 10s >>> >>> external_labels: >>> monitor: 'prometheus-local' >>> account: 'es-newconstruction-pre' >>> >>> scrape_configs: >>> - job_name: 'prometheus' >>> static_configs: >>> - targets: ['localhost:8080'] >>> >>> - job_name: 'cloudwatch-exporter' >>> honor_labels: false >>> metrics_path: '/metrics' >>> static_configs: >>> - targets: >>> - 'localhost:8001' >>> >>> What I can see now is: >>> >>> - 20 ListMetrics per 5 minutes >>> - 60 GetMetricStatistics per 5 minutes >>> >>> I have *3 Cloudfront distributions* in that environment. >>> >> >> I would expect 1 ListMetrics then, and 3 GetMetricStatistics. If you >> manually hit /metrics how much do the request counters go up by? >> >> Brian >> >> >>> >>> Thanks for helping out on this, Brian. >>> >>> On Friday, March 6, 2020 at 1:52:02 AM UTC+1, Brian Brazil wrote: >>>> >>>> On Thu, 5 Mar 2020 at 19:37, Albert Serrallé Ríos <[email protected]> >>>> wrote: >>>> >>>>> I have 11 metrics configured, and my metrics show 100/105 ListMetrics >>>>> calls and 380/399 GetMetricStatistics calls *every 5 minutes*. >>>>> >>>>> Certainly, the scraping interval affects the number of calls to >>>>> GetMetricStatistics (I've tested 5 minute vs 30 minute scrape, and >>>>> the number of calls is is reduced by 1/3 aprox) but in a way that is not >>>>> obvious to me. >>>>> >>>>> Also, the number of ListMetrics calls seems unecessarily high... >>>>> >>>> >>>> Can you share your config? >>>> >>>> I'd also suspect that there's more than one Prometheus hitting the >>>> exporter. >>>> >>>> Brian >>>> >>>> >>>>> >>>>> On Thursday, March 5, 2020 at 8:21:44 PM UTC+1, Brian Brazil wrote: >>>>>> >>>>>> On Thu, 5 Mar 2020 at 19:10, Albert Serrallé Ríos < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> >>>>>>> I've raised the scrape_interval to 30m and I've noticed that >>>>>>> there's a minimum amount of calls that, no matter the value for >>>>>>> scrape_interval, will be executed. >>>>>>> >>>>>>> Is there any theoretical minimum of GetMetricStatistics and >>>>>>> ListMetrics calls? Could that be configured or is it "hardcoded"? >>>>>>> >>>>>> >>>>>> You can expect at least one ListMetrics per configured metric, and >>>>>> then one GetMetricStatistics per relevant metric it finds. These should >>>>>> only ever be happening due to a scrape. >>>>>> >>>>>> Brian >>>>>> >>>>>> >>>>>>> >>>>>>> Context: I'm trying to reduce the Cloudwatch costs as much as >>>>>>> possible in dev/staging environments. >>>>>>> >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Prometheus Users" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/prometheus-users/bb4258f4-56c4-45cd-8d4c-3cec48de0bd0%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/prometheus-users/bb4258f4-56c4-45cd-8d4c-3cec48de0bd0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Brian Brazil >>>>>> www.robustperception.io >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Prometheus Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/prometheus-users/691ad00d-ea13-4018-919d-053fbc2b4d90%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/prometheus-users/691ad00d-ea13-4018-919d-053fbc2b4d90%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> >>>> -- >>>> Brian Brazil >>>> www.robustperception.io >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/bd5ef7f3-d5eb-4128-8017-0b27f3a41098%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/prometheus-users/bd5ef7f3-d5eb-4128-8017-0b27f3a41098%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Brian Brazil >> www.robustperception.io >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/39069f79-ee1c-498e-a34f-b1ad5c0541e0%40googlegroups.com.
