Hi Brian,
Thanks for commands for troubleshooting. Finally, single quotes are not
added in some of the targets. I have added it and works fine.
These commands will be helpful in troubleshooting in future.
Regards,
Venkatraman N
On Mon, Jan 29, 2024 at 7:52 PM 'Brian Candler' via Prometheus Users <
prometheus-users@googlegroups.com> wrote:
> I can't see the image link you posted: it gives a 404.
>
> There's probably some config error in your prometheus.yml or your targets
> file, but seeing only snippets means I can't spot them.
>
> Try these commands (adjust paths as necessary):
>
> /path/to/promtool check service-discovery /path/to/prometheus.yml
> blackbox-fundconnect_retail
> /path/to/promtool check config /path/to/prometheus.yml
>
> And what does the query
> up{job="blackbox-fundconnect_retail"}
> show?
>
> Apart from that, I don't think there's much more I can do to help you.
> There's a problem with the configs on your system that you'll need to
> diagnose locally.
>
> On Monday 29 January 2024 at 13:57:19 UTC Venkatraman Natarajan wrote:
>
>> Hi Brian,
>>
>> Thanks for the response.
>>
>> *Are you saying that job blackbox-fundconnect_retail does not appear
>> there, but blackbox-fundconnect_retail_03 does? *
>>
>> Yes, Correct. We have 100 jobs in prometheus. Now we have added 5 jobs
>> additionally, in that it is showing only 4 jobs.
>>
>> blackbox-fundconnect_retail_03,
>> blackbox-fundconnect_retail_04
>> blackbox-fundconnect_retail_05
>> blackbox-fundconnect_retail_06
>> blackbox-fundconnect_retail - This one is not showing in prometheus UI.
>>
>> *Is it possible that blackbox-fundconnect_retail has zero targets
>> configured?*
>>
>> No, It has 48 targets configured.
>>
>> Note: We need to configure different scrape intervals for those jobs
>> that's why configuring different jobs instead of a single job.
>>
>> [image: image.png]
>>
>>
>> Thanks,
>> Venkatraman N
>>
>> On Thursday, January 25, 2024 at 7:13:23 PM UTC+5:30 Brian Candler wrote:
>>
>>> Sorry, I don't know what you mean by "not showing all the jobs". You
>>> have only shown a small portion of the targets page. Are you saying that
>>> job blackbox-fundconnect_retail does not appear there,
>>> but blackbox-fundconnect_retail_03 does? Is it possible
>>> that blackbox-fundconnect_retail has zero targets configured?
>>>
>>> The PromQL query "up" will show you all the targets. "count by (job)
>>> (up)" will show you the jobs, with the number of targets for each. Those
>>> won't show a job with zero targets though.
>>>
>>> Note that in the above case, both scrape jobs look to be identical, in
>>> which case you could have one job:
>>>
>>> file_sd_configs:
>>> - files:
>>> - './dynamic/blackbox/blackbox_retail_FundConnect_targets.yml'
>>> - './dynamic/blackbox/blackbox_retail_FundConnectHealth-03_targets.yml'
>>>
>>> (You can use target labels to distinguish them, if you wish)
>>>
>>> > Do we have limitations in prometheus jobs.?
>>>
>>> No.
>>>
>>> > Prometheus version: prometheus:v2.27.1
>>>
>>> That's pretty old (May 2021). Current LTS version is v2.45.2
>>>
>>> On Thursday 25 January 2024 at 13:11:27 UTC Venkatraman Natarajan wrote:
>>>
>>>> Hi Team,
>>>>
>>>> We have 100+ prometheus jobs in prometheus UI but not showing all the
>>>> jobs.
>>>>
>>>> The below are sample 2 jobs. In this, first job is not showing in UI
>>>> but second job metrics showing fine.
>>>> - job_name: 'blackbox-fundconnect_retail'
>>>> scrape_interval: 1020s
>>>> metrics_path: /probe
>>>> honor_timestamps: true
>>>> params:
>>>> module: [http_2xx]
>>>> file_sd_configs:
>>>> - files:
>>>> ['./dynamic/blackbox/blackbox_retail_FundConnect_targets.yml']
>>>> relabel_configs:
>>>> - source_labels: [__address__]
>>>> separator: ;
>>>> regex: (.*)
>>>> target_label: __param_target
>>>> replacement: $1
>>>> action: replace
>>>> - source_labels: [__param_target]
>>>> separator: ;
>>>> regex: (.*)
>>>> target_label: instance
>>>> replacement: $1
>>>> action: replace
>>>> - separator: ;
>>>> regex: (.*)
>>>> target_label: __address__
>>>> replacement: {{ XXX }}:9122
>>>> action: replace
>>>>
>>>> - job_name: 'blackbox-fundconnect_retail_03'
>>>> scrape_interval: 1020s
>>>> metrics_path: /probe
>>>> honor_timestamps: true
>>>> params:
>>>> module: [http_2xx]
>>>> file_sd_configs:
>>>> - files:
>>>> ['./dynamic/blackbox/blackbox_retail_FundConnectHealth-03_targets.yml']
>>>> relabel_configs:
>>>> - source_labels: [__address__]
>>>> separator: ;
>>>> regex: (.*)
>>>> target_label: __param_target
>>>> replacement: $1
>>>> action: replace
>>>> - source_labels: [__param_target]
>>>> separator: ;
>>>> regex: (.*)
>>>> target_label: instance
>>>> replacement: $1
>>>> action: replace
>>>> - separator: ;
>>>> regex: (.*)
>>>> target_label: __address__
>>>> replacement: {{ XXXXX}}:9122
>>>> action: replace
>>>>
>>>> Do we have limitations in prometheus jobs.?
>>>>
>>>> I would like to add more prometheus jobs to scrape the metrics.
>>>>
>>>> Please find attached screenshot which shows jobs in prometheus UI
>>>>
>>>> Prometheus version: prometheus:v2.27.1
>>>>
>>>> Could you please help me on this?
>>>>
>>>> Thanks,
>>>> Venkatraman N
>>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/c5aa0bb1-e86e-4842-90d2-e1aab4b76d18n%40googlegroups.com
> <https://groups.google.com/d/msgid/prometheus-users/c5aa0bb1-e86e-4842-90d2-e1aab4b76d18n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/CANSgTEZUc9Yb8N93%2Bj1hKiVFc9iJ1njqbLGO5LQFWG5wBfRHAg%40mail.gmail.com.
