I found out the root cause after that. We have duplicated API with sama
name so the one I added metrics is not seen. Legacy code drive me crazy.
About those out-of-date metrics, I delete the db through the admin-api and
those metrics are cleared. Thanks for looking into that.
On Wednesday, January 3, 2024 at 5:11:38 p.m. UTC-5 Brian Candler wrote:
> I suggest you take Prometheus entirely out of the picture, and scrape your
> metrics endpoint directly using curl:
>
> curl localhost:8881/metrics
>
> See if your expected custom metric is there. If it's not, then that's why
> Prometheus isn't showing it, and you need to debug your Python code.
>
> On Wednesday 3 January 2024 at 20:44:12 UTC Honsen Clock wrote:
>
>> Issue Description:
>> I have a Python Flask app and I wanna add a new metric
>> "by_path_counter_doc_quality" for an API, which I added by:
>> ```Python
>> # app.py
>>
>> by_path_counter = metrics.counter(
>> 'by_path_counter_doc_quality', 'Request count by request paths',
>> labels={'path': lambda: request.path, 'method': lambda:
>> request.method}
>> )
>> ...
>> @app.route("/document_quality/transformation", methods=["POST"])
>> @by_path_counter
>> def run_comp_metrics():
>> # code
>> ```
>> But when I start prometheus server and monitor, I cannot find this
>> metrics on the metrics list. The flask app itself is working and using
>> metric "flask_http_request_total" can find the requests.
>> [image: prometheus-flask-app.png]
>> - Moreover, I notice that the metrics list on the "metrics explorer" tab
>> shows out-of-dated metrics. See the attached image, "by_path_counter" and
>> "in_progress" were the metrics I added in previous scrape_config job.
>>
>> [image: prometheus-metrics list.png]
>>
>> My prometheus.yml:
>> ```yaml
>> # my global config
>> global:
>> scrape_interval: 15s
>> evaluation_interval: 15s
>>
>> alerting:
>> alertmanagers:
>> - static_configs:
>> - targets:
>> # - alertmanager:9093
>>
>> rule_files:
>> # - "first_rules.yml"
>> # - "second_rules.yml"
>>
>> scrape_configs:
>> - job_name: "doc-quality"
>> static_configs:
>> - targets: ["localhost:8881"]
>> ```
>> Some context about the issue:
>> In my prometheus.yaml, I created a job "flask_app", with metrics called
>> "by_path_counter" and "in_progress". The metrics works well. So I wanna
>> applied the same way to the current new job "doc-quality" and new metrics
>> "by_path_counter_doc_quality".
>>
>> My questions:
>> - What's the root cause for above issue? Why my new metric is now shown?
>> - Are there any ways that I can reset/refresh the metrics? Maybe clean
>> the time series DB? If so, how to to that.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/5256b926-2904-491a-b501-f277fe18156bn%40googlegroups.com.
