[
https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448812#comment-16448812
]
Bolke de Bruin commented on AIRFLOW-2367:
-----------------------------------------
You really need to provide more metrics and configuration options. The
scheduler can be really busy when you have a lot of dags. worker memory also
affects your performance. In other words let a DBA have a look at what you are
doing and report back.
> High POSTGRES DB CPU utilization
> --------------------------------
>
> Key: AIRFLOW-2367
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2367
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: Airflow 2.0, 1.9.0
> Reporter: John Arnold
> Priority: Major
> Attachments: cpu.png
>
>
> We are seeing steady state 70-90% CPU utilization. It feels like a missing
> index kind of problem, as our TPS rate is really low, I'm not seeing any long
> running queries, connection counts are reasonable (low hundreds) and locks
> also look reasonable (not many exclusive / write locks)
> We shut down the webserver and it doesn't go away, so it doesn't seem to be
> in that part of the code. My guess is either the scheduler has an inefficient
> query, or the (Celery) executor code path does.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
