michaelosthege opened a new issue, #53173: URL: https://github.com/apache/airflow/issues/53173
### Apache Airflow version 3.0.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Yesterday I upgraded my small home setup (Raspberry Pi, 2 dags, 3 tasks) to Airflow 3. In comparison to Airflow 2.9.0 the CPU load has increased dramatically: <img width="2768" height="530" alt="Image" src="https://github.com/user-attachments/assets/7ab02ab9-b3c5-4281-ae6b-69e6dccfe3b3"; /> A few minutes later I also stopped the scheduler and database CPU load went down too. ### What you think should happen instead? Airflow 3 should cause the same or lower CPU load to Airflow 2. ### How to reproduce My docker-compose setup uses this `Dockerfile` ``` FROM apache/airflow:3.0.2 ENV AIRFLOW__CORE__EXECUTOR=LocalExecutor ENV AIRFLOW__API__BASE_URL="http://junas.fritz.box:5000"; ENV AIRFLOW__CORE__AUTH_MANAGER="airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager" # Note: AIRFLOW__API_AUTH__JWT_SECRET is set in the .env file ENV AIRFLOW__DATABASE__SQL_ALCHEMY_CONN="postgresql+psycopg2://airflowu:xx...@raspi.fritz.box:5001/airflowdb" ENV AIRFLOW__CORE__LOAD_EXAMPLES="False" ENV AIRFLOW__SCHEDULER__ENABLE_HEALTH_CHECK="True" # Settings to reduce CPU load ENV AIRFLOW__API__WORKERS=1 ENV AIRFLOW__SCHEDULER__MIN_FILE_PROCESS_INTERVAL=120 ENV AIRFLOW__DAG_PROCESSOR__PARSING_PROCESSES=1 ENV AIRFLOW__DAG_PROCESSOR__BUNDLE_REFRESH_CHECK_INTERVAL=60 COPY requirements.txt /tmp/requirements.txt # one entry: paho-mqtt RUN pip install -r /tmp/requirements.txt ``` The docker-compose services: ```yaml airflow-db: container_name: airflow-db build: context: ./dockerfiles/airflow-postgres dockerfile: Dockerfile restart: unless-stopped ports: - "5001:5432" volumes: - af_pgdata:/var/lib/postgresql/data environment: POSTGRES_DB: airflowdb POSTGRES_USER: airflowu POSTGRES_PASSWORD: airflowp airflow-api-server: container_name: airflow-api-server build: context: ./dockerfiles/airflow dockerfile: Dockerfile depends_on: - airflow-db restart: unless-stopped env_file: ./dockerfiles/airflow/.env ports: - "5000:8080" user: "1000:0" volumes: - /home/admin/airflow/dags:/opt/airflow/dags - /home/admin/airflow/logs:/opt/airflow/logs command: bash -c "airflow db migrate && airflow api-server" healthcheck: test: ["CMD", "curl", "--fail", "http://localhost:8080/api/v2/version";] interval: 30s timeout: 10s retries: 5 start_period: 30s airflow-dag-processor: container_name: airflow-dag-processor build: context: ./dockerfiles/airflow dockerfile: Dockerfile depends_on: - airflow-db - airflow-api-server restart: unless-stopped env_file: ./dockerfiles/airflow/.env user: "1000:0" volumes: - /home/admin/airflow/dags:/opt/airflow/dags - /home/admin/airflow/logs:/opt/airflow/logs command: bash -c "airflow dag-processor" healthcheck: test: ["CMD-SHELL", 'airflow jobs check --job-type DagProcessorJob'] interval: 30s timeout: 20s retries: 5 start_period: 30s airflow-scheduler: container_name: airflow-scheduler build: context: ./dockerfiles/airflow dockerfile: Dockerfile depends_on: - airflow-db - airflow-api-server - airflow-dag-processor restart: unless-stopped env_file: ./dockerfiles/airflow/.env user: "1000:0" volumes: - /home/admin/airflow/dags:/opt/airflow/dags - /home/admin/airflow/logs:/opt/airflow/logs command: bash -c "airflow scheduler" healthcheck: test: ["CMD", "curl", "--fail", "http://localhost:8974/health";] interval: 30s timeout: 10s retries: 5 start_period: 30s ``` ### Operating System Debian GNU/Linux 12 (bookworm) ### Versions of Apache Airflow Providers _No response_ ### Deployment Official Apache Airflow Helm Chart ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
