TejasMorbagal commented on issue #54550:
URL: https://github.com/apache/airflow/issues/54550#issuecomment-3197016265
Hi @viiccwen. It didn't solve the issue. Tried the below and also increased
time for startup probe.
1. Disabling `OpenLineage` by setting
```
env:
- name: PYTHONPATH
value: "/opt/airflow/dags:$PYTHONPATH"
# Disable openlineage plugin
- name: AIRFLOW__OPENLINEAGE__DISABLED
value: "true"
```
2. Custom airflow image to uninstall `OpenLineage`
```
FROM apache/airflow:3.0.4
# Install only the providers your CONTROL PLANE needs (UI, S3 logging,
Postgres, K8s)
ARG AIRFLOW_VERSION=3.0.4
ARG PY=3.11
ARG
CONSTRAINTS_URL=https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PY}.txt
# Put your minimal control requirements here
# e.g. amazon for S3 logs, postgres for metadata DB, k8s for executor API
calls
COPY requirements.txt /requirements.txt
RUN pip install --no-cache-dir -r /requirements.txt --constraint
${CONSTRAINTS_URL}
# Make sure OpenLineage is NOT present
RUN pip uninstall -y openlineage-airflow
apache-airflow-providers-openlineage || true
# Hard-disable at runtime in case the package sneaks in
ENV OPENLINEAGE_DISABLED=true
USER airflow
````
Scheduler logs after using the custom image without OpenLinenage:
```
[2025-08-18T13:34:37.311+0000] {settings.py:345} DEBUG - Setting up DB
connection pool (PID 8)
[2025-08-18T13:34:37.312+0000] {settings.py:454} DEBUG -
settings.prepare_engine_args(): Using pool settings. pool_size=5,
max_overflow=10, pool_recycle=1800, pid=8
[2025-08-18T13:34:37.340+0000] {configuration.py:861} DEBUG - Could not
retrieve value from section core, for key asset_manager_kwargs. Skipping
redaction of this conf.
[2025-08-18T13:34:37.340+0000] {configuration.py:861} DEBUG - Could not
retrieve value from section database, for key sql_alchemy_engine_args. Skipping
redaction of this conf.
[2025-08-18T13:34:37.352+0000] {cli_action_loggers.py:50} DEBUG - Adding
<function default_action_log at 0x7f9f00931b20> to pre execution callback
[2025-08-18T13:34:38.662+0000] {cli_action_loggers.py:78} DEBUG - Calling
callbacks: [<function default_action_log at 0x7f9f00931b20>]
[2025-08-18T13:34:38.806+0000] {providers_manager.py:356} DEBUG -
Initializing Providers Manager[config]
[2025-08-18T13:34:38.806+0000] {providers_manager.py:356} DEBUG -
Initializing Providers Manager[list]
[2025-08-18T13:34:38.992+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.amazon.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-amazon
[2025-08-18T13:34:39.007+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.celery.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-celery
[2025-08-18T13:34:39.011+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.cncf.kubernetes.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package
apache-airflow-providers-cncf-kubernetes
[2025-08-18T13:34:39.015+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.common.compat.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package
apache-airflow-providers-common-compat
[2025-08-18T13:34:39.017+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.common.io.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-common-io
[2025-08-18T13:34:39.019+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.common.messaging.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package
apache-airflow-providers-common-messaging
[2025-08-18T13:34:39.021+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.common.sql.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package
apache-airflow-providers-common-sql
[2025-08-18T13:34:39.026+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.docker.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-docker
[2025-08-18T13:34:39.028+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.elasticsearch.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package
apache-airflow-providers-elasticsearch
[2025-08-18T13:34:39.031+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.fab.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-fab
[2025-08-18T13:34:39.034+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.ftp.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-ftp
[2025-08-18T13:34:39.037+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.git.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-git
[2025-08-18T13:34:39.039+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.google.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-google
[2025-08-18T13:34:39.053+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.grpc.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-grpc
[2025-08-18T13:34:39.055+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.hashicorp.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-hashicorp
[2025-08-18T13:34:39.057+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.http.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-http
[2025-08-18T13:34:39.059+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.microsoft.azure.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package
apache-airflow-providers-microsoft-azure
[2025-08-18T13:34:39.065+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.mysql.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-mysql
[2025-08-18T13:34:39.068+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.odbc.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-odbc
[2025-08-18T13:34:39.070+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.postgres.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-postgres
[2025-08-18T13:34:39.072+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.redis.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-redis
[2025-08-18T13:34:39.084+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.sendgrid.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-sendgrid
[2025-08-18T13:34:39.087+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.sftp.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-sftp
[2025-08-18T13:34:39.089+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.slack.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-slack
[2025-08-18T13:34:39.092+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.smtp.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-smtp
[2025-08-18T13:34:39.094+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.snowflake.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-snowflake
[2025-08-18T13:34:39.096+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.ssh.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-ssh
[2025-08-18T13:34:39.097+0000] {providers_manager.py:598} DEBUG - Loading
EntryPoint(name='provider_info',
value='airflow.providers.standard.get_provider_info:get_provider_info',
group='apache_airflow_provider') from package apache-airflow-providers-standard
[2025-08-18T13:34:39.099+0000] {providers_manager.py:359} DEBUG -
Initialization of Providers Manager[list] took 0.29 seconds
[2025-08-18T13:34:39.099+0000] {configuration.py:1871} DEBUG - Loading
providers configuration
[2025-08-18T13:34:39.128+0000] {providers_manager.py:359} DEBUG -
Initialization of Providers Manager[config] took 0.32 seconds
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2025-08-18T13:34:39.496+0000] {plugins_manager.py:353} DEBUG - Loading
plugins
[2025-08-18T13:34:39.496+0000] {plugins_manager.py:269} DEBUG - Loading
plugins from directory: /opt/airflow/plugins
[2025-08-18T13:34:39.496+0000] {plugins_manager.py:249} DEBUG - Loading
plugins from entrypoints
[2025-08-18T13:34:39.497+0000] {listener.py:37} DEBUG - Calling
'on_starting' with {'component': <airflow.jobs.job.Job object at
0x7f9f082bb650>}
[2025-08-18T13:34:39.497+0000] {listener.py:38} DEBUG - Hook impls: []
[2025-08-18T13:34:39.497+0000] {listener.py:42} DEBUG - Result from
'on_starting': []
[2025-08-18T13:34:39.756+0000] {scheduler_job_runner.py:996} INFO - Starting
the scheduler
[2025-08-18T13:34:39.761+0000] {scheduler_job_runner.py:1006} DEBUG - Using
DatabaseCallbackSink as callback sink.
[2025-08-18T13:34:39.761+0000] {executor_loader.py:257} DEBUG - Loading
executor :KubernetesExecutor: from core
[2025-08-18T13:45:58.757+0000] {scheduler_job_runner.py:288} INFO - Exiting
gracefully upon receiving signal 15
[2025-08-18T13:45:58.760+0000] {executor_loader.py:257} DEBUG - Loading
executor :KubernetesExecutor: from core
```
Any other clues or pointers ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
