GitHub user rcrchawla added a comment to the discussion: Airflow task failed
but spark kube app is running
Sure. I have suggested the above point to deployment team, they are currently
considering to the point and will keep you posted about the further updates.
Also today I have noticed in our api server same mysql issue occur @02:47 am
UTC which occured yesterday, in worker logs I see the below logs of it
2026-03-13 02:45:42.277543 [info ] [2ae514fc-eb08-4617-86b9-ae92e5a95011]
Executing workload in Celery: token='eyJ***'
ti=TaskInstance(id=UUID('019ce4a7-62a5-7a9d-9004-06b3f1f73368'),
task_id='etl_load_to_ods_group_id_2.reconcile_meta_group_id_2',
dag_id='segment-groupid-2', run_id='scheduled__2026-03-13T00:45:00+00:00',
try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=9,
executor_config=None, parent_context_carrier={}, context_carrier={},
queued_dttm=None) dag_rel_path=PurePosixPath('segment_groupid_2.py')
bundle_info=BundleInfo(name='dags-folder', version=None)
log_path='dag_id=segment-groupid-2/run_id=scheduled__2026-03-13T00:45:00+00:00/task_id=etl_load_to_ods_group_id_2.reconcile_meta_group_id_2/attempt=1.log'
type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils]
2026-03-13 02:45:42.309091 [info ] Secrets backends loaded for worker
[supervisor] backend_classes=['EnvironmentVariablesBackend'] count=1
Till 02:45 **2ae514fc-eb08-4617-86b9-ae92e5a95011** runs fine
2026-03-13 02:46:00.160867 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:00.342989 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:00.575250 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:02.209653 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:02.780170 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:03.383007 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:03.855494 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 1st time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:05.441315 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:05.606781 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:06.091219 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:07.350774 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:07.940790 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:09.082668 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:09.262903 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 2nd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:11.170876 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:11.614824 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:12.320942 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:13.038009 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:13.884969 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:15.234633 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:15.896211 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 3rd time calling it.
[airflow.sdk.api.client]
2026-03-13 02:46:17.023386 [warning ] Starting call to
'airflow.sdk.api.client.Client.request', this is the 4th time calling it.
[airflow.sdk.api.client]
[31mΓò░ΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓò»[0m
[1;91mReadTimeout: [0mtimed out
**2026-03-13 02:47:00.136773 [error ] Task
execute_workload[2ae514fc-eb08-4617-86b9-ae92e5a95011] raised unexpected:
ReadTimeout('timed out') [celery.app.trace]**
[31mΓò¡ΓöÇ[0m[31mΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇ[0m[31m
[0m[1;31mTraceback [0m[1;2;31m(most recent call last)[0m[31m [0m
GitHub link:
https://github.com/apache/airflow/discussions/63298#discussioncomment-16111675
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
