opeida commented on issue #44354:
URL: https://github.com/apache/airflow/issues/44354#issuecomment-3174227992

   @kaxil sure! The mentioned issue was observed on 3.0.3. I will update my 
environment to 3.0.4 and come back with the results.
   
   ### Issue description
   The issue was that, after functioning correctly for a few runs, some of the 
DAGs randomly disappeared from the UI and weren't scheduled to run. The 
dag-processor failed to import them and logged the following error in their log 
files:
   ```
   
{"timestamp":"2025-07-02T00:00:43.214131Z","level":"warning","event":"Process 
exited abnormally","exit_code":126,"logger":"processor"}
   ```
   At the same time, `kubectl logs <dag-processor-pod>` contained the following 
error:
   ```
   --- Supervised process Last chance exception handler ---
   Traceback (most recent call last):
     File 
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py",
 line 377, in _fork_main
       target()
     File 
"/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/processor.py",
 line 123, in _parse_file_entrypoint
       result = _parse_file(msg, log)
                ^^^^^^^^^^^^^^^^^^^^^
     File 
"/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/processor.py",
 line 139, in _parse_file
       _execute_callbacks(bag, msg.callback_requests, log)
     File 
"/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/processor.py",
 line 179, in _execute_callbacks
       raise NotImplementedError(
   NotImplementedError: Haven't coded Task callback yet - 
https://github.com/apache/airflow/issues/44354!
   ```
   ### Little research
   After some googling, I assumed the `126 error` may be a permission issue but 
found nothing here. The files were synced with `git-sync` sidecars without a 
persistence directory. They had the following permissions/ownership in the 
dag-processor container:
   ```
   
airflow@airflow-dag-processor-6b9ddd54db-cpxf6:/opt/airflow/dags/repo/dags/clicks_daily_report$
 ls -la
   total 24
   drwxr-sr-x 2 65533 root 4096 Jul 23 15:56 .
   drwxr-sr-x 5 65533 root 4096 Jul 23 15:56 ..
   -rw-r--r-- 1 65533 root    0 Jul 23 15:56 __init__.py
   -rw-r--r-- 1 65533 root 2212 Jul 23 15:56 clicks_daily_report.json
   -rw-r--r-- 1 65533 root 7116 Jul 23 15:56 clicks_daily_report.py
   -rw-r--r-- 1 65533 root 3717 Jul 23 15:56 helper.py
   ```
   The executable bits in permissions are missing but @potiuk told he doesn't 
believe Dag file processor uses executable bits of Dags for anything.
   
   ### Environment configuration
   The deployed Airflow instance was based on the official resources only: GKE 
cluster deployed with Helm chart v1.18.0 and an extended image based on 
`apache/airflow:slim-3.0.3-python3.12`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to