noelmcloughlin edited a comment on issue #10026: URL: https://github.com/apache/airflow/issues/10026#issuecomment-906811467
I see "_pid not matching_" issue in Airflow 2.13/CeleryWorker/RMQ when manually triggering DAGS. Was not happening in airflow 2.1.2 (+cherrypick from https://github.com/apache/airflow/pull/16860). I'm not using run_as_user. I cannot replicate the issue when DAG has single task, only with multiple tasks, my testing listed below. I'm have ntp enabled. Issue happens on branching into parallelization of tasks, not sequential DAG1 (BashOp) task1 Success DAG2 (BashOp) taskgroup1(task1): Recorded pid 14206 does not match the current pid 14207 taskgroup2(task1): Recorded pid 18247 does not match the current pid 18248 DAG3 (PyOp) Screenshot below, [this DAG](https://github.com/teamclairvoyant/airflow-maintenance-dags/blob/master/backup-configs/airflow-backup-configs.py) .. The green "backup-airflow-config" task terminated with "_State of this instance has been externally set to success. Terminating instance_" but I don't know why, so maybe different issue, the yellow tasks were terminated too.  DAG4 (PyOP,ShortCircuitOp) task1 >> task2 >> task3 success -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
