noelmcloughlin edited a comment on issue #10026: URL: https://github.com/apache/airflow/issues/10026#issuecomment-906811467
I see "_pid not matching_" issue in Airflow 2.1.3/CeleryWorker/RMQ when manually triggering DAGS. Was not happening in airflow 2.1.2 (+cherrypick from https://github.com/apache/airflow/pull/16860). I'm not using run_as_user. I cannot replicate the issue when DAG has single task, only with multiple tasks, my testing listed below. I'm have ntp enabled. Issue happens on branching into parallelization of tasks, not sequential DAG1 (BashOp) task1 Success DAG2 (BashOp) taskgroup1(task1): Recorded pid 14206 does not match the current pid 14207 taskgroup2(task1): Recorded pid 18247 does not match the current pid 18248 DAG3 (PyOP,ShortCircuitOp) task1 >> task2 >> task3 success -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
