nclaeys commented on issue #20982: URL: https://github.com/apache/airflow/issues/20982#issuecomment-1020530647
Well I do not think the problem is in the adopt_task_instances but I am not sure what the expected behavior should be. It only tries adopting task instances that have the queued_by_job_id field filled in, in which case it changes the labels identifying the previous scheduler pod name with the new one in order to watch them (makes sense in my opinion). In the manually triggered run, we do not have a queued_by_job_id so it is not adopted. If I am not mistaken the run button triggers a local execution of the task within the web pod and creates a local kubernetes executor to launch the task, which is different than a task that is triggered from the scheduler pod itself. I am not sure what the expected behavior should be: - we could change the query to find orphaned tasks - we should make sure the queued_by_job_id is filled in even if it is a local execution of the job? But in that case we probably do not want it to be adopted by the scheduler pod as it is already managed by the web pod... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
