shubhgurav0590 commented on PR #67369:
URL: https://github.com/apache/airflow/pull/67369#issuecomment-4532093944
"Hi @jroachgolf84 — you're correct, I reverted my initial fix after CI
showed it created a DAG cycle.
My original fix added
create_databricks_workflow_task.set_upstream(root_task) inside the for
root_task in roots loop. But roots contains INTERNAL workflow tasks, so this
created a circular dependency: launch → inner_task → launch.
The actual bug from #51598 is that when external_task >> task_group is set
OUTSIDE the with block, the launch task ignores that external upstream
dependency and runs immediately.
My proposed correct fix would be to transfer the task group's upstream
dependencies to the launch task inside __exit__:
for upstream_task_id in self.upstream_task_ids:
upstream_task = self.dag.get_task(upstream_task_id)
create_databricks_workflow_task.set_upstream(upstream_task)
Could you confirm if this is the right approach before I implement it? I
want to make sure I'm solving the actual bug correctly."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
