ashb commented on a change in pull request #4396: [AIRFLOW-3585] - Add edges to
database
URL: https://github.com/apache/airflow/pull/4396#discussion_r256762858
##########
File path: airflow/models/__init__.py
##########
@@ -3118,6 +3150,50 @@ def date_range(self, start_date, num=None,
end_date=timezone.utcnow()):
start_date=start_date, end_date=end_date,
num=num, delta=self._schedule_interval)
+ def create_edges(self, graph_id):
+ """
+ This will create all tasks and edges for a single execution date.
+ This will not push to the database.
+ :param execution_date: Execution date of tasks
+ :return:
+ """
+ edges = []
+ for task in self.task_dict.values():
+ for down in task.downstream_task_ids:
+ edges.append(DagEdge(self.dag_id, graph_id, down,
task.task_id))
+ return edges
+
+ def create_tis(self, execution_date):
+ """
+ This will create all tasks and edges for a single execution date.
+ This will not push to the database.
+ :param execution_date: Execution date of tasks
+ :return:
+ """
+ tasks = []
+ tis = []
+ for task in self.task_dict.values():
+ if task.start_date is None:
+ tasks.append(task)
+ elif task.start_date <= execution_date:
+ tasks.append(task)
+ for task in tasks:
+ tis.append(TaskInstance(task=task, execution_date=execution_date))
+ return tis
+
+ def get_db_tis_and_edges(self, execution_date):
+ with create_session() as session:
+ dag_run = session.query(DagRun)\
+ .filter(DagRun.execution_date == execution_date)\
+ .filter(DagRun.dag_id == self.dag_id).one()
Review comment:
This function feels like it belongs on DagRun - since a DagRun is an
instance of a dag for a given date.
It already has `get_task_instances()` for instance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
