potiuk commented on a change in pull request #6596: [AIRFLOW-6004] Untangle Executors class to avoid cyclic imports URL: https://github.com/apache/airflow/pull/6596#discussion_r352306164
########## File path: airflow/jobs/scheduler_job.py ########## @@ -35,21 +35,24 @@ from sqlalchemy import and_, func, not_, or_ from sqlalchemy.orm.session import make_transient -from airflow import executors, models, settings +from airflow import models, settings from airflow.configuration import conf from airflow.exceptions import AirflowException +from airflow.executors.local_executor import LocalExecutor +from airflow.executors.sequential_executor import SequentialExecutor from airflow.jobs.base_job import BaseJob from airflow.models import DAG, DagRun, SlaMiss, errors +from airflow.models.taskinstance import SimpleTaskInstance from airflow.stats import Stats from airflow.ti_deps.dep_context import SCHEDULEABLE_STATES, SCHEDULED_DEPS, DepContext from airflow.ti_deps.deps.pool_slots_available_dep import STATES_TO_COUNT_AS_RUNNING from airflow.utils import asciiart, helpers, timezone from airflow.utils.dag_processing import ( - AbstractDagFileProcessor, DagFileProcessorAgent, SimpleDag, SimpleDagBag, SimpleTaskInstance, - list_py_file_paths, + AbstractDagFileProcessor, DagFileProcessorAgent, SimpleDag, SimpleDagBag, Review comment: Because I realised that SimpleTaskInstance is actually extension of the TaskInstance. Placing it in the dag_processing package was part of the dependency problem. SimpleTaskInstance is just a simpler representation of the Task Instance and it belongs there - not in the DagProcessor. It uses TaskInstance but does not use DagProcessing. There is a chain of dependencies that STI brings if you keep it in DagProcessing, because when you want to use STI you automatically use DagProcessing - which causes cyclic dependencies. Moving STI to TI made it so much simpler and got rid of a number of cyclic deps. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
