potiuk commented on issue #6601: [AIRFLOW-6010] Remove cyclic imports and pylint disables URL: https://github.com/apache/airflow/pull/6601#issuecomment-555870456 @ashb @feluelle - but also @kaxil and @coufon as a lot of cyclic import problems were caused by DAG serialization, so please take a close look - maybe I overlooked something. I indeed went back to the drawing board and I think I fixed the problems with cyclic imports in simple, and elegant way. I yet have to fix migration for serialization table and run the tests, but I have all the reasons to believe, that the changes I made (mainly in imports but also few other places) are good and they will lead to much nicer and easier-to-reason-about code. Here is the detailed description of the changes: ``` There were a number of problems involving cyclic imports in Airflow's core. Mainly about settings, DAG context management, base operator imports and serialisation. Some of those problems were workarounded by #pylint: disables (for pylint), some of them were bypassed with TYPE_CHECKING (for mypy) and some of them were just hidden because pylint check was splitting file lists while TravisiCI build. This commit fixes most of the problems (only executor problem is left) and removes all the workarounds. The fixes are: * Context for DAG context management was loaded from settings and Now context managemen is moved to DAG and 'import settings' is not needed in baseoperator, subdag_operator. * Serialized Fields are lazy initialised - they were previously initialized while parsing the python modules which made it impossible to avoid cycles. * SerializedDagModel is removed from 'airflow.models' and imported directly from serialization package. This is only internal class and does not need to be exposed via models * BaseOperator in core is imported from baseoperator package rather than from 'airflow.models'. This helps in importing the whole airflow __init__ of 'airflow' without having to pay attention to the sequence of imports there. * BaseOperator on the other hand is imported from 'airflowi.models' in operators/DAGs/hooks/sensors. This is important for Backporting (AIP-21) * The imports of BaseOperator are enforced with pre-commit. * All the pylint/mypy hacks related to cyclic imports are removed. ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
