[
https://issues.apache.org/jira/browse/AIRFLOW-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889507#comment-16889507
]
immerrr again commented on AIRFLOW-4982:
----------------------------------------
I have found a workaround for this circular dependency. It only works if the
extension classes reside in a package, and the workaround is to import
"airflow" from that package's __init__.py. It works as follows:
* you request loading "mypackage.myplugin"
* Python first loads "mypackage"
* "mypackage" imports "airflow" which indirectly imports
"airflow.plugins_manager"
* "airflow.plugins_manager" defines "AirflowPlugin" base class and then
proceeds to load plugins importing from "mypackage.plugin"
* at this point
** "mypackage" package is being loaded
** "airflow" package is being loaded
** "airflow.plugins_manager" is being loaded
* so technically the circular dependency is still there, but the difference is
that only now "mypackage.myplugin" starts being imported
* and by the moment "mypackage.myplugin" does "from airflow.plugins_manager
import AirflowPlugin" "airflow.plugins_manager" already has "AirflowPlugin"
class defined, so the inheritance works
Ideally, though, it should be possible to write plugins without invoking any of
the plugin loading magic and it would be postponed until it is actually
required. Looks like there are already some lazy loaders in place if you use
"airflow.operators.foo" as an attribute on "airflow" package rather than
"airflow.operators" module, so it could be a matter of one hook placed in just
the right spot.
> New entry_point load causes circular dependency
> -----------------------------------------------
>
> Key: AIRFLOW-4982
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4982
> Project: Apache Airflow
> Issue Type: Bug
> Components: plugins
> Affects Versions: 1.10.3
> Reporter: Ryan Pennell
> Priority: Minor
>
> I believe there is a bug with the current method used to load airflow plugins
> with setuptools entry_points.
> When creating a plugin, the current recommended method is (in myplugin.py):
> {code:python}
> from airflow.plugin_manager import AirflowPlugin
> class MyCustomPlugin(AirflowPlugin):
> name='myplugin'
> hooks = []
> {code}
> Also in plugins manager (~line 149) is the code to load the entry point.
> Unless I'm missing something, doesn't this lead to a circular load? If I try
> to use import airflow.operators.myplugin, the entry point will be loaded. So
> the entry point loads myplugin.py, which imports airflow.plugin_manager
> because it needs to inherit from AirflowPlugin. But when myplugin.py imports
> plugin_manager, it tries to load the entry points again. An because it
> hasn't completed the import of AirflowPlugin, the class MyCustomPlugin does
> not exists yet, so the import fails.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
