[
https://issues.apache.org/jira/browse/AIRFLOW-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889075#comment-16889075
]
Ryan Pennell edited comment on AIRFLOW-4982 at 7/19/19 6:04 PM:
----------------------------------------------------------------
I don't think that will work either. python 3 loads the \_\_init\_\_.py of the
complete path, so the entry point load will still be accessed before
AirflowPlugin can be imported. So airflow/\_\_init\_\_.py will be executed when
you "from airflow.plugin_manager import AirflowPlugin". The only way around
this would be to isolate the entry point loading from AirflowPlugin.
It could be placed in it's own file (airflow/bases.py?), but this would mean
the statement "import airflow" cannot execute the entry point load and is a big
disadvantage of having classes as extendable plugins.
Ideally, the plugin would not be a class, but discovered by the structure of a
package or module. This way, you don't have to import anything in order to make
an airflow plugin (no inheritance). This would also mean the
AirflowPlugin.on_load and AirflowPlugin.validate would have to be moved
(perhaps just in plugin_manager.py). You could even make it
backwards-compatable be just having the package/module validation discover the
same class variable (name, operators, sensors, etc)
I can contribute outside of work, but I would not be comfortable making these
changes without more input from the community. This change would also
effectively deprecate the "AirflowPlugin" class
was (Author: rpennell):
I don't think that will work either. python 3 loads the __init__.py of the
complete path, so the entry point load will still be accessed before
AirflowPlugin can be imported. So airflow/__init__.py will be executed when
you "from airflow.plugin_manager import AirflowPlugin". The only way around
this would be to isolate the entry point loading from AirflowPlugin.
It could be placed in it's own file (airflow/bases.py?), but this would mean
the statement "import airflow" cannot execute the entry point load and is a big
disadvantage of having classes as extendable plugins.
Ideally, the plugin would not be a class, but discovered by the structure of a
package or module. This way, you don't have to import anything in order to
make an airflow plugin (no inheritance). This would also mean the
AirflowPlugin.on_load and AirflowPlugin.validate would have to be moved
(perhaps just in plugin_manager.py). You could even make it
backwards-compatable be just having the package/module validation discover the
same class variable (name, operators, sensors, etc)
I can contribute outside of work, but I would not be comfortable making these
changes without more input from the community. This change would also
effectively deprecate the "AirflowPlugin" class
> New entry_point load causes circular dependency
> -----------------------------------------------
>
> Key: AIRFLOW-4982
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4982
> Project: Apache Airflow
> Issue Type: Bug
> Components: plugins
> Affects Versions: 1.10.3
> Reporter: Ryan Pennell
> Priority: Minor
>
> I believe there is a bug with the current method used to load airflow plugins
> with setuptools entry_points.
> When creating a plugin, the current recommended method is (in myplugin.py):
> {code:python}
> from airflow.plugin_manager import AirflowPlugin
> class MyCustomPlugin(AirflowPlugin):
> name='myplugin'
> hooks = []
> {code}
> Also in plugins manager (~line 149) is the code to load the entry point.
> Unless I'm missing something, doesn't this lead to a circular load? If I try
> to use import airflow.operators.myplugin, the entry point will be loaded. So
> the entry point loads myplugin.py, which imports airflow.plugin_manager
> because it needs to inherit from AirflowPlugin. But when myplugin.py imports
> plugin_manager, it tries to load the entry points again. An because it
> hasn't completed the import of AirflowPlugin, the class MyCustomPlugin does
> not exists yet, so the import fails.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
