jscheffl commented on code in PR #52952:
URL: https://github.com/apache/airflow/pull/52952#discussion_r2188628976
##########
providers/edge3/src/airflow/providers/edge3/plugins/edge_executor_plugin.py:
##########
@@ -213,12 +214,18 @@ def change_maintenance_comment(self, worker_name: str):
except AirflowConfigException:
EDGE_EXECUTOR_ACTIVE = False
+# Load the API endpoint only on api-server (Airflow 3.x) or webserver (Airflow
2.x)
+if AIRFLOW_V_3_0_PLUS:
+ RUNNING_ON_APISERVER = sys.argv[1] in ["api-server"] if len(sys.argv) > 1
else False
+else:
+ RUNNING_ON_APISERVER = "gunicorn" in sys.argv[0] and "airflow-webserver"
in sys.argv
Review Comment:
Thanks for the general comment. I agree and ad-hoc I have no "something"
better idea. I would call it a "bootstrap" problem of dependencies.
From my experience the plugins_manager is very central in the middle.
Whereas I am not sure whether the "@providers_configurations_loaded" is the
right approach. I feel like in a couple of areas we initialize too much.
Challenge is that we have passed the release of 3.0 and if we change this it
will be very fundamental- Calling for a 4.0 is too early so we need to be able
to accept short term the situation and if we want to make something "better" we
need to make it backwards-compatible. Which makes it even harder.
Calling for ideas on devlist would be a good option. But if you did already
it might end-up in the same. At least I will have some sleeps about it... but I
am not sure if I will have any better idea...
Immediate thinking: Yes, do NOT load all providers in plugins manager and if
especially requested only load the things needed, e.g. hooks.
If we would move-away from a pythonic loading of code, one option would be
to describe provider details only in YAML and load this as data/information and
execute the Python code only on demand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
