jscheffl commented on code in PR #44454:
URL: https://github.com/apache/airflow/pull/44454#discussion_r1864175163
##########
providers/src/airflow/providers/edge/cli/edge_command.py:
##########
@@ -59,29 +57,6 @@
)
-@providers_configuration_loaded
-def force_use_internal_api_on_edge_worker():
- """
- Ensure that the environment is configured for the internal API without
needing to declare it outside.
-
- This is only required for an Edge worker and must to be done before the
Click CLI wrapper is initiated.
- That is because the CLI wrapper will attempt to establish a DB connection,
which will fail before the
- function call can take effect. In an Edge worker, we need to "patch" the
environment before starting.
- """
- if "airflow" in sys.argv[0] and sys.argv[1:3] == ["edge", "worker"]:
- api_url = conf.get("edge", "api_url")
- if not api_url:
- raise SystemExit("Error: API URL is not configured, please correct
configuration.")
- logger.info("Starting worker with API endpoint %s", api_url)
- # export Edge API to be used for internal API
- os.environ["AIRFLOW_ENABLE_AIP_44"] = "True"
- os.environ["AIRFLOW__CORE__INTERNAL_API_URL"] = api_url
- InternalApiConfig.set_use_internal_api("edge-worker")
Review Comment:
Yes, it should. Theproblem is that in Airflow 2.10 there is a bit of
validation at point of CLI parsing which impliocitly invokes ExecutorLoader and
this indirectly checks for which executor is used and this is compatible in
regards of SQLite. This patching was needed to work-around these validations.
As of the removal - which totally make sense in main/Airflow 3 - the
entrypoint was broken in Airflow 2.10 which we do not clean or repair at this
stage. Scratched my head 2 hours yesterday trying to make it / restore it such
that it is working with Airflow 2.10 again.
I did not complete a deployment test with Airflow 2.10 and Edge Executor,
really running a CLI worker in the CI together with the server... individually
all Unit tests are working but you see how it is broken if you call `breeze
down && breeze release-management prepare-provider-packages
--include-not-ready-providers edge && breeze start-airflow --python 3.12
--load-example-dags --backend postgres --executor EdgeExecutor --answer y
--use-airflow-version 2.10.3 --use-packages-from-dist`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
