o-nikolas commented on issue #27929:
URL: https://github.com/apache/airflow/issues/27929#issuecomment-1352642516
> I tried using the below code to get the Executor, which leads to a
circular dependency error.
Hey @utkarsharma2!
Unfortunately it's not too surprising, `conf` and `settings` are both used
in **many** places (the latter importing the former and using it quite a bit)
and importing an executor is causing a bridge between the two. You can see
`settings` imports `conf` which when it's loaded calls `validate()` on itself,
during which an executor is now imported with your change (in this case the
sequential executor) which then imports the `BaseExecutor` (of course) and that
thing imports a bunch of stuff which all use `settings` in the end. So tl;dr,
importing a module inside conf turns out to be dangerous stuff! :laughing:
You can try some things like locally importing modules only where they're
needed, or limiting imports only if they're used for type checking. I played
around quickly and didn't find a clean way out of the loop but I do encourage
you to have a look as well now that you have more context.
The ultimate solution may be to move the executor config validation to
runtime. i.e. the executor will check if it's incompatible with the DB being
used during startup and throw if it's now (which can actually be pretty well
encapsulated in the base executor actually).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
