Hello everyone, After thinking through whether the dask provider should be pre-installed or not, it got me thinking about whether we should even be pre-installing the celery and k8s providers. Core Airflow so far hasn't had the dependencies required to make those executors functional either - users either had to use the extra or install the provider directly. So that doesn't really change if we choose not to preinstall the providers.
Why would it be beneficial to not pre-install them? The most obvious benefit is you don't get a handful of new required deps you aren't using (like celery/flower/billiard/etc if you don't use CeleryExecutor). However another benefit is we wouldn't have to set a minimum version for these 2 providers to ensure we have the version with the executors, allowing an "old" k8s provider (KPO itself is pretty common after all) to be used if you aren't using KubernetesExecutor. The one downside here is how do we make sure users have a new enough provider, one that has their executor of choice in it. After chatting with Jarek, he brought up that we could also make the error messaging more helpful in this situation. This messaging would basically solve the problem without needing minimum version pinning in core. All that said, I think we should strongly consider not pre-installing the celery and k8s providers as well. Thoughts? Jed
