Subject: RE: [EXTERNAL][DISCUSS] Should we pre-install celery/k8s providers?
CAUTION: This email originated from outside of the organization. Do not click
links or open attachments unless you can confirm the sender and know the
content is safe.
FYI. I will hold on with the formal vote until we have
FYI. I will hold on with the formal vote until we have a few more days
for discussion. This is an easy change.
J.
On Fri, Jul 21, 2023 at 8:38 PM Daniel Standish
wrote:
>
> Yeah it ain't the easiest of decisions.
>
> @niko, when I think about the staged approach, it feels like it is maybe
>
Yeah it ain't the easiest of decisions.
@niko, when I think about the staged approach, it feels like it is maybe
more disruptive than doing it all at once.
stage 1: force everyone to install k8s and celery libs -- that's one
disruption / risk
stage 2: remove pre-install -- another disruption
it
Yeah. I thought a bit more about that and the risks of forcing Airflow
2.7 users to use the latest k8s provider even if they do not use K8S
executor is much bigger (and much more difficult to handle) than risks
of not installing the provider in the right version.
Also I think there is one more
I second Jed's proposal and believe that the implications and benefits, as laid
out, make a lot of sense.
> we wouldn't have to set a minimum version for these 2 providers to ensure we
> have the version with the executors, allowing an "old" k8s provider (KPO
> itself is pretty common after
, 2023 10:07:44 AM
To: dev@airflow.apache.org
Subject: RE: [EXTERNAL][DISCUSS] Should we pre-install celery/k8s providers?
CAUTION: This email originated from outside of the organization. Do not click
links or open attachments unless you can confirm the sender and know the
content is safe
* *provider* extras or provider optional features
Yeah I think this is an important point:
>
> Core Airflow so far hasn't had the dependencies
> required to make those executors functional either - users either had to
> use the extra or install the provider directly. So that doesn't really
> change if we choose not to preinstall the providers.
>
I like this solution conceptually. I have not considered every angle, but
as long as there aren't any big gotchas, this seems like the right thing to
do.
Yes. I think this makes much more sense.
On Fri, Jul 21, 2023 at 6:19 PM Jed Cunningham wrote:
>
> Hello everyone,
>
> After thinking through whether the dask provider should be pre-installed or
> not, it got me thinking about whether we should even be pre-installing the
> celery and k8s
Hello everyone,
After thinking through whether the dask provider should be pre-installed or
not, it got me thinking about whether we should even be pre-installing the
celery and k8s providers. Core Airflow so far hasn't had the dependencies
required to make those executors functional either -
11 matches
Mail list logo
