Going through the discussions above, I was leaning towards the idea of removing it completely initially but came across Pierre's idea after. I like that!
Thanks & Regards, Amogh Desai On Wed, Aug 30, 2023 at 1:58 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Whoa.. I am glad I started it... I see some really good ideas here :). > > On Wed, Aug 30, 2023 at 9:20 AM Pierre Jeambrun <pierrejb...@gmail.com> > wrote: > > > Maybe we can introduce a global “DEBUG” config option/env variable. This > > could control some more verbose logging but most importantly only when > this > > is turned on, we could use the sequential executor, debug executor, > > _PIP_ADDITIONAL_REQUIREMENTS > > and any other “debug/development” purpose options. > > > > Like for other frameworks this would be documented and user would have a > > warning when using it. > > > > Then there is nothing more we can do if the user puts in production a > > cluster in debug mode, this would be deliberate. (Like set the debug > option > > to True and then use the sequential executor). > > > > This could also automatically start the web server in debug mode and > other > > components in a similar way when detected to true. > > > > We could even limit the number of workers to 1 or other things that would > > make it unsuitable in a production environment. > > > > Just an idea. > > > > On Wed 30 Aug 2023 at 03:55, Pankaj Koti <pankaj.k...@astronomer.io > > .invalid> > > wrote: > > > > > I agree with Maciej's rationale here and inclined towards removing it > > > straight away. > > > > > > Also, is it the case that users have discovered this flag themselves > > > without us documenting it anywhere as a feature to use? > > > > > > *We have a leading underscore for the variable and it hints that it's > for > > > internal use.* > > > > > > I would be up for failing the image without any deprecation/warning if > we > > > have not documented it as a feature to use. > > > > > > On Wed, 30 Aug 2023, 03:16 Oliveira, Niko, <oniko...@amazon.com.invalid > > > > > wrote: > > > > > > > I'd vote for a period of time with warnings (either in the logs > and/or > > in > > > > the Airflow UI), as a deprecation warning of sorts. Followed by > > removing > > > > the feature later on, unless we find that the warnings are enough to > > > lower > > > > the operational load this causes us, but I think that's unlikely. > > > > > > > > Cheers, > > > > Niko > > > > > > > > ________________________________ > > > > From: Jed Cunningham <jedcunning...@apache.org> > > > > Sent: Tuesday, August 29, 2023 10:05:01 AM > > > > To: dev@airflow.apache.org > > > > Subject: RE: [EXTERNAL] [DISCUSS] Preventing users from misusing > > > > _PIP_ADDITIONAL_REQUIREMENTS ? > > > > > > > > CAUTION: This email originated from outside of the organization. Do > not > > > > click links or open attachments unless you can confirm the sender and > > > know > > > > the content is safe. > > > > > > > > > > > > > > > > I also don't like the 10 minute thing. I'd rather we remove it, or > > > display > > > > a message like we do sequential executor (we can only do so much, > this > > is > > > > as visible as we can make it really), I think in that order? > > > > > > > > > >
