Whoa.. I am glad I started it... I see some really good ideas here :). On Wed, Aug 30, 2023 at 9:20 AM Pierre Jeambrun <pierrejb...@gmail.com> wrote:
> Maybe we can introduce a global “DEBUG” config option/env variable. This > could control some more verbose logging but most importantly only when this > is turned on, we could use the sequential executor, debug executor, > _PIP_ADDITIONAL_REQUIREMENTS > and any other “debug/development” purpose options. > > Like for other frameworks this would be documented and user would have a > warning when using it. > > Then there is nothing more we can do if the user puts in production a > cluster in debug mode, this would be deliberate. (Like set the debug option > to True and then use the sequential executor). > > This could also automatically start the web server in debug mode and other > components in a similar way when detected to true. > > We could even limit the number of workers to 1 or other things that would > make it unsuitable in a production environment. > > Just an idea. > > On Wed 30 Aug 2023 at 03:55, Pankaj Koti <pankaj.k...@astronomer.io > .invalid> > wrote: > > > I agree with Maciej's rationale here and inclined towards removing it > > straight away. > > > > Also, is it the case that users have discovered this flag themselves > > without us documenting it anywhere as a feature to use? > > > > *We have a leading underscore for the variable and it hints that it's for > > internal use.* > > > > I would be up for failing the image without any deprecation/warning if we > > have not documented it as a feature to use. > > > > On Wed, 30 Aug 2023, 03:16 Oliveira, Niko, <oniko...@amazon.com.invalid> > > wrote: > > > > > I'd vote for a period of time with warnings (either in the logs and/or > in > > > the Airflow UI), as a deprecation warning of sorts. Followed by > removing > > > the feature later on, unless we find that the warnings are enough to > > lower > > > the operational load this causes us, but I think that's unlikely. > > > > > > Cheers, > > > Niko > > > > > > ________________________________ > > > From: Jed Cunningham <jedcunning...@apache.org> > > > Sent: Tuesday, August 29, 2023 10:05:01 AM > > > To: dev@airflow.apache.org > > > Subject: RE: [EXTERNAL] [DISCUSS] Preventing users from misusing > > > _PIP_ADDITIONAL_REQUIREMENTS ? > > > > > > CAUTION: This email originated from outside of the organization. Do not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > I also don't like the 10 minute thing. I'd rather we remove it, or > > display > > > a message like we do sequential executor (we can only do so much, this > is > > > as visible as we can make it really), I think in that order? > > > > > >
