Maybe we can introduce a global “DEBUG” config option/env variable. This could control some more verbose logging but most importantly only when this is turned on, we could use the sequential executor, debug executor, _PIP_ADDITIONAL_REQUIREMENTS and any other “debug/development” purpose options.
Like for other frameworks this would be documented and user would have a warning when using it. Then there is nothing more we can do if the user puts in production a cluster in debug mode, this would be deliberate. (Like set the debug option to True and then use the sequential executor). This could also automatically start the web server in debug mode and other components in a similar way when detected to true. We could even limit the number of workers to 1 or other things that would make it unsuitable in a production environment. Just an idea. On Wed 30 Aug 2023 at 03:55, Pankaj Koti <pankaj.k...@astronomer.io.invalid> wrote: > I agree with Maciej's rationale here and inclined towards removing it > straight away. > > Also, is it the case that users have discovered this flag themselves > without us documenting it anywhere as a feature to use? > > *We have a leading underscore for the variable and it hints that it's for > internal use.* > > I would be up for failing the image without any deprecation/warning if we > have not documented it as a feature to use. > > On Wed, 30 Aug 2023, 03:16 Oliveira, Niko, <oniko...@amazon.com.invalid> > wrote: > > > I'd vote for a period of time with warnings (either in the logs and/or in > > the Airflow UI), as a deprecation warning of sorts. Followed by removing > > the feature later on, unless we find that the warnings are enough to > lower > > the operational load this causes us, but I think that's unlikely. > > > > Cheers, > > Niko > > > > ________________________________ > > From: Jed Cunningham <jedcunning...@apache.org> > > Sent: Tuesday, August 29, 2023 10:05:01 AM > > To: dev@airflow.apache.org > > Subject: RE: [EXTERNAL] [DISCUSS] Preventing users from misusing > > _PIP_ADDITIONAL_REQUIREMENTS ? > > > > CAUTION: This email originated from outside of the organization. Do not > > click links or open attachments unless you can confirm the sender and > know > > the content is safe. > > > > > > > > I also don't like the 10 minute thing. I'd rather we remove it, or > display > > a message like we do sequential executor (we can only do so much, this is > > as visible as we can make it really), I think in that order? > > >
