> This doesn't sound right to me. I think it could be right if we do two things:
1) ask the user at migration time "Are you using this feature? Are you ok with changing the default to switch it off" (we can also revert that question and ask them "are you ok with switching old default to new one"). 2) we have the new default for users who are installing airflow for the first time, I think currently we silently assume that both users who migrate from the old version of airflow have the same defaults as "new users". I **THINK** we can make a distinctions. a) old users could keep the default from the version they are migrating from and can get the question during migration (are you willing to keep the setting as it was before)? b) new users have the new default and can manually change it J, On Wed, Sep 6, 2023 at 7:57 PM Daniel Standish <daniel.stand...@astronomer.io.invalid> wrote: > > > > would definitely be in favour of that approach and using it more > > liberally. I think SemVer does not say anything about this case - "the > > software still supports it but you need to flip a flg" sounds like a nice > > way of introducing behavioural changes without breaking changes. > > > This doesn't sound right to me. > > According to what you're proposing, we could change the default to > `catchup=False` and add an option to flip it back. But then when a user > upgrades, many dags in the wild would stop "catching up", since it's not a > required parameter. > > This feels like it is totally forbidden by semver. I think it's different > from e.g., moving the executors out and forcing user to install them, > because that's effectively an "installation instructions" change whereas > this is a behavior change. > > > > > > > On Sat, Sep 2, 2023 at 1:21 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > What about more frequently using news fragments + using configuration > > settings as a way to introduce breaking changes that users can revert? > > > > I would definitely be in favour of that approach and using it more > > liberally. I think SemVer does not say anything about this case - "the > > software still supports it but you need to flip a flg" sounds like a nice > > way of introducing behavioural changes without breaking changes. > > > > We could even introduce some way (adding interactive `airflow db > migrate`) > > - to ask the user during migration what to do. That would make the > > automated and generally unattended Helm Chart migrations, but I am sure > we > > can figure out something there but I easily imagine that when we detect > > that user migrates from 2.7.0 to 2.8.0 and we have some feature disabled > > there by default we could ask the question "This feature in 2.8.0 is now > > disabled by default, do you want to enable it ? + link to documentation > > explaining what it is and why we disabled it". > > > > J. > > > > > > On Fri, Sep 1, 2023 at 11:30 PM Ryan Hatter > > <ryan.hat...@astronomer.io.invalid> wrote: > > > > > What about more frequently using news fragments + using configuration > > > settings as a way to introduce breaking changes that users can revert? > > > Disabling > > > "trigger dag with config" without Params by default > > > < > > > > > > https://airflow.apache.org/docs/apache-airflow/2.7.0/release_notes.html#the-trigger-ui-form-is-skipped-in-web-ui-if-no-parameters-are-defined-in-a-dag-33351 > > > > > > > is more or less what I'm referring to. While this was a breaking > change, > > > users were given the option to "roll back". > > > > > > On Thu, Aug 31, 2023 at 12:14 AM Amogh Desai <amoghdesai....@gmail.com > > > > > wrote: > > > > > > > Very good discussions going on here. > > > > > > > > Semver has been a point of concern for us too in our internal > product. > > > > Some ideas emerging out of this could help me there. Thanks, Jarek > and > > > > Niko. > > > > > > > > There are two points I'd like to stress on to say why semver is that > > > > important: > > > > > > > > - Compatibility: Without versioning that indicates compatibility, > > > > users might hesitate to upgrade due to concerns about potential > > > > breaking changes. This could lead to fragmentation, where different > > > > users are using different versions of the software, making support > and > > > > maintenance more challenging. > > > > > > > > - Release Communication: Semantic versioning helps communicate the > > > > impact of a release to users, maintainers, and contributors. Without > > > > clear versioning, understanding the significance of a release becomes > > > > more difficult. > > > > > > > > > > > > Thanks & Regards, > > > > Amogh Desai > > > > > > > > > > > > On Thu, Aug 31, 2023 at 3:56 AM Jarek Potiuk <ja...@potiuk.com> > wrote: > > > > > > > > > > > Now, one elephant in the room - the 5 year security patches thing > > > Jarek > > > > > brought up. I freely admit I haven't tracked this at all, so please > > > > correct > > > > > me if I'm wrong. If that ends up panning out though, I think we > will > > > have > > > > > to reassess our strategy with providers too. > > > > > > > > > > Just to answer the last point. Yes. I believe so. I was never a big > > fan > > > > of > > > > > introducing breaking changes in providers to make maintainers' > lives > > a > > > > bit > > > > > easier. While I was a big fan of doing it when we had a very good > > > reason. > > > > > > > > > > I think we have too many breaking changes in providers now. And we > do > > > it > > > > > because we "can" - we mostly only consider maintainer's lives > easier > > > > > currently. But I think when those regulations are in place we will > > have > > > > to > > > > > make a much more deliberate judgment on when we make a major > release > > > > > in providers and take on a deliberate decision "is it worth doing > it, > > > > > knowing that we will have to deliver patches for previous major > > > > versions". > > > > > This will be something that the regulations might make us think > about > > > > when > > > > > making a decision "should we break?". And when we do - we should be > > > > > prepared to cherry-pick security fixes to those old versions. We > > > > currently > > > > > can have a process for it - and it is off-loaded mainly to > > > stakeholders. > > > > > > > > > > > > > > > https://github.com/apache/airflow/blob/main/PROVIDERS.rst#mixed-governance-model > > > > > - where we mainly take care about releasing cherry-picked code. > > > > > > > > > > I believe the overwhelming majority of those "breaking" releases > that > > > we > > > > > really needed, were in providers where there is an active > stakeholder > > > > > already cooperating with us. I have - honestly quite an expectation > > > that > > > > > this will stay like that. In the proposed regulations, the > > stakeholders > > > > are > > > > > (much more than the OSS foundations) responsible for security of > the > > > > > software they provide to their users and they will have incentive > to > > > fix > > > > > and provide fixes also for past releases of those integration. And > I > > > > think > > > > > we can work out a collaborative model on that - very close to the > > > "mixed > > > > > governance" we already have. And in other cases where we have no > > active > > > > > stakeholders, we might simply refrain from making breaking changes > if > > > not > > > > > absolutely needed. > > > > > > > > > > That would be my approach to the elephant. > > > > > > > > > > J, > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > > > > > > >