eschutho opened a new issue #12566: URL: https://github.com/apache/superset/issues/12566
### Motivation As Superset moves to version 1.0, we want to keep this and future versions stable for users, while also communicating clearly to users when upgrading if they should expect any breaking changes, and how to prepare for new versions in updating their own code or usage of the application. ### Proposed Change The principles of Semantic Versioning are outlined [in this document](https://semver.org/) by Tom Preston-Werner co-founder of GitHub. The proposal is to follow these guidelines with regard to public releases of Superset. The definitions for what is a major/minor and patch release are described in the document, but in this document, we should clearly define what a breaking change on Superset is and then how to communicate it. We also want to set up guidelines for deprecating features and announcing those deprecations in the changelog and release notes, and then fully removing the feature in a breaking change at some later date with a major version release. The goal is to have fewer breaking changes and to aim for changes to be backward compatible and then grouped together in a major release. **What is a breaking change?** * CLI * Removal of a command * Addition of required argument flag * Removal of an argument flag in a way that removes functionality and could break any integrations * API * Removing/changing the url of a route * Adding/renaming required parameters * Addition of a required header * Feature or Config Flags * Removing a flag or switching the default in the config * Setting a new flag where the default would break any existing functionality * Database migrations * Migrations that take a lot of time and downtime or alternative methods of deployment needs to be accounted for or planned * UI * changes that would break existing charts, dashboards, queries, or favorites * User roles/permissions * Changes that would remove current functionality for a user * With discretion, any changes that give users a significant amount of permissions where they didn’t have permissions before could also be considered a breaking change and users should be given sufficient notification. * Python version upgrades * Any changes that are version bumps in Python that would require users to upgrade their systems. We currently support the last three minor versions of Python. * SQLAlchemy version upgrades * Any changes that are major version bumps in SQLAlchemy that would require users to upgrade their databases * Database connectors * Any changes that would remove functionality for a particular database that is currently supported **Best practices to make changes backward compatible:** * CLI * Add a new command and keep the existing command. Prompt users of the old command that it is deprecated and encourage them to use the new one. * API * Add new routes and leave the old routes intact. Message users in the changelog that the old API endpoints are deprecated and will be removed in the x.0 release. * Alternatively for larger systemic changes, create a new API version. * Feature or Config Flags * Before removing a flag that would create a breaking change, notate in the changelog and release notes that the flag/feature is deprecated and will be removed in the x.0 release. * When adding a new flag, the existing functionality should remain the default. Even though the new flag is in the base config file, it’s best to set the default to the “false” value of the flag. * Switching the default to “on” is a breaking change if the flag removes some functionality or breaks the UI (see other breaking change guidelines). Setting a flag to default to “on” should only be done in major release versions if the resulting change is a breaking change. * Database Migrations * Optimize migrations and scripts to run without creating too much load on the database. Split them up and run separately when possible. * UI * Changing the look and adding features is considered backward compatible, but removing critical features is not. Put old functionality in a feature flag (default the flag with the old functionality to “false”) and remove the functionality with the “true” value of the flag. In the changelog, alert users of upcoming changes before removing the flag or changing the default to the new flag. * User roles/permissions * Instead of removing permissions, put the old functionality in a feature flag (default the old functionality to “false”) and remove the functionality with the “true” value of the flag. * Python version upgrades * Since we only support the last three minor versions, any minor Python version upgrades could require some users to upgrade their Python version. Therefore, minor Python version bumps should be considered breaking changes. * SQLAlchemy major version upgrades * SQLAlchemy major upgrades should be carefully tested for cases where the upgrade would break integration for users that may be on an older database version that is no longer supported or with any loss of functionality. When in doubt, add the upgrade to a Superset major version release. * Database Connectors * Instead of removing database functionality, put the old functionality in a feature flag (default the old functionality to “false”) and remove the functionality with the “true” value of the flag. **How to announce a breaking change in a PR?** We should add a “breaking” title prefix that should be used in PRs when there is a breaking change, i.e., `breaking: remove functionality for this feature`. Reviewers should only allow this to be merged when the next release planned is a breaking change. This change should be notated in the changelog. Option2: add a “breaking” label, but contributors would not have access to this functionality, only committers A “TODO” should be added in the code with a greppable tag of “TODO: DEPRECATION NOTES” with any helpful instructions for what code needs to be changed or removed in order to remove the functionality at the next major version. Keep in mind that someone else other than the author may be removing the functionality at a later date, so clear instructions are helpful. UPDATING.MD should be updated with a “Deprecated” section for each release, and for major bumps, a “Breaking Change” section should be added. This document adds more context than what we can get from the PR commit message, and will be partially copied into the changelog. The full format of the changelog with deprecation and breaking change notifications is TBD. **How to announce deprecations to users** The changelog should contain information about features that are deprecated, and clearly notate when it is going to be permanently removed. I.e., “The feature flag “ANIMATED_WINDOWS” has been deprecated and will be removed in the 2.0 release. We should be able to automatically pull this from UPDATING.MD into the changelog. When possible, it is encouraged to log a message to users when they are using a feature that is deprecated, and let them know that the feature will be removed soon. CLI logs are a good example. We should try to give users a sufficient amount of time between deprecating and removing functionality, to allow them to update their integrations. **How to announce breaking changes to contributors** The changelog for the major version bumps should contain information about any necessary changes that users will need to make to their codebase, integrations, databases, or charts before upgrading so that there are no interruptions in their usage of the platform. We can continue to point contributors to UPDATING.MD for more specific information on how to make the update. The release notes should also mention deprecations and breaking changes, but likely with less technical information, so it may be a subset of what is communicated in the changelog. **How to implement breaking changes upon a major version bump** Just before a major version branch/rc branch is cut, a commit (or multiple commits) should be merged into master that remove(s) any deprecated features/flags/backward incompatible changes. This sweep through the code and changelog for deprecated features ensures that we keep our code clean and are diligent about removing deprecated functionality on each major version bump. If the changelog is well documented with deprecation notes, we should be able to find what needs to be removed from that list. **What if something is inadvertently added to a minor or patch release that is a breaking change?** If a breaking change is released, a patch fix should be released as soon as possible correcting/reverting the breaking change and a note for the affected version in the changelog should highlight that a breaking change is in that release, and encourage users to take the next patched version. **What if a security issue requires a change that is a breaking change? ** If there is no way to correct the security issue without breaking functionality or dependencies and if it is unreasonable due to time constraints to add it to a major version, then the security fix should be put into a minor version release, and then plan to fast-follow with a major version. Add extra notes in the changelog to bring attention to the breaking change on the minor version, and also add the changes to the following major version. If you have to remove some functionality that was deprecated and users haven’t been notified of the deprecation in advance, try to be as clear as possible in the changelog and release notes so that users aren’t caught off guard. An extra communication to the community may also be necessary in some cases. ### Rejected Alternatives Describe alternative approaches that were considered and rejected. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
