turbaszek commented on a change in pull request #13767: URL: https://github.com/apache/airflow/pull/13767#discussion_r567697537
########## File path: dev/provider_packages/README_BACKPORT_PACKAGES.md ########## @@ -0,0 +1,260 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +<!-- START doctoc generated TOC please keep comment here to allow auto update --> +<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [Backport packages](#backport-packages) +- [What the backport packages are](#what-the-backport-packages-are) +- [Content of the release notes](#content-of-the-release-notes) + - [Generating release notes](#generating-release-notes) + - [Preparing backport packages](#preparing-backport-packages) +- [Debugging the backport provider packages](#debugging-the-backport-provider-packages) +- [Testing backport provider packages](#testing-backport-provider-packages) + +<!-- END doctoc generated TOC please keep comment here to allow auto update --> + +# Backport packages + +# What the backport packages are + +The Backport Provider packages are packages (per provider) that make it possible to easily use Hooks, +Operators, Sensors, and Secrets from the 2.0 version of Airflow in the 1.10.* series. + +The release manager prepares backport packages separately from the main Airflow Release, using +`breeze` commands and accompanying scripts. This document provides an overview of the command line tools +needed to prepare backport packages. + +# Content of the release notes + +Each of the backport packages contains Release notes in the form of the README.md file that is +automatically generated from history of the changes and code of the provider. + +The script generates all the necessary information: + +* summary of requirements for each backport package +* list of dependencies (including extras to install them) when package + depends on other providers packages +* table of new hooks/operators/sensors/protocols/secrets +* table of moved hooks/operators/sensors/protocols/secrets with the + information where they were moved from +* changelog of all the changes to the provider package. This will be + automatically updated with an incremental changelog whenever we decide to + release separate packages. + +The script generates two types of files: + +* BACKPORT_PROVIDERS_CHANGES_YYYY.MM.DD.md which keeps information about changes (commits) in a particular + version of the provider package. The file for latest release gets updated when you iterate with + the same new date/version, but it never changes automatically for already released packages. + This way - just before the final release, you can manually correct the changes file if you + want to remove some changes from the file. + +* README.md which is regenerated every time you run the script (unless there are no changes since + the last time you generated the release notes + +Note that our CI system builds the release notes for backport packages automatically with every build and +current date - this way you might be sure the automated generation of the release notes continues to +work. You can also preview the generated readme files (by downloading artifacts from GitHub Actions). +The script does not modify the README and CHANGES files if there is no change in the repo for that provider. + + +## Generating release notes + +When you want to prepare release notes for a package, you need to run: + +``` +./breeze --backports prepare-provider-documentation [YYYY.MM.DD] <PACKAGE_ID> ... +``` + + +* YYYY.MM.DD - is the CALVER version of the package to prepare. Note that this date cannot be earlier + than the already released version (the script will fail if it will be). It can be set in the future + anticipating the future release date. If you do not specify date, the date will be taken from the last + generated readme - the last generated CHANGES file will be updated. + +* <PACKAGE_ID> is usually directory in the `airflow/providers` folder (for example `google` but in several + cases, it might be one level deeper separated with `.` for example `apache.hive` + +You can run the script with multiple package names if you want to prepare several packages at the same time. +Before you specify a new version, the last released version is update in case you have any bug fixes +merged in the master recently, they will be automatically taken into account. + +Typically, the first time you run release before release, you run it with target release.date: Review comment: ```suggestion Typically, the first time you run the script before release, you run it with target release date: ``` ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
