potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427236160
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can
be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package.
It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to
PyPI.
Review comment:
We don't have to. Let me explain.
When we use the `--version-suffix-for-svn rc1` we generate packages without
any suffix and we rename them to contain the rc1 in the name. This is so that
when we release we can rename the packages back.
However when you use the `--version-suffix-for-pypi`, the suffix is added
dynamically during package generation - it's appended to the version of
package.
The version is not read from version.py, it is read from the most recent
PROVIDERS_CHANGES_YYYY_MM_DD.md -> YYYY.[M]M.[D]D
Theoretically, it can be a different version for each package being prepared
(depending which PROVIDERS_CHANGES_YYYY_MM_DD.md is most recent (by the date).
It works like this:
1) Prepare release notes
```
./breeze prepare-backport-readme -- 2020.05.20 PACKAGE1 PACKAGE2
```
Prepares the README.md and PROVIDERS_CHANGES_2020.05.20.md for the packages
specified.
You commit that and only tag the release after it has been merged into
master.
2) Then from that tagged release. when you want to generate those packages
you run:
```
./breeze prepare-backport-packages --version-suffix-for-pypi rc1 -- PACKAGE1
PACKAGE2
```
and the packages are generated with 2020.5.20rc1 version. Under the hood
`setup_backport_packages.py` is run with --version-suffix rc1 flag and it will
simply append the suffix to the package version. There is no need to modify
any sources temporarily, the suffix is added dynamically via result o this
method call:
```
def get_package_release_version(provider_package_id: str, version_suffix:
str = "") -> str:
"""
Returns release version including optional suffix.
:param provider_package_id: package id
:param version_suffix: optional suffix (rc1, rc2 etc).
:return:
"""
return get_latest_release(
get_package_path(provider_package_id=provider_package_id)).release_version +
version_suffix
```
3) Later when you want to release final packages for PyPI you run (from the
same tag/commit) :
```
./breeze prepare-backport-packages -- PACKAGE1 PACKAGE2
```
And no version suffix is added in this case.
4) For SVN packages you simply rename the packages already in SVN. They do
not contain the rc1 suffix internally - only in the name.
Currently, I am only using it to generate all packages so there is no risk
of mistakes but later I want to add additional switch
`prepare-backport-packages --package-version 2020.05.20`. This will verify if
the packages being released have all the 'CHANGES' prepared (via
`prepare-backport-readme`) and that they are the latest ''CHANGES' and that
this version has not already been released (checking for tag version in the git
log and possibly pypi). But I will only be able to test it after we release
officially the first wave of packages so I wanted to implement it later. This
will protect against the accidental generation of the same package version as
already released.
Eventually, I want to get rid of the manual operations and have all the
steps automated within breeze so that you do not have to copy&paste commands
from BACKPORT_PACKAGES.md, but this will require at least few iterations of
releasing packages to get it consistently right.
I want the release process to take a few minutes and have it fully
repeatable by anyone without setting up anything else but the local key and
pypirc configuration. I want to be able to release quite a bit more often in
the future - especially if we go AIP-8 route and split Airflow 2.0 into
separate "core" and "providers" packages from day one. I treat backport
packages as a testing ground for that approach to see if it is workable, so
that I can propose it.
We can likely adopt it later to release airflow in the same way - without
having to modify and discard version.py and add breeze command to do the same -
but I planned to do it later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
