amoghrajesh opened a new pull request, #58578:
URL: https://github.com/apache/airflow/pull/58578
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
### Context
Airflow releases happen often and certifying a release for contributors is
relatively a one step process - they install the release bits and validate if
their changes / the RC looks OK.
For PMC members, it is a bit more. It involves a series of ASF defined
checks to ensure the release bits are upto a standard - reproducible builds,
checksums, licenses, signatures, etc. For example:
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmc-members.
This in addition to validation of a RC can be time consuming and repetitive.
### What
I have had this idea in my mind that we have variety of releases we certify,
with most of the checks being similar:
1. reproducible builds
2. license checks
3. checksum checks
4. license checks
5. SVN checks
What if these could be baked into a `breeze` command? That's the motive
behind this change.
With this change, PMC members can validate Airflow release candidates using
a single command instead of manually running multiple verification steps. The
command automates SVN file checks, GPG signature verification, SHA512 checksum
validation, reproducible build verification, and Apache RAT license checks.
This reduces PMC verification time from atleast 30 minutes to under 5
minutes RC checks, ensures consistent validation across all PMC members, and
reduces the chance of missing verification steps.
All they have to run is this:
```shell script
breeze release-management validate-rc-by-pmc \
--distribution airflow \
--version 3.1.3rc1 \
--task-sdk-version 1.1.3rc1 \
--svn-path ~/asf-dist/dev/airflow \
```
### Testing
I tested this command out with the last airflow release: 3.1.3, task sdk
1.1.3, and this is how it looks:
```python
(airflow) ➜ airflow git:(breeze-release-validator) ✗ breeze
release-management validate-rc-by-pmc \
--distribution airflow \
--version 3.1.3rc1 \
--task-sdk-version 1.1.3rc1 \
--svn-path ~/Documents/OSS/repos/airflow-pmc/airflow \
--checks svn,signatures,checksums,licenses
Validating Apache Airflow 3.1.3rc1
SVN Path: /Users/amoghdesai/Documents/OSS/repos/airflow-pmc/airflow
Airflow Root: /Users/amoghdesai/Documents/OSS/repos/airflow
SVN File Verification
Checking Airflow directory:
/Users/amoghdesai/Documents/OSS/repos/airflow-pmc/airflow/3.1.3rc1
Checking Task SDK directory:
/Users/amoghdesai/Documents/OSS/repos/airflow-pmc/airflow/task-sdk/1.1.3rc1
Status: PASSED - All 21 expected files present
Duration: 0.0s
GPG Signature Verification
Status: PASSED - All 7 signatures verified
Duration: 0.2s
SHA512 Checksum Verification
Status: PASSED - All 7 checksums valid
Duration: 0.2s
Apache RAT License Verification
Apache RAT already present
Extracting source to /tmp/apache-airflow-src...
Running Apache RAT...
Status: PASSED - All files have approved licenses
Duration: 32.0s
======================================================================
ALL CHECKS PASSED (4/4)
You can vote +1 (binding) on this release.
Total validation time: 32.4s
======================================================================
```
P.S: skipped the reproducible build checks here because I didn't want the
command to delete my dist, airflow 3.1.3 is already released. And it worked
pretty well I'd say
I have started with airflow core & task sdk release, if it spans well, we
will extend it to providers, airflow ctl, python client
## Future Work
- [ ] Extend for provider releases
- [ ] Extend for airflow-ctl releases
- [ ] Extend for python client releases
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
