amaechler opened a new pull request, #20736:
URL: https://github.com/apache/druid/pull/20736
Part of #20732.
### Description
`licenses.yaml` records an exact version for every bundled dependency, so
every dependency bump needs a hand edit before `check-licenses.py` passes. This
PR adds `distribution/bin/refresh-license-versions.py`, which makes that edit
from the same Maven dependency reports `check-licenses.py` reads. It is the
first `licenses.yaml` item in #20732. It can be run on a Dependabot branch
today, and the release-time step builds on it.
```
distribution/bin/generate-license-dependency-reports.py . target
--clean-maven-artifact-transfer --parallel 2
distribution/bin/refresh-license-versions.py licenses.yaml
target/license-reports [--dry-run]
```
#### What it changes
For each entry with `libraries`, it looks up each library in the report of
the entry's own module. When all libraries of the entry moved to the same new
version, it rewrites that entry's `version:` line. Nothing else in the file
changes, so comments, ordering and `notice` blocks stay byte-identical. The new
value keeps the old quotes, and gets quotes when YAML would otherwise read it
as a number (`2.22`).
A version that another entry of the same module registers is never taken.
That keeps entries such as `jakarta.activation-api` 1.2.2 and 2.1.4 apart,
while the same library in different modules (for example PostgreSQL in three
extensions) is updated in each.
Entries with `skip_dependency_report_check`, `web-console` entries and
`extensions-contrib/` entries are skipped, since no report covers them.
#### What it leaves for a person
It prints these entries instead of guessing, and exits non-zero:
- libraries of one entry moved to different versions, or only some of them
moved: the entry needs splitting;
- a library has several new versions in the report;
- some libraries moved while others are in no report, so the entry may be
stale;
- two entries of one module would move the same library to the same version:
one of them should go.
#### Shared report reading
The loop that reads the dependency reports moves out of `check_licenses()`
into `read_dependency_reports()` in `check-licenses.py`, so both scripts use
it. `check-licenses.py` behaves as before.
#### How it was checked
- Unit tests for each rule above, in `test_check_licenses.py`, which the
`apache-release` build already runs.
- Ran it against real reports generated from current master:
- **Replay of recent Dependabot PRs:** I took `licenses.yaml` as it was at
three earlier commits and compared the script's output with the version edits
that were made by hand since.
- From #20707's version: the same 3 edits (#20699, #20703).
- From #20673's version: the same 8 edits.
- From #20671's version: the same 16 edits, including the Google entries
realigned in #20673.
- In each replay it also lists the "Jetbrains jdk7 jdk 8" entry for a
manual edit, because three of its libraries are in no report, and fixes the 6
entries of existing drift described below.
- On the #20707 version, `check-licenses.py` goes from 15 missing
licenses to passing after the refresh.
- **Current master:** it updates 6 entries and lists 4. These are existing
per-module drift that `check-licenses.py` does not catch, since it matches
versions across all modules (for example `Protocol Buffers` in `java-core`
registered at 4.36.0 while `java-core` ships 4.36.2). This PR does not change
`licenses.yaml`; that cleanup is a separate change.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
