amaechler opened a new pull request, #20707: URL: https://github.com/apache/druid/pull/20707
### Description The release build generates LICENSE.BINARY and NOTICE.BINARY with `--exclude-module-prefix=extensions-contrib/ --exclude-module-prefix=extensions/druid-testcontainers`, which drops every `licenses.yaml` entry tagged with those modules. `check-licenses.py` only checked that each reported dependency had an entry somewhere, so a jar that ships in the release but is registered only under a contrib module passed the check and was left out of the release LICENSE. On current master this affects `net.java.dev.jna:jna-platform` 5.19.1, which ships in `extensions/druid-azure-extensions` but is registered only under `extensions-contrib/druid-ranger-security`. Every other jar in the binary tarball has an entry the release keeps. #### Required a release-kept entry for shipped dependencies - `check-licenses.py` takes the same `--exclude-module-prefix` flags as `generate-binary-license.py`, and the CI script and the `apache-release` profile pass the same values the release generators use. - A dependency reported by a module that ships in the release must be registered under at least one module that is not excluded. Dependencies reported only by excluded modules (`druid-testcontainers`) keep the existing rule: an entry anywhere. - The check now records every module that reports a dependency. Before, the module was overwritten by whichever report was parsed last. - Report directories map to `licenses.yaml` module tags (`java-core`, `extensions/<artifactId>`), so the `druid_module` in error lines is the tag to use in the fix. `generate-license-dependency-reports.py` now names extension reports after the artifactId instead of the source directory (`ext/druid-avro-extensions` rather than `ext/avro-extensions`), which is also the extension's directory in the binary distribution. - Mismatch and unchecked handling are unchanged, and so are the exit codes. - The exclusion prefixes are now repeated in four places: the two generator executions and the check execution in `distribution/pom.xml`, and the CI script. They must change together. #### Registered jna-platform under druid-azure-extensions Added an `extensions/druid-azure-extensions` entry copied from the existing one (same name, license and version). #### Added unit tests for the rule The missing-license logic moved into `find_missing_licenses` and is covered by `distribution/bin/test_check_licenses.py` (stdlib `unittest`, no new dependencies), which a new `test-check-licenses` execution in the `apache-release` profile runs right before `check-licenses`, so it runs wherever the check runs, including the packaging-check CI job. `license_checks_script.sh` is no longer called by CI but gets the same flags, with a comment, so manual runs match the release. #### Verification Reports were generated from current master, and the binary tarball was built to compare against. - Unpatched checker, current `licenses.yaml`: passes. - Patched checker, current `licenses.yaml`: fails with one missing entry, `jna-platform` in `extensions/druid-azure-extensions`. - Patched checker, updated `licenses.yaml`: passes, with the same 158 unchecked warnings as master. - Every third-party jar in `lib/` and `extensions/*/` of the tarball has an entry that the release LICENSE keeps. - `generate-binary-license.py` and `generate-binary-notice.py`, run with the release flags, both succeed. The only change to LICENSE.BINARY is `jna-platform` under `EXTENSIONS/druid-azure-extensions`. - The unit tests fail when the new rule is removed, and when dependencies reported only by excluded modules lose their exemption. <hr> This PR has: - [x] been self-reviewed. - [x] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md) - [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
