rangareddy opened a new pull request, #857:
URL: https://github.com/apache/incubator-xtable/pull/857
## What
The bundled (shaded) jars list their third-party dependencies in
`META-INF/LICENSE` (`LICENSE-bundled`), and the banner says *"See licenses/ for
text of these licenses"* — but many bundled non-Apache-2.0 dependencies shipped
**no license text at all**. This adds the missing texts and makes the tooling
keep them complete and enforced.
Closes #700
## Background
The earlier tooling (`generate_shaded_license_metadata.py` +
`validate_shaded_license_coverage.sh`) already made the **coordinate listing**
in `LICENSE-bundled` complete and validated. The remaining gap (raised in the
issue's follow-up comment) was the **actual license text files** under
`META-INF/licenses/` — e.g. `xtable-hive-metastore` bundled ~20 non-AL2
dependencies (protobuf, asm, jamon/MPL-2.0, javax.mail/CDDL, junit/CPL, …) with
no bundled license text, and nothing validated their presence.
## Changes
- **`generate_shaded_license_metadata.py`** now also regenerates each
module's `META-INF/licenses/` directory. For every non-Apache-2.0 bundled
dependency it writes `LICENSE-<artifactId>`, sourced from:
1. a curated override (human-verified), then
2. the dependency's own jar (`META-INF/LICENSE*`, `LICENSE*`, …).
It **fails loudly** if a dependency has neither, so an unattributed
dependency can't slip through. Override-first matters because some jars bundle
a *misleading* license — e.g. `junit:4.11`'s `LICENSE.txt` is actually
Hamcrest's BSD text, not junit's Common Public License.
- **`release/scripts/license_overrides/`** — curated texts for dependencies
whose jars carry no (or a misleading) license: the previously hand-maintained
texts (paranamer, javolution, leveldbjni, jruby-jcodings/joni, jersey) plus
canonical texts for `asm`, `protobuf-java`, `jline`, `antlr`/`ST4`, `xmlenc`,
`zstd-jni`, `reactive-streams` (MIT-0), `stax-api`, `jamon-runtime` (MPL 2.0,
extracted verbatim), `junit` (CPL 1.0, extracted verbatim) and `aopalliance`
(public domain).
- **`validate_shaded_license_coverage.sh`** now **fails** when any
non-Apache-2.0 dependency listed in `LICENSE-bundled` has no
`META-INF/licenses/LICENSE-<artifactId>`, so this gap cannot reopen.
- Generator robustness for future regenerations: map `EPL 1.0`, and treat
`org.apache.hadoop` as Apache-2.0 (consistent with the existing
`hbase`/`zookeeper`/`velocity` group overrides).
Result: every non-Apache-2.0 bundled dependency now has a license text —
`xtable-aws` (5/5), `xtable-hive-metastore` (29/29),
`xtable-hudi-support-extensions` (3/3).
## Notes / scope
- The coordinate lists in `LICENSE-bundled`/`NOTICE-bundled` are
intentionally **unchanged**. The license texts are keyed off the committed
(canonical) dependency list rather than a freshly resolved tree, so the result
is reproducible regardless of local `~/.m2` state.
- `xtable-utilities` (unpublished, already skipped by the validator) is out
of scope; the generator now skips shade modules with no built dependency tree.
## Testing
- New validator coverage check reports OK for all three published shaded
modules, and correctly fails when a text is removed (verified).
- `bash -n` on the validator and `python3 -m py_compile` on the generator
pass.
- License texts for the boilerplate licenses (MPL 2.0, CPL 1.0, CDDL) were
extracted verbatim from license files already present in Maven artifacts rather
than hand-typed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
