rangareddy opened a new pull request, #19491:
URL: https://github.com/apache/hudi/pull/19491
### Describe the issue this Pull Request addresses
Closes #19468, a follow-up from #19433.
#19433 fixed a regression where `hudi-presto-bundle` silently shipped **0**
instead of **109**
`org/apache/hudi/hadoop/**` entries, losing `HoodieParquetInputFormat` — the
class the bundle exists to
provide. `maven-shade-plugin` does not fail when an `artifactSet` include
matches nothing, so the build stayed
green and the jar just got quietly smaller.
Nothing in CI could have caught it, for two independent reasons:
1. **No presto coverage at all.** `git grep -il presto
packaging/bundle-validation` returns nothing, and the
same holds for trino. Of the seven bundles touched by #19433, only
`hudi-hadoop-mr-bundle` appears in
`bundle-validation`.
2. **`-am` hides this class of bug.** Every `-pl packaging/...` invocation
under `.github/workflows` passes
`-am`. With `-am` the dependency bundles are built in-reactor and Maven's
`ReactorReader` serves the
effective model rather than the published dependency-reduced POM, so the
resolution path that actually
broke is never exercised.
There is a third detail worth stating, which the issue did not: the job
builds with `mvn package`, never
`install`, so `hudi-hadoop-mr-bundle` is not in the local repository either.
Simply dropping `-am` would fail
to resolve it. The new step installs it first.
### Summary and Changelog
- `packaging/bundle-validation/validate_presto_bundle.sh` asserts the bundle
carries
`HoodieParquetInputFormat`, `HoodieParquetRealtimeInputFormat` and
`HoodieTableMetaClient`, and that it has
at least 100 `org/apache/hudi/hadoop/**` entries. A **floor** rather than
an exact count, so ordinary
additions do not fail the build while a collapse to zero still does.
Failures print what to check rather
than just a diff of numbers.
- A `Validate Presto Bundle Contents` step in the existing
`validate-bundles` job — not a new job, so no
extra runner. It installs `hudi-hadoop-mr-bundle`, then builds the presto
bundle **without `-am`** so the
dependency resolves from the repository, then runs the script.
### Verification
The script is exercised both ways locally. Against the real bundle:
```
::warning::validate_presto_bundle.sh validating
hudi-presto-bundle-1.3.0-SNAPSHOT.jar
found org/apache/hudi/hadoop/HoodieParquetInputFormat.class
found
org/apache/hudi/hadoop/realtime/HoodieParquetRealtimeInputFormat.class
found org/apache/hudi/common/table/HoodieTableMetaClient.class
109 org/apache/hudi/hadoop/** entries (floor 100)
validation was successful.
exit=0
```
Against a jar with `org/apache/hudi/hadoop/` stripped — i.e. the #19433
regression reproduced:
```
::error::org/apache/hudi/hadoop/HoodieParquetInputFormat.class is missing
from presto-regressed.jar.
::error::An artifactSet include probably matched no artifact. ...
exit=1
```
The step's exact command sequence — `install -pl
packaging/hudi-hadoop-mr-bundle -am`, then
`package -pl packaging/hudi-presto-bundle` with no `-am`, then the script —
was run locally and passes.
`bot.yml` parses as YAML with the step present, and `apache-rat:check` run
the way `validate-source` does
(source-release copy, tracked files only) reports **0 Unknown Licenses**
with the new script recognised as
`AL2`.
**What CI alone can confirm:** that the step is wired into the right job and
runs in the GitHub Actions
environment. The script logic and the Maven sequence are verified locally;
the YAML wiring is not something I
can execute here.
### Deliberately not attempted
The issue's broader suggestion — *"a check that fails the build when a shade
`artifactSet` include matches no
artifact would catch this whole class of silent shrinkage across all 16
bundles, rather than one bundle at a
time"*. That is the better fix and I agree with it, but shade has no such
option, so it needs either a
custom enforcer rule or a plugin change, and it would want its own PR and
its own discussion about what to do
with includes that are intentionally speculative. This PR closes the
specific hole; that idea deserves to stay
open.
### Impact
CI only — one new script and one step in an existing job. No production
code, no published artifact change.
The step adds an install of one bundle plus a rebuild of the presto bundle
to the `validate-bundles` job.
### Risk Level
low — a new assertion in CI. The failure mode is a red build on a bundle
that has genuinely lost classes,
which is the intent.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
- [x] CI passes on my PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
