rangareddy opened a new pull request, #19491:
URL: https://github.com/apache/hudi/pull/19491
### Describe the issue this Pull Request addresses
Closes #19468, a follow-up from #19433.
#19433 fixed a regression where `hudi-presto-bundle` silently shipped **0**
instead of **109**
`org/apache/hudi/hadoop/**` entries, losing `HoodieParquetInputFormat` — the
class the bundle exists to
provide. `maven-shade-plugin` does not fail when an `artifactSet` include
matches nothing, so the build stayed
green and the jar just got quietly smaller.
Nothing in CI could have caught it, for two independent reasons:
1. **No presto coverage at all.** `git grep -il presto
packaging/bundle-validation` returns nothing, and the
same holds for trino. Of the seven bundles touched by #19433, only
`hudi-hadoop-mr-bundle` appears in
`bundle-validation`.
2. **`-am` hides this class of bug.** Every `-pl packaging/...` invocation
under `.github/workflows` passes
`-am`. With `-am` the dependency bundles are built in-reactor and Maven's
`ReactorReader` serves the
effective model rather than the published dependency-reduced POM, so the
resolution path that actually
broke is never exercised.
There is a third detail worth stating, which the issue did not: the job
builds with `mvn package`, never
`install`, so `hudi-hadoop-mr-bundle` is not in the local repository either.
Simply dropping `-am` would fail
to resolve it. The new step installs it first.
### Summary and Changelog
- `packaging/bundle-validation/validate_presto_bundle.sh` asserts the bundle
carries
`HoodieParquetInputFormat`, `HoodieParquetRealtimeInputFormat` and
`HoodieTableMetaClient`, and that it has
at least 100 `org/apache/hudi/hadoop/**` entries. A **floor** rather than
an exact count, so ordinary
additions do not fail the build while a collapse to zero still does.
Failures print what to check rather
than just a diff of numbers.
- A `Validate Presto Bundle Contents` step in the existing
`validate-bundles` job — not a new job, so no
extra runner. It installs `hudi-hadoop-mr-bundle`, then builds the presto
bundle **without `-am`** so the
dependency resolves from the repository, then runs the script.
The two commits on this branch are squashed into one. The squashed content
is byte-identical to the previous
head; only the history changed, and the message now describes the change as
it stands rather than the
selector bug that was fixed along the way.
### Verification
The script is exercised both ways locally. Against the real bundle:
```
::warning::validate_presto_bundle.sh validating
hudi-presto-bundle-1.3.0-SNAPSHOT.jar
found org/apache/hudi/hadoop/HoodieParquetInputFormat.class
found
org/apache/hudi/hadoop/realtime/HoodieParquetRealtimeInputFormat.class
found org/apache/hudi/common/table/HoodieTableMetaClient.class
109 org/apache/hudi/hadoop/** entries (floor 100)
validation was successful.
exit=0
```
Against a jar with `org/apache/hudi/hadoop/` stripped — i.e. the #19433
regression reproduced:
```
::error::org/apache/hudi/hadoop/HoodieParquetInputFormat.class is missing
from presto-regressed.jar.
::error::An artifactSet include probably matched no artifact. ...
exit=1
```
The step's exact command sequence — `install -pl
packaging/hudi-hadoop-mr-bundle -am`, then
`package -pl packaging/hudi-presto-bundle` with no `-am`, then the script —
was run locally and passes.
`bot.yml` parses as YAML with the step present, and `apache-rat:check` run
the way `validate-source` does
(source-release copy, tracked files only) reports **0 Unknown Licenses**
with the new script recognised as
`AL2`.
**CI then found a real bug in this step, now fixed.** The first run failed —
and it was the step's own jar
selection, not the bundle:
```
::warning::validate_presto_bundle.sh validating
hudi-presto-bundle-1.3.0-SNAPSHOT-javadoc.jar
::error::org/apache/hudi/hadoop/HoodieParquetInputFormat.class is missing
```
The selector was `ls .../hudi-presto-bundle-*.jar | grep -v sources | head
-1`, and this job builds with
`-DdeployArtifacts=true` so a javadoc jar exists too. `-` (0x2D) sorts
before `.` (0x2E), so
`hudi-presto-bundle-<version>-javadoc.jar` comes first and `grep -v sources`
did not exclude it. A plain
`mvn package` produces no javadoc jar, which is exactly why this passed
locally and failed in CI — the
wiring was the one part not executable outside CI.
Fixed two ways: the step now names the jar exactly from the project version,
with no globbing at all (the
same idiom the job already uses for `ci_run.sh`), and the script refuses a
`-sources`/`-javadoc`/`-tests`
jar with a message saying what to pass, so a future caller gets the real
error rather than a misleading
missing-class one. Re-verified locally under CI's conditions, building with
`-DdeployArtifacts=true` so all
three jars exist: the old selector picks the javadoc jar, the script now
rejects it by name (`exit=1`), and
the exact path passes with 109 entries (`exit=0`).
### Current CI status — stated plainly
**The fixed step has not yet run in CI.** `validate-bundles` executed on
this PR exactly once, on the
revision *before* the selector fix, and that is the run that found the bug.
ASF Infra then disabled GitHub
Actions on apache/hudi over runner-time usage (see #19514), so since the fix
only the cheap gates have
reported. The green checks on this PR are those gates, not the new assertion.
Standing in for the missing CI run, the script was exercised locally against
a bundle built from current
master, on all four of its paths plus a negative test:
```
main artifact -> exit 0 3/3 required classes, 109
org/apache/hudi/hadoop/** (floor 100)
-sources.jar -> exit 1 refused by name, with the message
telling the caller what to pass
missing jar path -> exit 1 "presto bundle jar not found: ..."
no argument -> exit 1 usage
jar with those entries
stripped (zip -d) -> exit 1 "HoodieParquetInputFormat.class is
missing ... An artifactSet
include probably matched no artifact"
```
The last line is the one that matters: it is the #19433 shape reproduced
deliberately, so the gate is not
vacuous — it fails on exactly the bundle it exists to reject. What remains
unverified outside CI is only the
workflow wiring, which is what bit last time; I would rather say so than
carry a checked "CI passes" box.
**Interaction with #19514**, which trims this same job: that PR is now
**merged** (2026-08-05), and on
current master `validate-bundles` has exactly one matrix leg left,
`scala-2.13` / `flink1.20` / `spark3.5`.
Two consequences, both checked rather than assumed:
- This step is unaffected by the trimming. That leg's `Build Project` branch
builds a `-pl`-limited set that
does **not** include `packaging/hudi-presto-bundle`, but the step builds
the bundle itself (deliberately
without `-am`, so bundle dependencies resolve from the repository) rather
than relying on the preceding
step's module list.
- It runs once per PR, not four times, because only one leg survives. An
earlier revision of this
description said a rebase would be needed for a textual conflict in
`bot.yml`; that is not the case,
`git merge-tree` against current master reports no conflict and GitHub
reports the PR mergeable.
For the record, I briefly gated this step to a single matrix leg to avoid
validating a byte-identical jar
four times, then reverted it: this branch's merge base still has all four
legs, so the redundancy looked
real, but on master there is only one leg and the condition would have
bought nothing while risking the
check silently not running if the commented-out legs are ever restored.
### Deliberately not attempted
The issue's broader suggestion — *"a check that fails the build when a shade
`artifactSet` include matches no
artifact would catch this whole class of silent shrinkage across all 16
bundles, rather than one bundle at a
time"*. That is the better fix and I agree with it, but shade has no such
option, so it needs either a
custom enforcer rule or a plugin change, and it would want its own PR and
its own discussion about what to do
with includes that are intentionally speculative. This PR closes the
specific hole; that idea deserves to stay
open.
### Impact
CI only — one new script and one step in an existing job. No production
code, no published artifact change.
The step adds an install of one bundle plus a rebuild of the presto bundle
to the `validate-bundles` job.
### Risk Level
low — a new assertion in CI. The failure mode is a red build on a bundle
that has genuinely lost classes,
which is the intent.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
- [ ] CI passes on my PR — the cheap gates pass; the Java CI matrix cannot
run while Actions is disabled
on the repo (#19514). The new step's own behaviour is covered locally,
including a negative test, in
"Current CI status" above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
