rangareddy opened a new pull request, #19490:
URL: https://github.com/apache/hudi/pull/19490
### Describe the issue this Pull Request addresses
Closes #19469, a follow-up from #19433.
`hudi-hadoop-mr-bundle` is a shaded fat jar, and `hudi-presto-bundle`
depended on it while shading classes
that jar already contains. #19433 declared `hudi-hadoop-mr` and
`hudi-hadoop-common` directly, which left the
bundle dependency looking redundant. The task was to check whether it is,
and drop it if so.
**It is not redundant.** Removing it on its own drops **623 entries** from
the presto jar, all
`org/codehaus/jackson/**`:
```
total entries: 7313 -> 6690 (109 org/apache/hudi/hadoop/** in both)
```
The `artifactSet` include `org.codehaus.jackson:*` resolved only through
`hudi-hadoop-mr-bundle`'s published
POM, which lists `jackson-core-asl` and `jackson-mapper-asl` because #19433
turned on
`promoteTransitiveDependencies`. There is no other compile-scope path to
them: the hadoop and hive
dependencies that bring Jackson 1.x are `provided`, so they do not propagate
transitively to this bundle.
This is the same silent-shrinkage shape as the regression #19433 fixed —
shade does not fail when an
`artifactSet` include matches nothing.
### Summary and Changelog
- `packaging/hudi-presto-bundle` drops the `hudi-hadoop-mr-bundle`
dependency and declares
`org.codehaus.jackson:jackson-core-asl` and `jackson-mapper-asl` directly,
at `compile` scope because they
are shaded in. All of this bundle's shade inputs now come from libraries
rather than from another bundle's
published dependency list.
- The root `pom.xml` manages their version behind a new
`jackson.asl.version` property, at `provided` scope,
so a second bundle needing Jackson 1.x does not re-pin it. Nothing else in
the repo declared these
artifacts, so a version had to come from somewhere; the root pom is where
this repo keeps them.
### Verification
Using the install-then-resolve recipe from #19433, since an in-reactor build
cannot see published reduced
POMs, and building the presto bundle **without `-am`** so
`hudi-hadoop-mr-bundle` resolves from the
repository:
```
mvn install -pl packaging/hudi-hadoop-mr-bundle -am -DskipTests
mvn package -pl packaging/hudi-presto-bundle -DskipTests
```
| state | total jar entries | `org/apache/hudi/hadoop/**` |
| --- | --- | --- |
| before (master) | 7313 | 109 |
| bundle dependency simply removed | 6690 | 109 |
| this PR | **7313** | **109** |
The full entry listing is diffed, not just counted, and is **identical** to
master — the bar the issue set.
The published reduced POM no longer mentions `hudi-hadoop-mr-bundle`, and
does not mention the two Jackson
artifacts either, since they are absorbed into the jar.
`apache-rat:check` reports only `target/` build outputs, which
`validate-source` excludes; this PR edits two
tracked POMs and adds no files.
### Not addressed here
Those 623 classes ship **unrelocated** under `org/codehaus/jackson/`, and no
relocation rule in this bundle
or the root pom covers them, so the bundle puts Jackson 1.9.13 on Presto's
classpath in its original package.
Whether that is intended is a separate question from where the dependency is
declared, and changing it would
alter jar contents rather than preserve them, so it is out of scope for this
PR. Worth its own issue if it is
not deliberate.
Question 3 of the issue — whether bundles should depend on library modules
only as a general rule — is
answered in the affirmative for this bundle. Applying it across the other
bundles is #19466's territory.
### Impact
No change to the presto bundle's jar contents. Its published POM loses
`hudi-hadoop-mr-bundle`, which is
correct: consumers were being told to resolve a fat jar this bundle no
longer needs. The root pom gains two
managed versions, which is inert for every module that does not declare them.
### Risk Level
low — dependency declarations only, with the jar entry listing proven
identical to master.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
- [x] CI passes on my PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
