voonhous opened a new issue, #19469:
URL: https://github.com/apache/hudi/issues/19469
## Task Description
**What needs to be done:**
Review whether `packaging/hudi-presto-bundle` should keep depending on
`hudi-hadoop-mr-bundle`, and drop it if not.
**Why this task is needed:**
`hudi-hadoop-mr-bundle` is itself a shaded fat jar. `hudi-presto-bundle`
depends on it and then shades a set of classes that jar already contains, so
the same classes are pulled in from a bundle instead of from the library
modules that produce them.
After #19433 the dependency block declares all three:
```xml
<dependency>
<groupId>org.apache.hudi</groupId>
<artifactId>hudi-hadoop-mr-bundle</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hudi</groupId>
<artifactId>hudi-hadoop-mr</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hudi</groupId>
<artifactId>hudi-hadoop-common</artifactId>
<version>${project.version}</version>
</dependency>
```
`hudi-hadoop-mr` and `hudi-hadoop-common` were added by #19433 because the
bundle shades both but reached them only transitively through
`hudi-hadoop-mr-bundle`. Once that bundle started publishing a reduced POM the
includes matched nothing and the presto jar lost 109
`org/apache/hudi/hadoop/**` entries. Declaring them directly fixed it and
decoupled this bundle's shade inputs from another bundle's published dependency
list.
That fix leaves `hudi-hadoop-mr-bundle` looking redundant, but #19433
deliberately did not remove it -- that PR was scoped to published POM metadata,
and dropping a dependency changes shade inputs, which needs its own
verification.
**What to check:**
1. Whether anything in the presto bundle's `artifactSet` still resolves only
through `hudi-hadoop-mr-bundle` after the direct declarations were added.
2. Whether removing it changes jar contents at all. The bar is a
byte-identical listing, or a difference that is understood and intended.
3. Whether the presto bundle should depend on library modules only, as a
general rule -- one fat jar depending on another is worth avoiding regardless
of whether it currently causes a problem.
**Verification:** the install-then-resolve recipe from #19433, since an
in-reactor build cannot see published reduced POMs:
```
mvn install -pl packaging/hudi-hadoop-mr-bundle -DskipTests
mvn package -pl packaging/hudi-presto-bundle -DskipTests # no -am
unzip -l packaging/hudi-presto-bundle/target/hudi-presto-bundle-*.jar | grep
-c org/apache/hudi/hadoop/
```
Expect 109 before and after. Note that CI cannot currently catch a
regression here -- see #19468.
## Task Type
Code improvement/refactoring
## Related Issues
**Parent feature issue:** #16407
**Related issues:** #19433, #19466, #19468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
