junegunn opened a new pull request, #8559:
URL: https://github.com/apache/hbase/pull/8559
## Description
`hbase-shaded-client` ships a
`META-INF/services/java.net.spi.InetAddressResolverProvider` naming a class
that no classloader can load. On Java 18+ the JVM reads that declaration on the
first name lookup, so every `InetAddress` call in the process throws, not just
HBase calls.
```
java.util.ServiceConfigurationError:
java.net.spi.InetAddressResolverProvider:
Provider
org.apache.hadoop.hbase.shaded.org.xbill.DNS.spi.DnsjavaInetAddressResolverProvider
not found
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
at java.base/java.net.InetAddress.loadResolver(InetAddress.java:508)
```
Three things combine:
1. dnsjava 3.6.1 is a multi-release jar. Its service file sits at the root,
but `DnsjavaInetAddressResolverProvider` ships only under
`META-INF/versions/18/`.
2. Shade relocates `org.xbill`. It rewrites the versioned class's bytecode
to the relocated name but leaves its jar entry path alone, so the name and the
path disagree. `ServicesResourceTransformer` then writes the merged service
file naming the relocated class.
3. The shaded jar's manifest has no `Multi-Release: true`, so everything
under `META-INF/versions/**` is invisible to the JVM regardless.
JEP 418 introduced the SPI in Java 18, which is why 11 and 17 are unaffected.
## Affected versions
dnsjava 3.6.0 added the provider, and Hadoop 3.4.1 is the first Hadoop
release to depend on dnsjava 3.6.1. Any HBase artifact built against Hadoop
3.4.1 or later carries it. Verified by inspecting the published jars on Maven
Central:
| artifact | state |
| --- | --- |
| `2.5.11-hadoop3` through `2.5.15-hadoop3` | affected |
| `2.6.2-hadoop3` through `2.6.6-hadoop3` | affected |
| `3.0.0-beta-2`, `3.0.0` | affected |
| `2.5.10-hadoop3` and earlier, `3.0.0-beta-1` and earlier | clean |
| plain Hadoop 2 builds of `2.5.x` and `2.6.x`, and all `2.4.x` | clean,
older dnsjava |
`hbase-shaded-testing-util` is affected on the same builds.
`hbase-shaded-client-byo-hadoop` and `hbase-shaded-mapreduce` bundle no
dnsjava, which is why switching to byo-hadoop is the downstream workaround.
## Reproduction
Any JDK 18+, no cluster needed, with an affected `hbase-shaded-client` on
the classpath:
```java
System.out.println(java.net.InetAddress.getByName("localhost"));
```
A script that inspects a given jar and runs the lookup, with captured output
on Java 11 and Java 21 against both a published jar and a patched build:
https://gist.github.com/junegunn/5e51baf73f32ecbfcd4a9116b7df775a
A minimal project with only dnsjava 3.6.1 and maven-shade-plugin 3.6.0
reproduces it byte for byte, so nothing here is HBase specific. Shade 3.6.2,
the latest release, produces the same broken output, so a plugin bump is not an
alternative.
## Fix
Extend the existing `dnsjava:dnsjava` filter in `hbase-shaded/pom.xml` to
drop the service file and the versioned entries. The filter lives in the parent
pluginManagement, so one edit covers every shaded artifact. The provider is
opt-in behind the `org.dnsjava.spi.enable` system property, so removing it
costs nothing and all 315 relocated dnsjava classes stay.
Marking the uber jar `Multi-Release: true` instead does not work, since
shade still leaves the versioned entry path unrelocated, and a relocated shaded
client should not be installing a JVM wide DNS resolver anyway.
The commit also adds a check to `ensure-jars-have-correct-contents.sh` in
both invariants modules, failing the build when a shaded jar declares this SPI.
Static rather than runtime, because precommit and nightly build on JDK 8, 11
and 17, where the SPI is never consulted and no runtime test can see the
breakage. A general check, rejecting any service provider that is not loadable
from its own jar, is left as future work.
Unrelated but worth flagging: the surrounding content check in that script
is a no-op on master and branch-3, from an `allowed_expr` assignment bug in
HBASE-29226, so it prints `grep: empty (sub)expression`. Filing separately.
## Test result
Rebuilt `hbase-shaded-client` and `hbase-shaded-testing-util`: the service
file and the `versions/18` entries are gone, all 315 relocated dnsjava classes
are retained, and `InetAddress.getByName` succeeds on Java 21.
`mvn verify` passes on both invariants modules. The new check exits 1 on an
affected published jar and 0 on the rebuilt artifacts.
## Context
Depended on by HBASE-29546.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
