[
https://issues.apache.org/jira/browse/HBASE-26492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449862#comment-17449862
]
Andrew Kyle Purtell edited comment on HBASE-26492 at 11/27/21, 4:02 PM:
------------------------------------------------------------------------
Here is a good resource on the issue:
https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
[~zhangduo]
{noformat}
Implementation-Title: Apache HBase Patched and Relocated (Shaded) Prot
obuf
Build-Jdk-Spec: 1.8
{noformat}
The issue isn't the JDK that hbase-shaded was created with. The issue is how
the protobuf classes were created. We are shading them, not recompiling them,
so we import the linkage of ByteBuffer#position(int) with the JDK 9+ return
type by leaving it as is when otherwise transforming the protobuf classes
during the shade step.
And now I recall I filed HBASE-25465 for our own builds, to use --release=8,
when I first learned of this issue. Seems protobuf developers need to do the
same, or we should do it for them when building hbase-thirdparty, i.e.
1. Check out the protobuf sources
2. Build with --release=8 (not needed, and not supported anyway, if building
with Java 8)
3. Shade the result into org.apache.hbase.thirdparty
was (Author: apurtell):
Here is a good resource on the issue:
https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/
[~zhangduo]
{noformat}
Implementation-Title: Apache HBase Patched and Relocated (Shaded) Prot
obuf
Build-Jdk-Spec: 1.8
{noformat}
The issue isn't the JDK that hbase-shaded was created with. The issue is how
the protobuf classes were created. We are shading them, not recompiling them,
so we import the linkage of ByteBuffer#position(int) with the JDK 9+ return
type by leaving it as is when otherwise transforming the protobuf classes
during the shade step.
And now I recall I filed HBASE-25465 for our own builds, to use --release=8,
when I first learned of this issue. Seems protobuf developers need to do the
same, or we should do it for them when building hbase-thirdparty, i.e.
1. Check out the protobuf sources
2. Build with --release=8
3. Shade the result into org.apache.hbase.thirdparty
> [branch-2, hbase-thirdparty, Java 8] TestUnloadAccessController and other
> unit tests fail to start due to ByteBuffer link error
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-26492
> URL: https://issues.apache.org/jira/browse/HBASE-26492
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 2.4.8, 2.4.9
> Environment: Java 1.8.0_312, vendor: Azul Systems, Inc., runtime:
> /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
> OS name: "mac os x", version: "12.0.1", arch: "aarch64", family: "mac"
> Reporter: Andrew Kyle Purtell
> Priority: Major
> Fix For: 2.5.0, 2.4.9
>
>
> org.apache.hadoop.hbase.security.access.TestUnloadAccessController
> Hang in setUpBeforeClass. Master will not initialize. Root cause is a
> NoSuchMethodError.
> {noformat}
> 2021-11-26 19:09:56,465 WARN
> [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=62950]
> ipc.RpcExecutor$Handler(370):
> Handler errors java.lang.NoSuchMethodError:
> java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
> at
> org.apache.hbase.thirdparty.com.google.protobuf.CodedOutputStream$HeapNioEncoder.flush(CodedOutputStream.java:1546)
> at
> org.apache.hadoop.hbase.ipc.ServerCall.writeToCOS(ServerCall.java:378)
> at
> org.apache.hadoop.hbase.ipc.ServerCall.createHeaderAndMessageBytes(ServerCall.java:385)
> at
> org.apache.hadoop.hbase.ipc.ServerCall.createHeaderAndMessageBytes(ServerCall.java:363)
> at
> org.apache.hadoop.hbase.ipc.ServerCall.setResponse(ServerCall.java:267)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:168)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:354)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:334)
> {noformat}
> This is a known issue with ByteBuffer in JDK 8 vs ByteBuffer in later
> versions. When code is compiled with Java 9 or later using a specific subset
> of ByteBuffer APIs, the resulting bytecode will not link with Java 8's
> runtime. It works fine the other way. When compiled with Java 8, the bytecode
> will link with later Java runtimes just fine.
> protobuf included into hbase-thirdparty was likely compiled with Java 9 or
> later. We shade that bytecode as is into hbase-thirdparty. Tests were
> attempted with Java 8, so this failure case manifested.
> Apache Maven 3.8.3 (ff8e977a158738155dc465c6a97ffaf31982d739)
> Java version: 1.8.0_312, vendor: Azul Systems, Inc., runtime:
> /Library/Java/JavaVirtualMachines/zulu-8.jdk/Contents/Home/jre
> OS name: "mac os x", version: "12.0.1", arch: "aarch64", family: "mac"
> We should be able to fix this problem by compiling protobuf with Java 8 and
> then shading the result when building hbase-thirdparty.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
