[
https://issues.apache.org/jira/browse/HBASE-23833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062290#comment-17062290
]
Michael Stack commented on HBASE-23833:
---------------------------------------
Thanks [~weichiu]
So, need two versions of FanOutOneBlock* class. One for use with hadoop-3.3 and
another for older versions?
We used to have to do crazy stuff when hadoop1 vs hadoop2. We had whole system
for making it so hbase would select either hadoop1 compat metrics system or
hadoop2 at compile time using profiles; a hadoop1 and hadoop2 profile. You can
see the vestiges still; we have hbase-hadoop-compat module which is
"Interfaces" and they "implementations" of which there is only one now,
hbase-hadoop2-compat. Should we figure how to move the FanOut* stuff to
hbase-hadoop-compat and then add a hbase-hadoop33-compat w/ impl that uses the
hadoop internal pbs?
I left some comments on your PR too.
Thanks for having a go at this one.
> The relocated hadoop-thirdparty protobuf breaks HBase asyncwal
> --------------------------------------------------------------
>
> Key: HBASE-23833
> URL: https://issues.apache.org/jira/browse/HBASE-23833
> Project: HBase
> Issue Type: Bug
> Components: wal
> Affects Versions: 3.0.0
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Priority: Major
>
> Hadoop trunk (3.3.0) shaded protobuf and moved it to hadoop-thirdparty. As
> the result, hbase asyncwal fails to compile because asyncwal uses the
> Hadoop's protobuf objects.
> The following command
> {code}
> mvn clean install -Dhadoop.profile=3.0 -Dhadoop.version=3.3.0-SNAPSHOT
> {code}
> fails with the following error:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile
> (default-compile) on project hbase-server: Compilation failure: Compilation
> failure:
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java:[361,44]
> cannot access org.apache.hadoop.thirdparty.protobuf.MessageOrBuilder
> [ERROR] class file for
> org.apache.hadoop.thirdparty.protobuf.MessageOrBuilder not found
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java:[362,14]
> cannot access org.apache.hadoop.thirdparty.protobuf.GeneratedMessageV3
> [ERROR] class file for
> org.apache.hadoop.thirdparty.protobuf.GeneratedMessageV3 not found
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java:[366,16]
> cannot access org.apache.hadoop.thirdparty.protobuf.ByteString
> [ERROR] class file for org.apache.hadoop.thirdparty.protobuf.ByteString not
> found
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java:[375,12]
> cannot find symbol
> [ERROR] symbol: method
> writeDelimitedTo(org.apache.hbase.thirdparty.io.netty.buffer.ByteBufOutputStream)
> [ERROR] location: variable proto of type
> org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.DataTransferEncryptorMessageProto
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java:[702,81]
> incompatible types:
> org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.DataTransferEncryptorMessageProto
> cannot be converted to com.google.protobuf.MessageLite
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:[314,66]
> incompatible types:
> org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.BlockOpResponseProto
> cannot be converted to com.google.protobuf.MessageLite
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:[330,81]
> cannot access org.apache.hadoop.thirdparty.protobuf.ProtocolMessageEnum
> [ERROR] class file for
> org.apache.hadoop.thirdparty.protobuf.ProtocolMessageEnum not found
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:[380,10]
> cannot find symbol
> [ERROR] symbol: method
> writeDelimitedTo(org.apache.hbase.thirdparty.io.netty.buffer.ByteBufOutputStream)
> [ERROR] location: variable proto of type
> org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.OpWriteBlockProto
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:[422,77]
> cannot access org.apache.hadoop.thirdparty.protobuf.Descriptors
> [ERROR] class file for org.apache.hadoop.thirdparty.protobuf.Descriptors
> not found
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutput.java:[323,64]
> incompatible types:
> org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.PipelineAckProto
> cannot be converted to com.google.protobuf.MessageLite
> [ERROR]
> /Users/weichiu/sandbox/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/SyncReplicationWALProvider.java:[209,68]
> invalid method reference
> [ERROR] non-static method get() cannot be referenced from a static context
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
