Wenning Ding created HUDI-1180:
----------------------------------
Summary: Upgrade HBase to 2.3.3
Key: HUDI-1180
URL: https://issues.apache.org/jira/browse/HUDI-1180
Project: Apache Hudi
Issue Type: Improvement
Reporter: Wenning Ding
Trying to upgrade HBase to 2.3.3 but ran into several issues.
According to the Hadoop version support matrix:
[http://hbase.apache.org/book.html#hadoop], also need to upgrade Hadoop to
2.8.5+.
There are several API conflicts between HBase 2.2.3 and HBase 1.2.3, we need to
resolve this first. After resolving conflicts, I am able to compile it but then
I ran into a tricky jetty version issue during the testing:
{code:java}
[ERROR] TestHBaseIndex.testDelete() Time elapsed: 4.705 s <<< ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] TestHBaseIndex.testSimpleTagLocationAndUpdate() Time elapsed: 0.174 s
<<< ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] TestHBaseIndex.testSimpleTagLocationAndUpdateWithRollback() Time
elapsed: 0.076 s <<< ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] TestHBaseIndex.testSmallBatchSize() Time elapsed: 0.122 s <<< ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] TestHBaseIndex.testTagLocationAndDuplicateUpdate() Time elapsed: 0.16
s <<< ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] TestHBaseIndex.testTotalGetsBatching() Time elapsed: 1.771 s <<<
ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] TestHBaseIndex.testTotalPutsBatching() Time elapsed: 0.082 s <<<
ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
34206 [Thread-260] WARN
org.apache.hadoop.hdfs.server.datanode.DirectoryScanner - DirectoryScanner:
shutdown has been called
34240 [BP-1058834949-10.0.0.2-1597189606506 heartbeating to
localhost/127.0.0.1:55924] WARN
org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager -
IncrementalBlockReportManager interrupted
34240 [BP-1058834949-10.0.0.2-1597189606506 heartbeating to
localhost/127.0.0.1:55924] WARN
org.apache.hadoop.hdfs.server.datanode.DataNode - Ending block pool service
for: Block pool BP-1058834949-10.0.0.2-1597189606506 (Datanode Uuid
cb7bd8aa-5d79-4955-b1ec-bdaf7f1b6431) service to localhost/127.0.0.1:55924
34246
[refreshUsed-/private/var/folders/98/mxq3vc_n6l5728rf1wmcwrqs52lpwg/T/temp1791820148926982977/dfs/data/data1/current/BP-1058834949-10.0.0.2-1597189606506]
WARN org.apache.hadoop.fs.CachingGetSpaceUsed - Thread Interrupted waiting
to refresh disk information: sleep interrupted
34247
[refreshUsed-/private/var/folders/98/mxq3vc_n6l5728rf1wmcwrqs52lpwg/T/temp1791820148926982977/dfs/data/data2/current/BP-1058834949-10.0.0.2-1597189606506]
WARN org.apache.hadoop.fs.CachingGetSpaceUsed - Thread Interrupted waiting
to refresh disk information: sleep interrupted
37192 [HBase-Metrics2-1] WARN org.apache.hadoop.metrics2.impl.MetricsConfig -
Cannot locate configuration: tried
hadoop-metrics2-datanode.properties,hadoop-metrics2.properties
43904 [master/iad1-ws-cor-r12:0:becomeActiveMaster-SendThread(localhost:58768)]
WARN org.apache.zookeeper.ClientCnxn - Session 0x173dfeb0c8b0004 for server
null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
[INFO]
[ERROR] Tests run: 10, Failures: 0, Errors: 7, Skipped: 0
[INFO]
{code}
Basically currently Hudi and it's dependency Javalin depend on Jetty 9.4.x but
Hbase depends on jetty 9.3.x. And they have incompatible APIs which could not
be easily resolved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
