[
https://issues.apache.org/jira/browse/HBASE-25166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17217865#comment-17217865
]
Hudson commented on HBASE-25166:
--------------------------------
Results for branch master
[build #101 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/101/]:
(/) *{color:green}+1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/101/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/101/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/101/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> MobFileCompactionChore is closing the master's shared cluster connection
> ------------------------------------------------------------------------
>
> Key: HBASE-25166
> URL: https://issues.apache.org/jira/browse/HBASE-25166
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 3.0.0-alpha-1
> Reporter: Ankit Singhal
> Assignee: Ankit Singhal
> Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> Code which is doing so in MobFileCompactionChore
> {code:java}
> try (Connection conn = master.getConnection();
> Admin admin = conn.getAdmin();) { {code}
> As master uses this connection to read the meta or other system tables, so
> none of the meta operations through master will work.
> Symptoms in master logs:-
> {code:java}
> s, events=841, succcessCount=123, totalEvents=12824192,
> totalSuccessCount=1891300
> 2020-10-05 16:34:25,062 INFO
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor: Unable
> to get remote Address
> 2020-10-05 16:34:25,062 ERROR
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore: Failed to
> normalize regions.
> java.io.IOException: connection is closed
> at
> org.apache.hadoop.hbase.MetaTableAccessor.getMetaHTable(MetaTableAccessor.java:241)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:768)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:727)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:215)
> at
> org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:189)
> at
> org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1821)
> at
> org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48)
> at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:188)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) {code}
> Symptoms at the client:-
> {code:java}
> RpcRetryingCaller{globalStartTime=1602099132430, pause=100, maxAttempts=11},
> java.io.IOException: java.io.IOException: connection is closed
> at
> org.apache.hadoop.hbase.MetaTableAccessor.getMetaHTable(MetaTableAccessor.java:241)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1116)
> at
> org.apache.hadoop.hbase.master.TableStateManager.readMetaState(TableStateManager.java:258)
> at
> org.apache.hadoop.hbase.master.TableStateManager.isTablePresent(TableStateManager.java:175)
> at
> org.apache.hadoop.hbase.master.HMaster.getTableDescriptors(HMaster.java:3277)
> at
> org.apache.hadoop.hbase.master.HMaster.listTableDescriptors(HMaster.java:3221)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.getTableDescriptors(MasterRpcServices.java:1064)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:418)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
