[hbase] branch branch-2.3 updated: HBASE-25562 ReplicationSourceWALReader log and handle exception immediately without retrying (#2966)

2021-03-24 Thread sunxin
This is an automated email from the ASF dual-hosted git repository.

sunxin pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2.3 by this push:
 new 754caba  HBASE-25562 ReplicationSourceWALReader log and handle 
exception immediately without retrying (#2966)
754caba is described below

commit 754caba4796d24fbc5a0eae1d8205f601d78a961
Author: XinSun 
AuthorDate: Thu Mar 25 14:16:21 2021 +0800

HBASE-25562 ReplicationSourceWALReader log and handle exception immediately 
without retrying (#2966)

Signed-off-by: sandeepvinayak
---
 .../regionserver/ReplicationSourceWALReader.java   | 30 +-
 1 file changed, 18 insertions(+), 12 deletions(-)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
index 4a32962..d3c44a5 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
@@ -147,14 +147,15 @@ class ReplicationSourceWALReader extends Thread {
   }
 }
   } catch (IOException e) { // stream related
-if (sleepMultiplier < maxRetriesMultiplier) {
-  LOG.debug("Failed to read stream of replication entries: " + e);
-  sleepMultiplier++;
+if (handleEofException(e)) {
+  sleepMultiplier = 1;
 } else {
-  LOG.error("Failed to read stream of replication entries", e);
-  handleEofException(e);
+  LOG.warn("Failed to read stream of replication entries", e);
+  if (sleepMultiplier < maxRetriesMultiplier) {
+sleepMultiplier ++;
+  }
+  Threads.sleep(sleepForRetries * sleepMultiplier);
 }
-Threads.sleep(sleepForRetries * sleepMultiplier);
   } catch (InterruptedException e) {
 LOG.trace("Interrupted while sleeping between WAL reads");
 Thread.currentThread().interrupt();
@@ -241,24 +242,29 @@ class ReplicationSourceWALReader extends Thread {
 }
   }
 
-  // if we get an EOF due to a zero-length log, and there are other logs in 
queue
-  // (highly likely we've closed the current log), we've hit the max retries, 
and autorecovery is
-  // enabled, then dump the log
-  private void handleEofException(IOException e) {
+  /**
+   * if we get an EOF due to a zero-length log, and there are other logs in 
queue
+   * (highly likely we've closed the current log), and autorecovery is
+   * enabled, then dump the log
+   * @return true only the IOE can be handled
+   */
+  private boolean handleEofException(IOException e) {
 // Dump the log even if logQueue size is 1 if the source is from recovered 
Source
 // since we don't add current log to recovered source queue so it is safe 
to remove.
 if ((e instanceof EOFException || e.getCause() instanceof EOFException) &&
   (source.isRecovered() || logQueue.size() > 1) && this.eofAutoRecovery) {
   try {
 if (fs.getFileStatus(logQueue.peek()).getLen() == 0) {
-  LOG.warn("Forcing removal of 0 length log in queue: " + 
logQueue.peek());
+  LOG.warn("Forcing removal of 0 length log in queue: {}", 
logQueue.peek());
   logQueue.remove();
   currentPosition = 0;
+  return true;
 }
   } catch (IOException ioe) {
-LOG.warn("Couldn't get file length information about log " + 
logQueue.peek());
+LOG.warn("Couldn't get file length information about log {}", 
logQueue.peek());
   }
 }
+return false;
   }
 
   public Path getCurrentPath() {


[hbase] branch branch-2.4 updated: HBASE-25693 NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) (#3091)

2021-03-24 Thread apurtell
This is an automated email from the ASF dual-hosted git repository.

apurtell pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2.4 by this push:
 new a11220d  HBASE-25693 NPE getting metrics from standby masters 
(MetricsMasterWrapperImpl.getMergePlanCount) (#3091)
a11220d is described below

commit a11220d3bf508daebd271ea03c5d3558ff6cf603
Author: Andrew Purtell 
AuthorDate: Wed Mar 24 19:09:10 2021 -0700

HBASE-25693 NPE getting metrics from standby masters 
(MetricsMasterWrapperImpl.getMergePlanCount) (#3091)

Signed-off-by: Duo Zhang 
Signed-off-by: Nick Dimiduk 
---
 .../org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java| 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
index 1bc5dd4..c4e1a2e 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
@@ -50,11 +50,17 @@ public class MetricsMasterWrapperImpl implements 
MetricsMasterWrapper {
 
   @Override
   public long getSplitPlanCount() {
+if (master.getRegionNormalizerManager() == null) {
+  return 0;
+}
 return master.getRegionNormalizerManager().getSplitPlanCount();
   }
 
   @Override
   public long getMergePlanCount() {
+if (master.getRegionNormalizerManager() == null) {
+  return 0;
+}
 return master.getRegionNormalizerManager().getMergePlanCount();
   }
 


[hbase] branch branch-2 updated: HBASE-25693 NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) (#3091)

2021-03-24 Thread apurtell
This is an automated email from the ASF dual-hosted git repository.

apurtell pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2 by this push:
 new 124ea5e  HBASE-25693 NPE getting metrics from standby masters 
(MetricsMasterWrapperImpl.getMergePlanCount) (#3091)
124ea5e is described below

commit 124ea5eee327643fd23104915d9844a3c9365366
Author: Andrew Purtell 
AuthorDate: Wed Mar 24 19:09:10 2021 -0700

HBASE-25693 NPE getting metrics from standby masters 
(MetricsMasterWrapperImpl.getMergePlanCount) (#3091)

Signed-off-by: Duo Zhang 
Signed-off-by: Nick Dimiduk 
---
 .../org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java| 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
index 1bc5dd4..c4e1a2e 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
@@ -50,11 +50,17 @@ public class MetricsMasterWrapperImpl implements 
MetricsMasterWrapper {
 
   @Override
   public long getSplitPlanCount() {
+if (master.getRegionNormalizerManager() == null) {
+  return 0;
+}
 return master.getRegionNormalizerManager().getSplitPlanCount();
   }
 
   @Override
   public long getMergePlanCount() {
+if (master.getRegionNormalizerManager() == null) {
+  return 0;
+}
 return master.getRegionNormalizerManager().getMergePlanCount();
   }
 


[hbase] branch master updated: HBASE-25693 NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) (#3091)

2021-03-24 Thread apurtell
This is an automated email from the ASF dual-hosted git repository.

apurtell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/master by this push:
 new f6bb4bb  HBASE-25693 NPE getting metrics from standby masters 
(MetricsMasterWrapperImpl.getMergePlanCount) (#3091)
f6bb4bb is described below

commit f6bb4bb93ecf24dcff15d915fb52c9819ceffcba
Author: Andrew Purtell 
AuthorDate: Wed Mar 24 19:09:10 2021 -0700

HBASE-25693 NPE getting metrics from standby masters 
(MetricsMasterWrapperImpl.getMergePlanCount) (#3091)

Signed-off-by: Duo Zhang 
Signed-off-by: Nick Dimiduk 
---
 .../org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java| 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
index e03c4df..d65b029 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsMasterWrapperImpl.java
@@ -55,11 +55,17 @@ public class MetricsMasterWrapperImpl implements 
MetricsMasterWrapper {
 
   @Override
   public long getSplitPlanCount() {
+if (master.getRegionNormalizerManager() == null) {
+  return 0;
+}
 return master.getRegionNormalizerManager().getSplitPlanCount();
   }
 
   @Override
   public long getMergePlanCount() {
+if (master.getRegionNormalizerManager() == null) {
+  return 0;
+}
 return master.getRegionNormalizerManager().getMergePlanCount();
   }
 


[hbase] branch branch-2.3 updated: HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3089)

2021-03-24 Thread huaxiangsun
This is an automated email from the ASF dual-hosted git repository.

huaxiangsun pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2.3 by this push:
 new 8710772  HBASE-25691 Test failure: 
TestVerifyBucketCacheFile.testRetrieveFromFile (#3089)
8710772 is described below

commit 871077241b24a01b6cfcaec81e7981525792dacf
Author: huaxiangsun 
AuthorDate: Wed Mar 24 14:41:11 2021 -0700

HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile 
(#3089)

The issue is that FileInputStream is created with try-with-resources, so 
its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected, 
its close() is called again.
To avoid this double-free resources, add guard against it.

Signed-off-by: stack 
---
 .../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
index fca1ffa..8e5f558 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@@ -1118,14 +1118,27 @@ public class BucketCache implements BlockCache, 
HeapSize {
*/
   private FileInputStream deleteFileOnClose(final File file) throws 
IOException {
 return new FileInputStream(file) {
+  private File myFile;
+  private FileInputStream init(File file) {
+myFile = file;
+return this;
+  }
   @Override
   public void close() throws IOException {
+// close() will be called during try-with-resources and it will be
+// called by finalizer thread during GC. To avoid double-free resource,
+// set myFile to null after the first call.
+if (myFile == null) {
+  return;
+}
+
 super.close();
-if (!file.delete()) {
-  throw new IOException("Failed deleting persistence file " + 
file.getAbsolutePath());
+if (!myFile.delete()) {
+  throw new IOException("Failed deleting persistence file " + 
myFile.getAbsolutePath());
 }
+myFile = null;
   }
-};
+}.init(file);
   }
 
   private void verifyCapacityAndClasses(long capacitySize, String ioclass, 
String mapclass)


[hbase] branch branch-2.4 updated: HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3088)

2021-03-24 Thread huaxiangsun
This is an automated email from the ASF dual-hosted git repository.

huaxiangsun pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2.4 by this push:
 new 91205bf  HBASE-25691 Test failure: 
TestVerifyBucketCacheFile.testRetrieveFromFile (#3088)
91205bf is described below

commit 91205bf227e8d5df21e34cd257514886f9a18365
Author: huaxiangsun 
AuthorDate: Wed Mar 24 14:39:35 2021 -0700

HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile 
(#3088)

The issue is that FileInputStream is created with try-with-resources, so 
its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected, 
its close() is called again.
To avoid this double-free resources, add guard against it.

Signed-off-by: stack 
---
 .../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
index de44ad7..1f9aac4 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@@ -1101,14 +1101,27 @@ public class BucketCache implements BlockCache, 
HeapSize {
*/
   private FileInputStream deleteFileOnClose(final File file) throws 
IOException {
 return new FileInputStream(file) {
+  private File myFile;
+  private FileInputStream init(File file) {
+myFile = file;
+return this;
+  }
   @Override
   public void close() throws IOException {
+// close() will be called during try-with-resources and it will be
+// called by finalizer thread during GC. To avoid double-free resource,
+// set myFile to null after the first call.
+if (myFile == null) {
+  return;
+}
+
 super.close();
-if (!file.delete()) {
-  throw new IOException("Failed deleting persistence file " + 
file.getAbsolutePath());
+if (!myFile.delete()) {
+  throw new IOException("Failed deleting persistence file " + 
myFile.getAbsolutePath());
 }
+myFile = null;
   }
-};
+}.init(file);
   }
 
   private void verifyCapacityAndClasses(long capacitySize, String ioclass, 
String mapclass)


[hbase] branch branch-2 updated (c8c043a -> 80529af)

2021-03-24 Thread huaxiangsun
This is an automated email from the ASF dual-hosted git repository.

huaxiangsun pushed a change to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git.


from c8c043a  HBASE-25685 asyncprofiler2.0 no longer supports svg; wants 
html (#3079)
 add 80529af  HBASE-25691 Test failure: 
TestVerifyBucketCacheFile.testRetrieveFromFile (#3087)

No new revisions were added by this update.

Summary of changes:
 .../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)


[hbase-site] branch asf-site updated: INFRA-10751 Empty commit

2021-03-24 Thread git-site-role
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hbase-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new bb50ccf  INFRA-10751 Empty commit
bb50ccf is described below

commit bb50ccfd27e19b329391b365d3b831d5095b39a8
Author: jenkins 
AuthorDate: Wed Mar 24 20:17:36 2021 +

INFRA-10751 Empty commit


[hbase] branch branch-1 updated: HBASE-25593 Backport changes from HBASE-24418 to branch-1 (#2991)

2021-03-24 Thread vjasani
This is an automated email from the ASF dual-hosted git repository.

vjasani pushed a commit to branch branch-1
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-1 by this push:
 new 35bb776  HBASE-25593 Backport changes from HBASE-24418 to branch-1 
(#2991)
35bb776 is described below

commit 35bb776f5568c7ad38c7ab96beb7ae119e719528
Author: Aman Poonia 
AuthorDate: Thu Mar 25 01:07:42 2021 +0530

HBASE-25593 Backport changes from HBASE-24418 to branch-1 (#2991)

Signed-off-by: Viraj Jasani 
---
 hbase-common/src/main/resources/hbase-default.xml  |  32 +-
 .../org/apache/hadoop/hbase/master/HMaster.java|  69 +--
 .../hbase/master/normalizer/RegionNormalizer.java  |   9 +-
 .../master/normalizer/SimpleRegionNormalizer.java  | 514 +++--
 .../normalizer/TestSimpleRegionNormalizer.java | 503 +++-
 .../TestSimpleRegionNormalizerOnCluster.java   | 143 +-
 6 files changed, 857 insertions(+), 413 deletions(-)

diff --git a/hbase-common/src/main/resources/hbase-default.xml 
b/hbase-common/src/main/resources/hbase-default.xml
index 009b841..3cde97a 100644
--- a/hbase-common/src/main/resources/hbase-default.xml
+++ b/hbase-common/src/main/resources/hbase-default.xml
@@ -611,8 +611,36 @@ possible configurations would overwhelm and obscure the 
important.
 hbase.regions.slop
 0.001
 Rebalance if any regionserver has average + (average * slop) 
regions. 
-  The default value of this parameter is 0.001 in StochasticLoadBalancer 
(the default load balancer),
-  while the default is 0.2 in other load balancers (i.e., 
SimpleLoadBalancer).
+  The default value of this parameter is 0.001 in StochasticLoadBalancer 
(the default
+  load balancer), while the default is 0.2 in other load balancers (i.e., 
SimpleLoadBalancer).
+
+  
+  
+hbase.normalizer.split.enabled
+true
+Whether to split a region as part of 
normalization.
+  
+  
+hbase.normalizer.merge.enabled
+true
+Whether to merge a region as part of 
normalization.
+  
+  
+hbase.normalizer.min.region.count
+3
+The minimum number of regions in a table to consider it for 
merge normalization.
+
+  
+  
+hbase.normalizer.merge.min_region_age.days
+3
+The minimum age for a region to be considered for a merge, in 
days.
+  
+  
+hbase.normalizer.merge.min_region_size.mb
+1
+The minimum size for a region to be considered for a merge, 
in whole MBs.
+
   
   
 hbase.server.thread.wakefrequency
diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
index e2fe330..1342ce9 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
@@ -41,6 +41,7 @@ import java.util.Set;
 import java.util.concurrent.CountDownLatch;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicReference;
+import java.util.concurrent.locks.ReentrantLock;
 import java.util.regex.Pattern;
 import javax.servlet.ServletException;
 import javax.servlet.http.HttpServlet;
@@ -106,6 +107,8 @@ import 
org.apache.hadoop.hbase.master.cleaner.ReplicationZKNodeCleaner;
 import org.apache.hadoop.hbase.master.cleaner.ReplicationZKNodeCleanerChore;
 import org.apache.hadoop.hbase.master.cleaner.SnapshotCleanerChore;
 import org.apache.hadoop.hbase.master.handler.DispatchMergingRegionHandler;
+import org.apache.hadoop.hbase.master.normalizer.NormalizationPlan;
+import org.apache.hadoop.hbase.master.normalizer.NormalizationPlan.PlanType;
 import org.apache.hadoop.hbase.master.normalizer.RegionNormalizer;
 import org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore;
 import org.apache.hadoop.hbase.master.normalizer.RegionNormalizerFactory;
@@ -121,9 +124,9 @@ import 
org.apache.hadoop.hbase.master.procedure.MasterDDLOperationHelper;
 import org.apache.hadoop.hbase.master.procedure.MasterProcedureConstants;
 import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
 import 
org.apache.hadoop.hbase.master.procedure.MasterProcedureScheduler.ProcedureEvent;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil;
 import org.apache.hadoop.hbase.master.procedure.ModifyColumnFamilyProcedure;
 import org.apache.hadoop.hbase.master.procedure.ModifyNamespaceProcedure;
-import org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil;
 import org.apache.hadoop.hbase.master.procedure.ModifyTableProcedure;
 import org.apache.hadoop.hbase.master.procedure.ProcedurePrepareLatch;
 import org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait;
@@ -133,9 +136,6 @@ import 
org.apache.hadoop.hbase.master.snapshot.SnapshotManager;
 import org.apache.hadoop.hbase.monitoring.MemoryBoundedLogMessageBuffer;
 import org.apache.hadoop.hbase.monitoring

[hbase] branch master updated: HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3081)

2021-03-24 Thread huaxiangsun
This is an automated email from the ASF dual-hosted git repository.

huaxiangsun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/master by this push:
 new 1e3fe3c  HBASE-25691 Test failure: 
TestVerifyBucketCacheFile.testRetrieveFromFile (#3081)
1e3fe3c is described below

commit 1e3fe3ceac323925d895d5e24b323bd056fc4b8e
Author: huaxiangsun 
AuthorDate: Wed Mar 24 09:01:17 2021 -0700

HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile 
(#3081)

The issue is that FileInputStream is created with try-with-resources, so 
its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected, 
its close() is called again.
To avoid this double-free resources, add guard against it.

Signed-off-by: stack 
---
 .../hadoop/hbase/io/hfile/bucket/BucketCache.java | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
index a84d812..0187f5e 100644
--- 
a/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
+++ 
b/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
@@ -1112,14 +1112,27 @@ public class BucketCache implements BlockCache, 
HeapSize {
*/
   private FileInputStream deleteFileOnClose(final File file) throws 
IOException {
 return new FileInputStream(file) {
+  private File myFile;
+  private FileInputStream init(File file) {
+myFile = file;
+return this;
+  }
   @Override
   public void close() throws IOException {
+// close() will be called during try-with-resources and it will be
+// called by finalizer thread during GC. To avoid double-free resource,
+// set myFile to null after the first call.
+if (myFile == null) {
+  return;
+}
+
 super.close();
-if (!file.delete()) {
-  throw new IOException("Failed deleting persistence file " + 
file.getAbsolutePath());
+if (!myFile.delete()) {
+  throw new IOException("Failed deleting persistence file " + 
myFile.getAbsolutePath());
 }
+myFile = null;
   }
-};
+}.init(file);
   }
 
   private void verifyCapacityAndClasses(long capacitySize, String ioclass, 
String mapclass)


[hbase] branch HBASE-22120 updated (6f72d03 -> e5e6c47)

2021-03-24 Thread zhangduo
This is an automated email from the ASF dual-hosted git repository.

zhangduo pushed a change to branch HBASE-22120
in repository https://gitbox.apache.org/repos/asf/hbase.git.


 discard 6f72d03  HBASE-25616 Upgrade opentelemetry to 1.0.0 (#3034)
 discard 3aabc2e  HBASE-25617 Revisit the span names (#2998)
 discard e4c1b1a  HBASE-25591 Upgrade opentelemetry to 0.17.1 (#2971)
 discard 534f87a  HBASE-25535 Set span kind to CLIENT in AbstractRpcClient 
(#2907)
 discard 7105111  HBASE-25484 Add trace support for WAL sync (#2892)
 discard cc36e7b  HBASE-25455 Add trace support for HRegion read/write 
operation (#2861)
 discard b6621a2  HBASE-25481 Add host and port attribute when tracing rpc call 
at client side (#2857)
 discard 6268a32  HBASE-25454 Add trace support for connection registry (#2828)
 discard 5a96f0a  HBASE-23898 Add trace support for simple apis in async client 
(#2813)
 discard 89eb1d4  HBASE-25424 Find a way to config OpenTelemetry tracing 
without direct… (#2808)
 discard 1be0ff0  HBASE-25401 Add trace support for async call in rpc client 
(#2790)
 discard 56992ca  HBASE-25373 Remove HTrace completely in code base and try to 
make use of OpenTelemetry
 add 95342a2  HBASE-25654 [Documentation] Fix format error in security.adoc
 add 373dc77  HBASE-25548 Optionally allow snapshots to preserve cluster's 
max file… (#2923)
 add d79019b  HBASE-25629 Reimplement TestCurrentHourProvider to not depend 
on unstable TZs (#3013)
 add 0e6c2c4  HBASE-25636 Expose HBCK report as metrics (#3031)
 add 0cc1ae4  HBASE-25587 [hbck2] Schedule SCP for all unknown servers 
(#2978)
 add cc61714  HBASE-25566 RoundRobinTableInputFormat (#2947)
 add 1a69a52  HBASE-25570 On largish cluster, "CleanerChore: Could not 
delete dir..." makes master log unreadable (#2949)
 add 7386fb6  HBASE-25622 Result#compareResults should compare tags. (#3026)
 add 876fec1  HBASE-25657 Fix spotbugs warnings after upgrading spotbugs to 
4.x (#3041)
 add aeec8ca  HBASE-25635 CandidateGenerator may miss some region balance 
actions (#3024)
 add 8337fb2  HBASE-25662 Fix spotbugs warning in 
RoundRobinTableInputFormat (#3050)
 add f4e1ab7  HBASE-25663 Make graceful_stop localhostname compare match 
even if fqdn (#3048)
 add 630f47e   HBASE-25660 Print split policy in use on Region open (as 
well as split policy vitals) (#3044)
 add 21409bf  HBASE-25573 release script generated vote template has 
incorrect staging area (#2952)
 add 625bea3  HBASE-25595 TestLruBlockCache.testBackgroundEvictionThread is 
flaky (#2974)
 add 0ef892b  HBASE-25621 Balancer should check region plan source to avoid 
misplace region groups (#3002)
 add db2e6d8  HBASE-25597 Add row info in Exception when cell size exceeds 
maxCellSize (#2976)
 add 5457554  HBASE-25374 Make REST Client connection and socket time out 
configurable (#2752)
 add c36e40e  Revert "HBASE-25663 Make graceful_stop localhostname compare 
match even if fqdn (#3048)"
 add 59ec375  HBASE-25594 graceful_stop.sh fails to unload regions when ran 
at localhost
 add d74ae15  HBASE-25568 Upgrade Thrift jar to fix CVE-2020-13949 (#3043)
 add 75931b4  HBASE-25669 Fix typo of hbase.mob.compaction.chore.period in 
the docs (#3056)
 add ebb0adf  HBASE-25665 Option to use hostname instead of canonical 
hostname for secure HBase cluster connection (#3051)
 add bcf503e  HBASE-25653 Add units and round off region size to 2 digits 
after decimal (#3046)
 add 976629c  HBASE-25608 Support HFileOutputFormat locality sensitive even 
destination cluster is different from source cluster (#2988)
 add ff38218  HBASE-25627: HBase replication should have a metric to 
represent if the source is stuck getting initialized (#3018)
 add d200a67  Update 2.4.x download link to release 2.4.2
 add a698b1e  HBASE-25673 Wrong log regarding current active master at 
ZKLeaderManager#waitToBecomeLeader (#3057)
 add 7ac1c8b  HBASE-25677 Server+table counters on each scan #nextRaw 
invocation becomes a bottleneck when heavy load (#3061)
 add 3ebb978  HBASE-25674 - RegionInfo.parseFrom(DataInputStream) sometimes 
fails to read the protobuf magic marker (#3062)
 add 0cead10  HBOPS-25594 Make easier to use graceful_stop on localhost 
mode (#3054)
 add d93035a  HBASE-25643 The delayed FlushRegionEntry should be removed 
when we ne… (#3049)
 add 585aca1  HBASE-25518 Support separate child regions to different 
region servers (#3001)
 add 82dfa27  Revert "HBOPS-25594 Make easier to use graceful_stop on 
localhost mode (#3054)"
 add cc6c14a  HBASE-25594 Make easier to use graceful_stop on localhost 
mode (#3054)
 add f405990  HBASE-25678 Support nonce operations for Increment/Append in 
RowMutations and CheckAndMutate (#3064)
 add fea4bd1  HBASE-25679 Size of log queue metric is incorrect (#3071)
 add a3938c8  HBASE-25681 Add a switch for server/table queryMeter (#3070)
 add ba3610d  HBASE-1957