[hbase-filesystem] branch master updated: HBASE-25572 add a 'release' profile to do common project release tasks.
This is an automated email from the ASF dual-hosted git repository.
busbey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase-filesystem.git
The following commit(s) were added to refs/heads/master by this push:
new b23665d HBASE-25572 add a 'release' profile to do common project
release tasks.
b23665d is described below
commit b23665d1b33fcb59956ab4d6c562fe9b2fdbb07c
Author: Sean Busbey
AuthorDate: Fri Feb 12 21:26:07 2021 -0600
HBASE-25572 add a 'release' profile to do common project release tasks.
Closes #20
Signed-off-by: Peter Somogyi
---
pom.xml | 57 +++--
1 file changed, 55 insertions(+), 2 deletions(-)
diff --git a/pom.xml b/pom.xml
index 09ff24b..555878a 100644
--- a/pom.xml
+++ b/pom.xml
@@ -38,8 +38,9 @@
UTF-8
-1.8
-1.8
+1.8
+${compileSource}
+${compileSource}
+
+ release
+
+
+
+org.apache.rat
+apache-rat-plugin
+
+
+package
+
+ check
+
+
+
+
+
+org.apache.maven.plugins
+maven-enforcer-plugin
+
+
+
+ ${compileSource}
+ HBase has unsupported dependencies.
+ HBase requires that all dependencies be compiled with version
${compileSource} or earlier
+ of the JDK to properly build from source. You appear to be using a newer
dependency. You can use
+ either "mvn -version" or "mvn enforcer:display-info" to verify what version
is active.
+ Non-release builds can temporarily build with a newer JDK version by setting
the
+ 'compileSource' property (eg. mvn -DcompileSource=1.8 clean package).
+
+
+
+
+
+
+
+
[hbase-operator-tools] branch master updated: HBASE-25577 HBase operator tools pom should include nexus staging repo management
This is an automated email from the ASF dual-hosted git repository. busbey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hbase-operator-tools.git The following commit(s) were added to refs/heads/master by this push: new b4f4676 HBASE-25577 HBase operator tools pom should include nexus staging repo management b4f4676 is described below commit b4f4676992a0d1b2e121c57eb5658898713d2754 Author: Sean Busbey AuthorDate: Sat Feb 13 12:41:58 2021 -0600 HBASE-25577 HBase operator tools pom should include nexus staging repo management closes #82 Signed-off-by: Peter Somogyi --- pom.xml | 22 ++ 1 file changed, 22 insertions(+) diff --git a/pom.xml b/pom.xml index 8a48889..3b5a64b 100644 --- a/pom.xml +++ b/pom.xml @@ -444,5 +444,27 @@ + + + apache-release + + + + +org.sonatype.plugins +nexus-staging-maven-plugin +1.6.8 +true + + https://repository.apache.org/ + apache.releases.https + + + + +
[hbase-operator-tools] annotated tag rel/1.1.0 updated (6e7626e -> 074c0b0)
This is an automated email from the ASF dual-hosted git repository. busbey pushed a change to annotated tag rel/1.1.0 in repository https://gitbox.apache.org/repos/asf/hbase-operator-tools.git. *** WARNING: tag rel/1.1.0 was modified! *** from 6e7626e (commit) to 074c0b0 (tag) tagging 6e7626e4701d3b50c313ce310c04e570dc138dfa (commit) replaces rel/1.0.0 by Sean Busbey on Sat Feb 20 01:12:33 2021 -0600 - Log - HBASE-25589 release hbase-operator-tools 1.1.0 vote thread: https://s.apache.org/hbase-operator-tools-1.1.0-vote-passes -BEGIN PGP SIGNATURE- iQIzBAABCAAdFiEEtQLIf9bwyPyEePmgqSb9BRAWQC0FAmAwtowACgkQqSb9BRAW QC2DCA//WQMWICb4VNBwsTZNUlOraMvd/CF3ZKtpnc24GpP681NVxRk842QymUL8 8OhknG882tzbzvnqB/s74fAR9EoUyGUTzALdcp4+pA6HJvGiamBPqIurns8WBllh BThJ1mY3jIsMyU++wG2MWaTiZah78gX0f4YhkqDNkoJJXBXuBM2rFncyO+lBhw3O rIf/1ZibO/cJIi5zXRHWKcy8sBAP+MZsMU+Xkit3aeFv4iQs+YsJ2KfPf2fUW/CJ C3GmLV8k4PMTw+FsQZSkzIz1Aey3rylB3pFO+kcqbHagF58FZ4zqK7fQ/b5MAuAz JQ03+aoSvM7KhTSdUAVr3LYPGZ1XuaTpkBgfkQdFAu/oPCIzM8aRtqI1i4VvqdXU zZd05w5pQVh1z7X0qlPUL5xP8aZyFsSMYQFqAfPYJqy+fnTy5L89mPdekMGpwxKk C2OhBwtlJ7RitnUm8vjyKWI2l1u+dVpR5gHlshuk0BuAUo/ehfokyBBAVtttbti5 puWeYx13Gja6VegvQ+0/gdUwmLyYVcmO+8+nAcvBd1deh+L66HCCX9pyR9ffBkH/ ttb4keEcVGyB5soEE/zPTrta4r8vwhT7ccSEOI7xU90Fmxv01G+ITI47WH2c8e3j xeIObNuWTbIXIcFpOiJJsIT/ktE6u1SJy3XgkpvRiV4AwTTycic= =a4kc -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[hbase-thirdparty] annotated tag rel/3.5.0 updated (f691e59 -> 8b7b2c8)
This is an automated email from the ASF dual-hosted git repository. busbey pushed a change to annotated tag rel/3.5.0 in repository https://gitbox.apache.org/repos/asf/hbase-thirdparty.git. *** WARNING: tag rel/3.5.0 was modified! *** from f691e59 (commit) to 8b7b2c8 (tag) tagging f691e59158ee0889b9659958d37145fe5e3b25d5 (commit) replaces rel/3.4.1 by Sean Busbey on Sat Feb 20 01:05:03 2021 -0600 - Log - HBASE-2 Release hbase-thirdparty 3.5.0 vote thread: https://s.apache.org/hbase-thirdparty-3.5.0-vote-passes -BEGIN PGP SIGNATURE- iQIzBAABCAAdFiEEtQLIf9bwyPyEePmgqSb9BRAWQC0FAmAwtUIACgkQqSb9BRAW QC1Iyw//dYTr1N3CPkoWoPn+PJe4BezsXzY4GeZGbAMduDuZqz+Dv9JfLoJgkBge zbYHy0tsQt2dmqbTrYDvV7dO7QX5YVmGYK6OMDmBVNKW9sq7y6tcUEVJYC8E8d2s ZQY2XhAPgmX9k1sfaI7+GtG9qPrMpWG8qanNTrsMEAjdd9G43XrdV1xIa8oRxvpT aomGEhWSsRJGUg3JVW6M3z+39k/lUX2Uj8us752ZbPgBpkpsesnNvcFTxxemNFi8 00f49cYhdBe/yQzH6lToEiq7zACCnIDfFyg5TGg3RM3bqFaSbbnVrw3hpFUXZxsH qV9TBanBBJ9j2RY80pwSXYasS9bU5kvUH0sReTqvPVn8cYvhia4kIxLDuuwEVQgJ X3MDQhS29v+QIy4NZ+nbt6JxRe+dYK8RHZKRZlFK4yuXxOukCTUCZoLogSx8zLOP hMh3WzxYvVdgO+bBt45YtH/I1U9ho2KXv8w5qwjoeAO2I+P4fIm64UGzu42WVB4b hpEbZ72+NAX6c9PTSsfnqLYHfl4+sObWneTFJKPff6riyiv4IVGU6pWB9KVK+fDU VAUw0g5U+BrwB1e7Z9+LPGKzhOlm5JHSYG332QyTAsOWeZmVhabg/Y++Ym3y3Nfa 49GCc1BAv5rVU3LfaKuHOXQejEXYs+gjNj30uEEU9W4mI+fGeyc= =ubFd -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r46214 - /release/hbase/hbase-operator-tools-1.0.0/
Author: busbey Date: Sat Feb 20 06:55:15 2021 New Revision: 46214 Log: removing non-current release of hbase operator tools Removed: release/hbase/hbase-operator-tools-1.0.0/
svn commit: r46213 - /dev/hbase/hbase-operator-tools-1.1.0RC0/ /release/hbase/hbase-operator-tools-1.1.0/
Author: busbey Date: Sat Feb 20 06:54:17 2021 New Revision: 46213 Log: Release Apache HBase Operator Tools 1.1.0 vote thread: https://s.apache.org/hbase-operator-tools-1.1.0-vote-passes Added: release/hbase/hbase-operator-tools-1.1.0/ - copied from r46212, dev/hbase/hbase-operator-tools-1.1.0RC0/ Removed: dev/hbase/hbase-operator-tools-1.1.0RC0/
svn commit: r46212 - /release/hbase/hbase-thirdparty-3.4.1/
Author: busbey Date: Sat Feb 20 06:49:32 2021 New Revision: 46212 Log: removing non-current version of hbase-thirdparty Removed: release/hbase/hbase-thirdparty-3.4.1/
svn commit: r46211 - /dev/hbase/hbase-thirdparty-3.5.0RC0/ /release/hbase/hbase-thirdparty-3.5.0/
Author: busbey Date: Sat Feb 20 06:48:21 2021 New Revision: 46211 Log: Release Apache HBase Thirdparty 3.5.0 vote thread: https://s.apache.org/hbase-thirdparty-3.5.0-vote-passes Added: release/hbase/hbase-thirdparty-3.5.0/ - copied from r46210, dev/hbase/hbase-thirdparty-3.5.0RC0/ Removed: dev/hbase/hbase-thirdparty-3.5.0RC0/
[hbase] branch branch-2.4 updated: HBASE-25562 ReplicationSourceWALReader log and handle exception immediately without retrying (#2943)
This is an automated email from the ASF dual-hosted git repository.
sunxin pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new c1a9e89 HBASE-25562 ReplicationSourceWALReader log and handle
exception immediately without retrying (#2943)
c1a9e89 is described below
commit c1a9e8973a10d9ae484a3ec30964f2c99e74b0d5
Author: XinSun
AuthorDate: Sat Feb 20 10:20:54 2021 +0800
HBASE-25562 ReplicationSourceWALReader log and handle exception immediately
without retrying (#2943)
Signed-off-by: Wellington Chevreuil
Signed-off-by: stack
Signed-off-by: shahrs87
---
.../regionserver/ReplicationSourceWALReader.java | 30 --
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
index 52ac144..f52a83a 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
@@ -150,14 +150,13 @@ class ReplicationSourceWALReader extends Thread {
}
}
} catch (IOException e) { // stream related
-if (sleepMultiplier < maxRetriesMultiplier) {
- LOG.debug("Failed to read stream of replication entries: " + e);
- sleepMultiplier++;
-} else {
- LOG.error("Failed to read stream of replication entries", e);
- handleEofException(e);
+if (!handleEofException(e)) {
+ LOG.warn("Failed to read stream of replication entries", e);
+ if (sleepMultiplier < maxRetriesMultiplier) {
+sleepMultiplier ++;
+ }
+ Threads.sleep(sleepForRetries * sleepMultiplier);
}
-Threads.sleep(sleepForRetries * sleepMultiplier);
} catch (InterruptedException e) {
LOG.trace("Interrupted while sleeping between WAL reads");
Thread.currentThread().interrupt();
@@ -244,10 +243,13 @@ class ReplicationSourceWALReader extends Thread {
}
}
- // if we get an EOF due to a zero-length log, and there are other logs in
queue
- // (highly likely we've closed the current log), we've hit the max retries,
and autorecovery is
- // enabled, then dump the log
- private void handleEofException(IOException e) {
+ /**
+ * if we get an EOF due to a zero-length log, and there are other logs in
queue
+ * (highly likely we've closed the current log), and autorecovery is
+ * enabled, then dump the log
+ * @return true only the IOE can be handled
+ */
+ private boolean handleEofException(IOException e) {
PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
// Dump the log even if logQueue size is 1 if the source is from recovered
Source
// since we don't add current log to recovered source queue so it is safe
to remove.
@@ -255,14 +257,16 @@ class ReplicationSourceWALReader extends Thread {
(source.isRecovered() || queue.size() > 1) && this.eofAutoRecovery) {
try {
if (fs.getFileStatus(queue.peek()).getLen() == 0) {
- LOG.warn("Forcing removal of 0 length log in queue: " +
queue.peek());
+ LOG.warn("Forcing removal of 0 length log in queue: {}",
queue.peek());
logQueue.remove(walGroupId);
currentPosition = 0;
+ return true;
}
} catch (IOException ioe) {
-LOG.warn("Couldn't get file length information about log " +
queue.peek());
+LOG.warn("Couldn't get file length information about log {}",
queue.peek());
}
}
+return false;
}
public Path getCurrentPath() {
[hbase] branch branch-2 updated: HBASE-25562 ReplicationSourceWALReader log and handle exception immediately without retrying (#2943)
This is an automated email from the ASF dual-hosted git repository.
sunxin pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new a3edcc2 HBASE-25562 ReplicationSourceWALReader log and handle
exception immediately without retrying (#2943)
a3edcc2 is described below
commit a3edcc285481357446ed02effbb83a4142402c8f
Author: XinSun
AuthorDate: Sat Feb 20 10:20:54 2021 +0800
HBASE-25562 ReplicationSourceWALReader log and handle exception immediately
without retrying (#2943)
Signed-off-by: Wellington Chevreuil
Signed-off-by: stack
Signed-off-by: shahrs87
---
.../regionserver/ReplicationSourceWALReader.java | 30 --
1 file changed, 17 insertions(+), 13 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
index 52ac144..f52a83a 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
@@ -150,14 +150,13 @@ class ReplicationSourceWALReader extends Thread {
}
}
} catch (IOException e) { // stream related
-if (sleepMultiplier < maxRetriesMultiplier) {
- LOG.debug("Failed to read stream of replication entries: " + e);
- sleepMultiplier++;
-} else {
- LOG.error("Failed to read stream of replication entries", e);
- handleEofException(e);
+if (!handleEofException(e)) {
+ LOG.warn("Failed to read stream of replication entries", e);
+ if (sleepMultiplier < maxRetriesMultiplier) {
+sleepMultiplier ++;
+ }
+ Threads.sleep(sleepForRetries * sleepMultiplier);
}
-Threads.sleep(sleepForRetries * sleepMultiplier);
} catch (InterruptedException e) {
LOG.trace("Interrupted while sleeping between WAL reads");
Thread.currentThread().interrupt();
@@ -244,10 +243,13 @@ class ReplicationSourceWALReader extends Thread {
}
}
- // if we get an EOF due to a zero-length log, and there are other logs in
queue
- // (highly likely we've closed the current log), we've hit the max retries,
and autorecovery is
- // enabled, then dump the log
- private void handleEofException(IOException e) {
+ /**
+ * if we get an EOF due to a zero-length log, and there are other logs in
queue
+ * (highly likely we've closed the current log), and autorecovery is
+ * enabled, then dump the log
+ * @return true only the IOE can be handled
+ */
+ private boolean handleEofException(IOException e) {
PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
// Dump the log even if logQueue size is 1 if the source is from recovered
Source
// since we don't add current log to recovered source queue so it is safe
to remove.
@@ -255,14 +257,16 @@ class ReplicationSourceWALReader extends Thread {
(source.isRecovered() || queue.size() > 1) && this.eofAutoRecovery) {
try {
if (fs.getFileStatus(queue.peek()).getLen() == 0) {
- LOG.warn("Forcing removal of 0 length log in queue: " +
queue.peek());
+ LOG.warn("Forcing removal of 0 length log in queue: {}",
queue.peek());
logQueue.remove(walGroupId);
currentPosition = 0;
+ return true;
}
} catch (IOException ioe) {
-LOG.warn("Couldn't get file length information about log " +
queue.peek());
+LOG.warn("Couldn't get file length information about log {}",
queue.peek());
}
}
+return false;
}
public Path getCurrentPath() {
[hbase] branch master updated (88057d8 -> ed90a14)
This is an automated email from the ASF dual-hosted git repository. sunxin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from 88057d8 HBASE-25539: Add age of oldest wal metric (#2945) add ed90a14 HBASE-25562 ReplicationSourceWALReader log and handle exception immediately without retrying (#2943) No new revisions were added by this update. Summary of changes: .../regionserver/ReplicationSourceWALReader.java | 30 -- 1 file changed, 17 insertions(+), 13 deletions(-)
[hbase-site] branch asf-site updated: INFRA-10751 Empty commit
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hbase-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 6ab7495 INFRA-10751 Empty commit 6ab7495 is described below commit 6ab74951a756f47fed2c7a36fa08b251d79e65e9 Author: jenkins AuthorDate: Fri Feb 19 20:19:49 2021 + INFRA-10751 Empty commit
[hbase] branch branch-1 updated: HBASE-25583: Handle the running replication source gracefully with replication nodes deleted (#2960)
This is an automated email from the ASF dual-hosted git repository.
apurtell pushed a commit to branch branch-1
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-1 by this push:
new f9a9148 HBASE-25583: Handle the running replication source gracefully
with replication nodes deleted (#2960)
f9a9148 is described below
commit f9a91488b2c39320bed502619bf7adb765c79de6
Author: Sandeep Pal <50725353+sandeepvina...@users.noreply.github.com>
AuthorDate: Fri Feb 19 10:15:45 2021 -0800
HBASE-25583: Handle the running replication source gracefully with
replication nodes deleted (#2960)
Signed-off-by: Wellington Chevreuil
Signed-off-by: Andrew Purtell
---
.../hbase/replication/ReplicationQueues.java | 2 +-
.../hbase/replication/ReplicationQueuesZKImpl.java | 50 --
.../ReplicationSourceWithoutPeerException.java | 39 +
.../regionserver/ReplicationSource.java| 21 ++-
.../regionserver/ReplicationSourceManager.java | 38 +++--
.../hbase/replication/ReplicationSourceDummy.java | 1 +
.../ReplicationSourceDummyWithNoTermination.java | 26 +++
.../hbase/replication/TestReplicationSource.java | 80 -
.../replication/TestReplicationStateBasic.java | 21 ++-
.../replication/TestReplicationStateZKImpl.java| 2 +-
.../regionserver/TestReplicationSourceBase.java| 150 +
.../regionserver/TestReplicationSourceManager.java | 180 -
...tReplicationSourceWithoutReplicationZnodes.java | 85 ++
.../regionserver/helper/DummyServer.java | 95 +++
14 files changed, 594 insertions(+), 196 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
index ccc7172..05ae794 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
@@ -59,7 +59,7 @@ public interface ReplicationQueues {
* @param queueId a String that identifies the queue.
* @param filename name of the WAL
*/
- void removeLog(String queueId, String filename);
+ void removeLog(String queueId, String filename) throws
ReplicationSourceWithoutPeerException;
/**
* Set the current position for a specific WAL in a given queue.
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
index 4121fa3..83e3680 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
@@ -25,19 +25,19 @@ import java.util.TreeSet;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
-import org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.Abortable;
import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.classification.InterfaceAudience;
import org.apache.hadoop.hbase.exceptions.DeserializationException;
import org.apache.hadoop.hbase.protobuf.ProtobufUtil;
import org.apache.hadoop.hbase.protobuf.generated.ZooKeeperProtos;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.Pair;
import org.apache.hadoop.hbase.zookeeper.ZKUtil;
-import org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher;
import org.apache.hadoop.hbase.zookeeper.ZKUtil.ZKUtilOp;
+import org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher;
import org.apache.zookeeper.KeeperException;
/**
@@ -132,17 +132,44 @@ public class ReplicationQueuesZKImpl extends
ReplicationStateZKBase implements R
}
@Override
- public void removeLog(String queueId, String filename) {
+ public void removeLog(String queueId, String filename)
+throws ReplicationSourceWithoutPeerException {
try {
- String znode = ZKUtil.joinZNode(this.myQueuesZnode, queueId);
- znode = ZKUtil.joinZNode(znode, filename);
- ZKUtil.deleteNode(this.zookeeper, znode);
-} catch (KeeperException e) {
- this.abortable.abort("Failed to remove wal from queue (queueId=" +
queueId + ", filename="
- + filename + ")", e);
+ try {
+String znode = ZKUtil.joinZNode(this.myQueuesZnode, queueId);
+znode = ZKUtil.joinZNode(znode, filename);
+ZKUtil.deleteNode(this.zookeeper, znode);
+ } catch (KeeperException.NoNodeException e) {
+// in case of no node exception we should not crash the region server
+// but instead check if the replication peer has been removed.
+// If so, we
