[GitHub] [hbase] Apache-HBase commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
Apache-HBase commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi… URL: https://github.com/apache/hbase/pull/323#issuecomment-509899559 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 24 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 257 | master passed | | +1 | compile | 24 | master passed | | +1 | checkstyle | 12 | master passed | | +1 | shadedjars | 266 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 40 | master passed | | +1 | javadoc | 19 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 241 | the patch passed | | +1 | compile | 23 | the patch passed | | +1 | javac | 23 | the patch passed | | -1 | checkstyle | 11 | hbase-rsgroup: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 255 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 719 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 47 | the patch passed | | +1 | javadoc | 19 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 260 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 12 | The patch does not generate ASF License warnings. | | | | 2545 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/323 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 96360d1fd185 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 9ac9505f2a | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/12/artifact/out/diff-checkstyle-hbase-rsgroup.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/12/testReport/ | | Max. process+thread count | 4558 (vs. ulimit of 1) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/12/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sunhelly commented on a change in pull request #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
sunhelly commented on a change in pull request #323: HBASE-22414 Interruption
of moving regions in RSGroup will cause regi…
URL: https://github.com/apache/hbase/pull/323#discussion_r301872555
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
##
@@ -459,4 +463,197 @@ public boolean evaluate() throws Exception {
Assert.assertEquals(null, rsGroupAdmin.getRSGroupInfo(fooGroup.getName()));
}
+ @Test
+ public void testFailedMoveWhenMoveServer() throws Exception {
+final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
1);
+final byte[] familyNameBytes = Bytes.toBytes("f");
+final int tableRegionCount = 10;
+// All the regions created below will be assigned to the default group.
+TEST_UTIL.createMultiRegionTable(tableName, familyNameBytes,
tableRegionCount);
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() throws Exception {
+List regions = getTableRegionMap().get(tableName);
+if (regions == null) {
+ return false;
+}
+return getTableRegionMap().get(tableName).size() >= tableRegionCount;
+ }
+});
+
+// get target server to move, which should has more than one regions
+// randomly set a region state to SPLITTING
+Map> assignMap =
getTableServerRegionMap().get(tableName);
+String rregion = null;
+ServerName toMoveServer = null;
+for (ServerName server : assignMap.keySet()) {
+ rregion = assignMap.get(server).size() > 1 &&
!newGroup.containsServer(server.getAddress()) ?
+ assignMap.get(server).get(0) :
+ null;
+ if (rregion != null) {
+toMoveServer = server;
+break;
+ }
+}
+assert toMoveServer != null;
+RegionInfo ri =
TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().
+getRegionInfo(Bytes.toBytesBinary(rregion));
+RegionStateNode rsn =
+
TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().getRegionStates()
+.getRegionStateNode(ri);
+rsn.setState(RegionState.State.SPLITTING);
+
+// start thread to recover region state
+final ServerName movedServer = toMoveServer;
+final String sregion = rregion;
+AtomicBoolean changed = new AtomicBoolean(false);
+Thread t1 = new Thread(() -> {
+ LOG.debug("thread1 start running, will recover region state");
+ long current = System.currentTimeMillis();
+ while (System.currentTimeMillis() - current <= 5) {
+List regions =
master.getAssignmentManager().getRegionsOnServer(movedServer);
+LOG.debug("server region size is:{}", regions.size());
+assert regions.size() >= 1;
+// when there is exactly one region left, we can determine the move
operation encountered
+// exception caused by the strange region state.
+if (regions.size() == 1) {
+ assertEquals(regions.get(0).getRegionNameAsString(), sregion);
+ rsn.setState(RegionState.State.OPEN);
+ LOG.info("set region {} state OPEN", sregion);
+ changed.set(true);
+ break;
+}
+sleep(5000);
+ }
+});
+t1.start();
+
+// move target server to group
+Thread t2 = new Thread(() -> {
+ LOG.info("thread2 start running, to move regions");
+ try {
+rsGroupAdmin.moveServers(Sets.newHashSet(movedServer.getAddress()),
newGroup.getName());
+ } catch (IOException e) {
+LOG.error("move server error", e);
+ }
+});
+t2.start();
+
+t1.join();
+t2.join();
+
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() {
+if (changed.get()) {
+ return
master.getAssignmentManager().getRegionsOnServer(movedServer).size() == 0 &&
!rsn
+ .getRegionLocation().equals(movedServer);
+}
+return false;
+ }
+});
+ }
+
+ @Test
+ public void testFailedMoveWhenMoveTable() throws Exception {
Review comment:
I will add a new skeleton function recoverRegionStateThread at the newly
attached patch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] sunhelly commented on a change in pull request #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
sunhelly commented on a change in pull request #323: HBASE-22414 Interruption
of moving regions in RSGroup will cause regi…
URL: https://github.com/apache/hbase/pull/323#discussion_r301868475
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
##
@@ -459,4 +463,197 @@ public boolean evaluate() throws Exception {
Assert.assertEquals(null, rsGroupAdmin.getRSGroupInfo(fooGroup.getName()));
}
+ @Test
+ public void testFailedMoveWhenMoveServer() throws Exception {
+final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
1);
+final byte[] familyNameBytes = Bytes.toBytes("f");
+final int tableRegionCount = 10;
+// All the regions created below will be assigned to the default group.
+TEST_UTIL.createMultiRegionTable(tableName, familyNameBytes,
tableRegionCount);
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() throws Exception {
+List regions = getTableRegionMap().get(tableName);
+if (regions == null) {
+ return false;
+}
+return getTableRegionMap().get(tableName).size() >= tableRegionCount;
+ }
+});
+
+// get target server to move, which should has more than one regions
+// randomly set a region state to SPLITTING
+Map> assignMap =
getTableServerRegionMap().get(tableName);
+String rregion = null;
+ServerName toMoveServer = null;
+for (ServerName server : assignMap.keySet()) {
+ rregion = assignMap.get(server).size() > 1 &&
!newGroup.containsServer(server.getAddress()) ?
+ assignMap.get(server).get(0) :
+ null;
+ if (rregion != null) {
+toMoveServer = server;
+break;
+ }
+}
+assert toMoveServer != null;
+RegionInfo ri =
TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().
+getRegionInfo(Bytes.toBytesBinary(rregion));
+RegionStateNode rsn =
+
TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().getRegionStates()
+.getRegionStateNode(ri);
+rsn.setState(RegionState.State.SPLITTING);
+
+// start thread to recover region state
+final ServerName movedServer = toMoveServer;
+final String sregion = rregion;
+AtomicBoolean changed = new AtomicBoolean(false);
+Thread t1 = new Thread(() -> {
+ LOG.debug("thread1 start running, will recover region state");
+ long current = System.currentTimeMillis();
+ while (System.currentTimeMillis() - current <= 5) {
+List regions =
master.getAssignmentManager().getRegionsOnServer(movedServer);
+LOG.debug("server region size is:{}", regions.size());
+assert regions.size() >= 1;
+// when there is exactly one region left, we can determine the move
operation encountered
+// exception caused by the strange region state.
+if (regions.size() == 1) {
+ assertEquals(regions.get(0).getRegionNameAsString(), sregion);
+ rsn.setState(RegionState.State.OPEN);
+ LOG.info("set region {} state OPEN", sregion);
+ changed.set(true);
+ break;
+}
+sleep(5000);
+ }
+});
+t1.start();
+
+// move target server to group
+Thread t2 = new Thread(() -> {
+ LOG.info("thread2 start running, to move regions");
+ try {
+rsGroupAdmin.moveServers(Sets.newHashSet(movedServer.getAddress()),
newGroup.getName());
+ } catch (IOException e) {
+LOG.error("move server error", e);
+ }
+});
+t2.start();
+
+t1.join();
+t2.join();
+
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() {
+if (changed.get()) {
+ return
master.getAssignmentManager().getRegionsOnServer(movedServer).size() == 0 &&
!rsn
+ .getRegionLocation().equals(movedServer);
+}
+return false;
+ }
+});
+ }
+
+ @Test
+ public void testFailedMoveWhenMoveTable() throws Exception {
+final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
1);
+final byte[] familyNameBytes = Bytes.toBytes("f");
+final int tableRegionCount = 5;
+// All the regions created below will be assigned to the default group.
+TEST_UTIL.createMultiRegionTable(tableName, familyNameBytes,
tableRegionCount);
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() throws Exception {
+List regions = getTableRegionMap().get(tableName);
+if (regions == null) {
+ return false;
+}
+return getTableRegionMap().get(tableName).size() >= tableRegionCount;
+ }
+});
+
+// randomly set a region state to SPLITTING
+Map> assignMap =
getTableServerRegionMap().get(tableName);
+String rregion = null;
+ServerName srcServer = null;
+for
[GitHub] [hbase] sunhelly commented on a change in pull request #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
sunhelly commented on a change in pull request #323: HBASE-22414 Interruption
of moving regions in RSGroup will cause regi…
URL: https://github.com/apache/hbase/pull/323#discussion_r301868062
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
##
@@ -459,4 +463,197 @@ public boolean evaluate() throws Exception {
Assert.assertEquals(null, rsGroupAdmin.getRSGroupInfo(fooGroup.getName()));
}
+ @Test
+ public void testFailedMoveWhenMoveServer() throws Exception {
+final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
1);
+final byte[] familyNameBytes = Bytes.toBytes("f");
+final int tableRegionCount = 10;
+// All the regions created below will be assigned to the default group.
+TEST_UTIL.createMultiRegionTable(tableName, familyNameBytes,
tableRegionCount);
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() throws Exception {
+List regions = getTableRegionMap().get(tableName);
+if (regions == null) {
+ return false;
+}
+return getTableRegionMap().get(tableName).size() >= tableRegionCount;
+ }
+});
+
+// get target server to move, which should has more than one regions
+// randomly set a region state to SPLITTING
+Map> assignMap =
getTableServerRegionMap().get(tableName);
+String rregion = null;
+ServerName toMoveServer = null;
+for (ServerName server : assignMap.keySet()) {
+ rregion = assignMap.get(server).size() > 1 &&
!newGroup.containsServer(server.getAddress()) ?
+ assignMap.get(server).get(0) :
+ null;
+ if (rregion != null) {
+toMoveServer = server;
+break;
+ }
+}
+assert toMoveServer != null;
+RegionInfo ri =
TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().
+getRegionInfo(Bytes.toBytesBinary(rregion));
+RegionStateNode rsn =
+
TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().getRegionStates()
+.getRegionStateNode(ri);
+rsn.setState(RegionState.State.SPLITTING);
+
+// start thread to recover region state
+final ServerName movedServer = toMoveServer;
+final String sregion = rregion;
+AtomicBoolean changed = new AtomicBoolean(false);
+Thread t1 = new Thread(() -> {
+ LOG.debug("thread1 start running, will recover region state");
+ long current = System.currentTimeMillis();
+ while (System.currentTimeMillis() - current <= 5) {
+List regions =
master.getAssignmentManager().getRegionsOnServer(movedServer);
+LOG.debug("server region size is:{}", regions.size());
+assert regions.size() >= 1;
+// when there is exactly one region left, we can determine the move
operation encountered
+// exception caused by the strange region state.
+if (regions.size() == 1) {
+ assertEquals(regions.get(0).getRegionNameAsString(), sregion);
+ rsn.setState(RegionState.State.OPEN);
+ LOG.info("set region {} state OPEN", sregion);
+ changed.set(true);
+ break;
+}
+sleep(5000);
+ }
+});
+t1.start();
+
+// move target server to group
+Thread t2 = new Thread(() -> {
+ LOG.info("thread2 start running, to move regions");
+ try {
+rsGroupAdmin.moveServers(Sets.newHashSet(movedServer.getAddress()),
newGroup.getName());
+ } catch (IOException e) {
+LOG.error("move server error", e);
+ }
+});
+t2.start();
+
+t1.join();
+t2.join();
+
+TEST_UTIL.waitFor(WAIT_TIMEOUT, new Waiter.Predicate() {
+ @Override
+ public boolean evaluate() {
+if (changed.get()) {
+ return
master.getAssignmentManager().getRegionsOnServer(movedServer).size() == 0 &&
!rsn
+ .getRegionLocation().equals(movedServer);
+}
+return false;
+ }
+});
+ }
+
+ @Test
+ public void testFailedMoveWhenMoveTable() throws Exception {
Review comment:
I make a function named setARegionState to randomly choosing a region and
set its state. The difference of the two UTs is at adding group and moving
operation. Is this OK?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] sunhelly commented on a change in pull request #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
sunhelly commented on a change in pull request #323: HBASE-22414 Interruption
of moving regions in RSGroup will cause regi…
URL: https://github.com/apache/hbase/pull/323#discussion_r301867559
##
File path:
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
##
@@ -459,4 +463,197 @@ public boolean evaluate() throws Exception {
Assert.assertEquals(null, rsGroupAdmin.getRSGroupInfo(fooGroup.getName()));
}
+ @Test
+ public void testFailedMoveWhenMoveServer() throws Exception {
+final RSGroupInfo newGroup = addGroup(getGroupName(name.getMethodName()),
1);
Review comment:
In this case, I have used rsGroupAdmin.addRSGroup to add a new group without
adding server. Then I create a multi-region table, choose a region to change
state, and record the server that the region we changed on. In the last call
RSGroupAdminServer.moveServers to move the recorded server, and check if the
error message contains the region name.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] sunhelly commented on a change in pull request #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
sunhelly commented on a change in pull request #323: HBASE-22414 Interruption
of moving regions in RSGroup will cause regi…
URL: https://github.com/apache/hbase/pull/323#discussion_r301866362
##
File path:
hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminServer.java
##
@@ -215,7 +216,12 @@ private void moveServerRegionsFromGroup(Set
servers, String targetGroup
if (!targetGrp.containsTable(region.getTable())) {
LOG.info("Moving server region {}, which do not belong to RSGroup
{}",
region.getShortNameToLog(), targetGroupName);
-this.master.getAssignmentManager().move(region);
+try {
+ this.master.getAssignmentManager().move(region);
+}catch (IOException ioe){
+ LOG.error("Move region {} from group failed, will retry, current
retry time is {}",
Review comment:
I have changed this log level to DEBUG, and added a failed regions list in
the ex message when max number of reties has been reached.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22665) RegionServer abort failed when AbstractFSWAL.shutdown hang
[
https://issues.apache.org/jira/browse/HBASE-22665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881637#comment-16881637
]
Duo Zhang commented on HBASE-22665:
---
Yes, it should be something like this [~wchevreuil]. But I want to write a UT
to reproduce it first... Give me some time.
> RegionServer abort failed when AbstractFSWAL.shutdown hang
> --
>
> Key: HBASE-22665
> URL: https://issues.apache.org/jira/browse/HBASE-22665
> Project: HBase
> Issue Type: Bug
> Environment: HBase 2.1.2
> Hadoop 3.1.x
> centos 7.4
>Reporter: Yechao Chen
>Priority: Major
> Attachments: image-2019-07-08-16-07-37-664.png,
> image-2019-07-08-16-08-26-777.png, image-2019-07-08-16-14-43-455.png,
> jstack_20190625, jstack_20190704_1, jstack_20190704_2, rs.log.part1
>
>
> We use hbase 2.1.2,when the rs with heavy qps and rs abort with error like
> "Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to
> get sync result after 30 ms for txid=36380334, WAL system stuck?"
>
> RegionServer aborted failed when AbstractFSWAL.shutdown hang
>
> jstack info always show the regionserver hang with "AbstractFSWAL.shutdown"
> "regionserver/hbase-slave-216-99:16020" #25 daemon prio=5 os_prio=0
> tid=0x7f204282c600 nid=0x34aa waiting on condition [0x7f0fe044d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f18a49b2bb8> (a
> java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:224)
> {color:#FF}at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285){color}
> {color:#FF} at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:815){color}
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.shutdown(AbstractFSWALProvider.java:168)
> at
> org.apache.hadoop.hbase.wal.RegionGroupingProvider.shutdown(RegionGroupingProvider.java:221)
> at org.apache.hadoop.hbase.wal.WALFactory.shutdown(WALFactory.java:239)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.shutdownWAL(HRegionServer.java:1445)
> {color:#FF}at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1117){color}
> {color:#FF} at java.lang.Thread.run(Thread.java:745){color}
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22645) Backport HBASE-19893 to all 2.x branches
[ https://issues.apache.org/jira/browse/HBASE-22645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881566#comment-16881566 ] Sakthi commented on HBASE-22645: Thanks [~stack] for pushing this in! > Backport HBASE-19893 to all 2.x branches > > > Key: HBASE-22645 > URL: https://issues.apache.org/jira/browse/HBASE-22645 > Project: HBase > Issue Type: Task > Components: snapshots >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Fix For: 2.0.6, 2.1.6 > > Attachments: hbase-22645.branch-2.1.001.patch, > hbase-22645.branch-2.1.001.patch > > > HBASE-19893 is present in 2.3, 2.2 but not in 2.1 or 2.0. This Jira is to > backport it to both these branches as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] Apache-HBase commented on issue #360: HBASE-15666 shaded dependencies for hbase-testing-util
Apache-HBase commented on issue #360: HBASE-15666 shaded dependencies for hbase-testing-util URL: https://github.com/apache/hbase/pull/360#issuecomment-509815326 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 23 | Docker mode activated. | ||| _ Prechecks _ | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 11 | Maven dependency ordering for branch | | +1 | mvninstall | 251 | master passed | | +1 | compile | 165 | master passed | | +1 | checkstyle | 125 | master passed | | +1 | shadedjars | 260 | branch has no errors when building our shaded downstream artifacts. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hbase-resource-bundle hbase-shaded hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants . | | +1 | findbugs | 0 | master passed | | +1 | javadoc | 196 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 13 | Maven dependency ordering for patch | | +1 | mvninstall | 298 | the patch passed | | +1 | compile | 166 | the patch passed | | +1 | javac | 166 | the patch passed | | -1 | checkstyle | 133 | root: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | shellcheck | 1 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 9 | The patch has no ill-formed XML file. | | +1 | shadedjars | 265 | patch has no errors when building our shaded downstream artifacts. | | -1 | hadoopcheck | 589 | The patch causes 12 errors with Hadoop v3.1.2. | | 0 | findbugs | 0 | Skipped patched modules with no Java source: hbase-resource-bundle hbase-shaded hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants . | | +1 | findbugs | 0 | the patch passed | | +1 | javadoc | 240 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 12283 | root in the patch passed. | | +1 | asflicense | 185 | The patch does not generate ASF License warnings. | | | | 15297 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-360/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/360 | | Optional Tests | dupname asflicense javac javadoc unit xml shellcheck shelldocs shadedjars hadoopcheck compile findbugs hbaseanti checkstyle | | uname | Linux 16d12ca43f7e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 9ac9505f2a | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | shellcheck | v0.4.4 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-360/2/artifact/out/diff-checkstyle-root.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-360/2/artifact/out/patch-javac-3.1.2.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-360/2/testReport/ | | Max. process+thread count | 5185 (vs. ulimit of 1) | | modules | C: hbase-resource-bundle hbase-shaded hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester hbase-shaded/hbase-shaded-check-invariants hbase-shaded/hbase-shaded-with-hadoop-check-invariants . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-360/2/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #368: HBASE-22662 Move RSGroupInfoManager to hbase-server
Apache-HBase commented on issue #368: HBASE-22662 Move RSGroupInfoManager to hbase-server URL: https://github.com/apache/hbase/pull/368#issuecomment-509790148 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 77 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for branch | | +1 | mvninstall | 282 | HBASE-22514 passed | | +1 | compile | 83 | HBASE-22514 passed | | +1 | checkstyle | 96 | HBASE-22514 passed | | +1 | shadedjars | 304 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 272 | HBASE-22514 passed | | +1 | javadoc | 50 | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 289 | the patch passed | | +1 | compile | 91 | the patch passed | | +1 | javac | 91 | the patch passed | | +1 | checkstyle | 97 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedjars | 307 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 878 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 279 | the patch passed | | +1 | javadoc | 49 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 14822 | hbase-server in the patch failed. | | +1 | unit | 426 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 50 | The patch does not generate ASF License warnings. | | | | 18865 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.quotas.TestSpaceQuotas | | | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/368 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile findbugs hbaseanti checkstyle | | uname | Linux 4abf8615b256 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | HBASE-22514 / 83db67204d | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/2/testReport/ | | Max. process+thread count | 4913 (vs. ulimit of 1) | | modules | C: hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/2/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] joshelser commented on a change in pull request #3: Hbase 22567
joshelser commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r301755347
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -164,6 +171,86 @@ int setRegionState(String region, RegionState.State
newState)
return EXIT_FAILURE;
}
+ int addMissingRegionsInMeta(String... tableNames) throws IOException {
+ExecutorService executorService = Executors.newFixedThreadPool(
+ tableNames.length > Runtime.getRuntime().availableProcessors() ?
+Runtime.getRuntime().availableProcessors() : tableNames.length);
+final CountDownLatch countDownLatch = new
CountDownLatch(tableNames.length);
+final List encodedRegionNames = new ArrayList<>();
+String result = "No regions added.";
+try(final MetaFixer metaFixer = new MetaFixer(this.conf)){
+ //reducing number of retries in case disable fails due to namespace
table region also missing
+ this.conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);
+ try(Connection conn = ConnectionFactory.createConnection(this.conf)) {
+final Admin admin = conn.getAdmin();
+for (String table : tableNames) {
+ final TableName tableName = TableName.valueOf(table);
+ if(admin.tableExists(tableName)) {
+executorService.submit(new Runnable() {
+ @Override public void run() {
+try {
+ LOG.debug("running thread for {}", table);
+ List missingRegions =
metaFixer.findMissingRegionsInMETA(table);
Review comment:
Yup, the use of Futures makes that much cleaner. Will have to dig in about
the issues of hbase:namespace being messed up :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] xcangCRM commented on a change in pull request #369: HBASE-21606 document meta table load metrics
xcangCRM commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301731650 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can Review comment: Actually, it's available since 1.3 Could you please double check that? thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #370: HBASE-19230 Write up fixVersion policy from dev discussion in refguide
Apache-HBase commented on issue #370: HBASE-19230 Write up fixVersion policy from dev discussion in refguide URL: https://github.com/apache/hbase/pull/370#issuecomment-509753488 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 25 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 | mvninstall | 243 | master passed | | 0 | refguide | 436 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 233 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | 0 | refguide | 436 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 | asflicense | 16 | The patch does not generate ASF License warnings. | | | | 1454 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-370/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/370 | | Optional Tests | dupname asflicense refguide | | uname | Linux 5631f8c3ad81 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 9ac9505f2a | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-370/1/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-370/1/artifact/out/patch-site/book.html | | Max. process+thread count | 81 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-370/1/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] xcangCRM commented on a change in pull request #369: HBASE-21606 document meta table load metrics
xcangCRM commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301727808 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can +affect the performance of the cluster. When it is enabled, it helps to monitor client access patterns by collecting +the following statistics: + +* number of get, put and delete operations on the `hbase:meta` table +* number of get, put and delete operations made by the top-N clients +* number of operations related to each table +* number of operations related to the top-N regions + +When to use the feature:: + This feature can help to identify hot spots in the meta table by showing the regions or tables where the meta info is + modified (e.g. by create, drop, split or move tables) or retrieved most frequently. It can also help to find misbehaving + client applications by showing which clients are using the meta table most heavily, which can for example suggest the + lack of meta table buffering or the lack of re-using open client connections in the client application. + +.Possible side-effects of enabling this feature +[WARNING] + +Having large number of clients and regions in the cluster can cause the registration and tracking of a large amount of +metrics, which can increase the memory and CPU footprint of the HBase region server handling the `hbase:meta` table. +It can also cause the significant increase of the JMX dump size, which can affect the monitoring or log aggregation +system you use beside HBase. It is recommended to turn on this feature only during debugging. + + +Where to find the metrics:: + Each metric attribute name will start with the ‘MetaTable_’ prefix. For all the metrics you will see five different + JMX attributes: count, mean rate, 1 minute rate, 5 minute rate and 15 minute rate. You will find these metrics in JMX + under the following MBean: + `Hadoop -> HBase -> RegionServer -> Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MetaTableMetrics` + +Configuration:: + To turn on this feature, you have to enable a custom coprocessor by adding the following section to hbase-site.xml. + This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on Review comment: It will not send metric when they don't host hbase:meta. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] syedmurtazahassan commented on issue #370: HBASE-19230 Write up fixVersion policy from dev discussion in refguide
syedmurtazahassan commented on issue #370: HBASE-19230 Write up fixVersion policy from dev discussion in refguide URL: https://github.com/apache/hbase/pull/370#issuecomment-509745639 @apurtell Kindly have a review when you have time. Thank you. I could not add you in reviewers and now i am not able to edit it to include you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] syedmurtazahassan opened a new pull request #370: HBASE-19230 Write up fixVersion policy from dev discussion in refguide
syedmurtazahassan opened a new pull request #370: HBASE-19230 Write up fixVersion policy from dev discussion in refguide URL: https://github.com/apache/hbase/pull/370 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
wchevreuil commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi… URL: https://github.com/apache/hbase/pull/323#issuecomment-509742126 Thanks for the updates, @sunhelly ! LGTM for the latest commits, had already resolved the conversation about exception handler when retries exhausted. Can you share your thoughts on the remaining comments? Let me know if those make sense. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[ https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881423#comment-16881423 ] Xu Cang commented on HBASE-21606: - Will do [~symat] thanks! > Document use of the meta table load metrics added in HBASE-19722 > > > Key: HBASE-21606 > URL: https://issues.apache.org/jira/browse/HBASE-21606 > Project: HBase > Issue Type: Task > Components: documentation, meta, metrics, Operability >Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3 >Reporter: Sean Busbey >Assignee: Szalay-Beko Mate >Priority: Critical > Attachments: HBASE-21606-v1.png > > > HBASE-19722 added a great new tool for figuring out where cluster load is > coming from. Needs a section in the ref guide > * When should I use this? > * Why shouldn't I use it all the time? > * What does using it look like? > * How do I use it? > I think all the needed info for making something to answer these questions is > in the discussion on HBASE-19722 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util
busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util URL: https://github.com/apache/hbase/pull/360#discussion_r301700745 ## File path: hbase-resource-bundle/src/main/resources/supplemental-models.xml ## @@ -332,6 +332,19 @@ under the License. + + + javax.servlet.jsp + jsp-api Review comment: weren't we already packaging this in the assembly? are we getting more than one of them from different groups or something? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util
busbey commented on a change in pull request #360: HBASE-15666 shaded
dependencies for hbase-testing-util
URL: https://github.com/apache/hbase/pull/360#discussion_r301698905
##
File path:
hbase-shaded/hbase-shaded-testing-util-tester/src/test/java/org/apache/hbase/shaded/TestShadedHBaseTestingUtility.java
##
@@ -0,0 +1,61 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase.shaded;
+
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.testclassification.ClientTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import static org.junit.Assert.assertEquals;
+
+@Category({ ClientTests.class, MediumTests.class })
+public class TestShadedHBaseTestingUtility {
+ private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestShadedHBaseTestingUtility.class);
+
+ @BeforeClass
+ public static void setUp() throws Exception {
+TEST_UTIL.startMiniCluster();
+ }
+
+ @AfterClass
+ public static void tearDown() throws Exception {
+TEST_UTIL.shutdownMiniCluster();
+ }
+
+ @Test
+ public void testCreateTable() throws Exception {
+TableName tableName = TableName.valueOf("test");
+
+TEST_UTIL.createTable(tableName, "family");
+
+int rows = TEST_UTIL.countRows(tableName);
Review comment:
probably best to actually do some puts so that we know the non-admin part of
the client works.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util
busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util URL: https://github.com/apache/hbase/pull/360#discussion_r301699356 ## File path: hbase-shaded/hbase-shaded-testing-util/src/main/resources/org/apache/hadoop/hbase/shaded/org/eclipse/jetty/webapp/webdefault.xml ## @@ -0,0 +1,534 @@ + Review comment: shouldn't we be able to add a proper license header for this file? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util
busbey commented on a change in pull request #360: HBASE-15666 shaded dependencies for hbase-testing-util URL: https://github.com/apache/hbase/pull/360#discussion_r301700356 ## File path: pom.xml ## @@ -2257,6 +2257,7 @@ junit junit + test Review comment: previously this caused issues with our assembly, because the ITs relied on it in order to run against a live cluster or standalone instance. please make sure that still works. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on issue #360: HBASE-15666 shaded dependencies for hbase-testing-util
busbey commented on issue #360: HBASE-15666 shaded dependencies for hbase-testing-util URL: https://github.com/apache/hbase/pull/360#issuecomment-509727436 I presume the dependency information will go in a release note? Should we call this out in the ref guide now? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #356: HBASE-15666 shaded dependencies for hbase-testing-util
busbey commented on a change in pull request #356: HBASE-15666 shaded dependencies for hbase-testing-util URL: https://github.com/apache/hbase/pull/356#discussion_r301677983 ## File path: hbase-shaded/hbase-shaded-testing-util/src/main/resources/org/apache/hadoop/hbase/shaded/org/mortbay/jetty/webapp/webdefault.xml ## @@ -0,0 +1,410 @@ + Review comment: needs a license header. presuming it is a template copied from somewhere it needs attribution. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics
busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301695838 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can +affect the performance of the cluster. When it is enabled, it helps to monitor client access patterns by collecting +the following statistics: + +* number of get, put and delete operations on the `hbase:meta` table +* number of get, put and delete operations made by the top-N clients +* number of operations related to each table +* number of operations related to the top-N regions + +When to use the feature:: + This feature can help to identify hot spots in the meta table by showing the regions or tables where the meta info is + modified (e.g. by create, drop, split or move tables) or retrieved most frequently. It can also help to find misbehaving + client applications by showing which clients are using the meta table most heavily, which can for example suggest the + lack of meta table buffering or the lack of re-using open client connections in the client application. + +.Possible side-effects of enabling this feature +[WARNING] + +Having large number of clients and regions in the cluster can cause the registration and tracking of a large amount of +metrics, which can increase the memory and CPU footprint of the HBase region server handling the `hbase:meta` table. +It can also cause the significant increase of the JMX dump size, which can affect the monitoring or log aggregation +system you use beside HBase. It is recommended to turn on this feature only during debugging. + + +Where to find the metrics:: + Each metric attribute name will start with the ‘MetaTable_’ prefix. For all the metrics you will see five different + JMX attributes: count, mean rate, 1 minute rate, 5 minute rate and 15 minute rate. You will find these metrics in JMX + under the following MBean: + `Hadoop -> HBase -> RegionServer -> Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MetaTableMetrics` + +Configuration:: + To turn on this feature, you have to enable a custom coprocessor by adding the following section to hbase-site.xml. + This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on + the region, where the `hbase:meta` table is located. It will produce JMX metrics which can be downloaded from the + web UI of the given RegionServer or by a simple REST call. + +.Enabling the Meta Table Metrics feature +[source,xml] + + +hbase.coprocessor.region.classes +org.apache.hadoop.hbase.coprocessor.MetaTableMetrics + + + +.How the top-N metrics are calculated? +[NOTE] + +The 'top-N' type of metrics will be counted using the lossy count algorithm, which is about to identify elements in a +data stream whose frequency count exceed a user-given threshold. The frequency computed by this algorithm is not always +accurate, but has an error threshold that can be specified by the user as a configuration parameter. +The run time space required by the algorithm is inversely proportional to the specified error threshold, hence larger +the error parameter, the smaller the footprint and the less accurate are the metrics. (see the following paper: +link:http://www.vldb.org/conf/2002/S10P03.pdf[Motwani, R; Manku, G.S (2002). "Approximate frequency counts over data streams"]) + +You can specify the error rate of the algorithm as a floating-point value between 0 and 1 (exclusive), it's default +value is 0.02. Having the error rate set to `E` and having `N` as the total number of meta table operations, then +(assuming the random distribution of the activity of low frequency elements) at most `7 / E` meters will be kept and Review comment: "assuming a uniform distribution for the activity of low frequency elements" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics
busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301694772 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can +affect the performance of the cluster. When it is enabled, it helps to monitor client access patterns by collecting +the following statistics: + +* number of get, put and delete operations on the `hbase:meta` table +* number of get, put and delete operations made by the top-N clients +* number of operations related to each table +* number of operations related to the top-N regions + +When to use the feature:: + This feature can help to identify hot spots in the meta table by showing the regions or tables where the meta info is + modified (e.g. by create, drop, split or move tables) or retrieved most frequently. It can also help to find misbehaving + client applications by showing which clients are using the meta table most heavily, which can for example suggest the + lack of meta table buffering or the lack of re-using open client connections in the client application. + +.Possible side-effects of enabling this feature +[WARNING] + +Having large number of clients and regions in the cluster can cause the registration and tracking of a large amount of +metrics, which can increase the memory and CPU footprint of the HBase region server handling the `hbase:meta` table. +It can also cause the significant increase of the JMX dump size, which can affect the monitoring or log aggregation +system you use beside HBase. It is recommended to turn on this feature only during debugging. + + +Where to find the metrics:: + Each metric attribute name will start with the ‘MetaTable_’ prefix. For all the metrics you will see five different + JMX attributes: count, mean rate, 1 minute rate, 5 minute rate and 15 minute rate. You will find these metrics in JMX + under the following MBean: + `Hadoop -> HBase -> RegionServer -> Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MetaTableMetrics` + +Configuration:: + To turn on this feature, you have to enable a custom coprocessor by adding the following section to hbase-site.xml. + This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on + the region, where the `hbase:meta` table is located. It will produce JMX metrics which can be downloaded from the + web UI of the given RegionServer or by a simple REST call. + +.Enabling the Meta Table Metrics feature +[source,xml] + + +hbase.coprocessor.region.classes +org.apache.hadoop.hbase.coprocessor.MetaTableMetrics + + + +.How the top-N metrics are calculated? +[NOTE] + +The 'top-N' type of metrics will be counted using the lossy count algorithm, which is about to identify elements in a +data stream whose frequency count exceed a user-given threshold. The frequency computed by this algorithm is not always +accurate, but has an error threshold that can be specified by the user as a configuration parameter. Review comment: no comma between "accurate" and "but" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics
busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301692459 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can +affect the performance of the cluster. When it is enabled, it helps to monitor client access patterns by collecting +the following statistics: + +* number of get, put and delete operations on the `hbase:meta` table +* number of get, put and delete operations made by the top-N clients +* number of operations related to each table +* number of operations related to the top-N regions + +When to use the feature:: + This feature can help to identify hot spots in the meta table by showing the regions or tables where the meta info is + modified (e.g. by create, drop, split or move tables) or retrieved most frequently. It can also help to find misbehaving + client applications by showing which clients are using the meta table most heavily, which can for example suggest the + lack of meta table buffering or the lack of re-using open client connections in the client application. + +.Possible side-effects of enabling this feature +[WARNING] + +Having large number of clients and regions in the cluster can cause the registration and tracking of a large amount of +metrics, which can increase the memory and CPU footprint of the HBase region server handling the `hbase:meta` table. +It can also cause the significant increase of the JMX dump size, which can affect the monitoring or log aggregation +system you use beside HBase. It is recommended to turn on this feature only during debugging. + + +Where to find the metrics:: + Each metric attribute name will start with the ‘MetaTable_’ prefix. For all the metrics you will see five different + JMX attributes: count, mean rate, 1 minute rate, 5 minute rate and 15 minute rate. You will find these metrics in JMX + under the following MBean: + `Hadoop -> HBase -> RegionServer -> Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MetaTableMetrics` + +Configuration:: + To turn on this feature, you have to enable a custom coprocessor by adding the following section to hbase-site.xml. + This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on + the region, where the `hbase:meta` table is located. It will produce JMX metrics which can be downloaded from the Review comment: will only be active on the server right? so maybe > This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on the server where the `hbase:meta` table's region is located. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics
busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301694511 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can +affect the performance of the cluster. When it is enabled, it helps to monitor client access patterns by collecting +the following statistics: + +* number of get, put and delete operations on the `hbase:meta` table +* number of get, put and delete operations made by the top-N clients +* number of operations related to each table +* number of operations related to the top-N regions + +When to use the feature:: + This feature can help to identify hot spots in the meta table by showing the regions or tables where the meta info is + modified (e.g. by create, drop, split or move tables) or retrieved most frequently. It can also help to find misbehaving + client applications by showing which clients are using the meta table most heavily, which can for example suggest the + lack of meta table buffering or the lack of re-using open client connections in the client application. + +.Possible side-effects of enabling this feature +[WARNING] + +Having large number of clients and regions in the cluster can cause the registration and tracking of a large amount of +metrics, which can increase the memory and CPU footprint of the HBase region server handling the `hbase:meta` table. +It can also cause the significant increase of the JMX dump size, which can affect the monitoring or log aggregation +system you use beside HBase. It is recommended to turn on this feature only during debugging. + + +Where to find the metrics:: + Each metric attribute name will start with the ‘MetaTable_’ prefix. For all the metrics you will see five different + JMX attributes: count, mean rate, 1 minute rate, 5 minute rate and 15 minute rate. You will find these metrics in JMX + under the following MBean: + `Hadoop -> HBase -> RegionServer -> Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MetaTableMetrics` + +Configuration:: + To turn on this feature, you have to enable a custom coprocessor by adding the following section to hbase-site.xml. + This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on + the region, where the `hbase:meta` table is located. It will produce JMX metrics which can be downloaded from the + web UI of the given RegionServer or by a simple REST call. + +.Enabling the Meta Table Metrics feature +[source,xml] + + +hbase.coprocessor.region.classes +org.apache.hadoop.hbase.coprocessor.MetaTableMetrics + + + +.How the top-N metrics are calculated? +[NOTE] + +The 'top-N' type of metrics will be counted using the lossy count algorithm, which is about to identify elements in a Review comment: either > using a lossy counting algorithm, which is designed to identify elements in a data stream whose... or > using the Lossy Counting Algorithm (as defined in link:http://www.vldb.org/conf/2002/S10P03.pdf[Motwani, R; Manku, G.S (2002). "Approximate frequency counts over data streams"]), which is designed to identify elements in a data stream whose... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics
busbey commented on a change in pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#discussion_r301696454 ## File path: src/main/asciidoc/_chapters/ops_mgt.adoc ## @@ -1738,6 +1738,83 @@ hbase.regionserver.authenticationFailures:: hbase.regionserver.mutationsWithoutWALCount :: Count of writes submitted with a flag indicating they should bypass the write ahead log +[[rs_meta_metrics]] +=== Meta Table Load Metrics + +HBase meta table metrics collection feature is available in HBase 1.4+ but it is disabled by default, as it can +affect the performance of the cluster. When it is enabled, it helps to monitor client access patterns by collecting +the following statistics: + +* number of get, put and delete operations on the `hbase:meta` table +* number of get, put and delete operations made by the top-N clients +* number of operations related to each table +* number of operations related to the top-N regions + +When to use the feature:: + This feature can help to identify hot spots in the meta table by showing the regions or tables where the meta info is + modified (e.g. by create, drop, split or move tables) or retrieved most frequently. It can also help to find misbehaving + client applications by showing which clients are using the meta table most heavily, which can for example suggest the + lack of meta table buffering or the lack of re-using open client connections in the client application. + +.Possible side-effects of enabling this feature +[WARNING] + +Having large number of clients and regions in the cluster can cause the registration and tracking of a large amount of +metrics, which can increase the memory and CPU footprint of the HBase region server handling the `hbase:meta` table. +It can also cause the significant increase of the JMX dump size, which can affect the monitoring or log aggregation +system you use beside HBase. It is recommended to turn on this feature only during debugging. + + +Where to find the metrics:: + Each metric attribute name will start with the ‘MetaTable_’ prefix. For all the metrics you will see five different + JMX attributes: count, mean rate, 1 minute rate, 5 minute rate and 15 minute rate. You will find these metrics in JMX + under the following MBean: + `Hadoop -> HBase -> RegionServer -> Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MetaTableMetrics` + +Configuration:: + To turn on this feature, you have to enable a custom coprocessor by adding the following section to hbase-site.xml. + This coprocessor will run on all the HBase RegionServers, but will be active (i.e. consume memory / CPU) only on Review comment: just to be clear, are the metrics totally missing from other RegionServers? or are they just always 0? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on issue #369: HBASE-21606 document meta table load metrics
busbey commented on issue #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#issuecomment-509720635 here's the new section in the rendering the QA bot did: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-369/1/artifact/out/patch-site/book.html#rs_meta_metrics This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-connectors] busbey commented on a change in pull request #34: HBASE-22599 Let hbase-connectors compile against HBase 2.2.0
busbey commented on a change in pull request #34: HBASE-22599 Let
hbase-connectors compile against HBase 2.2.0
URL: https://github.com/apache/hbase-connectors/pull/34#discussion_r301687377
##
File path:
kafka/hbase-kafka-proxy/src/main/java/org/apache/hadoop/hbase/kafka/KafkaBridgeConnection.java
##
@@ -40,22 +40,15 @@
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.yetus.audience.InterfaceAudience;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
/**
* a alternative implementation of a connection object that forwards the
mutations to a kafka queue
* depending on the routing rules (see kafka-route-rules.xml).
* */
@InterfaceAudience.Private
public class KafkaBridgeConnection implements Connection {
- private static final Logger LOG =
LoggerFactory.getLogger(KafkaBridgeConnection.class);
Review comment:
why are we dropping the LOG definition?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase-connectors] busbey commented on a change in pull request #34: HBASE-22599 Let hbase-connectors compile against HBase 2.2.0
busbey commented on a change in pull request #34: HBASE-22599 Let
hbase-connectors compile against HBase 2.2.0
URL: https://github.com/apache/hbase-connectors/pull/34#discussion_r301687016
##
File path:
kafka/hbase-kafka-proxy/src/main/java/org/apache/hadoop/hbase/kafka/KafkaBridgeConnection.java
##
@@ -161,6 +145,10 @@ public RegionLocator getRegionLocator(TableName
tableName) throws IOException {
return null;
}
+ @Override
+ public void clearRegionLocationCache() {
Review comment:
what happens when building against HBase 2.1.z here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
Apache-HBase commented on issue #363: Add unit tests for org.apache.hadoop.hbase.util.Strings URL: https://github.com/apache/hbase/pull/363#issuecomment-509712837 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 54 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 261 | master passed | | +1 | compile | 20 | master passed | | +1 | checkstyle | 23 | master passed | | +1 | shadedjars | 270 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 38 | master passed | | +1 | javadoc | 20 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 239 | the patch passed | | +1 | compile | 21 | the patch passed | | +1 | javac | 21 | the patch passed | | +1 | checkstyle | 22 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 267 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 738 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 47 | the patch passed | | +1 | javadoc | 19 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 172 | hbase-common in the patch passed. | | +1 | asflicense | 10 | The patch does not generate ASF License warnings. | | | | 2541 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-363/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/363 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ef07f7fc5201 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 9ac9505f2a | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-363/3/testReport/ | | Max. process+thread count | 327 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-363/3/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #369: HBASE-21606 document meta table load metrics
Apache-HBase commented on issue #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369#issuecomment-509704464 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 28 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 | mvninstall | 266 | master passed | | 0 | refguide | 475 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 237 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | 0 | refguide | 482 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 | asflicense | 14 | The patch does not generate ASF License warnings. | | | | 1568 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-369/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/369 | | Optional Tests | dupname asflicense refguide | | uname | Linux 4ddc4bc5658f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 9ac9505f2a | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-369/1/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-369/1/artifact/out/patch-site/book.html | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-369/1/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22665) RegionServer abort failed when AbstractFSWAL.shutdown hang
[
https://issues.apache.org/jira/browse/HBASE-22665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881338#comment-16881338
]
Wellington Chevreuil commented on HBASE-22665:
--
I guess we can assume it's fine to sign the _readyForRollingCond_ if:
{code:java}
writerBroken(currentEpochAndState) and waitingRoll(currentEpochAndState){code}
After all, the given writer won't be able to _sync_ anymore, right? Right now,
we are just checking on the [_writerBroken_ condition
first|https://github.com/apache/hbase/blob/branch-2.1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java#L479],
ending consumer thread without doing nothing. Thus, if no more writes happen,
_LogRoller_ thread may never move on.
In the hypothetical situation I wondered about in my previous messages, where
1) sync failed, then 2) LogRoller kicked out, we have two state values that
currently will prevent _waitForSafePoint_ from being awakened: 1)
_writerBroken_ is true and 2) _unackedAppends_ not empty.
We may get this fixed by change [this
block|https://github.com/apache/hbase/blob/branch-2.1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java#L479]
as follows:
{code:java}
if (writerBroken(currentEpochAndState)&&!waitingRoll(currentEpochAndState)) {
return;
}
if (waitingRoll(currentEpochAndState)) {
if (writer.getLength() > fileLengthAtLastSync) {
// issue a sync
sync(writer);
} else {
if (unackedAppends.isEmpty()||writerBroken(currentEpochAndState)) {
readyForRolling = true;
readyForRollingCond.signalAll();
}
}
return;
}
{code}
> RegionServer abort failed when AbstractFSWAL.shutdown hang
> --
>
> Key: HBASE-22665
> URL: https://issues.apache.org/jira/browse/HBASE-22665
> Project: HBase
> Issue Type: Bug
> Environment: HBase 2.1.2
> Hadoop 3.1.x
> centos 7.4
>Reporter: Yechao Chen
>Priority: Major
> Attachments: image-2019-07-08-16-07-37-664.png,
> image-2019-07-08-16-08-26-777.png, image-2019-07-08-16-14-43-455.png,
> jstack_20190625, jstack_20190704_1, jstack_20190704_2, rs.log.part1
>
>
> We use hbase 2.1.2,when the rs with heavy qps and rs abort with error like
> "Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to
> get sync result after 30 ms for txid=36380334, WAL system stuck?"
>
> RegionServer aborted failed when AbstractFSWAL.shutdown hang
>
> jstack info always show the regionserver hang with "AbstractFSWAL.shutdown"
> "regionserver/hbase-slave-216-99:16020" #25 daemon prio=5 os_prio=0
> tid=0x7f204282c600 nid=0x34aa waiting on condition [0x7f0fe044d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f18a49b2bb8> (a
> java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:224)
> {color:#FF}at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285){color}
> {color:#FF} at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:815){color}
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.shutdown(AbstractFSWALProvider.java:168)
> at
> org.apache.hadoop.hbase.wal.RegionGroupingProvider.shutdown(RegionGroupingProvider.java:221)
> at org.apache.hadoop.hbase.wal.WALFactory.shutdown(WALFactory.java:239)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.shutdownWAL(HRegionServer.java:1445)
> {color:#FF}at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1117){color}
> {color:#FF} at java.lang.Thread.run(Thread.java:745){color}
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22669) Add unit tests for org.apache.hadoop.hbase.util.Strings
Eric Hettiaratchi created HBASE-22669:
-
Summary: Add unit tests for org.apache.hadoop.hbase.util.Strings
Key: HBASE-22669
URL: https://issues.apache.org/jira/browse/HBASE-22669
Project: HBase
Issue Type: Improvement
Components: java
Environment: Junit
Reporter: Eric Hettiaratchi
I've analysed your codebase and noticed that
{{org.apache.hadoop.hbase.util.Strings}} is not fully tested.
I've written some tests for the methods in this class with the help of
[Diffblue Cover|https://www.diffblue.com/opensource].
Hopefully, these tests will help you detect any regressions caused by future
code changes. If you would find it useful to have additional tests written for
this repository, I would be more than happy to look at other classes that you
consider important in a subsequent PR.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[ https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881323#comment-16881323 ] Szalay-Beko Mate commented on HBASE-21606: -- [~xucang], [~jatsakthi] can you take a look on the pull request if you have time? Thanks! > Document use of the meta table load metrics added in HBASE-19722 > > > Key: HBASE-21606 > URL: https://issues.apache.org/jira/browse/HBASE-21606 > Project: HBase > Issue Type: Task > Components: documentation, meta, metrics, Operability >Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3 >Reporter: Sean Busbey >Assignee: Szalay-Beko Mate >Priority: Critical > Attachments: HBASE-21606-v1.png > > > HBASE-19722 added a great new tool for figuring out where cluster load is > coming from. Needs a section in the ref guide > * When should I use this? > * Why shouldn't I use it all the time? > * What does using it look like? > * How do I use it? > I think all the needed info for making something to answer these questions is > in the discussion on HBASE-19722 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] symat opened a new pull request #369: HBASE-21606 document meta table load metrics
symat opened a new pull request #369: HBASE-21606 document meta table load metrics URL: https://github.com/apache/hbase/pull/369 I also attached a screenshot in jira (to see the formatting): [HBASE-21606-v1.png](https://issues.apache.org/jira/secure/attachment/12974064/HBASE-21606-v1.png) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-21606) Document use of the meta table load metrics added in HBASE-19722
[ https://issues.apache.org/jira/browse/HBASE-21606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szalay-Beko Mate updated HBASE-21606: - Attachment: HBASE-21606-v1.png > Document use of the meta table load metrics added in HBASE-19722 > > > Key: HBASE-21606 > URL: https://issues.apache.org/jira/browse/HBASE-21606 > Project: HBase > Issue Type: Task > Components: documentation, meta, metrics, Operability >Affects Versions: 3.0.0, 1.5.0, 1.4.6, 2.2.0, 2.0.2, 2.1.3 >Reporter: Sean Busbey >Assignee: Szalay-Beko Mate >Priority: Critical > Attachments: HBASE-21606-v1.png > > > HBASE-19722 added a great new tool for figuring out where cluster load is > coming from. Needs a section in the ref guide > * When should I use this? > * Why shouldn't I use it all the time? > * What does using it look like? > * How do I use it? > I think all the needed info for making something to answer these questions is > in the discussion on HBASE-19722 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22606) BucketCache improvement with additional tests
[
https://issues.apache.org/jira/browse/HBASE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881313#comment-16881313
]
Hudson commented on HBASE-22606:
Results for branch master
[build #1207 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1207/]: (/)
*{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1207//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1207//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1207//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> BucketCache improvement with additional tests
> -
>
> Key: HBASE-22606
> URL: https://issues.apache.org/jira/browse/HBASE-22606
> Project: HBase
> Issue Type: Task
>Affects Versions: 3.0.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0
>
>
> BucketCache code improvement with some additional test cases to cover up
> quite some logic implemented for Caching
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-22668) Integration Test for BulkLoad replication
[ https://issues.apache.org/jira/browse/HBASE-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Norbert Kalmar reassigned HBASE-22668: -- Assignee: Norbert Kalmar > Integration Test for BulkLoad replication > - > > Key: HBASE-22668 > URL: https://issues.apache.org/jira/browse/HBASE-22668 > Project: HBase > Issue Type: Test > Components: Replication >Reporter: Norbert Kalmar >Assignee: Norbert Kalmar >Priority: Major > > Add Integration Test to check if BulkLoad replication works, data is > available on a sink cluster once BulkLoad is complete on master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22668) Integration Test for BulkLoad replication
Norbert Kalmar created HBASE-22668: -- Summary: Integration Test for BulkLoad replication Key: HBASE-22668 URL: https://issues.apache.org/jira/browse/HBASE-22668 Project: HBase Issue Type: Test Components: Replication Reporter: Norbert Kalmar Add Integration Test to check if BulkLoad replication works, data is available on a sink cluster once BulkLoad is complete on master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-22668) Integration Test for BulkLoad replication
[ https://issues.apache.org/jira/browse/HBASE-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-22668 started by Norbert Kalmar. -- > Integration Test for BulkLoad replication > - > > Key: HBASE-22668 > URL: https://issues.apache.org/jira/browse/HBASE-22668 > Project: HBase > Issue Type: Test > Components: Replication >Reporter: Norbert Kalmar >Assignee: Norbert Kalmar >Priority: Major > > Add Integration Test to check if BulkLoad replication works, data is > available on a sink cluster once BulkLoad is complete on master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] Apache-HBase commented on issue #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
Apache-HBase commented on issue #363: Add unit tests for org.apache.hadoop.hbase.util.Strings URL: https://github.com/apache/hbase/pull/363#issuecomment-509659061 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 239 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 299 | master passed | | +1 | compile | 23 | master passed | | +1 | checkstyle | 25 | master passed | | +1 | shadedjars | 276 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 41 | master passed | | +1 | javadoc | 20 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 242 | the patch passed | | +1 | compile | 22 | the patch passed | | +1 | javac | 22 | the patch passed | | +1 | checkstyle | 24 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 267 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 742 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 49 | the patch passed | | +1 | javadoc | 17 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 180 | hbase-common in the patch passed. | | +1 | asflicense | 11 | The patch does not generate ASF License warnings. | | | | 2799 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-363/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/363 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ce32a708d3ba 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 9ac9505f2a | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-363/2/testReport/ | | Max. process+thread count | 290 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-363/2/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881238#comment-16881238
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #172 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/172/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/172//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/172//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/172//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [hbase] Braavos96 commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
Braavos96 commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301579600
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
+
+@Rule
+public final ExpectedException thrown = ExpectedException.none();
+
+@Test
+public void testAppendKeyValue() {
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz"));
+Assert.assertEquals(
+new StringBuilder("bar->baz"), Strings.appendKeyValue(
+new StringBuilder(), "bar", "baz", "->", "| "));
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "=", ", "));
+Assert.assertEquals(
+new StringBuilder("foo| bar->baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "->", "| "));
+}
+
+@Test
+public void testDomainNamePointerToHostName() {
+System.out.println(Strings.domainNamePointerToHostName(null));
Review comment:
Sorry forgot to change that to an assert statement
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Braavos96 commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
Braavos96 commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301579424
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
+
+@Rule
+public final ExpectedException thrown = ExpectedException.none();
+
+@Test
+public void testAppendKeyValue() {
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz"));
+Assert.assertEquals(
+new StringBuilder("bar->baz"), Strings.appendKeyValue(
+new StringBuilder(), "bar", "baz", "->", "| "));
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "=", ", "));
+Assert.assertEquals(
+new StringBuilder("foo| bar->baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "->", "| "));
+}
+
+@Test
+public void testDomainNamePointerToHostName() {
Review comment:
Sorry I seemed to have forgotten to add that test
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Braavos96 commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
Braavos96 commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301579132
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
+
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Braavos96 commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
Braavos96 commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301579080
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #368: HBASE-22662 Move RSGroupInfoManager to hbase-server
Apache-HBase commented on issue #368: HBASE-22662 Move RSGroupInfoManager to hbase-server URL: https://github.com/apache/hbase/pull/368#issuecomment-509616445 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 25 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | 0 | mvndep | 25 | Maven dependency ordering for branch | | +1 | mvninstall | 245 | HBASE-22514 passed | | +1 | compile | 73 | HBASE-22514 passed | | +1 | checkstyle | 82 | HBASE-22514 passed | | +1 | shadedjars | 275 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 244 | HBASE-22514 passed | | +1 | javadoc | 48 | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 240 | the patch passed | | +1 | compile | 74 | the patch passed | | +1 | javac | 74 | the patch passed | | -1 | checkstyle | 68 | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 272 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 723 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 259 | the patch passed | | -1 | javadoc | 32 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | -1 | unit | 8817 | hbase-server in the patch failed. | | +1 | unit | 207 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 52 | The patch does not generate ASF License warnings. | | | | 12149 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.assignment.TestOpenRegionProcedureHang | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/368 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 52e33b11b803 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | HBASE-22514 / 83db67204d | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/1/artifact/out/diff-checkstyle-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/1/artifact/out/diff-javadoc-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/1/testReport/ | | Max. process+thread count | 4883 (vs. ulimit of 1) | | modules | C: hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-368/1/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #354: HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM…
Apache-HBase commented on issue #354: HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM… URL: https://github.com/apache/hbase/pull/354#issuecomment-509610265 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 169 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 25 | Maven dependency ordering for branch | | +1 | mvninstall | 234 | master passed | | +1 | compile | 73 | master passed | | +1 | checkstyle | 74 | master passed | | +1 | shadedjars | 250 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 238 | master passed | | +1 | javadoc | 48 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for patch | | +1 | mvninstall | 219 | the patch passed | | +1 | compile | 74 | the patch passed | | +1 | javac | 74 | the patch passed | | +1 | checkstyle | 79 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 262 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 692 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 251 | the patch passed | | +1 | javadoc | 48 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 13982 | hbase-server in the patch failed. | | +1 | unit | 507 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 44 | The patch does not generate ASF License warnings. | | | | 17580 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/354 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c4d444d1664e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 605f8a15bb | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/5/testReport/ | | Max. process+thread count | 5148 (vs. ulimit of 1) | | modules | C: hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-354/5/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #365: HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because almost all of the heap allocation will happen in another separated allocator
Apache-HBase commented on issue #365: HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because almost all of the heap allocation will happen in another separated allocator named HEAP URL: https://github.com/apache/hbase/pull/365#issuecomment-509604114 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 72 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 35 | Maven dependency ordering for branch | | +1 | mvninstall | 297 | master passed | | +1 | compile | 79 | master passed | | +1 | checkstyle | 93 | master passed | | +1 | shadedjars | 283 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 256 | master passed | | +1 | javadoc | 57 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for patch | | +1 | mvninstall | 255 | the patch passed | | +1 | compile | 77 | the patch passed | | +1 | javac | 77 | the patch passed | | +1 | checkstyle | 94 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 283 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 828 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 284 | the patch passed | | +1 | javadoc | 55 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 179 | hbase-common in the patch passed. | | -1 | unit | 19855 | hbase-server in the patch failed. | | +1 | asflicense | 75 | The patch does not generate ASF License warnings. | | | | 23537 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs | | | hadoop.hbase.master.TestAssignmentManagerMetrics | | | hadoop.hbase.master.procedure.TestProcedurePriority | | | hadoop.hbase.replication.TestReplicationSyncUpTool | | | hadoop.hbase.namespace.TestNamespaceAuditor | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.util.TestFromClientSide3WoUnsafe | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.replication.TestReplicationSmallTests | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/365 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 162750ac65d7 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 605f8a15bb | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/1/testReport/ | | Max. process+thread count | 5183 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/1/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22606) BucketCache improvement with additional tests
[ https://issues.apache.org/jira/browse/HBASE-22606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-22606: - Resolution: Fixed Status: Resolved (was: Patch Available) Closing PR and marking this as resolved. Manually squashed PR 333 commits locally, then pushed to master. Thanks for the contribution, [~vjasani]! > BucketCache improvement with additional tests > - > > Key: HBASE-22606 > URL: https://issues.apache.org/jira/browse/HBASE-22606 > Project: HBase > Issue Type: Task >Affects Versions: 3.0.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0 > > > BucketCache code improvement with some additional test cases to cover up > quite some logic implemented for Caching -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] wchevreuil closed pull request #333: [HBASE-22606] : BucketCache additional tests
wchevreuil closed pull request #333: [HBASE-22606] : BucketCache additional tests URL: https://github.com/apache/hbase/pull/333 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] petersomogyi commented on issue #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
petersomogyi commented on issue #363: Add unit tests for org.apache.hadoop.hbase.util.Strings URL: https://github.com/apache/hbase/pull/363#issuecomment-509597822 Please create an HBASE Jira for this change at https://issues.apache.org/jira/secure/Dashboard.jspa and use it in the commit name. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #365: HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because almost all of the heap allocation will happen in another separated allocator
Apache-HBase commented on issue #365: HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because almost all of the heap allocation will happen in another separated allocator named HEAP URL: https://github.com/apache/hbase/pull/365#issuecomment-509594160 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 84 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 25 | Maven dependency ordering for branch | | +1 | mvninstall | 276 | master passed | | +1 | compile | 79 | master passed | | +1 | checkstyle | 109 | master passed | | +1 | shadedjars | 317 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 279 | master passed | | +1 | javadoc | 57 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 262 | the patch passed | | +1 | compile | 78 | the patch passed | | +1 | javac | 78 | the patch passed | | +1 | checkstyle | 106 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 299 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 895 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 301 | the patch passed | | +1 | javadoc | 59 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 187 | hbase-common in the patch passed. | | +1 | unit | 14728 | hbase-server in the patch passed. | | +1 | asflicense | 50 | The patch does not generate ASF License warnings. | | | | 18578 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/365 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c9f89faaca9d 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 605f8a15bb | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/2/testReport/ | | Max. process+thread count | 4923 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-365/2/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22665) RegionServer abort failed when AbstractFSWAL.shutdown hang
[
https://issues.apache.org/jira/browse/HBASE-22665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881118#comment-16881118
]
Duo Zhang commented on HBASE-22665:
---
Oh, you are right.
Let me think how to fix this. Need to add more comments.
Thanks a lot.
> RegionServer abort failed when AbstractFSWAL.shutdown hang
> --
>
> Key: HBASE-22665
> URL: https://issues.apache.org/jira/browse/HBASE-22665
> Project: HBase
> Issue Type: Bug
> Environment: HBase 2.1.2
> Hadoop 3.1.x
> centos 7.4
>Reporter: Yechao Chen
>Priority: Major
> Attachments: image-2019-07-08-16-07-37-664.png,
> image-2019-07-08-16-08-26-777.png, image-2019-07-08-16-14-43-455.png,
> jstack_20190625, jstack_20190704_1, jstack_20190704_2, rs.log.part1
>
>
> We use hbase 2.1.2,when the rs with heavy qps and rs abort with error like
> "Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to
> get sync result after 30 ms for txid=36380334, WAL system stuck?"
>
> RegionServer aborted failed when AbstractFSWAL.shutdown hang
>
> jstack info always show the regionserver hang with "AbstractFSWAL.shutdown"
> "regionserver/hbase-slave-216-99:16020" #25 daemon prio=5 os_prio=0
> tid=0x7f204282c600 nid=0x34aa waiting on condition [0x7f0fe044d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f18a49b2bb8> (a
> java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:224)
> {color:#FF}at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285){color}
> {color:#FF} at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:815){color}
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.shutdown(AbstractFSWALProvider.java:168)
> at
> org.apache.hadoop.hbase.wal.RegionGroupingProvider.shutdown(RegionGroupingProvider.java:221)
> at org.apache.hadoop.hbase.wal.WALFactory.shutdown(WALFactory.java:239)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.shutdownWAL(HRegionServer.java:1445)
> {color:#FF}at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1117){color}
> {color:#FF} at java.lang.Thread.run(Thread.java:745){color}
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [hbase] sunhelly commented on issue #354: HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM…
sunhelly commented on issue #354: HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM… URL: https://github.com/apache/hbase/pull/354#issuecomment-509573379 @jatsakthi I have updated testKillAllRSInGroup() to cover both null lastHost and group-server lastHost. Please review again. > However I still doubt about the requirement of the testKillRS() which just stops a rs from the group and moves one rs from the default group to the custom group. (However, both the restart & move case has been covered in testKillAllRSInGroup()). Are we trying to keep that test alive for the case where the lastHost is null? In testKillRS(), lastHost should not be null, because when creating test table, it adds group into to namespace config, as a result, all the table regions will be on group servers at the beginning, which is different from my moving table to group to make table regions on group servers. And I think this test case used disable/enable table to enforce reassign, which is different from my auto-reassign. This can be kept for testing force-reassign? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sunhelly commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
sunhelly commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi… URL: https://github.com/apache/hbase/pull/323#issuecomment-509567691 @wchevreuil I have updated the patch, please help to review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881082#comment-16881082
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #3 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/3/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/3//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/3//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/3//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22665) RegionServer abort failed when AbstractFSWAL.shutdown hang
[
https://issues.apache.org/jira/browse/HBASE-22665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881077#comment-16881077
]
Wellington Chevreuil commented on HBASE-22665:
--
{quote}We haven't done this? IIRC we will move the entries from unackedAppends
to toWriteAppends in syncFailed...
{quote}
Apparently, in
[_syncFailed_|https://github.com/apache/hbase/blob/branch-2.1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java#L314]
now, we just iterate over current _unackedAppends_, adding its elements to
_toWriteAppends_, but we never clean _unackedAppends_. From what I can see in
the code, we only remove elements from _unackedAppends_ in
[_finishSync_|https://github.com/apache/hbase/blob/branch-2.1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java#L416].
Looks like an edge case, where we syncFailed and no sync has ever been
triggered after that (maybe syncers are aborted? I need to analyse that further
yet), then log roll went up, LogRoller entered that _waitForSafePoint_ method,
it triggered the consumer thread in WAITING_ROLL state, reaching [this
condition|https://github.com/apache/hbase/blob/branch-2.1/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java#L487],
which is false, thus never notifying LogRoller thread.
I'm not sure we can simply clear _unackedAppends_ in _syncFailed_, as some
other methods are relying on this condition to decide what to do. Also, that
could allow a log roll to happen when we still have entries to be synced on a
present file.
> RegionServer abort failed when AbstractFSWAL.shutdown hang
> --
>
> Key: HBASE-22665
> URL: https://issues.apache.org/jira/browse/HBASE-22665
> Project: HBase
> Issue Type: Bug
> Environment: HBase 2.1.2
> Hadoop 3.1.x
> centos 7.4
>Reporter: Yechao Chen
>Priority: Major
> Attachments: image-2019-07-08-16-07-37-664.png,
> image-2019-07-08-16-08-26-777.png, image-2019-07-08-16-14-43-455.png,
> jstack_20190625, jstack_20190704_1, jstack_20190704_2, rs.log.part1
>
>
> We use hbase 2.1.2,when the rs with heavy qps and rs abort with error like
> "Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to
> get sync result after 30 ms for txid=36380334, WAL system stuck?"
>
> RegionServer aborted failed when AbstractFSWAL.shutdown hang
>
> jstack info always show the regionserver hang with "AbstractFSWAL.shutdown"
> "regionserver/hbase-slave-216-99:16020" #25 daemon prio=5 os_prio=0
> tid=0x7f204282c600 nid=0x34aa waiting on condition [0x7f0fe044d000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f18a49b2bb8> (a
> java.util.concurrent.locks.ReentrantLock$FairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
> at
> java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:224)
> {color:#FF}at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285){color}
> {color:#FF} at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:815){color}
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.shutdown(AbstractFSWALProvider.java:168)
> at
> org.apache.hadoop.hbase.wal.RegionGroupingProvider.shutdown(RegionGroupingProvider.java:221)
> at org.apache.hadoop.hbase.wal.WALFactory.shutdown(WALFactory.java:239)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.shutdownWAL(HRegionServer.java:1445)
> {color:#FF}at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1117){color}
> {color:#FF} at java.lang.Thread.run(Thread.java:745){color}
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881076#comment-16881076 ] kevin su commented on HBASE-22662: -- [~Apache9] sorry for wrong patch, and thanks for your reply, help me a lot > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task > Components: rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881073#comment-16881073
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #2 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/2/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/2//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/2//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/2//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Yechao Chen
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [hbase] Apache9 opened a new pull request #368: HBASE-22662 Move RSGroupInfoManager to hbase-server
Apache9 opened a new pull request #368: HBASE-22662 Move RSGroupInfoManager to hbase-server URL: https://github.com/apache/hbase/pull/368 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-connectors] asfgit commented on issue #34: HBASE-22599 Let hbase-connectors compile against HBase 2.2.0
asfgit commented on issue #34: HBASE-22599 Let hbase-connectors compile against HBase 2.2.0 URL: https://github.com/apache/hbase-connectors/pull/34#issuecomment-509551540 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-CONNECTORS-Build/51/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-22662: - Assignee: Duo Zhang > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22662: -- Component/s: rsgroup > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task > Components: rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase-connectors] meszibalu commented on issue #34: HBASE-22599 Let hbase-connectors compile against HBase 2.2.0
meszibalu commented on issue #34: HBASE-22599 Let hbase-connectors compile against HBase 2.2.0 URL: https://github.com/apache/hbase-connectors/pull/34#issuecomment-509549235 retest build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi…
Apache-HBase commented on issue #323: HBASE-22414 Interruption of moving regions in RSGroup will cause regi… URL: https://github.com/apache/hbase/pull/323#issuecomment-509533310 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 62 | Docker mode activated. | ||| _ Prechecks _ | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 293 | master passed | | +1 | compile | 22 | master passed | | +1 | checkstyle | 11 | master passed | | +1 | shadedjars | 289 | branch has no errors when building our shaded downstream artifacts. | | +1 | findbugs | 45 | master passed | | +1 | javadoc | 20 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 264 | the patch passed | | +1 | compile | 24 | the patch passed | | +1 | javac | 24 | the patch passed | | +1 | checkstyle | 13 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 290 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 853 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | findbugs | 46 | the patch passed | | +1 | javadoc | 21 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 635 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 11 | The patch does not generate ASF License warnings. | | | | 3214 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/323 | | Optional Tests | dupname asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 84c183858ba1 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | master / 605f8a15bb | | maven | version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) | | Default Java | 1.8.0_181 | | findbugs | v3.1.11 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/11/testReport/ | | Max. process+thread count | 4298 (vs. ulimit of 1) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-323/11/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881027#comment-16881027 ] Duo Zhang commented on HBASE-22662: --- Oh, seems not easy, we have a RSGroupInfo which is in the hbase-common module, and marked as IA.Public. And it is under o.a.h.h.rsgroup... > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22661) list_regions command in hbase shell is broken
[
https://issues.apache.org/jira/browse/HBASE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881021#comment-16881021
]
Duo Zhang commented on HBASE-22661:
---
I think the problem is that we use some methods only in HBaseAdmin but not in
the Admin interface, so after the removal of the HBaseAdmin the command is
broken.
We do not have a UT for the list_regions command?
> list_regions command in hbase shell is broken
> -
>
> Key: HBASE-22661
> URL: https://issues.apache.org/jira/browse/HBASE-22661
> Project: HBase
> Issue Type: Bug
>Reporter: Toshihiro Suzuki
>Priority: Major
>
> I faced the following error in the master branch:
> {code}
> hbase(main):001:0> create "test", "cf"
> 2019-07-07 23:24:15,254 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Created table test
> Took 6.5678 seconds
> => Hbase::Table - test
> hbase(main):002:0> list_regions "test"
> ERROR: undefined method `getClusterStatus' for
> #
> Did you mean? get_cluster_metrics
> For usage try 'help "list_regions"'
> Took 0.1997 seconds
> {code}
> I didn't check if the other branches have the same issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-15666) shaded dependencies for hbase-testing-util
[ https://issues.apache.org/jira/browse/HBASE-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-15666: Status: Patch Available (was: Open) > shaded dependencies for hbase-testing-util > -- > > Key: HBASE-15666 > URL: https://issues.apache.org/jira/browse/HBASE-15666 > Project: HBase > Issue Type: New Feature > Components: test >Affects Versions: 1.2.0, 1.1.0 >Reporter: Sean Busbey >Assignee: Balazs Meszaros >Priority: Critical > > Folks that make use of our shaded client but then want to test things using > the hbase-testing-util end up getting all of our dependencies again in the > test scope. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] petersomogyi commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
petersomogyi commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301431659
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
+
Review comment:
Add a ClassRule:
```
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestStrings.class);
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] petersomogyi commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
petersomogyi commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301435454
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
+
+@Rule
+public final ExpectedException thrown = ExpectedException.none();
+
+@Test
+public void testAppendKeyValue() {
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz"));
+Assert.assertEquals(
+new StringBuilder("bar->baz"), Strings.appendKeyValue(
+new StringBuilder(), "bar", "baz", "->", "| "));
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "=", ", "));
+Assert.assertEquals(
+new StringBuilder("foo| bar->baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "->", "| "));
+}
+
+@Test
+public void testDomainNamePointerToHostName() {
Review comment:
The domainNamePointerToHostName(String) method is removing the trailing `.`
from the input string. You did not cover this scenario in the test cases. Could
you create one?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] petersomogyi commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
petersomogyi commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301433702
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
+
+@Rule
+public final ExpectedException thrown = ExpectedException.none();
+
+@Test
+public void testAppendKeyValue() {
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz"));
+Assert.assertEquals(
+new StringBuilder("bar->baz"), Strings.appendKeyValue(
+new StringBuilder(), "bar", "baz", "->", "| "));
+Assert.assertEquals(
+new StringBuilder("foo, bar=baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "=", ", "));
+Assert.assertEquals(
+new StringBuilder("foo| bar->baz"), Strings.appendKeyValue(
+new StringBuilder("foo"), "bar", "baz", "->", "| "));
+}
+
+@Test
+public void testDomainNamePointerToHostName() {
+System.out.println(Strings.domainNamePointerToHostName(null));
Review comment:
Use Log instead of writing to stdout if needed.
Shouldn't this be an assertNull?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] petersomogyi commented on a change in pull request #363: Add unit tests for org.apache.hadoop.hbase.util.Strings
petersomogyi commented on a change in pull request #363: Add unit tests for
org.apache.hadoop.hbase.util.Strings
URL: https://github.com/apache/hbase/pull/363#discussion_r301431101
##
File path:
hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestStrings.java
##
@@ -0,0 +1,64 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.util;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.ExpectedException;
+
+public class TestStrings {
Review comment:
Add @Category(SmallTests.class)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881014#comment-16881014 ] kevin su commented on HBASE-22662: -- so the work will be # change package to o.a.h.h.master.rsgroup # move config in RSGroupInfoManager to HConstants anything I miss > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevin su reassigned HBASE-22662: Assignee: (was: kevin su) > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880986#comment-16880986 ] Duo Zhang commented on HBASE-22662: --- [~pingsutw] Sorry but it is not just a simple 'move', I plan to change the package to o.a.h.h.master.rsgroup, and also done some refactoring, as the hbase table related configs should not be placed in the interface, it is implementation specific. > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: kevin su >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22667) [Flush] NPE when region flushs
[
https://issues.apache.org/jira/browse/HBASE-22667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan resolved HBASE-22667.
---
Resolution: Duplicate
> [Flush] NPE when region flushs
> --
>
> Key: HBASE-22667
> URL: https://issues.apache.org/jira/browse/HBASE-22667
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Priority: Critical
> Fix For: 1.5.0, 1.4.7
>
>
> {code}
> 2019-07-09 08:02:14,262 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> hostname,16020,1562233574704: Replay of WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> namespace:table,963,1562296120996.b8e2f19748d374d192b93f106a0f73b3.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2646)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2284)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2095)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:508)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:478)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.(ArrayList.java:177)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:863)
> at
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1172)
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1145)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$900(HStore.java:122)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2505)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2600)
> ... 9 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22667) [Flush] NPE when region flushs
[
https://issues.apache.org/jira/browse/HBASE-22667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880977#comment-16880977
]
Reid Chan commented on HBASE-22667:
---
Confirmed, fixed in1.4.7.
> [Flush] NPE when region flushs
> --
>
> Key: HBASE-22667
> URL: https://issues.apache.org/jira/browse/HBASE-22667
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Priority: Critical
>
> {code}
> 2019-07-09 08:02:14,262 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> hostname,16020,1562233574704: Replay of WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> namespace:table,963,1562296120996.b8e2f19748d374d192b93f106a0f73b3.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2646)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2284)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2095)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:508)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:478)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.(ArrayList.java:177)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:863)
> at
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1172)
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1145)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$900(HStore.java:122)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2505)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2600)
> ... 9 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22667) [Flush] NPE when region flushs
[
https://issues.apache.org/jira/browse/HBASE-22667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-22667:
--
Fix Version/s: 1.4.7
1.5.0
> [Flush] NPE when region flushs
> --
>
> Key: HBASE-22667
> URL: https://issues.apache.org/jira/browse/HBASE-22667
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Priority: Critical
> Fix For: 1.5.0, 1.4.7
>
>
> {code}
> 2019-07-09 08:02:14,262 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> hostname,16020,1562233574704: Replay of WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> namespace:table,963,1562296120996.b8e2f19748d374d192b93f106a0f73b3.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2646)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2284)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2095)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:508)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:478)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.(ArrayList.java:177)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:863)
> at
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1172)
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1145)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$900(HStore.java:122)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2505)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2600)
> ... 9 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880970#comment-16880970 ] liang.feng commented on HBASE-22658: @[~stack] 1. This method name " isEnableRSGroup" is similar to "is_enabled" command of the hbase shell. 2. It is really better better to put code inside a try/finally. I will consider improving it. 3. It works very well in my hbase cluster. My cluster version is hbase 1.4.9. This patch is based on branch-1, and I've created a pull request at github. The URL is [https://github.com/apache/hbase/pull/367] . PS: You need to change scope of class RSGroupAdminClient to public, if its scope is private in your code in hbase-rsgroup-xxx.jar > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Improvement > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Attachments: HBASE-22658.branch-1.patch > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liang.feng updated HBASE-22658: --- Comment: was deleted (was: 1. This method name " isEnableRSGroup" is similar to "is_enabled" command of the hbase shell. 2. It is really better better to put code inside a try/finally. I will consider improving it. 3. It works very well in my hbase cluster. My cluster version is hbase 1.4.9. This patch is based on branch-1, and I've created a pull request at github. The URL is [https://github.com/apache/hbase/pull/367] . PS: You need to change scope of class RSGroupAdminClient to public, if its scope is private in your code in hbase-rsgroup-xxx.jar) > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Improvement > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Attachments: HBASE-22658.branch-1.patch > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22658) region_mover.rb should choose same rsgroup servers as target servers
[ https://issues.apache.org/jira/browse/HBASE-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880969#comment-16880969 ] liang.feng commented on HBASE-22658: 1. This method name " isEnableRSGroup" is similar to "is_enabled" command of the hbase shell. 2. It is really better better to put code inside a try/finally. I will consider improving it. 3. It works very well in my hbase cluster. My cluster version is hbase 1.4.9. This patch is based on branch-1, and I've created a pull request at github. The URL is [https://github.com/apache/hbase/pull/367] . PS: You need to change scope of class RSGroupAdminClient to public, if its scope is private in your code in hbase-rsgroup-xxx.jar > region_mover.rb should choose same rsgroup servers as target servers > -- > > Key: HBASE-22658 > URL: https://issues.apache.org/jira/browse/HBASE-22658 > Project: HBase > Issue Type: Improvement > Components: rsgroup, shell >Affects Versions: 1.4.10 >Reporter: liang.feng >Priority: Major > Labels: gracefulshutdown, region_mover, rsgroup > Attachments: HBASE-22658.branch-1.patch > > > There are many retries when i am using graceful_stop.sh to shutdown region > server after using regroup, because the target server in a different rsgroup. > This makes it slow to graceful shutdown a regionserver. So i think that > region_mover.rb should only choose same rsgroup servers as target servers. > Region mover is implemented by jruby in hbase1.x and is implemented by java > in hbase2.x . I tried to modify the RegionMover.java class to use the same > logic in hbase2.x, but mvn package failed due to hbase-server module and > hbase-rsgroup moudle needed to depend on each other, then maven throw a "The > projects in the reactor contain a cyclic reference". I couldn't solve it.So I > just uploaded patch for hbase1.x . > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [hbase] Apache-HBase commented on issue #367: region_mover.rb should choose same rsgroup servers as target servers
Apache-HBase commented on issue #367: region_mover.rb should choose same rsgroup servers as target servers URL: https://github.com/apache/hbase/pull/367#issuecomment-509506057 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 30 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ branch-1 Compile Tests _ | ||| _ Patch Compile Tests _ | | -1 | rubocop | 6 | The patch generated 37 new + 354 unchanged - 2 fixed = 391 total (was 356) | | +1 | ruby-lint | 1 | There were no new ruby-lint issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | ||| _ Other Tests _ | | +1 | asflicense | 24 | The patch does not generate ASF License warnings. | | | | 143 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-367/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/367 | | Optional Tests | dupname asflicense rubocop ruby_lint | | uname | Linux f552c4eac6cb 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | branch-1 / d2a4be9 | | maven | version: Apache Maven 3.0.5 | | rubocop | v0.72.0 | | rubocop | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-367/1/artifact/out/diff-patch-rubocop.txt | | ruby-lint | v2.3.1 | | Max. process+thread count | 37 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-367/1/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevin su updated HBASE-22662: - Attachment: HBASE-22662.v1.patch > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: kevin su >Priority: Major > Attachments: HBASE-22662.v0.patch, HBASE-22662.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880960#comment-16880960 ] kevin su commented on HBASE-22662: -- I got it, let me update the patch. > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: kevin su >Priority: Major > Attachments: HBASE-22662.v0.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22667) [Flush] NPE when region flushs
[
https://issues.apache.org/jira/browse/HBASE-22667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880959#comment-16880959
]
binlijin commented on HBASE-22667:
--
Check HBASE-21069
> [Flush] NPE when region flushs
> --
>
> Key: HBASE-22667
> URL: https://issues.apache.org/jira/browse/HBASE-22667
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.6
>Reporter: Reid Chan
>Priority: Critical
>
> {code}
> 2019-07-09 08:02:14,262 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> hostname,16020,1562233574704: Replay of WAL required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> namespace:table,963,1562296120996.b8e2f19748d374d192b93f106a0f73b3.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2646)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2322)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2284)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2170)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2095)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:508)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:478)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at java.util.ArrayList.(ArrayList.java:177)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.updateReaders(StoreScanner.java:863)
> at
> org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:1172)
> at
> org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1145)
> at
> org.apache.hadoop.hbase.regionserver.HStore.access$900(HStore.java:122)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:2505)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2600)
> ... 9 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22662) Move RSGroupInfoManager to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880955#comment-16880955 ] Duo Zhang commented on HBASE-22662: --- The work will be done on branch HBASE-22514, where we have already deleted the RSGroupProtobufUtil I believe... > Move RSGroupInfoManager to hbase-server > --- > > Key: HBASE-22662 > URL: https://issues.apache.org/jira/browse/HBASE-22662 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: kevin su >Priority: Major > Attachments: HBASE-22662.v0.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
